0
\$\begingroup\$

I'm trying to learn how to use logistic regression with Sklearn. After learning the theory, I tried implementing it using the Heart Attack Analysis datasheet from Kaggle.

Here's a snippet of the data:

enter image description here

I tried using just plain logistic regression after one-hot encoding two columns, and I got a score around 50%. The things I one-hot encoded I decided were categorical variables that had more than 2 categories (0 or 1) and were instead of forms like 0, 1, 2 and higher, so I figured one-hot encoding was necessary. I did this for the restecg and cp columns. Here's how my data sheet looked after the one-hot encoding, as well as obviously dropping out the output:

enter image description here

I'm trying to improve upon this. I tried some preprocessing with Sklearn, the reasons for which I'm meant to do this I'm still a bit unclear on but it's recommended, and my score is still quite poor. I'm trying to figure out if there is any way I can improve my score more:

# In[322]:


import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
from sklearn import preprocessing


# In[323]:


heart_df = pd.read_csv("heart.csv")
o2_sat_df = pd.read_csv("o2saturation.csv")

# not going to use the o2 sat df at this time as the data isn't given an id so I can't associate it with patients
# there isn't even an equal number of columns so I can't assume it goes with the same order as in heart_df


# In[324]:


heart_df


# # Clean up data
# This shouldn't need much cleaning at all - according to the accompanied information at https://www.kaggle.com/rashikrahmanpritom/heart-attack-analysis-prediction-dataset it seems that most features are numeric where appropriate and categorical where appropriate. I will need to give cp and restecg columns one-hot encoding as they are categorical with more than 2 categories. L-1 columns are needed for L categories for logit or there is over-prediction, so will need to drop_first.

# In[325]:


# restecg should probably be one-hot encoded as it appears to be categorical with values 0, 1, 2
# It could be the case that the labeling is intentional however and actually in fact numeric..


# In[326]:


outputs = heart_df['output']
heart_df = heart_df.drop(columns=['output'])


# In[327]:


new_restecg = pd.get_dummies(heart_df['restecg'], drop_first = True, prefix = 'restecg')
new_cp = pd.get_dummies(heart_df['cp'], drop_first = True, prefix = 'cp')


# In[328]:


heart_df = heart_df.drop(columns=['cp','restecg'])


# In[329]:


heart_df=heart_df.join(new_restecg)
heart_df=heart_df.join(new_cp)


# In[330]:


heart_df


# # Pre-process data

# In[331]:


patient_numerical = []
patient_categorical = []
indices = heart_df.index.values
train_part = round(0.8*len(indices))
for i in range(len(indices)):
    patient_num = np.asarray(heart_df.loc[heart_df.index == i].values[0][:-5])
    patient_cat = np.asarray(heart_df.loc[heart_df.index == i].values[0][-5:])
    patient_numerical.append(patient_num)
    patient_categorical.append(patient_cat)
    
    


# In[332]:


X_num = patient_numerical

len(X_num)


# In[333]:


# Trying first standard scaling

scaler = preprocessing.StandardScaler().fit(X_num)
X_num = scaler.transform(X_num)

test_x = X_num[train_part:]

# Can just try scaling to a range


# In[334]:


# Reinserting categorical data back

train_x = []
test_x =[]
for i in range(train_part):
    current_list = list(X_num[i])
    for j in range(len(patient_categorical[i])):
        current_list.append(patient_categorical[i][j])
    train_x.append(current_list)

for i in np.arange(train_part, len(X_num)):
    current_list = list(X_num[i])
    for j in range(len(patient_categorical[i])):
        current_list.append(patient_categorical[i][j])
    test_x.append(current_list)


# In[335]:


logit = LogisticRegression(random_state = 0, max_iter = 1000)


# In[336]:


logit.fit(train_x, train_y)


# In[337]:


predictions = logit.predict(test_x)


# In[338]:


accuracy_score(test_y, predictions)

What else can I do to get a higher score? Is preprocessing the key to doing this?

\$\endgroup\$
2
  • \$\begingroup\$ What is your expected score for using logistic regression? \$\endgroup\$
    – Chris Tang
    Jun 24 at 15:59
  • \$\begingroup\$ Maybe you could focus on feature engineering \$\endgroup\$
    – Chris Tang
    Jun 24 at 16:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.