Logistic regression using Sklearn

Question

I'm trying to learn how to use logistic regression with Sklearn. After learning the theory, I tried implementing it using the Heart Attack Analysis datasheet from Kaggle.

Here's a snippet of the data:

I tried using just plain logistic regression after one-hot encoding two columns, and I got a score around 50%. The things I one-hot encoded I decided were categorical variables that had more than 2 categories (0 or 1) and were instead of forms like 0, 1, 2 and higher, so I figured one-hot encoding was necessary. I did this for the restecg and cp columns. Here's how my data sheet looked after the one-hot encoding, as well as obviously dropping out the output:

I'm trying to improve upon this. I tried some preprocessing with Sklearn, the reasons for which I'm meant to do this I'm still a bit unclear on but it's recommended, and my score is still quite poor. I'm trying to figure out if there is any way I can improve my score more:

# In[322]:


import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
from sklearn import preprocessing


# In[323]:


heart_df = pd.read_csv("heart.csv")
o2_sat_df = pd.read_csv("o2saturation.csv")

# not going to use the o2 sat df at this time as the data isn't given an id so I can't associate it with patients
# there isn't even an equal number of columns so I can't assume it goes with the same order as in heart_df


# In[324]:


heart_df


# # Clean up data
# This shouldn't need much cleaning at all - according to the accompanied information at https://www.kaggle.com/rashikrahmanpritom/heart-attack-analysis-prediction-dataset it seems that most features are numeric where appropriate and categorical where appropriate. I will need to give cp and restecg columns one-hot encoding as they are categorical with more than 2 categories. L-1 columns are needed for L categories for logit or there is over-prediction, so will need to drop_first.

# In[325]:


# restecg should probably be one-hot encoded as it appears to be categorical with values 0, 1, 2
# It could be the case that the labeling is intentional however and actually in fact numeric..


# In[326]:


outputs = heart_df['output']
heart_df = heart_df.drop(columns=['output'])


# In[327]:


new_restecg = pd.get_dummies(heart_df['restecg'], drop_first = True, prefix = 'restecg')
new_cp = pd.get_dummies(heart_df['cp'], drop_first = True, prefix = 'cp')


# In[328]:


heart_df = heart_df.drop(columns=['cp','restecg'])


# In[329]:


heart_df=heart_df.join(new_restecg)
heart_df=heart_df.join(new_cp)


# In[330]:


heart_df


# # Pre-process data

# In[331]:


patient_numerical = []
patient_categorical = []
indices = heart_df.index.values
train_part = round(0.8*len(indices))
for i in range(len(indices)):
    patient_num = np.asarray(heart_df.loc[heart_df.index == i].values[0][:-5])
    patient_cat = np.asarray(heart_df.loc[heart_df.index == i].values[0][-5:])
    patient_numerical.append(patient_num)
    patient_categorical.append(patient_cat)
    
    


# In[332]:


X_num = patient_numerical

len(X_num)


# In[333]:


# Trying first standard scaling

scaler = preprocessing.StandardScaler().fit(X_num)
X_num = scaler.transform(X_num)

test_x = X_num[train_part:]

# Can just try scaling to a range


# In[334]:


# Reinserting categorical data back

train_x = []
test_x =[]
for i in range(train_part):
    current_list = list(X_num[i])
    for j in range(len(patient_categorical[i])):
        current_list.append(patient_categorical[i][j])
    train_x.append(current_list)

for i in np.arange(train_part, len(X_num)):
    current_list = list(X_num[i])
    for j in range(len(patient_categorical[i])):
        current_list.append(patient_categorical[i][j])
    test_x.append(current_list)


# In[335]:


logit = LogisticRegression(random_state = 0, max_iter = 1000)


# In[336]:


logit.fit(train_x, train_y)


# In[337]:


predictions = logit.predict(test_x)


# In[338]:


accuracy_score(test_y, predictions)

What else can I do to get a higher score? Is preprocessing the key to doing this?

Answer 1

Overview

You've done an good job:

• You leveraged code written by others with the imports
• Used meaningful names for many of the variables

Here are some adjustments for you to consider, mainly for coding style.

Layout

There are too many blank lines. Delete most of those, and the code is easier to understand just by having more of it on the screen at once.

Comments

There are many comments that look like:

# In[322]:
# In[323]:

They almost look like code, but they convey no meaning to anyone. You should delete them.

Documentation

Add a docstring to the top of the file to summarize the purpose of the code. Also describe the input files and the expected output of the code.

Unused code

This line does nothing, and it appears twice:

heart_df

It can be deleted.

This line does nothing and can be deleted:

len(X_num)

You don't use this variable, and you even say so in the comment that follows it:

o2_sat_df = pd.read_csv("o2saturation.csv")

# not going to use the o2 sat df at this time ...

Delete both.

Lint check

pylint identified a few issues.

There are some long lines that can be shortened, especially the line that starts:

# This shouldn't need much cleaning at all ...

If the comment is useful, make it a comment block.

Maybe the version of Python you use does not complain about these, but mine does:

E0602: Undefined variable 'test_y' 
E0602: Undefined variable 'train_y'

---

With the changes above, you will have a cleaner starting point for code to help get a better score.