# Logistic regression using Sklearn in Python

I'm trying to learn how to use logistic regression with Sklearn. After learning the theory, I tried implementing it using the Heart Attack Analysis datasheet from Kaggle.

Here's a snippet of the data:

I tried using just plain logistic regression after one-hot encoding two columns, and I got a score around 50%. The things I one-hot encoded I decided were categorical variables that had more than 2 categories (0 or 1) and were instead of forms like 0, 1, 2 and higher, so I figured one-hot encoding was necessary. I did this for the restecg and cp columns. Here's how my data sheet looked after the one-hot encoding, as well as obviously dropping out the output:

I'm trying to improve upon this. I tried some preprocessing with Sklearn, the reasons for which I'm meant to do this I'm still a bit unclear on but it's recommended, and my score is still quite poor. I'm trying to figure out if there is any way I can improve my score more:

# In[322]:

import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
from sklearn import preprocessing

# In[323]:

# not going to use the o2 sat df at this time as the data isn't given an id so I can't associate it with patients
# there isn't even an equal number of columns so I can't assume it goes with the same order as in heart_df

# In[324]:

heart_df

# # Clean up data
# This shouldn't need much cleaning at all - according to the accompanied information at https://www.kaggle.com/rashikrahmanpritom/heart-attack-analysis-prediction-dataset it seems that most features are numeric where appropriate and categorical where appropriate. I will need to give cp and restecg columns one-hot encoding as they are categorical with more than 2 categories. L-1 columns are needed for L categories for logit or there is over-prediction, so will need to drop_first.

# In[325]:

# restecg should probably be one-hot encoded as it appears to be categorical with values 0, 1, 2
# It could be the case that the labeling is intentional however and actually in fact numeric..

# In[326]:

outputs = heart_df['output']
heart_df = heart_df.drop(columns=['output'])

# In[327]:

new_restecg = pd.get_dummies(heart_df['restecg'], drop_first = True, prefix = 'restecg')
new_cp = pd.get_dummies(heart_df['cp'], drop_first = True, prefix = 'cp')

# In[328]:

heart_df = heart_df.drop(columns=['cp','restecg'])

# In[329]:

heart_df=heart_df.join(new_restecg)
heart_df=heart_df.join(new_cp)

# In[330]:

heart_df

# # Pre-process data

# In[331]:

patient_numerical = []
patient_categorical = []
indices = heart_df.index.values
train_part = round(0.8*len(indices))
for i in range(len(indices)):
patient_num = np.asarray(heart_df.loc[heart_df.index == i].values[0][:-5])
patient_cat = np.asarray(heart_df.loc[heart_df.index == i].values[0][-5:])
patient_numerical.append(patient_num)
patient_categorical.append(patient_cat)

# In[332]:

X_num = patient_numerical

len(X_num)

# In[333]:

# Trying first standard scaling

scaler = preprocessing.StandardScaler().fit(X_num)
X_num = scaler.transform(X_num)

test_x = X_num[train_part:]

# Can just try scaling to a range

# In[334]:

# Reinserting categorical data back

train_x = []
test_x =[]
for i in range(train_part):
current_list = list(X_num[i])
for j in range(len(patient_categorical[i])):
current_list.append(patient_categorical[i][j])
train_x.append(current_list)

for i in np.arange(train_part, len(X_num)):
current_list = list(X_num[i])
for j in range(len(patient_categorical[i])):
current_list.append(patient_categorical[i][j])
test_x.append(current_list)

# In[335]:

logit = LogisticRegression(random_state = 0, max_iter = 1000)

# In[336]:

logit.fit(train_x, train_y)

# In[337]:

predictions = logit.predict(test_x)

# In[338]:

accuracy_score(test_y, predictions)


What else can I do to get a higher score? Is preprocessing the key to doing this?

• What is your expected score for using logistic regression? Jun 24 at 15:59
• Maybe you could focus on feature engineering Jun 24 at 16:01