I'm trying to learn how to use logistic regression with Sklearn. After learning the theory, I tried implementing it using the Heart Attack Analysis datasheet from Kaggle.
Here's a snippet of the data:
I tried using just plain logistic regression after one-hot encoding two columns, and I got a score around 50%. The things I one-hot encoded I decided were categorical variables that had more than 2 categories (0 or 1) and were instead of forms like 0, 1, 2 and higher, so I figured one-hot encoding was necessary. I did this for the
cp columns. Here's how my data sheet looked after the one-hot encoding, as well as obviously dropping out the output:
I'm trying to improve upon this. I tried some preprocessing with Sklearn, the reasons for which I'm meant to do this I'm still a bit unclear on but it's recommended, and my score is still quite poor. I'm trying to figure out if there is any way I can improve my score more:
# In: import pandas as pd import numpy as np from sklearn.linear_model import LogisticRegression from sklearn.metrics import accuracy_score from sklearn import preprocessing # In: heart_df = pd.read_csv("heart.csv") o2_sat_df = pd.read_csv("o2saturation.csv") # not going to use the o2 sat df at this time as the data isn't given an id so I can't associate it with patients # there isn't even an equal number of columns so I can't assume it goes with the same order as in heart_df # In: heart_df # # Clean up data # This shouldn't need much cleaning at all - according to the accompanied information at https://www.kaggle.com/rashikrahmanpritom/heart-attack-analysis-prediction-dataset it seems that most features are numeric where appropriate and categorical where appropriate. I will need to give cp and restecg columns one-hot encoding as they are categorical with more than 2 categories. L-1 columns are needed for L categories for logit or there is over-prediction, so will need to drop_first. # In: # restecg should probably be one-hot encoded as it appears to be categorical with values 0, 1, 2 # It could be the case that the labeling is intentional however and actually in fact numeric.. # In: outputs = heart_df['output'] heart_df = heart_df.drop(columns=['output']) # In: new_restecg = pd.get_dummies(heart_df['restecg'], drop_first = True, prefix = 'restecg') new_cp = pd.get_dummies(heart_df['cp'], drop_first = True, prefix = 'cp') # In: heart_df = heart_df.drop(columns=['cp','restecg']) # In: heart_df=heart_df.join(new_restecg) heart_df=heart_df.join(new_cp) # In: heart_df # # Pre-process data # In: patient_numerical =  patient_categorical =  indices = heart_df.index.values train_part = round(0.8*len(indices)) for i in range(len(indices)): patient_num = np.asarray(heart_df.loc[heart_df.index == i].values[:-5]) patient_cat = np.asarray(heart_df.loc[heart_df.index == i].values[-5:]) patient_numerical.append(patient_num) patient_categorical.append(patient_cat) # In: X_num = patient_numerical len(X_num) # In: # Trying first standard scaling scaler = preprocessing.StandardScaler().fit(X_num) X_num = scaler.transform(X_num) test_x = X_num[train_part:] # Can just try scaling to a range # In: # Reinserting categorical data back train_x =  test_x = for i in range(train_part): current_list = list(X_num[i]) for j in range(len(patient_categorical[i])): current_list.append(patient_categorical[i][j]) train_x.append(current_list) for i in np.arange(train_part, len(X_num)): current_list = list(X_num[i]) for j in range(len(patient_categorical[i])): current_list.append(patient_categorical[i][j]) test_x.append(current_list) # In: logit = LogisticRegression(random_state = 0, max_iter = 1000) # In: logit.fit(train_x, train_y) # In: predictions = logit.predict(test_x) # In: accuracy_score(test_y, predictions)
What else can I do to get a higher score? Is preprocessing the key to doing this?