I'm trying to implement logistic regression and I believe my batch gradient descent is correct or at least it works well enough to give me decent accuracy for the dataset I'm using. When I use stochastic gradient descent I'm getting really poor accuracy so I'm not sure if it's my learning rate, epochs or just my code is incorrect. Also I'm wondering how would I add regularization to both of these? Do I add a variable lambda and multiply it by the learning rate or is the more to it?

BGD:

def batch_gradient(df, weights, bias, lr, epochs):
X = df.values
y = X[:,:1]
X = X[:,1:]
length = X.shape
for i in range(epochs):
output = (sigmoid((np.dot(weights, X.T)+bias)))
weights_tmp = (1/length) * (np.dot(X.T, (output - y.T).T))
bias_tmp = (1/length) * (np.sum(output - y.T))

weights -= (lr * (weights_tmp.T))
bias -= (lr * bias_tmp)

return weights, bias


SGD:

def stochastic_gradient(df, weights, bias, lr, epochs):
x_matrix = df.values
for i in range(epochs):
np.random.shuffle(x_matrix)
x_instance = x_matrix[np.random.choice(x_matrix.shape, 1, replace=True)]
y = x_instance[:,:1]

output = sigmoid(np.dot(weights, x_instance[:,1:].T) + bias)
weights_tmp = lr * np.dot(x_instance[:,1:].T, ((output - y)))

weights = (weights - weights_tmp.T)
bias -= lr * (output - y)

return weights, bias