Linear Regression on random data

Wrote a simple script to implement Linear regression and practice numpy/pandas. Uses random data, so obviously weights (thetas) have no significant meaning. Looking for feedback on

1. Performance
2. Python code style
3. Machine Learning code style
# Performs Linear Regression (from scratch) using randomized data
# Optimizes weights by using Gradient Descent Algorithm

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

np.random.seed(0)

features = 3
trainingSize = 10 ** 1
trainingSteps = 10 ** 3
learningRate = 10 ** -2

randData = np.random.rand(trainingSize, features + 1)
colNames = [f'feature{i}' for i in range(1, features + 1)]
colNames.append('labels')

dummy_column = pd.Series(np.ones(trainingSize), name='f0')
df = pd.DataFrame(randData, columns=colNames)

X = pd.concat([dummy_column, df.drop(columns='labels')], axis=1)
y = df['labels']
thetas = np.random.rand(features + 1)

cost = lambda thetas: np.mean((np.matmul(X, thetas) - y) ** 2) / 2
dJdtheta = lambda thetas, k: np.mean((np.matmul(X, thetas) - y) * X.iloc[:, k])
gradient = lambda thetas: np.array([dJdtheta(thetas, k) for k in range(X.shape)])

print(cost(thetas))

errors = np.zeros(trainingSteps)
for step in range(trainingSteps):
errors[step] = cost(thetas)

print(cost(thetas))

# Plots Cost function as gradient descent runs
plt.plot(errors)
plt.xlabel('Training Steps')
plt.ylabel('Cost Function')
plt.show()

Welcome!

Your first two lines are nice comments. Consider putting them in a module docstring:

"""Performs Linear Regression (from scratch) using randomized data.

Optimizes weights by using Gradient Descent Algorithm.
"""

Consider adding random noise to something linear (or to some "wrong model" sine or polynomial), rather than to a constant.

np.random.seed(0)

Nice - reproducibility is Good.

trainingSize = 10 ** 1
trainingSteps = 10 ** 3
learningRate = 10 ** -2

These expressions are correct and clear. But why evaluate a FP expression when you could just write it as a literal? 1e1, 1e3, 1e-2. (This answer would apply in many languages, including Python. And yes, I actually prefer seeing the two integers written as floating point, even if that forces me to call int() on them.)