Python comes with an official Style Guide often just called PEP8, and especially as a beginnger it's a good starting point to get going. Of course coding style is often a matter of choice, however, there are aspects you should definitely follow.
As you know, Python's code structure is build upon indentation, so some parts are already language-defined. The style guide also has recommendations on how to use whitespace within statements and expressions. For example,
, should always be followed be a single space character, while
= is preceded and followed by a single space when used in assignments (there should be no whitespace around
= if used as keyword arguments to functions such as
foo(batz='bar')). So you would go from
gradL_W, gradL_b = self.backward(y_i, y_hat, x_i)
self.w_, self.b_ = self.update(gradL_W, gradL_b)
which does look way cleaner IMHO.
It's always good practice to use descriptive variable names. Simply writing
self.weights instead of
self.w_ wont hurt that much. Apart from that, the Style Guide also recommends to use a single leading underscore for non-public variable names. Note, this is only a convention, since Python has no "real" private member variables as you may know from other languages.
There is a single function in your perceptron class that has a) documentation and b) also follows the docstring-style from the Style Guide. You should apply that to the other functions and maybe even to the class as well. Your current project might be simple, but if projects become more complex, good documentation will save you a lot of headache. Following the official docstring style also has the nice benefit that Python's
help(...) function as well as most IDEs will easily find it.
Apart from the stylistic issues mentioned above, there are some more direct code-focused aspects you should work on.
There is some duplicated code in your implementation of the perceptron model. E.g., you implement the scalar product at the input of the perceptron twice! The first time at
net_input, which is likely where it's supposed to go, and the second time in
predict also reimplements pretty much the whole point of
activation. It's pretty straightforward to implement
def predict(self, X):
s = self.net_input(X)
y_hat = list(map(self.activation, s))
If you use Python for a while you will also learn to use list comprehensions more often than the
list(map(...)) construct, simply because it's more flexible and readable. Rewriting the calculation of
y_hat as list comprehension would look like
y_hat = [self.activation(i) for i in s]. After you have arrived at
def predict(self, X):
s = self.net_input(X)
y_hat = [self.activation(i) for i in s]
you may also find the similarity between
forward quite striking.
forward can now be boiled down to simply
def forward(self, x):
Getting the "first" result of the return value using
 is mainly to ensure that the new version just returns a single number as it was before. Otherwise it would be a list with a single element.
Explicit is better than implicit
(Taken from the Zen of Python)
Using code like
heaviside = lambda x: (1, -1)[x < 0]
might be clever, but clever does not equal good or readable most of the time. The same thing can also be expressed as
return -1 if z < 0 else 1, which is almost a verbatim translation of the mathematical definition of the step function as presented by Rashka. If you don't like the inline-if, you may also write it as "normal" if condition:
if z < 0:
gradL_b = error * 1 is another instance of cleverness. This time, I am not even sure what you are actually trying to accomplish here. Depending on the type of your labels
error will either be of type
int or maybe also
float, and multiplying it with
1 will not change that.
errors += (y_hat != y_i) * 1 is more obvious, but could also be expressed explicitly as
errors += int(y_hat != y_i). Python should even be able to do
errors += y_hat != y_i without changing the end result.
I'm not 100% sure what you are trying to accomplish with some of your function return values. For example, I cannot see the point in
updateis another instance of that topic. I can see you're using it as
self.w_, self.b_ = self.update(gradL_W, gradL_b), but there is not a real reason to do this, since
update already modifies the weights internally.
You will likely have to think about that topic as you further progress in your journey into the depths of neural networks (pun intended). Often, the multilayer perceptron is one of the next logical steps in this process. At this stage, you will need the gradients of previous layers to be able to propagate errors through the network. But time will tell1.
Until then: Happy coding!
1You will also probably soon find out that modelling single neurons is not sufficient (and computationally efficient) for what there is to come. Depending on your learning materials, matrix operations will pop up more and more often. This is the time where the NumPy library will help you shine.