1. Introduction
This is for my ANN project.
Each building block of ANN, neuron, has activation function which determines how to get the output given the input (see formula in on repo I linked).
I have abstract class Neuron
which represents a neuron in ANN. The following points have been taken into account:
- Not only activation function is intended to be used but its derivative also
- Activation function (and its derivative) returns
double
and takesdouble
as the only argument - It should be easy and clear how to add and use new defined activation function
- Activation function should be initialized at the construction step of
Neuron
instance
2. Possible solution
2.1. Activation class
Each activation function is represented by an abstract class Activation
:
class Activation
{
public :
virtual double f( double x ) = 0;
virtual double fPrime( double x ) = 0;
};
2.2. Concrete activation
Concrete activation is represented by a derived class:
class Identity : public Activation
{
public :
double f( double x ) { return x; }
double fPrime( double x ) { return 1.; }
};
class Sin : public Activation
{
public :
double f( double x ) { return sin(x); }
double fPrime( double x ) { return cos(x); }
};
class Sigmoid : public Activation
{
public :
double f( double x ) { return 1. / ( 1. + exp( -x ) ); }
double fPrime( double x )
{
return ( 1. - f(x) ) / f(x);
}
};
2.3. Activation as a class member
I decided to use std::unique_ptr
to store activation function as a member of Neuron
class.
class Neuron
{
private :
std::unique_ptr<Activation> _act;
public :
Neuron( std::unique_ptr<Activation> act ) :
_act( std::move( act ) )
{ }
double Act( double x ) { return _act->f( x ); }//get output
double ActPrime( double x ) { return _act->fPrime( x ); }//get derivative of activation
//some other functions including pure virtual ones
};
2.4. Adding new activation
In order to add new activation function one "just" should create a class derived from Activation
class and overwrite functions f
and fPrime
.
2.5. Using
Neuron n( std::make_unique<Identity>() );
n.Act( 30. );
n.ActPrime( 30. );
3. Discussion
I am pretty satisfied with this solution but what I am thinking about is is any destructor needed in this case if I use unique_ptr
? If so, then what exactly?
Any critic, suggestion, correction, idea, advice etc. would be appreciate.