2
\$\begingroup\$

I have this small program for learning an artificial neuron to act as a simple signal counter: my cell has four input wires (also called dendrites) and a single output wire (also called axon). If at least two input signals are strong enough, the axon is activated. I consider the neuron to be learned as soon as it "guesses" the output sufficiently many times in a row.

Neuron.java:

package net.coderodde.ai.neural;

import java.util.Objects;

/**
 * This class models artificial neurons.
 * 
 * @author Rodion "rodde" Efremov
 */
public class Neuron {

    private final float[] dendriteWeightArray;
    private TransitionFunction transitionFunction;

    public Neuron(int dendriteAmount) {
        this.dendriteWeightArray = new float[dendriteAmount];
    }

    public float getDendriteWeight(int dendriteIndex) {
        checkDendriteIndex(dendriteIndex);
        return dendriteWeightArray[dendriteIndex];
    }

    public void setDendriteWeight(int dendriteIndex, float weight) {
        checkDendriteIndex(dendriteIndex);
        checkIsFinite(weight);
        dendriteWeightArray[dendriteIndex] = weight;
    }

    public void setTransitionFunction(TransitionFunction transitionFunction) {
        Objects.requireNonNull(transitionFunction, 
                               "The input transition function is null.");
        this.transitionFunction = transitionFunction;
    }

    public float process(float... input) {
        float sum = 0.0f;

        for (int i = 0; i < Math.min(input.length,
                                     dendriteWeightArray.length); ++i) {
            sum += input[i] * dendriteWeightArray[i];
        }

        return transitionFunction.process(sum);
    }

    public int getDendriteCount() {
        return dendriteWeightArray.length;
    }

    private void checkDendriteIndex(int index) {
        if (index < 0) {
            throw new IndexOutOfBoundsException(
                    "The dendrite index is negative: " + index);
        }

        if (index >= dendriteWeightArray.length) {
            throw new IndexOutOfBoundsException(
                    "The dendrite index is too large: " + index + ", the " +
                    "amount of dendrites: " + dendriteWeightArray.length);
        }
    }

    private static void checkIsFinite(float f) {
        if (Float.isNaN(f)) {
            throw new IllegalArgumentException("The value is NaN.");
        }

        if (Float.isInfinite(f)) {
            throw new IllegalArgumentException(
                    "The value is infinite in absolute value: " + f);
        }
    }
}

TransitionFunction.java:

package net.coderodde.ai.neural;

/**
 * This interface defines the API for a transition function in artificial 
 * neurons.
 * 
 * @author Rodion "rodde" Efremov
 * @version 1.6
 */
@FunctionalInterface
public interface TransitionFunction {

    /**
     * Maps the input signal to output.
     * 
     * @param input the input signal.
     * @return the output signal. 
     */
    public float process(float input);
}

NeuronLearner.java:

package net.coderodde.ai.neural;

/**
 * This class defines the API for neuron learning routines.
 * 
 * @author Rodion "rodde" Efremov
 * @version 1.6
 */
public interface NeuronLearner {

    /**
     * The actual learning method.
     * 
     * @param neuron the neuron to learn.
     */
    public void learn(Neuron neuron);
}

RandomNeuronLearner.java:

package net.coderodde.ai.neural;

import java.util.Random;

/**
 * This class implements fully random neuron learner routine.
 * 
 * @author Rodion "rodde" Efremov
 * @version 1.6
 */
public class RandomNeuronLearner implements NeuronLearner {

    private final Random random;

    public RandomNeuronLearner(Random random) {
        this.random = random;
    }

    @Override
    public void learn(Neuron neuron) {
        int dendriteCount = neuron.getDendriteCount();

        for (int i = 0; i < dendriteCount; ++i) {
            neuron.setDendriteWeight(i, 2.0f * random.nextFloat() - 1.0f);
        }

        float min = random.nextFloat();

        neuron.setTransitionFunction((f) -> {
            return f > min ? 1.0f : 0.f;
        });
    }
}

Demo.java:

package net.coderodde.ai.neural;

import java.util.Random;

public class Demo {

    private static final int MINIMUM_MATCHES = 50;

    public static void main(String[] args) {
        // This demonstration learns a neuron with four dendrites to act as a
        // "counter": if at least two dendrites receive a signal strong 
        // enough, the axon must be activated.
        Neuron neuron = new Neuron(4);
        Random random = new Random();
        NeuronLearner learner = new RandomNeuronLearner(new Random());

        long ta = System.currentTimeMillis();

        outer:
        for (;;) {
            learner.learn(neuron);

            for (int i = 0; i < MINIMUM_MATCHES; ++i) {
                float[] signal = getRandomSignal(neuron.getDendriteCount(), 
                                                 random);

                if (signalShouldPass(signal) != signalPasses(neuron.process(signal))) {
                    continue outer;
                }
            }

            break;
        }

        long tb = System.currentTimeMillis();

        System.out.println("The neuron learned in " + (tb - ta) + 
                           " milliseconds.");
    }

    /**
     * Generates a random signal.
     * 
     * @param size   the amount of dendrites of a neuron receiving the signal.
     * @param random the random number generator.
     * @return a random signal.
     */
    private static float[] getRandomSignal(int size, Random random) {
        float[] ret = new float[size];

        for (int i = 0; i < size; ++i) {
            ret[i] = random.nextFloat();
        }

        return ret;
    }

    /**
     * Checks whether the input signal should activate the axon.
     * 
     * @param signal the signal to check.
     * @return {@code true} if the input signal should activate the axon.
     */
    private static boolean signalShouldPass(float[] signal) {
        int count = 0;

        for (float f : signal) {
            if (f > 0.5) {
                count++;
            }
        }

        return count >= 2;
    }

    /**
     * Check whether the axon was activated.
     * 
     * @param output the output of the axon.
     * @return {@code true} if the axon was activated.
     */
    private static boolean signalPasses(float output) {
        return output > 0.7f;
    }
}

I get something like this:

The neuron learned in 307 milliseconds.

\$\endgroup\$

1 Answer 1

2
\$\begingroup\$

Doesn't really work

I don't like the concept of this program because mathematically speaking, your neural network can't actually produce the proper result that you are trying to achieve.

The current neural net is too primitive to be able to compute the required function. It simply sums up four weighted inputs and compares against a cutoff, which means that the best it can do is just make a guess equivalent to:

return (a+b+c+d > X) ? 1.0f : 0.0f;

So if you tried test cases like these (with permutations):

(0,0,0.5,0.5)       <-- Tests minimum succeeding case
(1,0.49,0.49,0.49)  <-- Tests maximum failing case

you would find that no neural net could pass every edge case test. As it is, all your program is doing is finding a neural net that can pass 50 random test cases, which means that the random test cases must not be a very good judge of success.

From a theoretical perspective, I think your neural net would need at least two layers. And it would be more interesting if the neural net actually "learned" instead of being randomly assigned new weights when it failed.

\$\endgroup\$
4
  • \$\begingroup\$ After posting my question, I tried to find out the success rate and it turned out to be around 80%. \$\endgroup\$
    – coderodde
    Jul 31, 2015 at 6:53
  • \$\begingroup\$ @coderodde What do you mean by "success rate"? Do you mean how many randomized tests pass? \$\endgroup\$
    – JS1
    Jul 31, 2015 at 7:08
  • \$\begingroup\$ That's correct. \$\endgroup\$
    – coderodde
    Jul 31, 2015 at 7:28
  • 1
    \$\begingroup\$ @coderodde Your code is doing something analogous to approximating a curve with a straight line. So if you pick the best straight line, you might pass 80% of queries of the form: is point P above or below the curve? But I don't really see the point of it, especially there no real learning going on, just random retries. \$\endgroup\$
    – JS1
    Jul 31, 2015 at 7:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.