# Basic neural network

I just programmed a basic neural network in F# to learn the logical OR function. As I am very new to F# and especially functional programming, I did it the imperative way. And even tho it works, I find it highly unattractive. I would like to improve it to make it as functional-like as possible. I though about overloading operators to make scalar products between weights and neuroninput but i don't think its the classier way to make my program nicer.

I want to make my code functional-like.

module nnbasic

let mutable neuroninput = [0.0;0.0]
let mutable weight = [0.4;0.6]
let rate = 0.2
let threeshold = 2.0

// [input1; input2; desiredoutput]
let matrix = [
[0.0;0.0;0.0];
[0.0;1.0;1.0];
[1.0;0.0;1.0];
[1.0;1.0;1.0]
]

let display output real =
if output = real then printfn "yes"
else printfn "no"

let output (_ni: float list, _wi: float list) =
if threeshold > _ni.[0]*_wi.[0] + _ni.[1]*_wi.[1] then 0.0 else 1.0

let mutable iter = 0
let mutable out = 0.0

while iter < 100 do
for row in matrix do
neuroninput <- [row.[0];row.[1]]

out <- output (neuroninput, weight)
weight <- [weight.[0]+rate*(row.[2]-out);weight.[1]]
display out row.[2]

out <- output (neuroninput, weight)
weight <- [weight.[0];weight.[1]+rate*(row.[2]-out)]

if threeshold > neuroninput.[0]*weight.[0] + neuroninput.[1]*weight.[1] then display 0.0 row.[2] else display 1.0 row.[2]
iter <- iter+1

• Tsk tsk... couldn't resist using mutable huh? :) Commented Mar 28, 2012 at 18:35

Printing a list of items to the console is inherently imperative. But keeping track of lots of state between iterations using higher-order functions is often inelegant. Regardless, there are some things you can do to clean this up and (if you're compelled) get rid of mutables. Most notably, you can use tuples instead of arrays/lists with an assumed width.

I'm not familiar with this algorithm, so I've merely translated what you have. You can implement it using a pair of mutually recursive functions. This allows you to "persist" the weight values between iterations.

let rate = 0.2
let threshold = 2.0

let inline output a b weightA weightB = if threshold > a * weightA + b * weightB then 0.0 else 1.0
let inline display output real = printfn <| if output = real then "yes" else "no"
let inline computeWeight weight c out = weight + rate * (c - out)

let matrix =
[|
0.0, 0.0, 0.0
0.0, 1.0, 1.0
1.0, 0.0, 1.0
1.0, 1.0, 1.0
|]

let rec iter n weightA weightB =
if n > 0 then
loop n 0 weightA weightB
and loop n i weightA weightB =
if i < matrix.Length then
let a, b, c = matrix.[i]
let out = output a b weightA weightB
let weightA = computeWeight weightA c out
display out c
let out = output a b weightA weightB
let weightB = computeWeight weightB c out
display out c
loop n (i + 1) weightA weightB
else iter (n - 1) weightA weightB


Usage:

let initialWeightA = 0.4
let initialWeightB = 0.6
iter 100 initialWeightA initialWeightB


loop could be implemented as a fold, but fold is not typically used with side-effects.

• Nice work. It appears that loop is nicely tail call optimized. I wonder if this can be made into a single tail call optimized function though. That way large values for n will play nicely. Commented Mar 28, 2012 at 20:07
• Why won't large ns play nicely? Commented Mar 28, 2012 at 20:12
• I couldn't tell you how large n could potentially be but when I looked at the IL produced by the F# compiler I think iter is vulnerable to a stack overflow for large values of n. Commented Mar 28, 2012 at 20:16
• loop could return weightA, weightB instead of calling iter. iter would tail-call itself with those values. Commented Mar 28, 2012 at 20:25
• oww, nice! I understand a little better the functional programming mindset now, thank you
– user12195
Commented Mar 28, 2012 at 21:39