Basic neural network

Question

I just programmed a basic neural network in F# to learn the logical OR function. As I am very new to F# and especially functional programming, I did it the imperative way. And even tho it works, I find it highly unattractive. I would like to improve it to make it as functional-like as possible. I though about overloading operators to make scalar products between weights and neuroninput but i don't think its the classier way to make my program nicer.

I want to make my code functional-like.

module nnbasic

let mutable neuroninput = [0.0;0.0]
let mutable weight = [0.4;0.6]
let rate = 0.2
let threeshold = 2.0

// [input1; input2; desiredoutput]
let matrix = [
             [0.0;0.0;0.0];
             [0.0;1.0;1.0];
             [1.0;0.0;1.0];
             [1.0;1.0;1.0]
             ]

let display output real =
    if output = real then printfn "yes"
    else printfn "no"

let output (_ni: float list, _wi: float list) =
    if threeshold > _ni.[0]*_wi.[0] + _ni.[1]*_wi.[1] then 0.0 else 1.0

let mutable iter = 0
let mutable out = 0.0

while iter < 100 do
    for row in matrix do
        neuroninput <- [row.[0];row.[1]]

        out <- output (neuroninput, weight)
        weight <- [weight.[0]+rate*(row.[2]-out);weight.[1]]
        display out row.[2]

        out <- output (neuroninput, weight)
        weight <- [weight.[0];weight.[1]+rate*(row.[2]-out)]

        if threeshold > neuroninput.[0]*weight.[0] + neuroninput.[1]*weight.[1] then display 0.0 row.[2] else display 1.0 row.[2]
        iter <- iter+1

Answer 1

Printing a list of items to the console is inherently imperative. But keeping track of lots of state between iterations using higher-order functions is often inelegant. Regardless, there are some things you can do to clean this up and (if you're compelled) get rid of mutables. Most notably, you can use tuples instead of arrays/lists with an assumed width.

I'm not familiar with this algorithm, so I've merely translated what you have. You can implement it using a pair of mutually recursive functions. This allows you to "persist" the weight values between iterations.

let rate = 0.2
let threshold = 2.0

let inline output a b weightA weightB = if threshold > a * weightA + b * weightB then 0.0 else 1.0
let inline display output real = printfn <| if output = real then "yes" else "no"
let inline computeWeight weight c out = weight + rate * (c - out)

let matrix = 
  [|
    0.0, 0.0, 0.0
    0.0, 1.0, 1.0
    1.0, 0.0, 1.0
    1.0, 1.0, 1.0
  |]

let rec iter n weightA weightB = 
  if n > 0 then
    loop n 0 weightA weightB
and loop n i weightA weightB =
  if i < matrix.Length then
    let a, b, c = matrix.[i]
    let out = output a b weightA weightB
    let weightA = computeWeight weightA c out
    display out c
    let out = output a b weightA weightB
    let weightB = computeWeight weightB c out
    display out c
    loop n (i + 1) weightA weightB
  else iter (n - 1) weightA weightB

Usage:

let initialWeightA = 0.4
let initialWeightB = 0.6
iter 100 initialWeightA initialWeightB

loop could be implemented as a fold, but fold is not typically used with side-effects.