1
\$\begingroup\$

I've written a fairly simple script in F# which reads structured data from binary. I've avoided using the built-in .NET BinaryReader class because it's stateful in that it maintains its index in the stream. Instead I wanted to write a purely functional implementation. I'm doing this for my own personal learning so YAGNI does not apply.

First I defined a binary buffer type and a module of functions to operate on it.

type Buffer = Buffer of uint8 []
type ReadResult<'a> =
  | Ok of 'a*Buffer // (value, remaining bytes)
  | BufferExhausted

module Buffer =
  type ByteOrder =
    | LittleEndian
    | BigEndian

  let private nativeOrder =
    if BitConverter.IsLittleEndian
      then LittleEndian
      else BigEndian

  let private swapOrder (x:ByteOrder) (y:ByteOrder) =
    if x <> y
      then Array.rev
      else id

  let private nativeOfLE = swapOrder LittleEndian nativeOrder
  let private nativeOfBE = swapOrder BigEndian nativeOrder

  let ofHexString (hex:string) =
    if (String.length hex % 2 = 0
      then
        hex.ToCharArray()
        |> Array.chunkBySize 2
        |> Array.map (fun nibbles -> Byte.Parse(String nibbles, NumberStyles.HexNumber))
        |> Buffer
        |> Result.Ok
      else
        Error "hex number length must be multiple of 2"

  let toHexString (Buffer buffer) =
    buffer
    |> Array.map (fun byte -> byte.ToString("x2")
    |> String.concat ""

  let private bytesToInt16 bytes = BitConverter.ToInt16(bytes, 0)
  let private bytesToInt32 bytes = BitConverter.ToInt32(bytes, 0)
  let private bytesToInt64 bytes = BitConverter.ToInt64(bytes, 0)
  let private bytesToUint16 bytes = BitConverter.ToUInt16(bytes, 0)
  let private bytesToUint32 bytes = BitConverter.ToUInt32(bytes, 0)
  let private bytesToUint64 bytes = BitConverter.ToUInt64(bytes, 0)

  let private read size convert (Buffer buffer) =
    if (Array.length buffer) >= size
      then
        let (bytes, remaining) = Array.splitAt size buffer
        Ok (convert bytes, Buffer remaining)
      else
        BufferExhausted

  let readInt16LE = read sizeof<int16> (nativeOfLE >> bytesToInt16)
  let readInt16BE = read sizeof<int16> (nativeOfBE >> bytesToInt16)
  let readInt32LE = read sizeof<int32> (nativeOfLE >> bytesToInt32)
  let readInt32BE = read sizeof<int32> (nativeOfBE >> bytesToInt32)
  let readInt64LE = read sizeof<int64> (nativeOfLE >> bytesToInt64)
  let readInt64BE = read sizeof<int64> (nativeOfBE >> bytesToInt64)
  // repeated for unsigned types...

So when it comes to read a structured message from the binary buffer, I've tried to build up a workflow..

type Message = {
  fieldA: int32
  fieldB: uint64
  fieldC: uint16
}

let bind expr fn =
  match expr with
    | BufferExhausted -> BufferExhausted
    | Ok (value, remaining) -> fn (value, remaining)

let readMessageWorkflow buffer =
  bind (buffer |> Buffer.readInt32LE) (fun (fieldA, buffer) ->
  bind (buffer |> Buffer.readUint64BE) (fun (fieldB, buffer) ->
  bind (buffer |> Buffer.readUint16BE) (fun (fieldC, buffer) ->
    let message =
      {
        fieldA = fieldA
        fieldB = fieldB
        fieldC = fieldC
      }
    ReadResult.Ok (message, buffer)
  )))

This workflow has the signature I want (Buffer->ReadResult<Message>) but the workflow definition becomes quite cumbersome for larger messages. After about a dozen fields it just feels very repetitive and verbose. I'm sure there's a way I could enhance this, maybe as a computation expression looking something like this..

read {
  let! fieldA = Buffer.readInt32LE
  let! fieldB = Buffer.readUint64BE
  let! fieldC = Buffer.readUint16BE

  rtn {
    fieldA = fieldA
    fieldB = fieldB
    fieldC = fieldC
  }
}

Alternatively, could I define a constructor function for the message and somehow thread it through the workflow, so that each step applies the next argument?

let buildMessage fieldC fieldB fieldA =
  {
    fieldA = fieldA
    fieldB = fieldB
    fieldC = fieldC
  }

I would really appreciate any ideas on where I could take this next. I feel like I'm heading in roughly the right direction but there's a whole lot further to go.

I'd also be very appreciative of any feedback on whether my general usage of F# is idiomatic or could be improved.

\$\endgroup\$
3
\$\begingroup\$

do you have to deal with little endian and big endian in the same message? Usually, it is one way or the other for the entire message.

If it's a well structured binary format, you may want to look at the way FSPickler does it -- which lends itself well to formulaic situations using a combinator approach.

It all depends on how much control you need.

Computational expressions are an interesting idea for this scenario. How likely is it that the stream will halt midway? I expect that a computational expression, although it'll give you Ok / Exhausted DU you want, will still be somewhat cluttered. There's just not a condensed syntax for them at this stage.

I'd prefer to read something like this given this is the desired approach:

let readMsg b =
  (Buffer.readInt32LE b,Buffer.readUint64BE b, Buffer.readUint16BE b) 
  |> fun a b c-> { fieldA = a; fieldB = b; fieldC = C }
\$\endgroup\$
  • \$\begingroup\$ Although the majority of fields are little endian, IP addresses and port numbers are encoded in network byte order so I must be prepared to handle both in the same message. Thanks for the heads up on Pickler. This part in particular looks similar to what I'm trying to do. \$\endgroup\$ – MattDavey Apr 12 '17 at 13:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.