# Will this converstion to/from fixed point cause me to lose precision?

I have a situation where I have a fixed point number that I want to convert to and from a floating point number. Specifically it is the SANE_Fixed type from the SANE API. Here is the what the documentation says (I grabbed the bits I think are relavant):

SANE_Fixed is used for variables that can take fixed point values in the range -32768 to 32767.9999 with a resolution of 1/65535.

#define SANE_FIXED_SCALE_SHIFT  16
typedef SANE_Word SANE_Fixed;


The macro SANE_FIXED_SCALE_SHIFT gives the location of the fixed binary point. This standard defines that value to be 16, which yields a resolution of 1/65536.

SANE_Word: A word is encoded as 4 bytes (32 bits). The bytes are ordered from most-significant to least-significant byte (big-endian byte-order).

So, over the wire I get 4 bytes which I convert into a .Net int like this (you can guess what GetByte and SendByte do):

public int GetWord()
{
int value = 0;
value += (GetByte() << 24);
value += (GetByte() << 16);
value += (GetByte() << 8);
value += (GetByte() << 0);
return value;
}


Then I want to convert that int into a floating point number (I went for decimal) like this:

public decimal ToFloating(int source)
{
decimal value = source / ((decimal)(1 << 16));
return value;
}


And I will also need to go back the other way so I would convert to fixed like this:

public int ToFixed(decimal source)
{
decimal value = source * ((decimal)(1 << 16));
return (int)value;
}


And then convert to four bytes like this to send it like so:

public void SendWord(int word)
{
SendByte((word >> 24) & 0xff);
SendByte((word >> 16) & 0xff);
SendByte((word >> 8) & 0xff);
SendByte((word >> 0) & 0xff);
}


So, here is my question:

This seems a pretty simplistic implementation to me and I read lots of stuff on the internet about creating custom fixed point classes and whatnot but, given the constraints of my scenario, is this a safe approach to take? Could I lose precision and is the maths correct?

• Have you tried it? Does it seem to work? – svick Mar 8 '13 at 9:17
• Yes, I have tried it and it does seem to work – kmp Mar 8 '13 at 10:13

If the SANE_Fixed type has a constantly fixed precision of 1/65536, that means it always stores the fractional part of the number as a 16 bit (two-byte) unsigned integer (hence the SANE_FIXED_SCALE_SHIFT of 16 bits). Since the whole number portion of the value can range from -32768 to 32767, that means that the whole part of the number is represented with a 16 bit signed integer (making the type use a total of 4 bytes).
The decimal type in .NET, while it is a floating point type, it is far more precise that the traditional float type. It stores a 96 bit integer (plus an additional bit for the sign). It then stores the position of the decimal point as a 5-bit integer, which allows the decimal point to be located after any digit in the 96-bit integer value. As such, the larger the value gets, the lower the fractional precision becomes.
In other words, the decimal type can store a value up to 79,228,162,514,264,337,593,543,950,335 (29 digits). If it does store a 29 digit number, however, it will be unable to store any fractional value with that. It will only store the whole part of the number. If, however, it is only storing a single digit value, the fractional part can be up to 28 digits long (e.g. 0.0000000000000000000000000001).
Bearing all of that in mind, if the maximum whole number that the SANE_Fixed type can store is 32767 (a five digit number), when cast to a decimal type, that leaves 24 digits of precision (e.g. 1/10^24) in the fractional part of the number. That is still far more precise than the 1/65536 precision provided by the SANE_Fixed type.
Therefore, you can be confident that the decimal type will accurately retain the precise value. Obviously, when casting from a decimal back into a SANE_Fixed type, you will lose precision, but since the value came from that type in the first place, your values are safe.