Questions tagged [floating-point]

For questions with concerns specifically related to the usage of floating point, such as accuracy and precision of calculations, handling of 0, infinity, and over/underflow, input/output, and binary representation. Not for code that casually happens to use floating point.

Filter by
Sorted by
Tagged with
5
votes
1answer
45 views

An IEEE half-float implementation in python similar to array.array, is there any way I could make this more efficient?

I've written this class to wrap a collection of bytes and interpret them as 16-bit floats. It's supposed to work like memoryview(buf).cast('f') or ...
2
votes
1answer
75 views

Safe runtime numeric casts

The rationale behind this code is to implement a runtime-safe number conversions in situations when precision loss is possible, but is not expected. Example: passing a ...
1
vote
1answer
81 views

Float to binary conversion

I wrote a program to convert float to it's binary IEEE754 representation in C. Any improvements to this is appreciated ...
4
votes
1answer
43 views

Locale-independent float print function

The objective was to write a locale-independent function that prints floating-point numbers (i.e. one that always uses . as a decimal point regardless of the locale)...
0
votes
1answer
25 views

GLSL atomic float add for architectures with out atomic float add

...
5
votes
1answer
72 views

Floating-point to String Conversion with Given Precision for Fractional Part

Faced with converting floating-point values from a sensor obtained to string on an embedded system to transmit over UART, I came up with the following dbl2str() to ...
7
votes
2answers
130 views

Accounting for floating point error when rounding

I have to implement some business logic that will round a number (typically a double) upwards. Given that this number comes from a third-party and I don't know how they come to calculate it I'm ...
7
votes
2answers
91 views

Pretty-print dollar amounts using custom precision for large amounts

Given a floating-point number, which is representing a dollar value, I'm aiming to return a rounded, human-readable string. Here are the rules for numbers equal to or greater than 1 million, equal to ...
2
votes
1answer
441 views

C# calculate total time worked with TimeSpan [closed]

So there is an existing codebase which calculates the total time worked per month. What the code is doing: iterates through all the working days entries retreives the value of net work time which is ...
3
votes
1answer
401 views

Plotting the integer values in text format in matplotlib piechart

I have done a clustering algorithm and represented the results in a pie chart as shown below. ...
2
votes
0answers
51 views

Suggestions on how to update equation?

After discovering Float Toy, my goal was to create my own version using Python and Tkinter. The results of the endeavor is shown below. Most of the functionality is complete, but the ...
2
votes
1answer
74 views

A program that reads 8763x2 values from a CSV file and initialises an array and few other variables

Relatively new to Python. Have some experience in C++. I have written a small program that reads a CSV file (SAMPLE) and initialises two arrays and few values. ...
0
votes
1answer
81 views

Function for comparing floating point numbers

I am trying to make a function for floating point numbers comparison. The goal is to "evaluate" all operators in single function: >, >=, =, <=, <. If X>Y, then F>0. If X==Y, F==0. If X<=Y, F&...
24
votes
4answers
3k views

IEEE 754 square root with Newton-Raphson

I have an implementation of the sqrt function that uses a combination of IEEE 754, packed bitfields, and the Newton-Raphson algorithm: ...
7
votes
2answers
158 views

Software floating-point multiplication

I wrote a floating-point multiplication function as an excercise. The program compares its result to the usual hardware multiplication result and for this purpose I use unspecified behavior, but the ...
-1
votes
2answers
142 views

Union of two double arrays

I would like to merge two double arrays and remove from the result array approximately equal items. My two input arrays have ...
3
votes
1answer
877 views

Fixed-Size Floating-Point Types

I have created my own personal data structure libraries for C and I've re-engineered it about two times now and I've added alot of preprocessor checks and defines concerning OS type and compiler type. ...
3
votes
1answer
330 views

Python maintain precision for currency

Working with currencies I am aware it is not good to work with floating point values due to rounding and maintaining precision. Better is to work with integers. Would the following Currency class make ...
4
votes
1answer
186 views

Java Mandelbrot Set Viewer

I'm writing my own version of a Mandelbrot set generator, with pan and zoom functionality. (There's not yet a color overlay of sets generated with various beta values; the beta value is changed ...
6
votes
3answers
668 views

Function to extract float from different price patterns

I've got a project where I get product data and need to get the prices which come in, in different formats. Some examples would be: US$17, USD17.00, 17,00€, 17€, GBP17, Only 17,-€, 17.000,00€, 17,000....
4
votes
1answer
110 views

Preperiodic, periodic or aperiodic binary string

This is extension of the rep-string task from Rosette code. I do not only want to check if the input string is the shortest periodic string (rep-string) or not, but also check if it is preperiodic. ...
0
votes
1answer
302 views

Converting Array of `UINT8` (`unsigned char`) to Array of `Float32` (`float`) Using AVX2

Given input array of UINT8 (unsigned char) with numElements how could one efficiently ...
5
votes
2answers
374 views

Calculating the number of coins in money change

I wrote a program that calculates the minimum number of coins required to give a user change. One concern I have is: When do we initialize the value of a float to be negative or positive? I recently ...
3
votes
2answers
589 views

Adding thousand separator to various number types

Am making a pure .net library with helper functions (GitHub). However I wanted to have a thousand separator for all number types and here is what I am currently doing ...
9
votes
1answer
220 views

Function to print a double - exactly

Rarely is the exact value of a double needed to be printed and only its leading significant digits, after rounding, are needed. It is a curiosity to see the exact ...
2
votes
2answers
170 views

Print a double as a decimal with a specified precision

How could I make this code more effective? ...
1
vote
2answers
31 views

Bad performance bash, changing display brightness

I have a script that changes the display brightness using xrandr. It is my first bash script. I mainly had a lot of trouble with dealing with the floating point ...
3
votes
0answers
77 views

Variable bit length lossy floating point compression

I am implementing a new compression algorithm for the weights of a neural network for the Leela Chess project. the weights are roughly 100Mb of ...
6
votes
3answers
1k views

Custom double parser optimized for performance

I'm trying to beat the native Double.TryParse for performance in parsing large multi-million row (simple) CSV files as much as possible. I do not have to support ...
3
votes
1answer
587 views

Convert string to double and check for errors

The following program converts a string to a double, and checks for errors. It uses the strtod() function to do the conversion, and follows the example given for ...
7
votes
3answers
530 views

std::isnormal in C#

Is this correct? Is this portable to .NET core and 32-bit ARM Linux? Is there a faster way, maybe something in the .NET framework? Is there a way to implement float32 version without using dynamic ...
1
vote
1answer
118 views

Scaling decimal numbers up within bounds of limit

This code is part of a simulation and inputs will be positive values less than one and inputs are constrained elsewhere. The goal of the algorithm is to scale the inputs up to much larger numbers, ...
7
votes
5answers
5k views

String to double without std::stod

I encountered a pesky error in Code::Blocks: the conversions functions (stod, stof, stoi, ......
1
vote
2answers
867 views

Store Float16 variables

I'm currently rolling a little Float16 implementation. It's minimalistic and can't handle infinity, NaN and will neither work properly with overflows nor with ...
5
votes
1answer
103 views

Handling float discrepancy in Unity3d

I know there are thousands of posts and questions around the topic of Float Discrepancy. And this is not really a question but a discussion for it. After sitting ...
3
votes
1answer
611 views

Base64 string ↔ float array

I need to convert f32 arrays with a fix length to base64 representation and back. My current code looks like this. It works, but it feels way too complicated. How ...
12
votes
2answers
727 views

Accurate modular arithmetic with double precision

I have a function ProdBigMod which computes the product of two double precise numbers \$x_1\$, \$x_2\$ (both of which are less than \$2^{53} - 1\$) and subsequently ...
2
votes
0answers
56 views

String to long double conversion using FBLD

I always wanted to try fbld x87 instruction to aid to convert string to floating point value. I sure a chance to face this instruction in disassembly of all the ...
1
vote
0answers
44 views

Trim real numbers to a set precision when the input is a string

The objective was to allow me to trim a real number to a set precision after the decimal if one exists but only if the value does not contain an exponent. My input is actually a ...
6
votes
3answers
2k views

Converting Array of Floats to UINT8 (`char`) or UINT16 (`unsigned short`) Using SSE4

The problem is given image in 32 Bit Floating Point Format (float) how to convert it to UINT8 (char) or UNIT16 (...
7
votes
2answers
267 views

Rewritten implementation of quadratic equation formula

I just started learning the C programming language at the university, and today we got a new assignment there is about Refinement of a C software there is designed to find the roots of the general ...
11
votes
2answers
3k views

Computing the integer square root of large numbers

The "integer square root" of a non-negative integer \$ n \$ is defined as the largest integer not greater than \$ \sqrt{n} \$: $$ \operatorname{isqrt}(n) = \lfloor \sqrt{n} \rfloor = \max \{ k \in \...
1
vote
1answer
246 views

Round a number and replace trailing zeros by spaces

Goal From a number x, I would like a string of the rounded value at n decimals and replace each trailing zeros by a space. ...
5
votes
2answers
98 views

Checking whether a number starts with any of a list of string prefixes

This code will return false if the long double rand begins with any string in the vector ...
1
vote
1answer
54 views

error management when parsing multiple floats

I am writing a handler to render a GIF of a Lorentz attractor. I need to parse floating point numbers from the some querystrings attached to the GIF path. If any of them are bad, I need to log an ...
4
votes
1answer
112 views

Calculating Polynomial: Computing very large numbers emulating double using 2 floats

I am working on a polynomial algorithm which requires calculations in large numbers, up to e+38. I am using a 32-bit system with compiler support of 32 bits for long...
5
votes
2answers
241 views

Cloning Python standard library's function math.frexp() that extracts exponent and mantissa from a floating point number

This is Python 3.5 code. I have attempted to develop my own frexp() function that mimics the functionality of Python standard library's ...
2
votes
1answer
535 views

WGS84-coordinate encoded as Integers

I want to store WGS84-coordinates in two integers for latitude and longitude instead of floats to get a higher precision than with floats. Here's my code: ...
3
votes
1answer
141 views

Indian currency format using C [closed]

Indian Currency format is 12345 → "12,345" (for odd length) and 123456 → "1,23,456" (for even length). I have included all ...
1
vote
0answers
596 views

Check narrowing floating point conversions

In some code I'm working on there is a (narrowing) conversion from double to float. The legacy code has some checks in place ...