Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
The results are in! See what nearly 90,000 developers picked as their most loved, dreaded, and desired coding languages and more in the 2019 Developer Survey.

Questions tagged [floating-point]

For questions with concerns specifically related to the usage of floating point, such as accuracy and precision of calculations, handling of 0, infinity, and over/underflow, input/output, and binary representation. Not for code that casually happens to use floating point.

5
votes
2answers
227 views

Calculating the number of coins in money change

I wrote a program that calculates the minimum number of coins required to give a user change. One concern I have is: When do we initialize the value of a float to be negative or positive? I recently ...
3
votes
2answers
245 views

Adding thousand separator to various number types

Am making a pure .net library with helper functions (GitHub). However I wanted to have a thousand separator for all number types and here is what I am currently doing ...
6
votes
1answer
84 views

Function to print a double - exactly

Rarely is the exact value of a double needed to be printed and only its leading significant digits, after rounding, are needed. It is a curiosity to see the exact ...
2
votes
2answers
64 views

Print a double as a decimal with a specified precision

How could I make this code more effective? ...
1
vote
2answers
26 views

Bad performance bash, changing display brightness

I have a script that changes the display brightness using xrandr. It is my first bash script. I mainly had a lot of trouble with dealing with the floating point ...
3
votes
0answers
58 views

Variable bit length lossy floating point compression

I am implementing a new compression algorithm for the weights of a neural network for the Leela Chess project. the weights are roughly 100Mb of ...
6
votes
3answers
323 views

Custom double parser optimized for performance

I'm trying to beat the native Double.TryParse for performance in parsing large multi-million row (simple) CSV files as much as possible. I do not have to support ...
3
votes
1answer
96 views

Convert string to double and check for errors

The following program converts a string to a double, and checks for errors. It uses the strtod() function to do the conversion, and follows the example given for ...
5
votes
2answers
384 views

std::isnormal in C#

Is this correct? Is this portable to .NET core and 32-bit ARM Linux? Is there a faster way, maybe something in the .NET framework? Is there a way to implement float32 version without using dynamic ...
1
vote
1answer
110 views

Scaling decimal numbers up within bounds of limit

This code is part of a simulation and inputs will be positive values less than one and inputs are constrained elsewhere. The goal of the algorithm is to scale the inputs up to much larger numbers, ...
6
votes
5answers
2k views

String to double without std::stod

I encountered a pesky error in Code::Blocks: the conversions functions (stod, stof, stoi, ......
1
vote
2answers
200 views

Store Float16 variables

I'm currently rolling a little Float16 implementation. It's minimalistic and can't handle infinity, NaN and will neither work properly with overflows nor with ...
5
votes
1answer
80 views

Handling float discrepancy in Unity3d

I know there are thousands of posts and questions around the topic of Float Discrepancy. And this is not really a question but a discussion for it. After sitting ...
3
votes
1answer
279 views

Base64 string ↔ float array

I need to convert f32 arrays with a fix length to base64 representation and back. My current code looks like this. It works, but it feels way too complicated. How ...
10
votes
1answer
400 views

Accurate modular arithmetic with double precision

I have a function ProdBigMod which computes the product of two double precise numbers \$x_1\$, \$x_2\$ (both of which are less than \$2^{53} - 1\$) and subsequently ...
2
votes
0answers
42 views

String to long double conversion using FBLD

I always wanted to try fbld x87 instruction to aid to convert string to floating point value. I sure a chance to face this instruction in disassembly of all the ...
1
vote
0answers
39 views

Trim real numbers to a set precision when the input is a string

The objective was to allow me to trim a real number to a set precision after the decimal if one exists but only if the value does not contain an exponent. My input is actually a ...
6
votes
3answers
687 views

Converting Array of Floats to UINT8 (`char`) or UINT16 (`unsigned short`) Using SSE4

The problem is given image in 32 Bit Floating Point Format (float) how to convert it to UINT8 (char) or UNIT16 (...
7
votes
2answers
190 views

Rewritten implementation of quadratic equation formula

I just started learning the C programming language at the university, and today we got a new assignment there is about Refinement of a C software there is designed to find the roots of the general ...
11
votes
2answers
1k views

Computing the integer square root of large numbers

The "integer square root" of a non-negative integer \$ n \$ is defined as the largest integer not greater than \$ \sqrt{n} \$: $$ \operatorname{isqrt}(n) = \lfloor \sqrt{n} \rfloor = \max \{ k \in \...
1
vote
1answer
169 views

Round a number and replace trailing zeros by spaces

Goal From a number x, I would like a string of the rounded value at n decimals and replace each trailing zeros by a space. ...
5
votes
2answers
85 views

Checking whether a number starts with any of a list of string prefixes

This code will return false if the long double rand begins with any string in the vector ...
1
vote
1answer
52 views

error management when parsing multiple floats

I am writing a handler to render a GIF of a Lorentz attractor. I need to parse floating point numbers from the some querystrings attached to the GIF path. If any of them are bad, I need to log an ...
4
votes
1answer
109 views

Calculating Polynomial: Computing very large numbers emulating double using 2 floats

I am working on a polynomial algorithm which requires calculations in large numbers, up to e+38. I am using a 32-bit system with compiler support of 32 bits for long...
5
votes
2answers
160 views

Cloning Python standard library's function math.frexp() that extracts exponent and mantissa from a floating point number

This is Python 3.5 code. I have attempted to develop my own frexp() function that mimics the functionality of Python standard library's ...
2
votes
1answer
276 views

WGS84-coordinate encoded as Integers

I want to store WGS84-coordinates in two integers for latitude and longitude instead of floats to get a higher precision than with floats. Here's my code: ...
3
votes
1answer
120 views

Indian currency format using C [closed]

Indian Currency format is 12345 → "12,345" (for odd length) and 123456 → "1,23,456" (for even length). I have included all ...
0
votes
0answers
230 views

Check narrowing floating point conversions

In some code I'm working on there is a (narrowing) conversion from double to float. The legacy code has some checks in place ...
6
votes
2answers
1k views

Convert float price to integer price without commas

My payment gateway wants the prices this way: "1050" instead of 10.50. So I created this function: https://repl.it/HgHI/3 ...
3
votes
1answer
609 views

Sorting Floating Point Values

I found sorting large arrays by a comparator that looks at the floating-point difference problematic, especially -ffast-math, (https://stackoverflow.com/questions/...
6
votes
3answers
1k views

User input validation that might be unbreakable for C++

I am new to programming and have been having a lot of trouble with user input validation. Everything I found online I could break and I finally put this function together that appears to be ...
5
votes
1answer
265 views

Efficiently finding approximate fraction, with tolerance for fp rounding

I needed to convert floating point numbers to int/int fractions. Python technically has this functionality, but has issues with rounding errors: ...
3
votes
4answers
997 views

Rounding JavaScript decimals

JavaScript has some inaccurate rounding behavior so I've been looking for a reliable solution without success. There are many answers in this SO post but none cover all the edge cases as far as I can ...
3
votes
2answers
2k views

C program to convert string to floating point

From K&R: Extend atof to handle scientific notation of the form 123.45e-6 where a floating-point number may be followed by ...
5
votes
1answer
136 views

Floating point comparison program

I made a small program to compare floating-point values by running multiple tests to a single pair of floats. I have added extra checks for under/overflow and allowing tolerance. How can I improve ...
1
vote
0answers
55 views

Numeric parser for integers, big integers, and doubles

I have written the following numeric parser. It may not be as fast as Boost's lexical_cast, nor as convenient, but it does get the job done and it does look good. I'...
2
votes
3answers
110 views

Generating a sinusoid sound signal in Java

Is there any legroom left for optimization (without switching to C)? ...
2
votes
1answer
6k views

Converting double to string without scientific notation and without losing digits

Requirements: Must convert a double to a string, without losing any digits. Cannot use scientific notation. In C#, no format ...
6
votes
1answer
51 views

Storing angles with overflow errors

Rather than checking if my angle is in the range of 0 to 2pi every time it gets set, I got the idea to store it as an unsigned short with 0xFFFF being +2pi, thus the standard overflow behavior for ...
2
votes
2answers
2k views

Pythonic way to convert XML attribute to float, or else get None

My Python code is converting XML attributes in an ElementTree to appropriate values. How do I best convert a value in this dict to a ...
5
votes
2answers
426 views

Get a floating point number from a string with a finite state machine

I wanted to restart the big-float project I had paused the moment I suddenly realized that the implementation of the main four rounding modes (which I postponed so long, that I got all of the IEEE-754 ...
0
votes
2answers
101 views

Parsing a string into a double

I'm trying to understand the strtod() function, and how I can handle any user input given to the function. I assume I tested for everything, however I am hoping for ...
2
votes
3answers
255 views

Solving quadratic equation in Clojure

I wrote a function in Clojure to solve quadratic equations using the quadratic formula The function ...
4
votes
1answer
9k views

Swift FloatingPoint rounded to places

I'm playing a bit with extensions for default protocols in Swift 3.0. I would like to implement rounded(toPlaces places: Int) -> Self for ...
3
votes
3answers
3k views

BigDecimal class with arbitrary precision

I recently came across a need to display very large decimal numbers and realized that I could use a BigDecimal class. After some tinkering I decided to leverage the ...
1
vote
3answers
213 views

Haskell function to format a floating-point number with some precision

I have not yet developed a satisfying coding style in Haskell. In the code snippet below, there is a lot wrong to my taste: Helper functions which should be local to ...
1
vote
4answers
2k views

Macro for rounding with variable precision

I use the following MY_ROUND macro for rounding: ...
1
vote
1answer
186 views

Make sure the user enters a valid floating-point number in C

I would like to know if this is a good way to make sure the user enters a valid number. ...
2
votes
3answers
4k views

Addition of two IEEE754-32bit single precision floating point numbers in C++

I made a C++ program which takes a list of triplets with S.No. from a text file, where the triplet, i.e. a,b,c correspond to a+b=c (addition done using float data type). Now I convert a and b from hex ...
5
votes
2answers
693 views

Atomic floating-point addition

I need to be able to atomically add a 32-bit floating point value to a location in memory. Here is what I came up with. While the code is Windows-specific, I'll extend it with Linux support using <...