Search code examples
Is floating-point math broken?...


mathfloating-pointlanguage-agnosticprecisionfloating-accuracy

Read More
Long double not giving the required precision...


c++cfilefloating-accuracy

Read More
How to handle very small numbers (<1e-15) in Python, passed to Scipy/Numpy functions?...


numpyscipyprecisionfloating-accuracympmath

Read More
Inaccuracy replicating Fortran mixed-precision expression in Rust...


rustfloating-pointprecisionfloating-accuracy

Read More
integer exact computation with logs...


floating-pointprecisionnumerical-methodsfloating-accuracy

Read More
C++ I've just read that floats are inexact and do not store exact integer values. What does this...


c++floating-accuracy

Read More
How to avoid floating point precision errors with floats or doubles in Java?...


javasumfloating-accuracy

Read More
Truncate (not round off) decimal numbers in JavaScript...


javascriptfloating-pointfloating-accuracy

Read More
How are FLT_DIG, DBL_DIG, and LDBL_DIG determined in C...


cfloating-pointprecisionfloating-accuracy

Read More
Cutover point between two CDF implementations...


numerical-methodsfloating-accuracynormal-distributionlibm

Read More
How to determine the accuracy of Pi (π)...


coptimizationtrigonometryfloating-accuracylookup-tables

Read More
Is "banker's rounding" really more numerically stable?...


floating-pointlanguage-agnosticroundingfloating-accuracybankers-rounding

Read More
Conversion accuracy issues of E57 to LAS in Python using pye57 and laspy...


pythonpython-3.xdata-conversionfloating-accuracypoint-clouds

Read More
Is there a floating point value of x, for which x-x == 0 is false?...


floating-pointfloating-accuracyieee-754

Read More
How to get bc to handle numbers in scientific (aka exponential) notation?...


bashnumericfloating-accuracybc

Read More
postgresql round half down function...


postgresqlfloating-pointplpgsqlroundingfloating-accuracy

Read More
Mystery behind presentation of Floating Point numbers...


swiftfloating-pointdecimalprecisionfloating-accuracy

Read More
Format floating-point numbers with sed to equal precision - add trailing zeros...


bashsedfloating-pointprecisionfloating-accuracy

Read More
Is Google Sheets less or more accurate than Microsoft Excel?...


excelgoogle-sheetsieee-754floating-accuracy

Read More
Whats the smallest de-/normalized number greater than 1?(64bit)...


floating-pointbinaryfloating-accuracyieee-754denormalization

Read More
PHP - Floating Number Precision...


phpfloating-pointprecisionfloating-accuracy

Read More
How to do an integer log2() in C++?...


c++floating-accuracylogarithm

Read More
Floating point less-than-equal comparisons after addition and subtraction...


rfloating-pointfloating-accuracyepsiloninexact-arithmetic

Read More
A more accurate data type than float or double?...


c++floating-pointfloating-accuracydecimal

Read More
How are mathematical libraries with infinite precision implemented?...


precisionfloating-accuracy

Read More
negative zero in python...


pythonfloating-pointfloating-accuracyzero

Read More
Accurate computation of scaled complementary error function, erfcx()...


cmathfloating-pointfloating-accuracy

Read More
How does rounding works in float multiplication?...


floating-pointroundingmultiplicationfloating-accuracyfixed-point

Read More
What is the most efficient way to round a float value to the nearest integer in java?...


javaoptimizationroundingfloating-accuracy

Read More
How does SQL Server store integer values of over 2^24 in a FLOAT(53) column?...


sql-serverfloating-pointfloating-accuracy

Read More
BackNext