Search code examples
integer exact computation with logs...


floating-pointprecisionnumerical-methodsfloating-accuracy

Read More
PHP7.1 json_encode() Float Issue...


phpjsonprecisionphp-7.1

Read More
Math.pow(64, 1/3) returns inexact 4...


javascriptmathprecision

Read More
How to 'cout' the correct number of decimal places of a double value?...


c++doubletostringprecision

Read More
Has anyone encounter this this stripping artifact during "RayTracing in one Weekend"?...


randomprecisionraytracingcompute-shader

Read More
Regex for Oracle NUMBER(10,8) field...


regexoracle-databaseprecisionfixed-point

Read More
Big numbers in c++ issue in cosine similarity...


c++precisionlong-integer

Read More
How are FLT_DIG, DBL_DIG, and LDBL_DIG determined in C...


cfloating-pointprecisionfloating-accuracy

Read More
Which single-precision floating point numbers need 9 significant decimal digits for unambiguous deci...


floating-pointprecision

Read More
Is the Java float to double cast lossy?...


javacastingfloating-pointprecision

Read More
Integer stored in 0 bytes...


sqlitememory-managementprecision

Read More
Python numpy float16 datatype operations, and float8?...


pythonnumpyfloating-pointprecision

Read More
log of unsigned long long int argument...


c++floating-pointprecisionlogarithm

Read More
Raising a float near 1 to an infinite power in Python...


pythonfloating-pointprecisioninfinity

Read More
How best to sum up lots of floating point numbers?...


algorithmlanguage-agnosticfloating-pointprecision

Read More
when to use expm1 instead of exp in java...


javaprecisionexp

Read More
Formatting floats: returning to default...


c++floating-pointprecisioniomanip

Read More
How do I print a double value with full precision using cout?...


c++floating-pointprecisioniostreamcout

Read More
Qt - Set global precision for all tables and models in a program...


c++qtsettingsprecision

Read More
C# double to decimal precision loss...


c#doubledecimalprecision

Read More
Precise Financial Calculation in JavaScript. What Are the Gotchas?...


javascriptdecimalprecisionfinance

Read More
Haskell's double behaviour...


haskelldoubleprecision

Read More
c++ long double (128-bit) precision...


x86-64precisionc++23long-double128-bit

Read More
What class to use for money representation?...


pythoncurrencyrounding-errorarbitrary-precisionprecision

Read More
efficient cumulative log_sum_exp...


rprecision

Read More
Why 5.0 / 2 returns 2.5000000000000000 (scale of 16) instead of 2.5 (scale of 1)?...


sqlpostgresqlprecisionnumericarbitrary-precision

Read More
float is overflowing in openGL despite being under 32 bits...


c++openglfloating-pointglslprecision

Read More
Intel x86_64 assembly compare signed double precision floats...


assemblyx86-64intelprecisionsse

Read More
How many digits can I rely on?...


floating-pointprecisionnumerical-computing

Read More
How to choose epsilon value for floating point?...


mathfloating-pointprecisionepsilon

Read More
BackNext