Mailing List Archive

Pre-RFC decimal data type
Since we're talking about wishes, would a decimal type be interesting? For example, in floating point, you can't represent .3. So the following evaluates to false:

if ( 0 == .1 + .2 - .3 ) { ... }

Fun fact: many COBOL-to-Java death marches have fallen down and not gotten back up because Java can't match COBOL's precision due to lacking COBOL's native decimal type (and simulating a Decimal type via classes meant that Java couldn't get the performance needed).
Imagine this evaluating as true (fake syntax):

if ( 0 == .1d + .2d - .3d ) { ... }

This would be a huge boon to anyone writing financial applications in Perl, or who otherwise needs to guarantee accuracy to a certain number of significant digits. Obviously, when decimal and floating points are mixed, we'd have to figure out the best way to address that. (C# requires casting, but I suspect we can't have nice toys).
Many languages currently support decimal data types natively (COBOL, C#) or via core libraries (Java, Python).
For what it's worth (not much), here's a Wikipedia page: https://en.wikipedia.org/wiki/Decimal_data_type
Best,Ovid-- IT consulting, training, specializing in Perl, databases, and agile developmenthttp://www.allaroundtheworld.fr/. 
Buy my book! - http://bit.ly/beginning_perl
Re: Pre-RFC decimal data type [ In reply to ]
On Thu, Aug 05, 2021 at 08:17:22AM +0000, Ovid via perl5-porters wrote:
> Since we're talking about wishes, would a decimal type be interesting? For example, in floating point, you can't represent .3. So the following evaluates to false:
>
> if ( 0 == .1 + .2 - .3 ) { ... }

Yes, it would be interesting, but I have no good idea how to implement this
in a performant and portable way.

> Fun fact: many COBOL-to-Java death marches have fallen down and not gotten back up because Java can't match COBOL's precision due to lacking COBOL's native decimal type (and simulating a Decimal type via classes meant that Java couldn't get the performance needed).

This I didn't know. Interesting. Thanks.

Nicholas Clark
RE: Pre-RFC decimal data type [ In reply to ]
From: Nicholas Clark <nick@ccl4.org>
>
> On Thu, Aug 05, 2021 at 08:17:22AM +0000, Ovid via perl5-porters wrote:
> > Since we're talking about wishes, would a decimal type be interesting?
For example, in floating point, you can't represent .3. So the following
evaluates to false:
> >
> > if ( 0 == .1 + .2 - .3 ) { ... }
>
> Yes, it would be interesting, but I have no good idea how to implement
this in a performant and portable way.

Use the Rational number implementation from Raku as inspiration? See
https://docs.raku.org/type/Rat

Paul
Re: Pre-RFC decimal data type [ In reply to ]
Raku's rational numbers aren't quite the same thing. The square root of 2 isn't rational, but you could use decimals for saying "I only need X significant digits." You could do 2.sqrt.Rat.perl in Raku and you get 1393/985. I don't want that. I want arbitrary precision decimal numbers where I specify how many significant digits I want and not have to always carry a numerator and denominator around.
(I could be misunderstanding what happens on the Raku side)
Best,Ovid-- IT consulting, training, specializing in Perl, databases, and agile developmenthttp://www.allaroundtheworld.fr/. 
Buy my book! - http://bit.ly/beginning_perl

On Thursday, 5 August 2021, 11:19:05 CEST, paul.marquess via perl5-porters <perl5-porters@perl.org> wrote:

From: Nicholas Clark <nick@ccl4.org>
>
> On Thu, Aug 05, 2021 at 08:17:22AM +0000, Ovid via perl5-porters wrote:
> > Since we're talking about wishes, would a decimal type be interesting?
For example, in floating point, you can't represent .3. So the following
evaluates to false:
> >
> > if ( 0 == .1 + .2 - .3 ) { ... }
>
> Yes, it would be interesting, but I have no good idea how to implement
this in a performant and portable way.

Use the Rational number implementation from Raku as inspiration? See
https://docs.raku.org/type/Rat

Paul
Re: Pre-RFC decimal data type [ In reply to ]
On Thu, Aug 05, 2021 at 10:18:39AM +0100, paul.marquess via perl5-porters wrote:
> From: Nicholas Clark <nick@ccl4.org>
> >
> > On Thu, Aug 05, 2021 at 08:17:22AM +0000, Ovid via perl5-porters wrote:
> > > Since we're talking about wishes, would a decimal type be interesting?
> For example, in floating point, you can't represent .3. So the following
> evaluates to false:
> > >
> > > if ( 0 == .1 + .2 - .3 ) { ... }
> >
> > Yes, it would be interesting, but I have no good idea how to implement
> this in a performant and portable way.
>
> Use the Rational number implementation from Raku as inspiration? See
> https://docs.raku.org/type/Rat

I don't know how much that relies on the bytecode specialiser to get decent
performance. Also, I suspect that it's not as easy to convert "idiomatic"*
NQP into sane C code. Compared with (say) a design for C, C++ or a similar
compiled language build around structures and explicit memory management.

Also, it's a full-on rational numbers system - in Rakudo, I believe that
0.2 is stored as 1/5 (not 2/10), and all the implementation is built for
arbitrary rationals (not just base 10), *and* can deal with "BigRats" - ie
big integers for numerator or denominator.

Possibly more useful would be a decimal arithmetic library written in C,
with a suitable licence.

Also, as there (IIRC) there's now an IEEE spec for decimal types, which might
actually be more useful is decimal floating point, particularly if some CPUs
can actually do it in hardware.

(So floating point rather than either arbitrary precision decimal
arithmetic, or fixed point decimal arithmetic - it wasn't clear what Ovid
was thinking, and I don't know any COBOL so don't know what it does. But
all three options would make 0.1 + 0.2 - 0.3 be zero)

Nicholas Clark

* Rakudo is written in NQP, not Raku. And most of that NQP is written for
speed, with knowledge of how MoarVM would compile it.
"This week"
And then gets changed when the optimiser changes. But maybe not until some
"weeks" later.
So it's not something beautiful and easy to learn from. It's not the swan
gliding smoothly over the water. It's more the legs paddling furiously
below the surface.
Re: Pre-RFC decimal data type [ In reply to ]
On Thursday, 5 August 2021, 10:29:29 CEST, Nicholas Clark <nick@ccl4.org> wrote:

> On Thu, Aug 05, 2021 at 08:17:22AM +0000, Ovid via perl5-porters wrote:
> > Since we're talking about wishes, would a decimal type be interesting? For example, in floating point, you can't represent .3. So the following evaluates to false:
> >
> > if ( 0 == .1 + .2 - .3 ) { ... }
>
> Yes, it would be interesting, but I have no good idea how to implement this
> in a performant and portable way.
I can't comment on portable, but as for performant, I don't think they can be as performant as floating point because you have to track how many significant digits you need (numbers automatically get zero padding) and with two decimals, compare the number of significant digits for each. However, you get developer performance and more correct code as a trade-off.

> > Fun fact: many COBOL-to-Java death marches have fallen down and not gotten back up because Java can't match COBOL's precision due to lacking COBOL's native decimal type (and simulating a Decimal type via classes meant that Java couldn't get the performance needed).>
> This I didn't know. Interesting. Thanks.

To be fair, many mainframes have hardware support for decimal numbers. That's a huge game changer.
Do GPUs?
Best,Ovid
Re: Pre-RFC decimal data type [ In reply to ]
On 2021-08-05 1:17 a.m., Ovid via perl5-porters wrote:
> Since we're talking about wishes, would a decimal type be interesting? For
> example, in floating point, you can't represent .3. So the following evaluates
> to false:
>
> Imagine this evaluating as true (fake syntax):
>
> if ( 0 == .1d + .2d - .3d ) { ... }
>
> Many languages currently support decimal data types natively (COBOL, C#) or via
> core libraries (Java, Python).

I would suggest that the simplest implementation is just as a scaled integer.

So its precision would be the same as a Perl integer because it would be
implementing using the same kind of machine integer, likely a 64-bit one.

Is about 18 digits of precision enough or do you need more?

I would expect that should handle fine the finance use case you mention.

In my experience finance apps often use scaled integers implemented at the app
level anyway in absence of a decimal type, say as an integer that counts in
100ths of a currency unit for example.

-- Darren Duncan
Re: Pre-RFC decimal data type [ In reply to ]
On Thu, Aug 5, 2021 at 4:36 AM Nicholas Clark <nick@ccl4.org> wrote:

> Possibly more useful would be a decimal arithmetic library written in C,
> with a suitable licence.

There is the decNumber package in GCC
(<https://github.com/gcc-mirror/gcc/tree/master/libdecnumber>) which
is presumably some version of GPL. It's also in ICU, which probably
has a more permissive license.

There is something called lipmpdec in Python:
<https://github.com/python/cpython/tree/main/Modules/_decimal/libmpdec>.
It's available independently at
<https://www.bytereef.org/mpdecimal/download.html> and says
"simplified BSD license."

One of the open source SQL databases may have an implementation of the
SQL NUMERIC data type that could be adapted.

Don't really know anything about these -- just did a few minutes of
web searching.

> Also, as there (IIRC) there's now an IEEE spec for decimal types,

It appears to be called x3.274-1996 but costs money from the ANSI store.
Re: Pre-RFC decimal data type [ In reply to ]
Ovid via perl5-porters <perl5-porters@perl.org> writes:

> Since we're talking about wishes, would a decimal type be interesting? For example, in floating point, you can't represent
> .3. So the following evaluates to false:
>
> if ( 0 == .1 + .2 - .3 ) { ... }
>
> Fun fact: many COBOL-to-Java death marches have fallen down and not gotten back up because Java can't match COBOL's
> precision due to lacking COBOL's native decimal type (and simulating a Decimal type via classes meant that Java couldn't
> get the performance needed).

In current Perl, you can get rather close like this:

perl -Mbignum=a,31 -E 'say (0 == 0.1 + 0.2 - 0.3)' # prints 1
perl -Mbignum=a,31 -E 'say (1 == 1/3 + 1/3 + 1/3)' # prints ''

If decimals were in core instead of objects of a core library, they
could avoid the bignum pitfall (https://perldoc.perl.org/bignum#Caveats)
and also the overhead for objects and overloading. Any further
performance improvement could be applied to bignum as well.

It might also encourage DB drivers to support the "decimal" data type
where the DB engine supports it.

For comparison:

COBOL's speed on mainframes comes from using an internal representation
(https://www.ibm.com/docs/en/i/7.4?topic=type-packed-decimal-format)
which is directly supported by hardware instructions operating on those
packed decimals. The hardware instructions handle integers up to 31
digits, but placing the decimal point is a task for programming
languages. Conversion from this format to and from strings is also
hardware supported and dead fast, the "zoned decimal format" happens to
be identical to the EBCDIC string.

C# uses a 96-bit integer which happens to have about the same numeric
range as 31 digits, plus flags for the sign and decimal point.

To add to the Java misery: If you retrieve a Java BigDecimal, e.g. from
a database, and want to add one, you need to create a new object:
value = value.add(new BigDecimal(1)). Compare that to Perl's $value++.
--
Cheers,
haj
Re: Pre-RFC decimal data type [ In reply to ]
On Thu, Aug 5, 2021 at 12:54 PM Harald Jörg <haj@posteo.de> wrote:

> Ovid via perl5-porters <perl5-porters@perl.org> writes:
>
> > Since we're talking about wishes, would a decimal type be interesting?
> For example, in floating point, you can't represent
> > .3. So the following evaluates to false:
> >
> > if ( 0 == .1 + .2 - .3 ) { ... }
> >
> > Fun fact: many COBOL-to-Java death marches have fallen down and not
> gotten back up because Java can't match COBOL's
> > precision due to lacking COBOL's native decimal type (and simulating a
> Decimal type via classes meant that Java couldn't
> > get the performance needed).
>
> In current Perl, you can get rather close like this:
>
> perl -Mbignum=a,31 -E 'say (0 == 0.1 + 0.2 - 0.3)' # prints 1
> perl -Mbignum=a,31 -E 'say (1 == 1/3 + 1/3 + 1/3)' # prints ''
>
> If decimals were in core instead of objects of a core library, they
> could avoid the bignum pitfall (https://perldoc.perl.org/bignum#Caveats)
> and also the overhead for objects and overloading. Any further
> performance improvement could be applied to bignum as well.
>
> It might also encourage DB drivers to support the "decimal" data type
> where the DB engine supports it.
>

Yeah this is Math::BigFloat under the hood and that is the way to do
arbitrary precision decimals. But that has significant performance
implications even beyond the bignum problem and object overhead. If there's
a way (and this is the big question) to support limited precision decimals
with better performance than arbitrary precision decimals, I think core
support for that would be a significant boon for specific applications.

-Dan
Re: Pre-RFC decimal data type [ In reply to ]
2021-8-6 1:54 Harald Jörg <haj@posteo.de> wrote:

>
> It might also encourage DB drivers to support the "decimal" data type
> where the DB engine supports it.
>
>
>
I have a question.

How does the database driver(mysql, postgresql, SQL server, oracle, etc)
now retrieve and store fixed-point data type values?
Re: Pre-RFC decimal data type [ In reply to ]
On 2021-08-05 9:53 a.m., Harald Jörg wrote:
> To add to the Java misery: If you retrieve a Java BigDecimal, e.g. from
> a database, and want to add one, you need to create a new object:
> value = value.add(new BigDecimal(1)). Compare that to Perl's $value++.

I would consider creating a new object to be a feature rather than a misery.
Numbers should be immutable. -- Darren Duncan
Re: Pre-RFC decimal data type [ In reply to ]
On 2021-08-05 5:03 p.m., Darren Duncan wrote:
> On 2021-08-05 9:53 a.m., Harald Jörg wrote:
>> To add to the Java misery: If you retrieve a Java BigDecimal, e.g. from
>> a database, and want to add one, you need to create a new object:
>> value = value.add(new BigDecimal(1)).  Compare that to Perl's $value++.
>
> I would consider creating a new object to be a feature rather than a misery.
> Numbers should be immutable. -- Darren Duncan

Also your $value++ example isn't the same thing, it isn't mutating the object,
its assigning a new thing to $value.

If someone had this:

$v1 = 42;
$v2 = $v1;
$v1++;

Then $v2 would still point to 42.

I would expect/desire even with bignum libraries, that adding one to $v1 would
NOT change the value in $v2, same as with small numbers, but if the number
object is mutable then it would.

-- Darren Duncan
Re: Pre-RFC decimal data type [ In reply to ]
Hi Ovid,
> Since we're talking about wishes, would a decimal type be interesting? […]For what it's worth (not much), here's a Wikipedia page: https://en.wikipedia.org/wiki/Decimal_data_type

Seems like this thread has fizzled out. Would it be fair to put this in a "would be nice" pot, rather than "want to seriously pursue"?

Feels like a candidate for experimentation on CPAN.

Also, take a look at
https://metacpan.org/pod/Math::Decimal

Neil
Re: Pre-RFC decimal data type [ In reply to ]
2021-10-15 5:57 Neil Bowers <neilb@neilb.org> wrote:

Hi Ovid,
>
> Since we're talking about wishes, would a decimal type be interesting? […]For
> what it's worth (not much), here's a Wikipedia page:
> https://en.wikipedia.org/wiki/Decimal_data_type
>
>
> Seems like this thread has fizzled out. Would it be fair to put this in a
> "would be nice" pot, rather than "want to seriously pursue"?
>
> Feels like a candidate for experimentation on CPAN.
>
> Also, take a look at
> https://metacpan.org/pod/Math::Decimal
>
> Neil
>

I see this.

Porters, I have one question. What do you do when you save a value to
mysql's DECIMAL type (fixed point)?

Using merely the string "100.2343"?