Mailing List Archive

Math::Big* to XS?
Hi all,

Not a pre-RFC because I'm not even sure what I'm asking. The Math::Big*
modules, such as Math::BigInt and friends are slow. They're tied scalars
that use overload and in, some cases, AUTOLOAD. I had a case in some
math-heavy code in the past where I've wanted to use Math::BigFloat for
pseudo-random number generation but it was so slow that I couldn't. Is it
possible for the various Math::Big* modules to be reimplemented in XS to
make their performance decent enough that people don't want you away from
them? Would that even really improve performance?

Are they not used enough that this isn't important? Or are they not used
enough because they're slow? I've no idea which.

Best,
Curtis "Ovid" Poe
CTO, All Around the World
World-class software development and consulting
https://allaroundtheworld.fr/
Re: Math::Big* to XS? [ In reply to ]
On Fri, Jun 17, 2022 at 2:14 PM Ovid <curtis.poe@gmail.com> wrote:

> Hi all,
>
> Not a pre-RFC because I'm not even sure what I'm asking. The Math::Big*
> modules, such as Math::BigInt and friends are slow. They're tied scalars
> that use overload and in, some cases, AUTOLOAD. I had a case in some
> math-heavy code in the past where I've wanted to use Math::BigFloat for
> pseudo-random number generation but it was so slow that I couldn't. Is it
> possible for the various Math::Big* modules to be reimplemented in XS to
> make their performance decent enough that people don't want you away from
> them? Would that even really improve performance?
>
> Are they not used enough that this isn't important? Or are they not used
> enough because they're slow? I've no idea which.
>

My impression is the two main performance issues are:

1. using the "bigint" or "bigrat" pragmas has an impact because it affects
every numeric literal in that scope, even ones that have no need to be
biggified. I avoid this by never using those pragmas

2. the actual precision math operations themselves, which depend on the
backend being used. The non-default backends are much better when dealing
with very large numbers, but they are less portable and packaged
separately. https://metacpan.org/pod/Math::BigInt#MATH-LIBRARY

I don't think the overhead of the pureperl code is comparable to these two
impacts but I haven't tested my assumptions and could be wrong.

-Dan
Re: Math::Big* to XS? [ In reply to ]
On 2022-06-17 11:14 a.m., Ovid wrote:
> Not a pre-RFC because I'm not even sure what I'm asking. The Math::Big* modules,
> such as Math::BigInt and friends are slow. They're tied scalars that use
> overload and in, some cases, AUTOLOAD. I had a case in some math-heavy code in
> the past where I've wanted to use Math::BigFloat for pseudo-random
> number generation but it was so slow that I couldn't. Is it possible for the
> various Math::Big* modules to be reimplemented in XS to make their performance
> decent enough that people don't want you away from them? Would that even
> really improve performance?
>
> Are they not used enough that this isn't important? Or are they not used enough
> because they're slow? I've no idea which.

I find it very important in principle to he unlimited precision exact math in
core as a base feature, big integers especially, and these would be strong
candidates to have XS/etc implementations available if they aren't already, but
otherwise having even a pure Perl implementation bundled with Perl is critical.

The nature of bignum math is such that it will always be significantly slower
than numbers that fit in single CPU registers, even if it was implemented in C.
However the tight loop logic its comprised of would be particularly sensitive
to pure Perl vs XS/etc differences, so having XS/etc would help make it about as
close to performance as is conceptually possible.

-- Darren Duncan
Re: Math::Big* to XS? [ In reply to ]
Hi there,

On Fri, 17 Jun 2022, Ovid wrote:

> Not a pre-RFC because I'm not even sure what I'm asking. The Math::Big*
> modules, such as Math::BigInt and friends are slow. They're tied scalars
> that use overload and in, some cases, AUTOLOAD. I had a case in some
> math-heavy code in the past where I've wanted to use Math::BigFloat for
> pseudo-random number generation but it was so slow that I couldn't. Is it
> possible for the various Math::Big* modules to be reimplemented in XS to
> make their performance decent enough that people don't want you away from
> them? Would that even really improve performance?
>
> Are they not used enough that this isn't important? Or are they not used
> enough because they're slow? I've no idea which.

Last time I tried I gave up not because they were slow but because
they didn't work. I was trying to store, retrieve and manipulate 64-
bit flag values and getting garbage after some of the manipulations.
The main problem appears to be that at some stage Perl decides to
convert my 64-bit bit fields from integers to floating point and then
back again. As you can imagine, if it's a bit field, any rounding
errors at all will throw a spanner in the works.

Admittedly I didn't spend a lot of time trying to find out exactly why
things went wrogn, I just gave up and did the bit-twiddling the hard
way using shifts etc. which seems perfectly reliable. There's a note
to self to take another look at it.

My 2c-worth is that it would be great to be able to do any logical
operations, on bit fields of arbitrary length, efficiently.

--

73,
Ged.
Re: Math::Big* to XS? [ In reply to ]
Hie

Perl ought to have modules for fast arbitrary-precision arithmetic, but
please don't naively re-implement Math::Big* into XS. Design something from
scratch, with a clean OO design. The Math::Big* modules are a design and
maintenance nightmare. I have so many times wanted to start off with the
best from these modules and create something like "Math::Big* done right",
but I haven't had the time. If anyone is interested, I'll be happy to share
my experience with the current implementation.

I'm not sure what you mean by Math::BigInt and friends being tied scalars.
They are objects, i.e., blessed references.

AUTOLOAD is used for two purposes. One is backwards compatibility with
those (if any) who still use fadd() etc. rather than badd() etc. The other
purpose is as a replacement for inheritance. The original author had
limited understanding of OO design and used AUTOLOAD to provide methods
that were directly available through inheritance anyway. I think this
second use could simply be removed, but I haven't given it priority.

The code could be made faster and cleaner, i.e., with a proper OO design
and not constantly checking global variables, but I think that will break
so much legacy code that I haven't dared changing it. In any case, it
obviously wouldn't be as fast as XS code.

Peter