Mailing List Archive

Christian Gollwitzer <auriocus@gmx.de> writes:

> Am 02.09.21 um 15:51 schrieb Hope Rouselle:
>> Just sharing a case of floating-point numbers. Nothing needed to be
>> solved or to be figured out. Just bringing up conversation.
>> (*) An introduction to me
>> I don't understand floating-point numbers from the inside out, but I
>> do
>> know how to work with base 2 and scientific notation. So the idea of
>> expressing a number as
>> mantissa * base^{power}
>> is not foreign to me. (If that helps you to perhaps instruct me on
>> what's going on here.)
>> (*) A presentation of the behavior
>>
>>>>> import sys
>>>>> sys.version
>> '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64
>> bit (AMD64)]'
>>
>>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
>>>>> sum(ls)
>> 39.599999999999994
>>
>>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
>>>>> sum(ls)
>> 39.60000000000001
>> All I did was to take the first number, 7.23, and move it to the
>> last
>> position in the list. (So we have a violation of the commutativity of
>> addition.)
>
> I believe it is not commutativity, but associativity, that is
> violated.

Shall we take this seriously? (I will disagree, but that doesn't mean I
am not grateful for your post. Quite the contary.) It in general
violates associativity too, but the example above couldn't be referring
to associativity because the second sum above could not be obtained from
associativity alone. Commutativity is required, applied to five pairs
of numbers. How can I go from

7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77

to

8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23?

Perhaps only through various application of commutativity, namely the
ones below. (I omit the parentheses for less typing. I suppose that
does not create much trouble. There is no use of associativity below,
except for the intented omission of parentheses.)

7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77
= 8.41 + 7.23 + 6.15 + 2.31 + 7.73 + 7.77
= 8.41 + 6.15 + 7.23 + 2.31 + 7.73 + 7.77
= 8.41 + 6.15 + 2.31 + 7.23 + 7.73 + 7.77
= 8.41 + 6.15 + 2.31 + 7.73 + 7.23 + 7.77
= 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23.

> Even for floating point, a+b=b+a except for maybe some extreme cases
> like denormliazed numbers etc.
>
> But in general (a+b)+c != a+ (b+c)
>
> Consider decimal floating point with 2 digits.
> a=1
> b=c=0.04
>
> Then you get LHS;
> (1 + 0.04) + 0.04 = 1 + 0.04 = 1
>
> RHS:
>
> 1 + (0.04 + 0.04) = 1 + 0.08 = 1.1
>
> Your sum is evaluated like (((a + b) + c) + ....) and hence, if you
> permute the numbers, it can be unequal. If you need better accuracy,
> there is the Kahan summation algorithm and other alternatives:
> https://en.wikipedia.org/wiki/Kahan_summation_algorithm

Thanks!
--
https://mail.python.org/mailman/listinfo/python-list

Chris Angelico <rosuav@gmail.com> writes:

> On Fri, Sep 3, 2021 at 4:29 AM Hope Rouselle <hrouselle@jevedi.com> wrote:
>>
>> Just sharing a case of floating-point numbers. Nothing needed to be
>> solved or to be figured out. Just bringing up conversation.
>>
>> (*) An introduction to me
>>
>> I don't understand floating-point numbers from the inside out, but I do
>> know how to work with base 2 and scientific notation. So the idea of
>> expressing a number as
>>
>> mantissa * base^{power}
>>
>> is not foreign to me. (If that helps you to perhaps instruct me on
>> what's going on here.)
>>
>> (*) A presentation of the behavior
>>
>> >>> import sys
>> >>> sys.version
>> '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64
>> bit (AMD64)]'
>>
>> >>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
>> >>> sum(ls)
>> 39.599999999999994
>>
>> >>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
>> >>> sum(ls)
>> 39.60000000000001
>>
>> All I did was to take the first number, 7.23, and move it to the last
>> position in the list. (So we have a violation of the commutativity of
>> addition.)
>
> It's not about the commutativity of any particular pair of operands -
> that's always guaranteed.

Shall we take this seriously? It has to be about the commutativity of
at least one particular pair because it is involved with the
commutavitity of a set of pairs. If various pairs are involved, then at
least one is involved. IOW, it is about the commutativity of some pair
of operands and so it could not be the case that it's not about the
commutativity of any. (Lol. I hope that's not too insubordinate. I
already protested against a claim for associativity in this thread and
now I'm going for the king of the hill, for whom I have always been so
grateful!)

> What you're seeing here is the results of intermediate rounding. Try
> this:

Indeed. Your post here is really great, as usual. It offers a nice
tool for investigation. Thank you so much.

>>>> def sum(stuff):
> ... total = 0
> ... for thing in stuff:
> ... total += thing
> ... print(thing, "-->", total)
> ... return total
> ...
>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
>>>> sum(ls)
> 7.23 --> 7.23
> 8.41 --> 15.64
> 6.15 --> 21.79
> 2.31 --> 24.099999999999998

Alright. Thanks so much for this example. Here's a new puzzle for me.
The REPL makes me think that both 21.79 and 2.31 *are* representable
exactly in Python's floating-point datatype because I see:

>>> 2.31
2.31
>>> 21.79
21.79

When I add them, the result obtained makes me think that the sum is
*not* representable exactly in Python's floating-point number:

>>> 21.79 + 2.31
24.099999999999998

However, when I type 24.10 explicitly, the REPL makes me think that
24.10 *is* representable exactly:

>>> 24.10
24.1

I suppose I cannot trust the appearance of the representation? What's
really going on there? (Perhaps the trouble appears while Python is
computing the sum of the numbers 21.79 and 2.31?) Thanks so much!
--
https://mail.python.org/mailman/listinfo/python-list

Chris Angelico <rosuav@gmail.com> writes:

> On Fri, Sep 3, 2021 at 4:58 AM Hope Rouselle <hrouselle@jevedi.com> wrote:
>>
>> Hope Rouselle <hrouselle@jevedi.com> writes:
>>
>> > Just sharing a case of floating-point numbers. Nothing needed to be
>> > solved or to be figured out. Just bringing up conversation.
>> >
>> > (*) An introduction to me
>> >
>> > I don't understand floating-point numbers from the inside out, but I do
>> > know how to work with base 2 and scientific notation. So the idea of
>> > expressing a number as
>> >
>> > mantissa * base^{power}
>> >
>> > is not foreign to me. (If that helps you to perhaps instruct me on
>> > what's going on here.)
>> >
>> > (*) A presentation of the behavior
>> >
>> >>>> import sys
>> >>>> sys.version
>> > '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64
>> > bit (AMD64)]'
>> >
>> >>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
>> >>>> sum(ls)
>> > 39.599999999999994
>> >
>> >>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
>> >>>> sum(ls)
>> > 39.60000000000001
>> >
>> > All I did was to take the first number, 7.23, and move it to the last
>> > position in the list. (So we have a violation of the commutativity of
>> > addition.)
>>
>> Suppose these numbers are prices in dollar, never going beyond cents.
>> Would it be safe to multiply each one of them by 100 and therefore work
>> with cents only? For instance
>
> Yes and no. It absolutely *is* safe to always work with cents, but to
> do that, you have to be consistent: ALWAYS work with cents, never with
> floating point dollars.
>
> (Or whatever other unit you choose to use. Most currencies have a
> smallest-normally-used-unit, with other currency units (where present)
> being whole number multiples of that minimal unit. Only in forex do
> you need to concern yourself with fractional cents or fractional yen.)
>
> But multiplying a set of floats by 100 won't necessarily solve your
> problem; you may have already fallen victim to the flaw of assuming
> that the numbers are represented accurately.

Hang on a second. I see it's always safe to work with cents, but I'm
only confident to say that when one gives me cents to start with. In
other words, if one gives me integers from the start. (Because then, of
course, I don't even have floats to worry about.) If I'm given 1.17,
say, I am not confident that I could turn this number into 117 by
multiplying it by 100. And that was the question. Can I always
multiply such IEEE 754 dollar amounts by 100?

Considering your last paragraph above, I should say: if one gives me an
accurate floating-point representation, can I assume a multiplication of
it by 100 remains accurately representable in IEEE 754?

>> --8<---------------cut here---------------start------------->8---
>> >>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
>> >>> sum(map(lambda x: int(x*100), ls)) / 100
>> 39.6
>>
>> >>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
>> >>> sum(map(lambda x: int(x*100), ls)) / 100
>> 39.6
>> --8<---------------cut here---------------end--------------->8---
>>
>> Or multiplication by 100 isn't quite ``safe'' to do with floating-point
>> numbers either? (It worked in this case.)
>
> You're multiplying and then truncating, which risks a round-down
> error. Try adding a half onto them first:
>
> int(x * 100 + 0.5)
>
> But that's still not a perfect guarantee. Far safer would be to
> consider monetary values to be a different type of value, not just a
> raw number. For instance, the value $7.23 could be stored internally
> as the integer 723, but you also know that it's a value in USD, not a
> simple scalar. It makes perfect sense to add USD+USD, it makes perfect
> sense to multiply USD*scalar, but it doesn't make sense to multiply
> USD*USD.

Because of the units? That would be USD squared? (Nice analysis.)

>> I suppose that if I multiply it by a power of two, that would be an
>> operation that I can be sure will not bring about any precision loss
>> with floating-point numbers. Do you agree?
>
> Assuming you're nowhere near 2**53, yes, that would be safe. But so
> would multiplying by a power of five. The problem isn't precision loss
> from the multiplication - the problem is that your input numbers
> aren't what you think they are. That number 7.23, for instance, is
> really....

Hm, I think I see what you're saying. You're saying multiplication and
division in IEEE 754 is perfectly safe --- so long as the numbers you
start with are accurately representable in IEEE 754 and assuming no
overflow or underflow would occur. (Addition and subtraction are not
safe.)

>>>> 7.23.as_integer_ratio()
> (2035064081618043, 281474976710656)
>
> ... the rational number 2035064081618043 / 281474976710656, which is
> very close to 7.23, but not exactly so. (The numerator would have to
> be ...8042.88 to be exactly correct.) There is nothing you can do at
> this point to regain the precision, although a bit of multiplication
> and rounding can cheat it and make it appear as if you did.
>
> Floating point is a very useful approximation to real numbers, but
> real numbers aren't the best way to represent financial data. Integers
> are.

I'm totally persuaded. Thanks.
--
https://mail.python.org/mailman/listinfo/python-list

Julio Di Egidio <julio@diegidio.name> writes:

> On Thursday, 2 September 2021 at 15:52:02 UTC+2, Hope Rouselle wrote:
>
>> I don't understand floating-point numbers from the inside out, but I do
>> know how to work with base 2 and scientific notation. So the idea of
>> expressing a number as
>>
>> mantissa * base^{power}
>
> That's the basic idea, but the actual (ISO) floating-point *encoding*
> is more complicated than that.
>
>> is not foreign to me. (If that helps you to perhaps instruct me on
>> what's going on here.)
>
> This is the "classic":
> DAVID GOLDBER
> What Every Computer Scientist Should Know About Floating-Point Arithmetic
> <http://perso.ens-lyon.fr/jean-michel.muller/goldberg.pdf>
>
> Here is some more introductory stuff:
> <https://en.wikipedia.org/wiki/Floating-point_arithmetic>
> <https://www.phys.uconn.edu/~rozman/Courses/P2200_15F/downloads/floating-point-guide-2015-10-15.pdf>

Rozman's was pretty concise and nice. Thank you.
--
https://mail.python.org/mailman/listinfo/python-list

On 9/4/21 9:40 AM, Hope Rouselle wrote:
> Chris Angelico <rosuav@gmail.com> writes:
>
>> On Fri, Sep 3, 2021 at 4:58 AM Hope Rouselle <hrouselle@jevedi.com> wrote:
>>>
>>> Hope Rouselle <hrouselle@jevedi.com> writes:
>>>
>>>> Just sharing a case of floating-point numbers. Nothing needed to be
>>>> solved or to be figured out. Just bringing up conversation.
>>>>
>>>> (*) An introduction to me
>>>>
>>>> I don't understand floating-point numbers from the inside out, but I do
>>>> know how to work with base 2 and scientific notation. So the idea of
>>>> expressing a number as
>>>>
>>>> mantissa * base^{power}
>>>>
>>>> is not foreign to me. (If that helps you to perhaps instruct me on
>>>> what's going on here.)
>>>>
>>>> (*) A presentation of the behavior
>>>>
>>>>>>> import sys
>>>>>>> sys.version
>>>> '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64
>>>> bit (AMD64)]'
>>>>
>>>>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
>>>>>>> sum(ls)
>>>> 39.599999999999994
>>>>
>>>>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
>>>>>>> sum(ls)
>>>> 39.60000000000001
>>>>
>>>> All I did was to take the first number, 7.23, and move it to the last
>>>> position in the list. (So we have a violation of the commutativity of
>>>> addition.)
>>>
>>> Suppose these numbers are prices in dollar, never going beyond cents.
>>> Would it be safe to multiply each one of them by 100 and therefore work
>>> with cents only? For instance
>>
>> Yes and no. It absolutely *is* safe to always work with cents, but to
>> do that, you have to be consistent: ALWAYS work with cents, never with
>> floating point dollars.
>>
>> (Or whatever other unit you choose to use. Most currencies have a
>> smallest-normally-used-unit, with other currency units (where present)
>> being whole number multiples of that minimal unit. Only in forex do
>> you need to concern yourself with fractional cents or fractional yen.)
>>
>> But multiplying a set of floats by 100 won't necessarily solve your
>> problem; you may have already fallen victim to the flaw of assuming
>> that the numbers are represented accurately.
>
> Hang on a second. I see it's always safe to work with cents, but I'm
> only confident to say that when one gives me cents to start with. In
> other words, if one gives me integers from the start. (Because then, of
> course, I don't even have floats to worry about.) If I'm given 1.17,
> say, I am not confident that I could turn this number into 117 by
> multiplying it by 100. And that was the question. Can I always
> multiply such IEEE 754 dollar amounts by 100?
>
> Considering your last paragraph above, I should say: if one gives me an
> accurate floating-point representation, can I assume a multiplication of
> it by 100 remains accurately representable in IEEE 754?
>

Multiplication by 100 might not be accurate if the number you are
starting with is close to the limit of precision, because 100 is
1.1001 x 64 so multiplying by 100 adds about 5 more 'bits' to the
representation of the number. In your case, the numbers are well below
that point.

>>> --8<---------------cut here---------------start------------->8---
>>>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
>>>>>> sum(map(lambda x: int(x*100), ls)) / 100
>>> 39.6
>>>
>>>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
>>>>>> sum(map(lambda x: int(x*100), ls)) / 100
>>> 39.6
>>> --8<---------------cut here---------------end--------------->8---
>>>
>>> Or multiplication by 100 isn't quite ``safe'' to do with floating-point
>>> numbers either? (It worked in this case.)
>>
>> You're multiplying and then truncating, which risks a round-down
>> error. Try adding a half onto them first:
>>
>> int(x * 100 + 0.5)
>>
>> But that's still not a perfect guarantee. Far safer would be to
>> consider monetary values to be a different type of value, not just a
>> raw number. For instance, the value $7.23 could be stored internally
>> as the integer 723, but you also know that it's a value in USD, not a
>> simple scalar. It makes perfect sense to add USD+USD, it makes perfect
>> sense to multiply USD*scalar, but it doesn't make sense to multiply
>> USD*USD.
>
> Because of the units? That would be USD squared? (Nice analysis.)
>
>>> I suppose that if I multiply it by a power of two, that would be an
>>> operation that I can be sure will not bring about any precision loss
>>> with floating-point numbers. Do you agree?
>>
>> Assuming you're nowhere near 2**53, yes, that would be safe. But so
>> would multiplying by a power of five. The problem isn't precision loss
>> from the multiplication - the problem is that your input numbers
>> aren't what you think they are. That number 7.23, for instance, is
>> really....
>
> Hm, I think I see what you're saying. You're saying multiplication and
> division in IEEE 754 is perfectly safe --- so long as the numbers you
> start with are accurately representable in IEEE 754 and assuming no
> overflow or underflow would occur. (Addition and subtraction are not
> safe.)
>

Addition and Subtraction are just as safe, as long as you stay within
the precision limits. Multiplication and division by powers of two are
the safest, not needing to add any precision, until you hit the limits
of the magnitude of numbers that can be expressed.

The problem is that a number like 0.1 isn't precisely represented, so it
ends up using ALL available precision to get the closest value to it so
ANY operations on it run the danger of precision loss.

>>>>> 7.23.as_integer_ratio()
>> (2035064081618043, 281474976710656)
>>
>> ... the rational number 2035064081618043 / 281474976710656, which is
>> very close to 7.23, but not exactly so. (The numerator would have to
>> be ...8042.88 to be exactly correct.) There is nothing you can do at
>> this point to regain the precision, although a bit of multiplication
>> and rounding can cheat it and make it appear as if you did.
>>
>> Floating point is a very useful approximation to real numbers, but
>> real numbers aren't the best way to represent financial data. Integers
>> are.
>
> I'm totally persuaded. Thanks.
>

--
https://mail.python.org/mailman/listinfo/python-list

Julio Di Egidio <julio@diegidio.name> writes:

> On Thursday, 2 September 2021 at 16:51:24 UTC+2, Christian Gollwitzer wrote:
>> Am 02.09.21 um 16:49 schrieb Julio Di Egidio:
>> > On Thursday, 2 September 2021 at 16:41:38 UTC+2, Peter Pearson wrote:
>> >> On Thu, 02 Sep 2021 10:51:03 -0300, Hope Rouselle wrote:
>> >
>> >>> 39.60000000000001
>> >>
>> >> Welcome to the exciting world of roundoff error:
>> >
>> > Welcome to the exiting world of Usenet.
>> >
>> > *Plonk*
>>
>> Pretty harsh, isn't it? He gave a concise example of the same inaccuracy
>> right afterwards.
>
> And I thought you were not seeing my posts...
>
> Given that I have already given a full explanation, you guys, that you
> realise it or not, are simply adding noise for the usual pub-level
> discussion I must most charitably guess.
>
> Anyway, just my opinion. (EOD.)

Which is certainly appreciated --- as a rule. Pub-level noise is pretty
much unavoidable in investigation, education. Being wrong is, too,
unavoidable in investigation, education. There is a point we eventually
publish at the most respected journals, but that's a whole other
interval of the time-line. IOW, chill out! :-D (Give us a C-k and meet
us up in the next thread. Oh, my, you're not a Gnus user: you are a
G2/1.0 user. That's pretty scary.)

By the way, how's sci.logic going? I, too, lost my patience there. :-)
--
https://mail.python.org/mailman/listinfo/python-list

Richard Damon <Richard@Damon-Family.org> writes:

> On 9/4/21 9:40 AM, Hope Rouselle wrote:
>> Chris Angelico <rosuav@gmail.com> writes:
>>
>>> On Fri, Sep 3, 2021 at 4:58 AM Hope Rouselle <hrouselle@jevedi.com> wrote:
>>>>
>>>> Hope Rouselle <hrouselle@jevedi.com> writes:
>>>>
>>>>> Just sharing a case of floating-point numbers. Nothing needed to be
>>>>> solved or to be figured out. Just bringing up conversation.
>>>>>
>>>>> (*) An introduction to me
>>>>>
>>>>> I don't understand floating-point numbers from the inside out, but I do
>>>>> know how to work with base 2 and scientific notation. So the idea of
>>>>> expressing a number as
>>>>>
>>>>> mantissa * base^{power}
>>>>>
>>>>> is not foreign to me. (If that helps you to perhaps instruct me on
>>>>> what's going on here.)
>>>>>
>>>>> (*) A presentation of the behavior
>>>>>
>>>>>>>> import sys
>>>>>>>> sys.version
>>>>> '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64
>>>>> bit (AMD64)]'
>>>>>
>>>>>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
>>>>>>>> sum(ls)
>>>>> 39.599999999999994
>>>>>
>>>>>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
>>>>>>>> sum(ls)
>>>>> 39.60000000000001
>>>>>
>>>>> All I did was to take the first number, 7.23, and move it to the last
>>>>> position in the list. (So we have a violation of the commutativity of
>>>>> addition.)
>>>>
>>>> Suppose these numbers are prices in dollar, never going beyond cents.
>>>> Would it be safe to multiply each one of them by 100 and therefore work
>>>> with cents only? For instance
>>>
>>> Yes and no. It absolutely *is* safe to always work with cents, but to
>>> do that, you have to be consistent: ALWAYS work with cents, never with
>>> floating point dollars.
>>>
>>> (Or whatever other unit you choose to use. Most currencies have a
>>> smallest-normally-used-unit, with other currency units (where present)
>>> being whole number multiples of that minimal unit. Only in forex do
>>> you need to concern yourself with fractional cents or fractional yen.)
>>>
>>> But multiplying a set of floats by 100 won't necessarily solve your
>>> problem; you may have already fallen victim to the flaw of assuming
>>> that the numbers are represented accurately.
>>
>> Hang on a second. I see it's always safe to work with cents, but I'm
>> only confident to say that when one gives me cents to start with. In
>> other words, if one gives me integers from the start. (Because then, of
>> course, I don't even have floats to worry about.) If I'm given 1.17,
>> say, I am not confident that I could turn this number into 117 by
>> multiplying it by 100. And that was the question. Can I always
>> multiply such IEEE 754 dollar amounts by 100?
>>
>> Considering your last paragraph above, I should say: if one gives me an
>> accurate floating-point representation, can I assume a multiplication of
>> it by 100 remains accurately representable in IEEE 754?
>>
>
> Multiplication by 100 might not be accurate if the number you are
> starting with is close to the limit of precision, because 100 is
> 1.1001 x 64 so multiplying by 100 adds about 5 more 'bits' to the
> representation of the number. In your case, the numbers are well below
> that point.

Alright. That's clear now. Thanks so much!

>>>> --8<---------------cut here---------------start------------->8---
>>>>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
>>>>>>> sum(map(lambda x: int(x*100), ls)) / 100
>>>> 39.6
>>>>
>>>>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
>>>>>>> sum(map(lambda x: int(x*100), ls)) / 100
>>>> 39.6
>>>> --8<---------------cut here---------------end--------------->8---
>>>>
>>>> Or multiplication by 100 isn't quite ``safe'' to do with floating-point
>>>> numbers either? (It worked in this case.)
>>>
>>> You're multiplying and then truncating, which risks a round-down
>>> error. Try adding a half onto them first:
>>>
>>> int(x * 100 + 0.5)
>>>
>>> But that's still not a perfect guarantee. Far safer would be to
>>> consider monetary values to be a different type of value, not just a
>>> raw number. For instance, the value $7.23 could be stored internally
>>> as the integer 723, but you also know that it's a value in USD, not a
>>> simple scalar. It makes perfect sense to add USD+USD, it makes perfect
>>> sense to multiply USD*scalar, but it doesn't make sense to multiply
>>> USD*USD.
>>
>> Because of the units? That would be USD squared? (Nice analysis.)
>>
>>>> I suppose that if I multiply it by a power of two, that would be an
>>>> operation that I can be sure will not bring about any precision loss
>>>> with floating-point numbers. Do you agree?
>>>
>>> Assuming you're nowhere near 2**53, yes, that would be safe. But so
>>> would multiplying by a power of five. The problem isn't precision loss
>>> from the multiplication - the problem is that your input numbers
>>> aren't what you think they are. That number 7.23, for instance, is
>>> really....
>>
>> Hm, I think I see what you're saying. You're saying multiplication and
>> division in IEEE 754 is perfectly safe --- so long as the numbers you
>> start with are accurately representable in IEEE 754 and assuming no
>> overflow or underflow would occur. (Addition and subtraction are not
>> safe.)
>>
>
> Addition and Subtraction are just as safe, as long as you stay within
> the precision limits. Multiplication and division by powers of two are
> the safest, not needing to add any precision, until you hit the limits
> of the magnitude of numbers that can be expressed.
>
> The problem is that a number like 0.1 isn't precisely represented, so it
> ends up using ALL available precision to get the closest value to it so
> ANY operations on it run the danger of precision loss.

Got it. That's clear now. It should've been before, but my attention
is that of a beginner, so some extra iterations turn up. As long as the
numbers involved are accurately representable, floating-points have no
other problems. I may, then, conclude that the whole difficulty with
floating-point is nothing but going beyond the reserved space for the
number.

However, I still lack an easy method to detect when a number is not
accurately representable by the floating-point datatype in use. For
instance, 0.1 is not representable accurately in IEEE 754. But I don't
know how to check that

>>> 0.1
0.1 # no clue
>>> 0.1 + 0.1
0.2 # no clue
>>> 0.1 + 0.1 + 0.1
0.30000000000000004 # there is the clue

How can I get a clearer and quicker evidence that 0.1 is not accurately
representable --- using the REPL?

I know

0.1 = 1/10 = 1 * 10^-1

and in base 2 that would have to be represented as... Let me calculate
it with my sophisticated skills:

0.1: 0 + 0.2
--> 0 + 0.4
--> 0 + 0.8
--> 1 + 0.6
--> 1 + 0.2, closing a cycle.

So 0.1 is representable poorly as 0.00011... In other words, 1/10 in
base 10 equals 1/2^4 + 1/2^5 + 1/2^9 + 1/2^10 + ...

The same question in other words --- what's a trivial way for the REPL
to show me such cycles occur?

>>>>>> 7.23.as_integer_ratio()
>>> (2035064081618043, 281474976710656)

Here's what I did on this case. The REPL is telling me that

7.23 = 2035064081618043/281474976710656

If that were true, then 7.23 * 281474976710656 would have to equal
2035064081618043. So I typed:

>>> 7.23 * 281474976710656
2035064081618043.0

That agrees with the falsehood. I'm getting no evidence of the problem.

When take control of my life out of the hands of misleading computers, I
calculate the sum:

844424930131968
+ 5629499534213120
197032483697459200
==================
203506408161804288
=/= 203506408161804300

How I can save the energy spent on manual verification?

Thanks very much.
--
https://mail.python.org/mailman/listinfo/python-list

Julio Di Egidio <julio@diegidio.name> writes:

[...]

>> I, too, lost my patience there. :-)
>
> As if I didn't know who's trolling...

I never trolled you. When we had our conversations in sci.logic, I was
Boris Dorestand --- you would remember if you have very good memory. We
talked for just a few days, I guess. The people trolling you there were
not me. So, though that's no proof, *I* know you didn't know. :-)
--
https://mail.python.org/mailman/listinfo/python-list

Am 04.09.21 um 14:48 schrieb Hope Rouselle:
> Christian Gollwitzer <auriocus@gmx.de> writes:
>
>> Am 02.09.21 um 15:51 schrieb Hope Rouselle:
>>> Just sharing a case of floating-point numbers. Nothing needed to be
>>> solved or to be figured out. Just bringing up conversation.
>>> (*) An introduction to me
>>> I don't understand floating-point numbers from the inside out, but I
>>> do
>>> know how to work with base 2 and scientific notation. So the idea of
>>> expressing a number as
>>> mantissa * base^{power}
>>> is not foreign to me. (If that helps you to perhaps instruct me on
>>> what's going on here.)
>>> (*) A presentation of the behavior
>>>
>>>>>> import sys
>>>>>> sys.version
>>> '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64
>>> bit (AMD64)]'
>>>
>>>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
>>>>>> sum(ls)
>>> 39.599999999999994
>>>
>>>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
>>>>>> sum(ls)
>>> 39.60000000000001
>>> All I did was to take the first number, 7.23, and move it to the
>>> last
>>> position in the list. (So we have a violation of the commutativity of
>>> addition.)
>>
>> I believe it is not commutativity, but associativity, that is
>> violated.
>
> Shall we take this seriously? (I will disagree, but that doesn't mean I
> am not grateful for your post. Quite the contary.) It in general
> violates associativity too, but the example above couldn't be referring
> to associativity because the second sum above could not be obtained from
> associativity alone. Commutativity is required, applied to five pairs
> of numbers. How can I go from
>
> 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77
>
> to
>
> 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23?
>
> Perhaps only through various application of commutativity, namely the
> ones below. (I omit the parentheses for less typing. I suppose that
> does not create much trouble. There is no use of associativity below,
> except for the intented omission of parentheses.)

With the parens it will become more obvious.

>
> 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77
> = 8.41 + 7.23 + 6.15 + 2.31 + 7.73 + 7.77

The sum is evaluated as

(((7.23 + 8.41) + 6.15 + ...)

For the first shift, you are correct that commutativity will result in

(((8.41 + 7.23) + 6.15 + ...)

But you can't go in one step to

(((8.41 + 6.15) + 7.23 + ...)

with the commutativity law alone. Instead, a sequence of associativity
and commutativity is required to move the 7.23 out of the first pair of
parentheses.

And what I was trying to say, the commutative steps *are* equal in
floating point arithmetics, whereas the associative steps are not.

Christian

--
https://mail.python.org/mailman/listinfo/python-list

On 5/09/21 2:42 am, Hope Rouselle wrote:
> Here's what I did on this case. The REPL is telling me that
>
> 7.23 = 2035064081618043/281474976710656

If 7.23 were exactly representable, you would have got
723/1000.

Contrast this with something that *is* exactly representable:

>>> 7.875.as_integer_ratio()
(63, 8)

and observe that 7875/1000 == 63/8:

>>> from fractions import Fraction
>>> Fraction(7875,1000)
Fraction(63, 8)

In general, to find out whether a decimal number is exactly
representable in binary, represent it as a ratio of integers
where the denominator is a power of 10, reduce that to lowest
terms, and compare with the result of as_integer_ratio().

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Christian Gollwitzer <auriocus@gmx.de> writes:

> Am 04.09.21 um 14:48 schrieb Hope Rouselle:
>> Christian Gollwitzer <auriocus@gmx.de> writes:
>>
>>> Am 02.09.21 um 15:51 schrieb Hope Rouselle:
>>>> Just sharing a case of floating-point numbers. Nothing needed to be
>>>> solved or to be figured out. Just bringing up conversation.
>>>> (*) An introduction to me
>>>> I don't understand floating-point numbers from the inside out, but I
>>>> do
>>>> know how to work with base 2 and scientific notation. So the idea of
>>>> expressing a number as
>>>> mantissa * base^{power}
>>>> is not foreign to me. (If that helps you to perhaps instruct me on
>>>> what's going on here.)
>>>> (*) A presentation of the behavior
>>>>
>>>>>>> import sys
>>>>>>> sys.version
>>>> '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64
>>>> bit (AMD64)]'
>>>>
>>>>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
>>>>>>> sum(ls)
>>>> 39.599999999999994
>>>>
>>>>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
>>>>>>> sum(ls)
>>>> 39.60000000000001
>>>> All I did was to take the first number, 7.23, and move it to the
>>>> last
>>>> position in the list. (So we have a violation of the commutativity of
>>>> addition.)
>>>
>>> I believe it is not commutativity, but associativity, that is
>>> violated.
>> Shall we take this seriously? (I will disagree, but that doesn't
>> mean I
>> am not grateful for your post. Quite the contary.) It in general
>> violates associativity too, but the example above couldn't be referring
>> to associativity because the second sum above could not be obtained from
>> associativity alone. Commutativity is required, applied to five pairs
>> of numbers. How can I go from
>> 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77
>> to
>> 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23?
>> Perhaps only through various application of commutativity, namely
>> the
>> ones below. (I omit the parentheses for less typing. I suppose that
>> does not create much trouble. There is no use of associativity below,
>> except for the intented omission of parentheses.)
>
> With the parens it will become more obvious.
>
>> 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77
>> = 8.41 + 7.23 + 6.15 + 2.31 + 7.73 + 7.77
>
> The sum is evaluated as
>
> (((7.23 + 8.41) + 6.15 + ...)
>
> For the first shift, you are correct that commutativity will result in
>
> (((8.41 + 7.23) + 6.15 + ...)
>
> But you can't go in one step to
>
> (((8.41 + 6.15) + 7.23 + ...)
>
> with the commutativity law alone. Instead, a sequence of
> associativity and commutativity is required to move the 7.23 out of
> the first pair of parentheses.
>
> And what I was trying to say, the commutative steps *are* equal in
> floating point arithmetics, whereas the associative steps are not.

Oh, I see it. Very good point! Lesson learned.
--
https://mail.python.org/mailman/listinfo/python-list

Greg Ewing <greg.ewing@canterbury.ac.nz> writes:

> On 5/09/21 2:42 am, Hope Rouselle wrote:
>> Here's what I did on this case. The REPL is telling me that
>> 7.23 = 2035064081618043/281474976710656
>
> If 7.23 were exactly representable, you would have got
> 723/1000.
>
> Contrast this with something that *is* exactly representable:
>
>>>> 7.875.as_integer_ratio()
> (63, 8)
>
> and observe that 7875/1000 == 63/8:
>
>>>> from fractions import Fraction
>>>> Fraction(7875,1000)
> Fraction(63, 8)
>
> In general, to find out whether a decimal number is exactly
> representable in binary, represent it as a ratio of integers
> where the denominator is a power of 10, reduce that to lowest
> terms, and compare with the result of as_integer_ratio().

That makes perfect sense and answers my question. I appreciate it.
--
https://mail.python.org/mailman/listinfo/python-list

Hope Rouselle <hrouselle@jevedi.com> writes:

> Greg Ewing <greg.ewing@canterbury.ac.nz> writes:
>
>> On 5/09/21 2:42 am, Hope Rouselle wrote:
>>> Here's what I did on this case. The REPL is telling me that
>>> 7.23 = 2035064081618043/281474976710656
>>
>> If 7.23 were exactly representable, you would have got
>> 723/1000.
>>
>> Contrast this with something that *is* exactly representable:
>>
>>>>> 7.875.as_integer_ratio()
>> (63, 8)
>>
>> and observe that 7875/1000 == 63/8:
>>
>>>>> from fractions import Fraction
>>>>> Fraction(7875,1000)
>> Fraction(63, 8)
>>
>> In general, to find out whether a decimal number is exactly
>> representable in binary, represent it as a ratio of integers
>> where the denominator is a power of 10, reduce that to lowest
>> terms, and compare with the result of as_integer_ratio().
>
> That makes perfect sense and answers my question. I appreciate it.

Here's my homework in high-precision. Thoughts? Thank you!

--8<---------------cut here---------------start------------->8---
def is_representable(s):
return in_lowest_terms(rat_power_of_10(s)) == float(s).as_integer_ratio()

# >>> is_representable("1.5")
# True
#
# >>> is_representable("0.1")
# False

def rat_power_of_10(s):
"""I assume s is a numeric string in the format <int>.<frac>"""
if "." not in s:
return int(s), 1
integral, fractional = s.split(".")
return int(integral + fractional), 10**(len(fractional))

# >>> rat_power_of_10("72.100")
# (72100, 1000)

def in_lowest_terms(rat):
from math import gcd
n, d = rat
return n//gcd(n, d), d//gcd(n, d)

# >>> in_lowest_terms( (72100, 1000) )
# (721, 10)
--8<---------------cut here---------------end--------------->8---
--
https://mail.python.org/mailman/listinfo/python-list

On Sun, Sep 5, 2021 at 12:44 PM Hope Rouselle <hrouselle@jevedi.com> wrote:
>
> Chris Angelico <rosuav@gmail.com> writes:
>
> > On Fri, Sep 3, 2021 at 4:29 AM Hope Rouselle <hrouselle@jevedi.com> wrote:
> >>
> >> Just sharing a case of floating-point numbers. Nothing needed to be
> >> solved or to be figured out. Just bringing up conversation.
> >>
> >> (*) An introduction to me
> >>
> >> I don't understand floating-point numbers from the inside out, but I do
> >> know how to work with base 2 and scientific notation. So the idea of
> >> expressing a number as
> >>
> >> mantissa * base^{power}
> >>
> >> is not foreign to me. (If that helps you to perhaps instruct me on
> >> what's going on here.)
> >>
> >> (*) A presentation of the behavior
> >>
> >> >>> import sys
> >> >>> sys.version
> >> '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64
> >> bit (AMD64)]'
> >>
> >> >>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
> >> >>> sum(ls)
> >> 39.599999999999994
> >>
> >> >>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
> >> >>> sum(ls)
> >> 39.60000000000001
> >>
> >> All I did was to take the first number, 7.23, and move it to the last
> >> position in the list. (So we have a violation of the commutativity of
> >> addition.)
> >
> > It's not about the commutativity of any particular pair of operands -
> > that's always guaranteed.
>
> Shall we take this seriously? It has to be about the commutativity of
> at least one particular pair because it is involved with the
> commutavitity of a set of pairs. If various pairs are involved, then at
> least one is involved. IOW, it is about the commutativity of some pair
> of operands and so it could not be the case that it's not about the
> commutativity of any. (Lol. I hope that's not too insubordinate. I
> already protested against a claim for associativity in this thread and
> now I'm going for the king of the hill, for whom I have always been so
> grateful!)

No, that is not the case. Look at the specific pairs of numbers that get added.

ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]

>>> 7.23 + 8.41
15.64
>>> _ + 6.15
21.79
>>> _ + 2.31
24.099999999999998
>>> _ + 7.73
31.83
>>> _ + 7.77
39.599999999999994

And with the other list:

ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]

>>> 8.41 + 6.15
14.56
>>> _ + 2.31
16.87
>>> _ + 7.73
24.6
>>> _ + 7.77
32.370000000000005
>>> _ + 7.23
39.60000000000001

If commutativity is being violated, then there should be some
situation where you could have written "7.73 + _" instead of "_ +
7.73" or equivalent, and gotten a different result. But that is simply
not the case. What you are seeing is NOT commutativity, but the
consequences of internal rounding, which is a matter of associativity.

> Alright. Thanks so much for this example. Here's a new puzzle for me.
> The REPL makes me think that both 21.79 and 2.31 *are* representable
> exactly in Python's floating-point datatype because I see:
>
> >>> 2.31
> 2.31
> >>> 21.79
> 21.79
>
> When I add them, the result obtained makes me think that the sum is
> *not* representable exactly in Python's floating-point number:
>
> >>> 21.79 + 2.31
> 24.099999999999998
>
> However, when I type 24.10 explicitly, the REPL makes me think that
> 24.10 *is* representable exactly:
>
> >>> 24.10
> 24.1
>
> I suppose I cannot trust the appearance of the representation? What's
> really going on there? (Perhaps the trouble appears while Python is
> computing the sum of the numbers 21.79 and 2.31?) Thanks so much!

The representation is a conversion from the internal format into
decimal digits. It is rounded for convenience of display, because you
don't want it to look like this:

>>> print(Fraction(24.10))
3391773469363405/140737488355328

Since that's useless, the repr of a float rounds it to the shortest
plausible number as represented in decimal digits. This has nothing to
do with whether it is exactly representable, and everything to do with
displaying things usefully in as many situations as possible :)

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On Sun, Sep 5, 2021 at 12:48 PM Hope Rouselle <hrouselle@jevedi.com> wrote:
>
> Chris Angelico <rosuav@gmail.com> writes:
>
> > On Fri, Sep 3, 2021 at 4:58 AM Hope Rouselle <hrouselle@jevedi.com> wrote:
> >>
> >> Hope Rouselle <hrouselle@jevedi.com> writes:
> >>
> >> > Just sharing a case of floating-point numbers. Nothing needed to be
> >> > solved or to be figured out. Just bringing up conversation.
> >> >
> >> > (*) An introduction to me
> >> >
> >> > I don't understand floating-point numbers from the inside out, but I do
> >> > know how to work with base 2 and scientific notation. So the idea of
> >> > expressing a number as
> >> >
> >> > mantissa * base^{power}
> >> >
> >> > is not foreign to me. (If that helps you to perhaps instruct me on
> >> > what's going on here.)
> >> >
> >> > (*) A presentation of the behavior
> >> >
> >> >>>> import sys
> >> >>>> sys.version
> >> > '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64
> >> > bit (AMD64)]'
> >> >
> >> >>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
> >> >>>> sum(ls)
> >> > 39.599999999999994
> >> >
> >> >>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
> >> >>>> sum(ls)
> >> > 39.60000000000001
> >> >
> >> > All I did was to take the first number, 7.23, and move it to the last
> >> > position in the list. (So we have a violation of the commutativity of
> >> > addition.)
> >>
> >> Suppose these numbers are prices in dollar, never going beyond cents.
> >> Would it be safe to multiply each one of them by 100 and therefore work
> >> with cents only? For instance
> >
> > Yes and no. It absolutely *is* safe to always work with cents, but to
> > do that, you have to be consistent: ALWAYS work with cents, never with
> > floating point dollars.
> >
> > (Or whatever other unit you choose to use. Most currencies have a
> > smallest-normally-used-unit, with other currency units (where present)
> > being whole number multiples of that minimal unit. Only in forex do
> > you need to concern yourself with fractional cents or fractional yen.)
> >
> > But multiplying a set of floats by 100 won't necessarily solve your
> > problem; you may have already fallen victim to the flaw of assuming
> > that the numbers are represented accurately.
>
> Hang on a second. I see it's always safe to work with cents, but I'm
> only confident to say that when one gives me cents to start with. In
> other words, if one gives me integers from the start. (Because then, of
> course, I don't even have floats to worry about.) If I'm given 1.17,
> say, I am not confident that I could turn this number into 117 by
> multiplying it by 100. And that was the question. Can I always
> multiply such IEEE 754 dollar amounts by 100?
>
> Considering your last paragraph above, I should say: if one gives me an
> accurate floating-point representation, can I assume a multiplication of
> it by 100 remains accurately representable in IEEE 754?

Humans usually won't give you IEEE 754 floats. What they'll usually
give you is a text string. Let's say you ask someone to type in the
prices of various items, the quantities thereof, and the shipping. You
take strings like "1.17" (or "$1.17"), and you parse that into the
integer 117.

> Hm, I think I see what you're saying. You're saying multiplication and
> division in IEEE 754 is perfectly safe --- so long as the numbers you
> start with are accurately representable in IEEE 754 and assuming no
> overflow or underflow would occur. (Addition and subtraction are not
> safe.)
>

All operations are equally valid. Anything that causes rounding can
cause loss of data, and that can happen with multiplication/division
as well as addition/subtraction. But yes, with the caveats you give,
everything is safe.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On Sun, Sep 5, 2021 at 12:50 PM Hope Rouselle <hrouselle@jevedi.com> wrote:
>
> Christian Gollwitzer <auriocus@gmx.de> writes:
>
> > Am 02.09.21 um 15:51 schrieb Hope Rouselle:
> >> Just sharing a case of floating-point numbers. Nothing needed to be
> >> solved or to be figured out. Just bringing up conversation.
> >> (*) An introduction to me
> >> I don't understand floating-point numbers from the inside out, but I
> >> do
> >> know how to work with base 2 and scientific notation. So the idea of
> >> expressing a number as
> >> mantissa * base^{power}
> >> is not foreign to me. (If that helps you to perhaps instruct me on
> >> what's going on here.)
> >> (*) A presentation of the behavior
> >>
> >>>>> import sys
> >>>>> sys.version
> >> '3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64
> >> bit (AMD64)]'
> >>
> >>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
> >>>>> sum(ls)
> >> 39.599999999999994
> >>
> >>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
> >>>>> sum(ls)
> >> 39.60000000000001
> >> All I did was to take the first number, 7.23, and move it to the
> >> last
> >> position in the list. (So we have a violation of the commutativity of
> >> addition.)
> >
> > I believe it is not commutativity, but associativity, that is
> > violated.
>
> Shall we take this seriously? (I will disagree, but that doesn't mean I
> am not grateful for your post. Quite the contary.) It in general
> violates associativity too, but the example above couldn't be referring
> to associativity because the second sum above could not be obtained from
> associativity alone. Commutativity is required, applied to five pairs
> of numbers. How can I go from
>
> 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77
>
> to
>
> 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23?
>
> Perhaps only through various application of commutativity, namely the
> ones below. (I omit the parentheses for less typing. I suppose that
> does not create much trouble. There is no use of associativity below,
> except for the intented omission of parentheses.)
>
> 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77
> = 8.41 + 7.23 + 6.15 + 2.31 + 7.73 + 7.77
> = 8.41 + 6.15 + 7.23 + 2.31 + 7.73 + 7.77
> = 8.41 + 6.15 + 2.31 + 7.23 + 7.73 + 7.77
> = 8.41 + 6.15 + 2.31 + 7.73 + 7.23 + 7.77
> = 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23.
>

Show me the pairs of numbers. You'll find that they are not the same
numbers. Commutativity is specifically that a+b == b+a and you won't
find any situation where that is violated.

As soon as you go to three or more numbers, what you're doing is
changing which numbers get added first, which is this:

a + (b + c) != (a + b) + c

and this can most certainly be violated due to intermediate rounding.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On Sun, Sep 5, 2021 at 12:55 PM Hope Rouselle <hrouselle@jevedi.com> wrote:
>
> Julio Di Egidio <julio@diegidio.name> writes:
>
> > On Thursday, 2 September 2021 at 16:51:24 UTC+2, Christian Gollwitzer wrote:
> >> Am 02.09.21 um 16:49 schrieb Julio Di Egidio:
> >> > On Thursday, 2 September 2021 at 16:41:38 UTC+2, Peter Pearson wrote:
> >> >> On Thu, 02 Sep 2021 10:51:03 -0300, Hope Rouselle wrote:
> >> >
> >> >>> 39.60000000000001
> >> >>
> >> >> Welcome to the exciting world of roundoff error:
> >> >
> >> > Welcome to the exiting world of Usenet.
> >> >
> >> > *Plonk*
> >>
> >> Pretty harsh, isn't it? He gave a concise example of the same inaccuracy
> >> right afterwards.
> >
> > And I thought you were not seeing my posts...
> >
> > Given that I have already given a full explanation, you guys, that you
> > realise it or not, are simply adding noise for the usual pub-level
> > discussion I must most charitably guess.
> >
> > Anyway, just my opinion. (EOD.)
>
> Which is certainly appreciated --- as a rule. Pub-level noise is pretty
> much unavoidable in investigation, education. Being wrong is, too,
> unavoidable in investigation, education. There is a point we eventually
> publish at the most respected journals, but that's a whole other
> interval of the time-line. IOW, chill out! :-D (Give us a C-k and meet
> us up in the next thread. Oh, my, you're not a Gnus user: you are a
> G2/1.0 user. That's pretty scary.)
>

I'm not a fan of the noise level in a pub, but I have absolutely no
problem with arguing these points out. And everyone (mostly) in this
thread is being respectful. I don't mind when someone else is wrong,
especially since - a lot of the time - I'm wrong too (or maybe I'm the
only one who's wrong).

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On Sun, Sep 5, 2021 at 12:58 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
>
> On 5/09/21 2:42 am, Hope Rouselle wrote:
> > Here's what I did on this case. The REPL is telling me that
> >
> > 7.23 = 2035064081618043/281474976710656
>
> If 7.23 were exactly representable, you would have got
> 723/1000.
>
> Contrast this with something that *is* exactly representable:
>
> >>> 7.875.as_integer_ratio()
> (63, 8)
>
> and observe that 7875/1000 == 63/8:
>
> >>> from fractions import Fraction
> >>> Fraction(7875,1000)
> Fraction(63, 8)
>
> In general, to find out whether a decimal number is exactly
> representable in binary, represent it as a ratio of integers
> where the denominator is a power of 10, reduce that to lowest
> terms, and compare with the result of as_integer_ratio().
>

Or let Python do that work for you!

>>> from fractions import Fraction
>>> Fraction("7.875") == Fraction(7.875)
True
>>> Fraction("7.8") == Fraction(7.8)
False

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On Sun, Sep 5, 2021 at 1:04 PM Hope Rouselle <hrouselle@jevedi.com> wrote:
> The same question in other words --- what's a trivial way for the REPL
> to show me such cycles occur?
>
> >>>>>> 7.23.as_integer_ratio()
> >>> (2035064081618043, 281474976710656)
>
> Here's what I did on this case. The REPL is telling me that
>
> 7.23 = 2035064081618043/281474976710656
>
> If that were true, then 7.23 * 281474976710656 would have to equal
> 2035064081618043. So I typed:
>
> >>> 7.23 * 281474976710656
> 2035064081618043.0
>
> That agrees with the falsehood. I'm getting no evidence of the problem.
>
> When take control of my life out of the hands of misleading computers, I
> calculate the sum:
>
> 844424930131968
> + 5629499534213120
> 197032483697459200
> ==================
> 203506408161804288
> =/= 203506408161804300
>
> How I can save the energy spent on manual verification?
>

What you've stumbled upon here is actually a neat elegance of
floating-point, and an often-forgotten fundamental of it: rounding
occurs exactly the same regardless of the scale. The number 7.23 is
represented with a certain mantissa, and multiplying it by some power
of two doesn't change the mantissa, only the exponent. So the rounding
happens exactly the same, and it comes out looking equal!

The easiest way, in Python, to probe this sort of thing is to use
either fractions.Fraction or decimal.Decimal. I prefer Fraction, since
a float is fundamentally a rational number, and you can easily see
what's happening. You can construct a Fraction from a string, and
it'll do what you would expect; or you can construct one from a float,
and it'll show you what that float truly represents.

It's often cleanest to print fractions out rather than just dumping
them to the console, since the str() of a fraction looks like a
fraction, but the repr() looks like a constructor call.

>>> Fraction(0.25)
Fraction(1, 4)
>>> Fraction(0.1)
Fraction(3602879701896397, 36028797018963968)

If it looks like the number you put in, it was perfectly
representable. If it looks like something of roughly that many digits,
it's probably not the number you started with.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On Sat, 04 Sep 2021 10:40:35 -0300, Hope Rouselle <hrouselle@jevedi.com>
declaimed the following:

>course, I don't even have floats to worry about.) If I'm given 1.17,
>say, I am not confident that I could turn this number into 117 by
>multiplying it by 100. And that was the question. Can I always
>multiply such IEEE 754 dollar amounts by 100?
>

HOW are you "given" that 1.17? If that is coming from some
user-readable source (keyboard entry, text-based file [CSV/TSV, even SYLK])
you do NOT have a number -- you have a string, which needs to be converted
by some algorithm.

For money, the best solution, again, is to use the Decimal module and
feed the /string/ to the initialization call. If you want to do it
yourself, to get a scaled integer, you will have to code a
parser/converter.

* strip extraneous punctuation ($, etc -- but not comma, decimal
point, or + and -)
* strip any grouping separators (commas, but beware, some countries
group using a period -- "1.234,56" vs "1,234.56"). "1,234.56" => "1234.56"
* ensure there is a decimal point (again, you may have to convert a
comma to decimal point), if not append a "." to the input
* append enough 0s to the end to ensure you have whatever scale
factor you are using behind the decimal point (as mentioned M$ Excel money
type uses four places) "1234.56" => "1234.5600"
* remove the decimal marker. "1234.5600" => "12345600"
* convert to native integer. int("12345600") => 12345600 [as integer]
{may fail if +/- are not in the required position for Python}

If the number is coming from some binary file format, it is already too
late. And you might need to study any DBMS being used. Some transfer values
as text strings, and reconvert to DBMS numeric types on receipt. Make sure
the DBMS is using a decimal number type and not a floating type for the
field (which leaves out SQLite3 <G> which will store anything in any field,
but uses some slightly obscure logic to determine what conversion is done)


--
Wulfraed Dennis Lee Bieber AF6VN
wlfraed@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/

--
https://mail.python.org/mailman/listinfo/python-list

On 2021-09-04 09:48:40 -0300, Hope Rouselle wrote:
> Christian Gollwitzer <auriocus@gmx.de> writes:
> > Am 02.09.21 um 15:51 schrieb Hope Rouselle:
> >>>>> ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
> >>>>> sum(ls)
> >> 39.599999999999994
> >>
> >>>>> ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
> >>>>> sum(ls)
> >> 39.60000000000001
> >> All I did was to take the first number, 7.23, and move it to the
> >> last
> >> position in the list. (So we have a violation of the commutativity of
> >> addition.)
> >
> > I believe it is not commutativity, but associativity, that is
> > violated.

I agree.


> Shall we take this seriously? (I will disagree, but that doesn't mean I
> am not grateful for your post. Quite the contary.) It in general
> violates associativity too, but the example above couldn't be referring
> to associativity because the second sum above could not be obtained from
> associativity alone. Commutativity is required, applied to five pairs
> of numbers. How can I go from
>
> 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77
>
> to
>
> 8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23?

Simple:

>>> 7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77
39.599999999999994
>>> 7.23 + (8.41 + 6.15 + 2.31 + 7.73 + 7.77)
39.60000000000001

Due to commutativity, this is the same as

>>> (8.41 + 6.15 + 2.31 + 7.73 + 7.77) + 7.23
39.60000000000001

So commutativity is preserved but associativity is lost. (Of course a
single example doesn't prove that this is always the case, but it can be
seen from the guarantees that IEEE-754 arithmetic gives you that this is
actually the case).

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"

On 2021-09-05 03:38:55 +1200, Greg Ewing wrote:
> If 7.23 were exactly representable, you would have got
> 723/1000.
>
> Contrast this with something that *is* exactly representable:
>
> >>> 7.875.as_integer_ratio()
> (63, 8)
>
> and observe that 7875/1000 == 63/8:
>
> >>> from fractions import Fraction
> >>> Fraction(7875,1000)
> Fraction(63, 8)
>
> In general, to find out whether a decimal number is exactly
> representable in binary, represent it as a ratio of integers
> where the denominator is a power of 10, reduce that to lowest
> terms,

... and check if the denominator is a power of two. If it isn't (e.g.
1000 == 2**3 * 5**3) then the number is not exactly representable as a
binary floating point number.

More generally, if the prime factorization of the denominator only
contains prime factors which are also prime factors of your base, then
the number can be exactle represented (unless either the denominator or
the enumerator get too big). So, for base 10 (2*5), all numbers which
have only powers of 2 and 5 in the denominator (e.g 1/10 == 1/(2*5),
1/8192 == 1/2**13, 1/1024000 == 1/(2**13 * 5**3)) can represented
exactly, but those with other prime factors (e.g. 1/3, 1/7,
1/24576 == 1/(2**13 * 3), 1/1024001 == 1/(11 * 127 * 733)) cannot.
Similarly, for base 12 (2*2*3) numbers with 2 and 3 in the denominator
can be represented and for base 60 (2*2*3*5), numbers with 2, 3 and 5.

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"

On 2021-09-04 10:01:23 -0400, Richard Damon wrote:
> On 9/4/21 9:40 AM, Hope Rouselle wrote:
> > Hm, I think I see what you're saying. You're saying multiplication and
> > division in IEEE 754 is perfectly safe --- so long as the numbers you
> > start with are accurately representable in IEEE 754 and assuming no
> > overflow or underflow would occur. (Addition and subtraction are not
> > safe.)
> >
>
> Addition and Subtraction are just as safe, as long as you stay within
> the precision limits.

That depends a lot on what you call "safe",

a * b / a will always be very close to b (unless there's an over- or
underflow), but a + b - a can be quite different from b.

In general when analyzing a numerical algorithm you have to pay a lot
more attention to addition and subtraction than to multiplication and
division.

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"

On 2021-09-05, Peter J. Holzer <hjp-python@hjp.at> wrote:
> On 2021-09-05 03:38:55 +1200, Greg Ewing wrote:
>> If 7.23 were exactly representable, you would have got
>> 723/1000.
>>
>> Contrast this with something that *is* exactly representable:
>>
>> >>> 7.875.as_integer_ratio()
>> (63, 8)
>>
>> and observe that 7875/1000 == 63/8:
>>
>> >>> from fractions import Fraction
>> >>> Fraction(7875,1000)
>> Fraction(63, 8)
>>
>> In general, to find out whether a decimal number is exactly
>> representable in binary, represent it as a ratio of integers where
>> the denominator is a power of 10, reduce that to lowest terms,
>
> ... and check if the denominator is a power of two. If it isn't
> (e.g. 1000 == 2**3 * 5**3) then the number is not exactly
> representable as a binary floating point number.
>
> More generally, if the prime factorization of the denominator only
> contains prime factors which are also prime factors of your base,
> then the number can be exactle represented (unless either the
> denominator or the enumerator get too big).

And once you understand that, ignore it and write code under the
assumumption that nothing can be exactly represented in floating
point.

If you like, you can assume that 0 can be exactly represented without
getting into too much trouble as long as it's a literal constant value
and not the result of any run-time FP operations.

If you want to live dangerously, you can assume that integers with
magnitude less than a million can be exactly represented. That
assumption is true for all the FP representations I've ever used, but
once you start depending on it, you're one stumble from the edge of
the cliff.

--
Grant




--
https://mail.python.org/mailman/listinfo/python-list

> On Sep 5, 2021, at 6:22 PM, Peter J. Holzer <hjp-python@hjp.at> wrote:
>
> ?On 2021-09-04 10:01:23 -0400, Richard Damon wrote:
>>> On 9/4/21 9:40 AM, Hope Rouselle wrote:
>>> Hm, I think I see what you're saying. You're saying multiplication and
>>> division in IEEE 754 is perfectly safe --- so long as the numbers you
>>> start with are accurately representable in IEEE 754 and assuming no
>>> overflow or underflow would occur. (Addition and subtraction are not
>>> safe.)
>>>
>>
>> Addition and Subtraction are just as safe, as long as you stay within
>> the precision limits.
>
> That depends a lot on what you call "safe",
>
> a * b / a will always be very close to b (unless there's an over- or
> underflow), but a + b - a can be quite different from b.
>
> In general when analyzing a numerical algorithm you have to pay a lot
> more attention to addition and subtraction than to multiplication and
> division.
>
> hp
>
> --
Yes, it depends on your definition of safe. If ‘close’ is good enough then multiplication is probably safer as the problems are in more extreme cases. If EXACT is the question, addition tends to be better. To have any chance, the numbers need to be somewhat low ‘precision’, which means the need to avoid arbitrary decimals. Once past that, as long as the numbers are of roughly the same magnitude, and are the sort of numbers you are apt to just write, you can tend to add a lot of them before you get enough bits to accumulate to have a problem. With multiplication, every multiply roughly adds the number of bits of precision, so you quickly run out, and one divide will have a chance to just end the process.

Remember, the question came up because the sum was’t associative because of fractional bits. That points to thinking of exact operations, and addition does better at that.
--
https://mail.python.org/mailman/listinfo/python-list