Mailing List Archive

On 2023-10-24, o1bigtenor via Python-list <python-list@python.org> wrote:

> Is there a way to verify that a program is going to do what it is
> supposed to do even before all the hardware has been assembled and
> installed and tested?

It depends on what you mean by "verify ...". If you want to prove a
program correct (in the mathematical sense), then the practical answer
is no. It's possible to prove _some_ programs correct, but they tend to
be uselessly trivial.

For real programs, the best you can do is choose a good set of test
cases and test them. If you can simulate the various inputs and
collect the outputs, then you can do testing before you have real
target hardware.


--
https://mail.python.org/mailman/listinfo/python-list

On 2023-10-24, o1bigtenor wrote:
> Greetings
>
> (Sorry for a nebulous subject but dunno how to have a short title for
> a complex question.)
> [...]
> Is there a way to verify that a program is going to do what it is
> supposed to do even
> before all the hardware has been assembled and installed and tested?

In short, no.

Reality is a mess, and even if you've programmed/perfectly/ to the
datasheets (and passed our unit-tests that are also based on those
datasheets), a piece of hardware may not actually conform to what's
written. Maybe the sheet is wrong, maybe the hardware is faulty, etc.



--
|_|O|_|
|_|_|O| Github: https://github.com/dpurgert
|O|O|O| PGP: DDAB 23FB 19FA 7D85 1CC1 E067 6D65 70E5 4CE7 2860
--
https://mail.python.org/mailman/listinfo/python-list

I don’t think there i a simple answer to this, although if you find something interesting, please share.

From my experience, industry is applying variety of testing methods. Starting from lowest level components and implementing unit tests, finishing with end-to-end testing platforms.

https://www.atlassian.com/continuous-delivery/software-testing/types-of-software-testing <https://www.atlassian.com/continuous-delivery/software-testing/types-of-software-testing>

Modular programming paradigm, IMO, is one of the solutions to this problem. Then, each component is a flexible program in itself that can be combined with others. This way, code is re-used by many people and code is well tested and issues are quick to surface.

As far as I know, unix/linux has a big emphasis on modularity in contrast with monolithic approach of windows, which could be one of the big reasons why (at least from my perspective) working in unix environment is so much more pleasant.

https://en.wikipedia.org/wiki/Unix_philosophy <https://en.wikipedia.org/wiki/Unix_philosophy>

Regards,
DG

> On 24 Oct 2023, at 15:22, o1bigtenor via Python-list <python-list@python.org> wrote:
>
> Greetings
>
> (Sorry for a nebulous subject but dunno how to have a short title for
> a complex question.)
>
> I have been using computers for a long time but am only beginning my
> foray into the
> galaxy of programming. Have done little to this point besides
> collection of information
> on sensors and working on the logic of what I wish to accomplish. Have
> been reading code that accompanies other's projects in the process of
> self development.
>
> Is there a way to verify that a program is going to do what it is
> supposed to do even
> before all the hardware has been assembled and installed and tested?
>
> (Many years ago I remember an article (if not an issue) in Byte magazine about
> mathematically proven constructs a.k.a. programs - - - this idea is
> what I'm pursuing.
> The concept is that in non-trivial programs there are plenty of places where a
> poorly placed symbol or lack of a character will result in at best an inaccurate
> result and at worst - - - no result. This is the kind of thing
> (correct code) that I'm
> hoping to accomplish - - - to rephrase the question - - - how do I
> test for that?)
>
> TIA
> --
> https://mail.python.org/mailman/listinfo/python-list

--
https://mail.python.org/mailman/listinfo/python-list

On 2023-10-24, Dan Purgert via Python-list <python-list@python.org> wrote:
> On 2023-10-24, o1bigtenor wrote:
>> Greetings
>>
>> (Sorry for a nebulous subject but dunno how to have a short title for
>> a complex question.)
>> [...]
>> Is there a way to verify that a program is going to do what it is
>> supposed to do even before all the hardware has been assembled and
>> installed and tested?
>
> In short, no.
>
> Reality is a mess, and even if you've programmed/perfectly/ to the
> datasheets (and passed our unit-tests that are also based on those
> datasheets), a piece of hardware may not actually conform to what's
> written. Maybe the sheet is wrong, maybe the hardware is faulty, etc.

And the specified customer requirements are usually wrong too. Sure,
the customer said it is supposed to do X, but what they actually
needed was Y.

And the protocol spec isn't quite right either. Sure, it says "when A
is received reply with B", but what everybody really does is slighty
different, and you need to do what everybody else does, or the widget
you're talking to won't cooperate.

And floating point doesn't really work the way you think it
does. Sometimes it does, close-enough, for the test-cases you happened
to choose...





--
https://mail.python.org/mailman/listinfo/python-list

> On 24 Oct 2023, at 18:25, o1bigtenor via Python-list <python-list@python.org> wrote:
>
> Is there a way to verify that a program is going to do what it is
> supposed to do

In the general case not proven to be not possible.
Have a read about the halting problem https://en.wikipedia.org/wiki/Halting_problem

It is common to simulate hardware that does not exist yet and run software in the simulated environment.

Barry

--
https://mail.python.org/mailman/listinfo/python-list

On Tue, Oct 24, 2023 at 4:54?PM Grant Edwards via Python-list
<python-list@python.org> wrote:
>
> On 2023-10-24, Dan Purgert via Python-list <python-list@python.org> wrote:
> > On 2023-10-24, o1bigtenor wrote:
> >> Greetings
> >>
> >> (Sorry for a nebulous subject but dunno how to have a short title for
> >> a complex question.)
> >> [...]
> >> Is there a way to verify that a program is going to do what it is
> >> supposed to do even before all the hardware has been assembled and
> >> installed and tested?
> >
> > In short, no.
> >
> > Reality is a mess, and even if you've programmed/perfectly/ to the
> > datasheets (and passed our unit-tests that are also based on those
> > datasheets), a piece of hardware may not actually conform to what's
> > written. Maybe the sheet is wrong, maybe the hardware is faulty, etc.
>
> And the specified customer requirements are usually wrong too. Sure,
> the customer said it is supposed to do X, but what they actually
> needed was Y.
>
> And the protocol spec isn't quite right either. Sure, it says "when A
> is received reply with B", but what everybody really does is slighty
> different, and you need to do what everybody else does, or the widget
> you're talking to won't cooperate.
>
> And floating point doesn't really work the way you think it
> does. Sometimes it does, close-enough, for the test-cases you happened
> to choose...
>
Fascinating - - - except here I get to wear almost all of the hats.
I'm putting together the hardware, I get to do the programming and I
will be running the completed equipment.

I am asking so that I'm not chasing my tail for inordinate amounts of time
- - - grin!

Interesting ideas so far.
--
https://mail.python.org/mailman/listinfo/python-list

On 10/24/2023 8:22 AM, o1bigtenor via Python-list wrote:
> Greetings
>
> (Sorry for a nebulous subject but dunno how to have a short title for
> a complex question.)
>
> I have been using computers for a long time but am only beginning my
> foray into the
> galaxy of programming. Have done little to this point besides
> collection of information
> on sensors and working on the logic of what I wish to accomplish. Have
> been reading code that accompanies other's projects in the process of
> self development.
>
> Is there a way to verify that a program is going to do what it is
> supposed to do even
> before all the hardware has been assembled and installed and tested?
>
> (Many years ago I remember an article (if not an issue) in Byte magazine about
> mathematically proven constructs a.k.a. programs - - - this idea is
> what I'm pursuing.
> The concept is that in non-trivial programs there are plenty of places where a
> poorly placed symbol or lack of a character will result in at best an inaccurate
> result and at worst - - - no result. This is the kind of thing
> (correct code) that I'm
> hoping to accomplish - - - to rephrase the question - - - how do I
> test for that?)
>
> TIA

By now you have read many responses that basically say that you cannot
prove that a given program has no errors, even apart from the hardware
question. Even if it could be done, the kind of specification that you
would need would in itself be difficult to create, read, and understand,
and would be subject to bugs itself.

Something less ambitious than a full proof of correctness of an
arbitrary program can sometimes be achieved. The programming team for
the Apollo moon mission developed a system which, if you would write
your requirements in a certain way, could generate correct C code for them.

You won't be doing that.

Here I want to point out something else. You say you are just getting
into programming. You are going to be making many mistakes and errors,
and there will be many things about programming you won't understand
until you get some good solid experience. That's not anything to do
with you personally, that's just how it will play out.

So be prepared to learn from your mistakes and bugs. They are how you
learn the nuts and bolts of the business of programming.

--
https://mail.python.org/mailman/listinfo/python-list

On Tue, Oct 24, 2023 at 5:28?PM Rob Cliffe <rob.cliffe@btinternet.com> wrote:
>
> There is no general way to prove that a program is "correct". Or even
> whether it will terminate or loop endlessly.
> These are of course theoretical statements of computer science. But
> they can be rigorously proven. (Sorry if I'm just saying this to show
> what a smart-ass I am. ????)
> In practice, of course, there is often a great deal that can be done to
> "verify" (a word whose meaning I intentionally leave vague) a program's
> correctness.
> In your case, it sounds as if you should
>
> Write programs or functions to simulate each piece of hardware and
> generate random, but reasonably realistic, data. (Python and most other
> programming languages provide means of generating random or
> pseudo-random data.)
> In your main program:
> Replace the bits of code that accept data from the hardware by
> bits of code that accept data from these simulation programs/functions.
> Write the decisions it makes to a log file (or files).
> Run the program as long as you can or until your patience is
> exhausted, and check from the log file(s) that it is behaving as you
> would expect.
>
> This is not guaranteed to catch all possible errors. (Nothing is.) E.g.
> The original code to accept data from the hardware (code that you
> remove in your test version of the program) might be wrong. Duh!
> There might be specific sets of input data that happen not to arise
> in your testing, but that your program logic does not cope with.
> Nonetheless, this sort of testing (if done diligently) can give you a
> high degree of confidence in your program.
> And it is a good idea to do it.
> When you come to run your program "for real", and you have to
> troubleshoot it (as in real life you probably will????), you will have
> eliminated simple bugs in your program, and can concentrate on the more
> likely sources of problems (e.g. misbehaving hardware).
>
Interesting - - - hopefully taken in the same vein as your second statement - -
I sorta sounds like programmers keep running around in the forest looking
for trees. (Grin!)

So how does one test software then?

Tia
--
https://mail.python.org/mailman/listinfo/python-list

On Tue, Oct 24, 2023 at 6:09?PM Thomas Passin via Python-list
<python-list@python.org> wrote:
>
snip
>
> By now you have read many responses that basically say that you cannot
> prove that a given program has no errors, even apart from the hardware
> question. Even if it could be done, the kind of specification that you
> would need would in itself be difficult to create, read, and understand,
> and would be subject to bugs itself.
>
> Something less ambitious than a full proof of correctness of an
> arbitrary program can sometimes be achieved. The programming team for
> the Apollo moon mission developed a system which, if you would write
> your requirements in a certain way, could generate correct C code for them.
>
> You won't be doing that.
>
> Here I want to point out something else. You say you are just getting
> into programming. You are going to be making many mistakes and errors,
> and there will be many things about programming you won't understand
> until you get some good solid experience. That's not anything to do
> with you personally, that's just how it will play out.
>
> So be prepared to learn from your mistakes and bugs. They are how you
> learn the nuts and bolts of the business of programming.
>

I am fully expecting to make mistakes (grin!).
I have a couple trades tickets - - - I've done more than a touch of technical
learning so mistakes are not scary.

What is interesting about this is the absolute certainty that it is impossible
to program so that that program is provably correct.
Somehow - - - well - - to me that sounds that programming is illogical.

If I set up a set of mathematical problems (linked) I can prove that the
logic structure of my answer is correct.

That's what I'm looking to do with the programming.

(Is that different than the question(s) that I've asked - - - dunno.)

Stimulating interaction for sure (grin!).
--
https://mail.python.org/mailman/listinfo/python-list

On 2023-10-24, Thomas Passin via Python-list <python-list@python.org> wrote:

> Something less ambitious than a full proof of correctness of an
> arbitrary program can sometimes be achieved. The programming team
> for the Apollo moon mission developed a system which, if you would
> write your requirements in a certain way, could generate correct C
> code for them.

Er, what?

C didnt' exist until after the Apollo program was done.

FORTRAN, perhaps?

--
https://mail.python.org/mailman/listinfo/python-list

On 2023-10-24, o1bigtenor via Python-list <python-list@python.org> wrote:

> So how does one test software then?

That's what customers are for!

<bah-dum zing>

[Actually, that's true more often than it should be.]


--
https://mail.python.org/mailman/listinfo/python-list

On 24/10/2023 22:51, Grant Edwards via Python-list wrote:

>>> Is there a way to verify that a program is going to do what it is
>>> supposed to do even before all the hardware has been assembled and
>>> installed and tested?
> And the specified customer requirements are usually wrong too. Sure,
> the customer said it is supposed to do X, but what they actually
> needed was Y.

And this is the hardest bit, specifying exactly what you want at
a level that can be formally verified. I worked on some safety
critical systems a while back(1990s) and we had to formally verify
the core (non UI) code. We did this, but it still failed in some
scenarios because we verified it against faulty specs which,
in turn, were based on the customer's incorrectly stated requirements.
Garbage-In-Garbage-Out still applies.

Was the 3 months of formal analysis a waste of time? No, we still
caught lots of subtle stuff that might have been missed, but it
wasn't 100%. The bugs we did have were caught and identified
during system tests. So far as I know, nobody has died as a
result of any bugs in that system.

But, to the OP, the effort in
a) Learning the math and gaining experience for formal analysis and
b) actually performing such an analysis of real design/code
is simply not worth the effort for 99% of the programs you will write.
It is much simpler and faster to just test. And test again. And again.
Especially if you use automated testing tools which is the norm nowadays.


--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


--
https://mail.python.org/mailman/listinfo/python-list

On 25/10/2023 00:08, o1bigtenor via Python-list wrote:

> So how does one test software then?

Testing is very different to proving!
As an industry we do a lot of testing at many different levels.
On bigger projects you'll find:
- Unit tests - testing small fragments of a bigger program
- Integration tests - testing that sub modules of code work
together (and code with hardware, if applicable)
- System testing - checking that the code(and hardware) as a
whole does what it should based on the specs (often done
by an independent team)
- Performance testing - checking the system runs as fast as it
should, using only the memory it should, for as long as it should.
- User testing - Can a real user drive it?
- security testing - Does it stop the bad guys from messing it
up or using it as a gateway?

And there are more levels if you are really keen.
Testing often(usually!) takes up more time than programming.
And there are many, many books written about how to do it.

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


--
https://mail.python.org/mailman/listinfo/python-list

On 10/24/2023 7:15 PM, o1bigtenor wrote:
> On Tue, Oct 24, 2023 at 6:09?PM Thomas Passin via Python-list
> <python-list@python.org> wrote:
>>
> snip
>>
>> By now you have read many responses that basically say that you cannot
>> prove that a given program has no errors, even apart from the hardware
>> question. Even if it could be done, the kind of specification that you
>> would need would in itself be difficult to create, read, and understand,
>> and would be subject to bugs itself.
>>
>> Something less ambitious than a full proof of correctness of an
>> arbitrary program can sometimes be achieved. The programming team for
>> the Apollo moon mission developed a system which, if you would write
>> your requirements in a certain way, could generate correct C code for them.
>>
>> You won't be doing that.
>>
>> Here I want to point out something else. You say you are just getting
>> into programming. You are going to be making many mistakes and errors,
>> and there will be many things about programming you won't understand
>> until you get some good solid experience. That's not anything to do
>> with you personally, that's just how it will play out.
>>
>> So be prepared to learn from your mistakes and bugs. They are how you
>> learn the nuts and bolts of the business of programming.
>>
>
> I am fully expecting to make mistakes (grin!).
> I have a couple trades tickets - - - I've done more than a touch of technical
> learning so mistakes are not scary.
>
> What is interesting about this is the absolute certainty that it is impossible
> to program so that that program is provably correct.
> Somehow - - - well - - to me that sounds that programming is illogical.
>
> If I set up a set of mathematical problems (linked) I can prove that the
> logic structure of my answer is correct.

In general, that's not the case - CF Godel's Theorem. There are true
arithmetical statements that cannot be proven to be true within the
axioms of arithmetic. There's a counterpart in programming called the
halting problem. Can an arbitrary computer program be proven to ever
finish - to come to a halt (meaning basically to spit out a computed
result)? Not in general. If it will never halt you can never check its
computation.

This doesn't mean that no program can ever be proven to halt, nor that
no program can never be proven correct by formal means. Will your
program be one of those? The answer may never come ...

> That's what I'm looking to do with the programming.
>
> (Is that different than the question(s) that I've asked - - - dunno.)
>
> Stimulating interaction for sure (grin!).
>

--
https://mail.python.org/mailman/listinfo/python-list

On 10/24/2023 7:37 PM, Grant Edwards via Python-list wrote:
> On 2023-10-24, Thomas Passin via Python-list <python-list@python.org> wrote:
>
>> Something less ambitious than a full proof of correctness of an
>> arbitrary program can sometimes be achieved. The programming team
>> for the Apollo moon mission developed a system which, if you would
>> write your requirements in a certain way, could generate correct C
>> code for them.
>
> Er, what?
>
> C didnt' exist until after the Apollo program was done.
>
> FORTRAN, perhaps?
>

Sorry, I mixed myself up. The head of the team continued to develop the
techniques and market them. It's todays's version that can output C
(going from memory a few years old here). Sorry to have confused
everyone and myself.

--
https://mail.python.org/mailman/listinfo/python-list

Whoa!

The question cannot be about whether it is possible to prove any abstract program will be correct and especially not on real hardware that can fail in various ways or have unexpected race conditions or interacts with other places such as over the internet.

It has been quite well proven (think Kurt Gödel) that any system as complex as just arithmetic can have propositions that can neither be proven as true or as false and could be either. So there will be logical setups, written perhaps into the form of programs, that cannot be proven to work right, or even just halt someday when done.

The real question is way more detailed and complex. How does one create a complex program while taking care to minimize as well as you can the chances it is flawed under some conditions. There are walls of books written on such topics and they range from ways to write the software, perhaps in small modules that can be tested and then combined into larger subunits that can also be tested. There are compilers/interpreters/linters and sometimes ways of declaring your intentions to them, that can catch some kinds of possible errors, or force you to find another way to do things. You can hire teams of people to create test cases and try them or automate them. You can fill the code with all kinds of tests and conditionals even at run time that guarantee to handle any kinds of data/arguments handed to it and do something valid or fail with stated reasons. You can generate all kinds of logs to help establish the right things are happening or catch some errors.

But all that gets you typically is fewer bugs and software that is very expensive to create and decades to produce and by that time, you have lost your market to others who settle for less.

Consider an example of bit rot. I mean what if your CPU or hard disk has a location where you can write a byte and read it back multiple times and sometimes get the wrong result. To be really cautions, you might need your software to write something in multiple locations and when it reads it back in, check all of them and if most agree, ignore the one or two that don't while blocking that memory area off and moving your data elsewhere. Or consider a memory leak that happens rarely but if a program runs for years or decades, may end up causing an unanticipated error.

You can only do so much. So once you have some idea what language you want to use and what development environment and so on, research what tools and methods are available and see what you can afford to do. But if you have also not chosen your target architecture and are being asked to GUARANTEE things from afar, that opens a whole new set of issues.

I was on a project once where we had a sort of networked system of machines exchanging things like email and we tested it. A while later, we decided to buy and add more machines of a new kind and had a heterogeneous network. Unfortunately, some tests had not been done with messages of a size that turned out to not be allowed on one set of machines as too big but were allowed on the other that had a higher limit. We caught the error in the field when a message of that size was sent and then got caught in junkmail later as the receiving or intermediate machine was not expecting to be the one dealing with it. We then lowered the maximum allowed size on all architectures to the capacity of the weakest one.

This reminds me a bit of questions about languages that are free and come pretty much without guarantees or support. Is it safe to use them? I mean could they be harboring back doors or spying on you? Will you get a guarantee they won't switch to a version 3.0 that is incompatible with some features your software used? The short answer is there are no guarantees albeit maybe you can purchase some assurances and services from some third party who might be able to help you with the open-source software.

Unless your project accepts the realities, why start?


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com@python.org> On Behalf Of o1bigtenor via Python-list
Sent: Tuesday, October 24, 2023 7:15 PM
To: Thomas Passin <list1@tompassin.net>
Cc: python-list@python.org
Subject: Re: Question(s)

On Tue, Oct 24, 2023 at 6:09?PM Thomas Passin via Python-list
<python-list@python.org> wrote:
>
snip
>
> By now you have read many responses that basically say that you cannot
> prove that a given program has no errors, even apart from the hardware
> question. Even if it could be done, the kind of specification that you
> would need would in itself be difficult to create, read, and understand,
> and would be subject to bugs itself.
>
> Something less ambitious than a full proof of correctness of an
> arbitrary program can sometimes be achieved. The programming team for
> the Apollo moon mission developed a system which, if you would write
> your requirements in a certain way, could generate correct C code for them.
>
> You won't be doing that.
>
> Here I want to point out something else. You say you are just getting
> into programming. You are going to be making many mistakes and errors,
> and there will be many things about programming you won't understand
> until you get some good solid experience. That's not anything to do
> with you personally, that's just how it will play out.
>
> So be prepared to learn from your mistakes and bugs. They are how you
> learn the nuts and bolts of the business of programming.
>

I am fully expecting to make mistakes (grin!).
I have a couple trades tickets - - - I've done more than a touch of technical
learning so mistakes are not scary.

What is interesting about this is the absolute certainty that it is impossible
to program so that that program is provably correct.
Somehow - - - well - - to me that sounds that programming is illogical.

If I set up a set of mathematical problems (linked) I can prove that the
logic structure of my answer is correct.

That's what I'm looking to do with the programming.

(Is that different than the question(s) that I've asked - - - dunno.)

Stimulating interaction for sure (grin!).
--
https://mail.python.org/mailman/listinfo/python-list

--
https://mail.python.org/mailman/listinfo/python-list

On Wed, 25 Oct 2023 at 12:11, Thomas Passin via Python-list
<python-list@python.org> wrote:
> This doesn't mean that no program can ever be proven to halt, nor that
> no program can never be proven correct by formal means. Will your
> program be one of those? The answer may never come ...

Indeed, and I would go further and say that, in any non-trivial
system, it is impossible to completely 100% prove that it is perfectly
correct. Sometimes you might have perfect mathematics and software,
but only subject to certain assumptions about the environment. Or
about the users. More commonly, you build a system so that failure
becomes vanishingly unlikely.

Take space flight as an example. Computers have been vital to the
safety of human lives in space pretty much since humans have been
going to space at all. How do you make sure that the Apollo Guidance
Computer works correctly when you need it to? Multiple layers of
protection. Error correcting memory, redundant systems, and human
monitoring, plus the ability to rewrite the guidance software on the
fly if they needed to. Even when people are being sent to the moon,
you can't completely guarantee that the software is perfect, so you
add other layers to give greater protection.

(And more recently, both India's "Chandrayaan 2" and Japan's
"Hakuto-R" unmanned moon missions crash-landed due to software issues.
A half century of improvements hasn't changed the fundamental fact
that building a perfect system is basically impossible.)

So is all hope lost? No. We learn from our mistakes, we add more
layers. And ultimately, we test until we're reasonably confident, and
then go with it, knowing that failures WILL happen. Your goal as a
programmer isn't to prevent failure altogether - if it were, you would
never be able to achieve anything. Your goal is to catch those
failures before they cause major issues.

1. Catch the failure as you're typing in code. Done, fixed, that's
what the Backspace key is for.
2. Catch the failure as you save. We have a lot of tools that can help
you to spot bugs.
3. Catch the failure before you commit and push. Unit tests are great for this.
4. Catch the failure collaboratively. Other developers can help. Or
you can use automated tests that run on a bot farm, checking your code
on a variety of different systems (see for example Python's
buildbots).
5. Catch the failure in alpha. Release to a small number of willing
users first. They get rewarded with cool new features before everyone
else does, in return for having fewer guarantees.
6. If all else fails, catch the failure before it kills someone.
Design your system so that failures are contained. That's easier for
some than others, but it's part of why I've been saying "system" here
rather than "program".

Eff up like it's your job.
https://thedailywtf.com/articles/eff-up-like-it-s-your-job

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On Wed, 25 Oct 2023 at 12:20, AVI GROSS via Python-list
<python-list@python.org> wrote:
> Consider an example of bit rot. I mean what if your CPU or hard disk has a location where you can write a byte and read it back multiple times and sometimes get the wrong result. To be really cautions, you might need your software to write something in multiple locations and when it reads it back in, check all of them and if most agree, ignore the one or two that don't while blocking that memory area off and moving your data elsewhere. Or consider a memory leak that happens rarely but if a program runs for years or decades, may end up causing an unanticipated error.
>

True, but there are FAR more efficient ways to do error correction :)
Hamming codes give you single-bit correction and two-bit detection at
a cost of log N bits, which is incredibly cheap - even if you were to
go for a granularity of 64 bytes (one cache line in a modern Intel
CPU), you would need just 11 bits of Hamming code for every 512 bits
of data and you can guarantee to fix any single-bit error in any cache
line. The "if most agree, ignore the one or two that don't" design
implies that you're writing to an absolute minimum of three places,
and in order to be able to ignore two that disagree, you'd probably
need five copies of everything - that is to say, to store 512 bits of
data, you would need 2560 bits of storage. But with a Hamming code,
you need just 523 bits to store 512 reliably.

Here's a great run-down on how efficiently this can be done, and how
easily. https://www.youtube.com/watch?v=X8jsijhllIA

Side note: If we assume that random bit flips occur at a rate of one
every X storage bits, having redundant copies of data will increase
the chances of a failure happening. For example, using a naive and
horrendously wasteful "store 256 copies of everything" strategy, you
would be 256 times more likely to have a random bitflip, which is
insane :) You would also be able to guarantee detection of up to 128
random bitflips. But as you can see, this puts a maximum on your
storage ratio.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Agreed, Chris. There are many methods way better than the sort of RAID
architecture I supplied as AN EXAMPLE easy to understand. But even so, if a
hard disk or memory chip is fried or a nuclear bomb takes out all servers in
or near a city, you would need some truly crazy architectures with info not
only distributed across the globe but perhaps also to various space
satellites or servers kept ever further out and eventually in hyperspace or
within a black hole (might be write-only, alas).

The point many of us keep saying is there can not easily or even with great
difficult, any perfect scheme that guarantees nothing will go wrong with the
software, hardware, the people using it and so on. And in the real world, as
compared to the reel world, many programs cannot remain static. Picture a
program that includes many tax laws and implementations that has to be
changed at least yearly as laws change. Some near-perfect code now has to
either be patched with lots of errors possible, or redesigned from scratch
and if it takes long enough, will come out after yet more changes and thus
be wrong.

A decent question you can ask is if the language this forum is supposed to
be on, is better in some ways to provide the kind of Teflon-coated code he
wants. Are there features better avoided? How do you make sure updates to
modules you use and trust are managed as they may break your code. Stuff
like that is not as abstract.

In my view, one consideration can be that when people can examine your
source code in the original language, that can open up ways others might
find ways to break it, more so than a compiled program that you only can
read in a more opaque way.


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com@python.org> On
Behalf Of Chris Angelico via Python-list
Sent: Tuesday, October 24, 2023 9:41 PM
To: python-list@python.org
Subject: Re: Question(s)

On Wed, 25 Oct 2023 at 12:20, AVI GROSS via Python-list
<python-list@python.org> wrote:
> Consider an example of bit rot. I mean what if your CPU or hard disk has a
location where you can write a byte and read it back multiple times and
sometimes get the wrong result. To be really cautions, you might need your
software to write something in multiple locations and when it reads it back
in, check all of them and if most agree, ignore the one or two that don't
while blocking that memory area off and moving your data elsewhere. Or
consider a memory leak that happens rarely but if a program runs for years
or decades, may end up causing an unanticipated error.
>

True, but there are FAR more efficient ways to do error correction :)
Hamming codes give you single-bit correction and two-bit detection at
a cost of log N bits, which is incredibly cheap - even if you were to
go for a granularity of 64 bytes (one cache line in a modern Intel
CPU), you would need just 11 bits of Hamming code for every 512 bits
of data and you can guarantee to fix any single-bit error in any cache
line. The "if most agree, ignore the one or two that don't" design
implies that you're writing to an absolute minimum of three places,
and in order to be able to ignore two that disagree, you'd probably
need five copies of everything - that is to say, to store 512 bits of
data, you would need 2560 bits of storage. But with a Hamming code,
you need just 523 bits to store 512 reliably.

Here's a great run-down on how efficiently this can be done, and how
easily. https://www.youtube.com/watch?v=X8jsijhllIA

Side note: If we assume that random bit flips occur at a rate of one
every X storage bits, having redundant copies of data will increase
the chances of a failure happening. For example, using a naive and
horrendously wasteful "store 256 copies of everything" strategy, you
would be 256 times more likely to have a random bitflip, which is
insane :) You would also be able to guarantee detection of up to 128
random bitflips. But as you can see, this puts a maximum on your
storage ratio.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

--
https://mail.python.org/mailman/listinfo/python-list

On 2023-10-24, o1bigtenor wrote:
> On Tue, Oct 24, 2023 at 5:28?PM Rob Cliffe <rob.cliffe@btinternet.com> wrote:
>>
>> There is no general way to prove that a program is "correct". Or even
>> whether it will terminate or loop endlessly.
>> [...]
>> When you come to run your program "for real", and you have to
>> troubleshoot it (as in real life you probably will????), you will have
>> eliminated simple bugs in your program, and can concentrate on the more
>> likely sources of problems (e.g. misbehaving hardware).
>>
> Interesting - - - hopefully taken in the same vein as your second
> statement - - I sorta sounds like programmers keep running around in
> the forest looking for trees. (Grin!)

No, you tend to know what parts of the spec are "wrong(tm)" (either in
the language you're working in, or the hardware).

If it comes to working with customers (as mentioned in one response),
you start to learn the questions to get at what they really want (but
that's better left to the architect :) )

>
> So how does one test software then?

You write unit tests (usually scripts or other software that can
interact with the main program to twiddle the knobs and such, and ensure
it's doing what was specified). Alternatively, you have to program your
hardware and test directly on that.


--
|_|O|_|
|_|_|O| Github: https://github.com/dpurgert
|O|O|O| PGP: DDAB 23FB 19FA 7D85 1CC1 E067 6D65 70E5 4CE7 2860
--
https://mail.python.org/mailman/listinfo/python-list

A post with useful ideas - - - - thanks (it generates some questions!
interleaved)

On Tue, Oct 24, 2023 at 8:35?PM Chris Angelico via Python-list
<python-list@python.org> wrote:
>
> On Wed, 25 Oct 2023 at 12:11, Thomas Passin via Python-list
> <python-list@python.org> wrote:
> > This doesn't mean that no program can ever be proven to halt, nor that
> > no program can never be proven correct by formal means. Will your
> > program be one of those? The answer may never come ...
>
snip
> So is all hope lost? No. We learn from our mistakes, we add more
> layers. And ultimately, we test until we're reasonably confident, and
> then go with it, knowing that failures WILL happen. Your goal as a
> programmer isn't to prevent failure altogether - if it were, you would
> never be able to achieve anything. Your goal is to catch those
> failures before they cause major issues.
>
> 1. Catch the failure as you're typing in code. Done, fixed, that's
> what the Backspace key is for.
> 2. Catch the failure as you save. We have a lot of tools that can help
> you to spot bugs.

Tools like this for python please.

> 3. Catch the failure before you commit and push. Unit tests are great for this.

Where might I find such please.

> 4. Catch the failure collaboratively. Other developers can help. Or
> you can use automated tests that run on a bot farm, checking your code
> on a variety of different systems (see for example Python's
> buildbots).

This is very interesting - - - grin - - - almost looks like another rabbit hole
to climb into though.

> 5. Catch the failure in alpha. Release to a small number of willing
> users first. They get rewarded with cool new features before everyone
> else does, in return for having fewer guarantees.

For here its software for use here so I get to wear all the hats.

> 6. If all else fails, catch the failure before it kills someone.
> Design your system so that failures are contained. That's easier for
> some than others, but it's part of why I've been saying "system" here
> rather than "program".

This will not be an issue here - - - at least not yet. This is software for
collecting data to enhance management of things that aren't generally
managed in most like outfits. (Or they utilize the 800# gorillas in the
industries tools which are bloody pricey.)
>
> Eff up like it's your job.
> https://thedailywtf.com/articles/eff-up-like-it-s-your-job
>
Very interesting article - - - thanks!

TIA
--
https://mail.python.org/mailman/listinfo/python-list

On Tue, Oct 24, 2023 at 8:43?PM Chris Angelico via Python-list
<python-list@python.org> wrote:
>
> On Wed, 25 Oct 2023 at 12:20, AVI GROSS via Python-list
> <python-list@python.org> wrote:
> > Consider an example of bit rot. I mean what if your CPU or hard disk has a location where you can write a byte and read it back multiple times and sometimes get the wrong result. To be really cautions, you might need your software to write something in multiple locations and when it reads it back in, check all of them and if most agree, ignore the one or two that don't while blocking that memory area off and moving your data elsewhere. Or consider a memory leak that happens rarely but if a program runs for years or decades, may end up causing an unanticipated error.
> >
>
> True, but there are FAR more efficient ways to do error correction :)
> Hamming codes give you single-bit correction and two-bit detection at
> a cost of log N bits, which is incredibly cheap - even if you were to
> go for a granularity of 64 bytes (one cache line in a modern Intel
> CPU), you would need just 11 bits of Hamming code for every 512 bits
> of data and you can guarantee to fix any single-bit error in any cache
> line. The "if most agree, ignore the one or two that don't" design
> implies that you're writing to an absolute minimum of three places,
> and in order to be able to ignore two that disagree, you'd probably
> need five copies of everything - that is to say, to store 512 bits of
> data, you would need 2560 bits of storage. But with a Hamming code,
> you need just 523 bits to store 512 reliably.
>
> Here's a great run-down on how efficiently this can be done, and how
> easily. https://www.youtube.com/watch?v=X8jsijhllIA
>
> Side note: If we assume that random bit flips occur at a rate of one
> every X storage bits, having redundant copies of data will increase
> the chances of a failure happening. For example, using a naive and
> horrendously wasteful "store 256 copies of everything" strategy, you
> would be 256 times more likely to have a random bitflip, which is
> insane :) You would also be able to guarantee detection of up to 128
> random bitflips. But as you can see, this puts a maximum on your
> storage ratio.
>

Hmmmmmmmmmm - - - - now how can I combine 'Hamming codes'
and a raid array?

TIA
--
https://mail.python.org/mailman/listinfo/python-list

On Tue, Oct 24, 2023 at 9:36?PM AVI GROSS via Python-list
<python-list@python.org> wrote:
>
> Agreed, Chris. There are many methods way better than the sort of RAID
> architecture I supplied as AN EXAMPLE easy to understand. But even so, if a
> hard disk or memory chip is fried or a nuclear bomb takes out all servers in
> or near a city, you would need some truly crazy architectures with info not
> only distributed across the globe but perhaps also to various space
> satellites or servers kept ever further out and eventually in hyperspace or
> within a black hole (might be write-only, alas).
>
> The point many of us keep saying is there can not easily or even with great
> difficult, any perfect scheme that guarantees nothing will go wrong with the
> software, hardware, the people using it and so on. And in the real world, as
> compared to the reel world, many programs cannot remain static. Picture a
> program that includes many tax laws and implementations that has to be
> changed at least yearly as laws change. Some near-perfect code now has to
> either be patched with lots of errors possible, or redesigned from scratch
> and if it takes long enough, will come out after yet more changes and thus
> be wrong.
>
> A decent question you can ask is if the language this forum is supposed to
> be on, is better in some ways to provide the kind of Teflon-coated code he
> wants. Are there features better avoided? How do you make sure updates to
> modules you use and trust are managed as they may break your code. Stuff
> like that is not as abstract.

The above are very interesting questions - - - - anyone care to tackle
one, or some?
>
> In my view, one consideration can be that when people can examine your
> source code in the original language, that can open up ways others might
> find ways to break it, more so than a compiled program that you only can
> read in a more opaque way.
>
(Tongue in cheek) Except doesn't one make more $$$$ when software in
hidden in an unreadable state? (That forces the user to go back to the
original dev or group - - yes?)

TIA
--
https://mail.python.org/mailman/listinfo/python-list

On Wed, 25 Oct 2023 at 21:46, o1bigtenor <o1bigtenor@gmail.com> wrote:
> > 2. Catch the failure as you save. We have a lot of tools that can help
> > you to spot bugs.
>
> Tools like this for python please.

Various ones. Type checkers like MyPy fall into this category if you
set your system up to run them when you save. Some editors do basic
syntax checks too.

> > 3. Catch the failure before you commit and push. Unit tests are great for this.
>
> Where might I find such please.

The same tools, but run as a pre-commit hook.

Any tool that can help you find bugs has the potential to be of value.
It's all a question of how much time it saves with earlier detection
of bugs versus how much it costs you in pacifying the tool. Some are
better than others.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list