Mailing List Archive

problem with Distributed File System Replication and Namespacing and different versions of Python 3
Hi,
I have a question about a problem we are facing in different Python versions, like Python 3.7.7, Python 3.8, Python 3.6 and Python 3.9.
This apparently doesnt happen in Python 3.10.2 and we were wondering how we could achieve the same behaviours without going deep into the nuts and bolts of the python import system.

We have python libraries that are distributed in a networked environment that when imported for the first time they are correctly imported but something happens in the meantime that prevents it from loading anymore. This networked paths are domain paths that are bound to a final "real" path in a specific file server from a specific site in the world.

What happens is, when injecting into the sys.path the domain names it doesnt import but when injecting into the sys.path the "real" file server path it works, generally speaking. We have been facing this issue in such different python 3 versions and i was wondering what makes this work in Python 3.10.2.

Apparently, in my opinion it is a mix of two things involved: one, something the DFS-R DFS-N does at a very low level, that python 3 versions dont like when importing and the other one, the fact that is a network path.

Can anybody shed some light about the improvements regarding this?

We will be tied to all these python versions for a very long time and we need to give solution to this.
One of the possibilities would be to distribute the libraries into C:/ local disc, this eliminating the network path variable. But it would be nice to know what is causing this in Python < 3.10.2.

Thanks in advance.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5EJ7FAWQSUOFFYSCBZF2NRQSY3SULVV5/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: problem with Distributed File System Replication and Namespacing and different versions of Python 3 [ In reply to ]
Not sure if this is relevant, but I know they updated the openssl version
in Python 3.10. I had the opposite problem trying to use Python 3.10 on
CentOS 7 which has the old openSSL. I had to back off to 3.9. maybe running
over the network invoked openSSL at some point and that's where the
disconnect occurs?

I would check which openSSL your host os uses.



On Thu, Oct 20, 2022, 2:36 PM <rainonthescarecrowhumanwheels@gmail.com>
wrote:

> Hi,
> I have a question about a problem we are facing in different Python
> versions, like Python 3.7.7, Python 3.8, Python 3.6 and Python 3.9.
> This apparently doesnt happen in Python 3.10.2 and we were wondering how
> we could achieve the same behaviours without going deep into the nuts and
> bolts of the python import system.
>
> We have python libraries that are distributed in a networked environment
> that when imported for the first time they are correctly imported but
> something happens in the meantime that prevents it from loading anymore.
> This networked paths are domain paths that are bound to a final "real" path
> in a specific file server from a specific site in the world.
>
> What happens is, when injecting into the sys.path the domain names it
> doesnt import but when injecting into the sys.path the "real" file server
> path it works, generally speaking. We have been facing this issue in such
> different python 3 versions and i was wondering what makes this work in
> Python 3.10.2.
>
> Apparently, in my opinion it is a mix of two things involved: one,
> something the DFS-R DFS-N does at a very low level, that python 3 versions
> dont like when importing and the other one, the fact that is a network path.
>
> Can anybody shed some light about the improvements regarding this?
>
> We will be tied to all these python versions for a very long time and we
> need to give solution to this.
> One of the possibilities would be to distribute the libraries into C:/
> local disc, this eliminating the network path variable. But it would be
> nice to know what is causing this in Python < 3.10.2.
>
> Thanks in advance.
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/5EJ7FAWQSUOFFYSCBZF2NRQSY3SULVV5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
Re: problem with Distributed File System Replication and Namespacing and different versions of Python 3 [ In reply to ]
Hi, not sure about this neither.
I was hoping someone with more experience deploying pythons and executing them over distributed file systems would shed a light why this happens.

I'm sorry but to be honest i dont see the correlation between OpenSSL and DFS-R/DFS-N, maybe there is one but im a bit lost here.

Thanks.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/UYXRISCKBZWRFCCOO4CB3JXX4LDD2IZ3/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: problem with Distributed File System Replication and Namespacing and different versions of Python 3 [ In reply to ]
I guess my simplified question would be:

¿how does Python 3 versions manage Distributed File Systems and in particular Windows' DFS?

Is it supported by the "import system" in Python? What differences are there?
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HS7KIAFPOZQ7RQY6DNDBM2ILYQ5SI7EP/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: problem with Distributed File System Replication and Namespacing and different versions of Python 3 [ In reply to ]
Import is implementing in Python itself via importlib, so you could try
walking through the code with a debugger to see where the difference may be
(I personally can't think of anything obvious that changed in importlib
between 3.9 and 3.10 that would affect this).

On Thu, Oct 20, 2022 at 11:36 AM <rainonthescarecrowhumanwheels@gmail.com>
wrote:

> Hi,
> I have a question about a problem we are facing in different Python
> versions, like Python 3.7.7, Python 3.8, Python 3.6 and Python 3.9.
> This apparently doesnt happen in Python 3.10.2 and we were wondering how
> we could achieve the same behaviours without going deep into the nuts and
> bolts of the python import system.
>
> We have python libraries that are distributed in a networked environment
> that when imported for the first time they are correctly imported but
> something happens in the meantime that prevents it from loading anymore.
> This networked paths are domain paths that are bound to a final "real" path
> in a specific file server from a specific site in the world.
>
> What happens is, when injecting into the sys.path the domain names it
> doesnt import but when injecting into the sys.path the "real" file server
> path it works, generally speaking. We have been facing this issue in such
> different python 3 versions and i was wondering what makes this work in
> Python 3.10.2.
>
> Apparently, in my opinion it is a mix of two things involved: one,
> something the DFS-R DFS-N does at a very low level, that python 3 versions
> dont like when importing and the other one, the fact that is a network path.
>
> Can anybody shed some light about the improvements regarding this?
>
> We will be tied to all these python versions for a very long time and we
> need to give solution to this.
> One of the possibilities would be to distribute the libraries into C:/
> local disc, this eliminating the network path variable. But it would be
> nice to know what is causing this in Python < 3.10.2.
>
> Thanks in advance.
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/5EJ7FAWQSUOFFYSCBZF2NRQSY3SULVV5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
Re: problem with Distributed File System Replication and Namespacing and different versions of Python 3 [ In reply to ]
On Fri, Oct 21, 2022 at 11:11 AM Juan Cristóbal Quesada <
rainonthescarecrowhumanwheels@gmail.com> wrote:

> I guess my simplified question would be:
>
> ¿how does Python 3 versions manage Distributed File Systems and in
> particular Windows' DFS?
>

In no specific way. We typically ask the OS to handle any file system
details.


>
> Is it supported by the "import system" in Python? What differences are
> there?
>

Not explicitly. Everything is done via file paths and things in the os
module (technically the posix or nt module depending on what OS you are
running on).

-Brett


> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/HS7KIAFPOZQ7RQY6DNDBM2ILYQ5SI7EP/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
Re: problem with Distributed File System Replication and Namespacing and different versions of Python 3 [ In reply to ]
Thanks, yes, that was my last resort: inspecting the code through debugging the import system.
Was hoping to get some light before that ;).
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MLDWDDD2AZ7LH5DQGEFT33R24BSHUZG7/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: problem with Distributed File System Replication and Namespacing and different versions of Python 3 [ In reply to ]
On 10/20/2022 1:07 PM, rainonthescarecrowhumanwheels@gmail.com wrote:
> What happens is, when injecting into the sys.path the domain names it doesnt import but when injecting into the sys.path the "real" file server path it works, generally speaking. We have been facing this issue in such different python 3 versions and i was wondering what makes this work in Python 3.10.2.

We added some path fixes into importlib in 3.10 that will result in
better path resolution during import on Windows, particularly when
resolving partial paths. You'll find them in
Lib/importlib/_bootstrap_external.py and Modules/posixmodule.c (they're
not real obvious - you'll be best to diff those files).

You're unlikely to easily backport them to earlier versions, so your
current workaround is probably best. If the problems are due to
*writing* bytecode files, you might also be able to use
PYTHONPYCACHEPREFIX to reference a local directory and avoid issues that
way.

Cheers,
Steve
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BYPWOPJV3SCYGX4ZMZWEY44WD5VP25QI/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: problem with Distributed File System Replication and Namespacing and different versions of Python 3 [ In reply to ]
Hi Steve,
thanks! Will definitely have a look at it as soon as i can.

Many thanks to all of you that replied. It was my first post in such python mailing lists and wasnt sure how accurate of a response i could have. You never know how active the mailing lists/forums are.

Best Regards,
JC
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EJK5AFATVF6D44TVQRGSIXFHWTUDE6IS/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: problem with Distributed File System Replication and Namespacing and different versions of Python 3 [ In reply to ]
I don't remember it being mentioned, but much of the traffic recently
migrated from this list to https://discuss.python.org/c/core-dev/23, which
you may wish to keep in touch with.

Kind regards,
Steve


On Tue, Oct 25, 2022 at 7:53 AM Juan Cristóbal Quesada <
rainonthescarecrowhumanwheels@gmail.com> wrote:

> Hi Steve,
> thanks! Will definitely have a look at it as soon as i can.
>
> Many thanks to all of you that replied. It was my first post in such
> python mailing lists and wasnt sure how accurate of a response i could
> have. You never know how active the mailing lists/forums are.
>
> Best Regards,
> JC
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/EJK5AFATVF6D44TVQRGSIXFHWTUDE6IS/
> Code of Conduct: http://python.org/psf/codeofconduct/
>