Mailing List Archive

On Mon, Mar 4, 2024 at 12:43?PM Tim Moody <tim@timmoody.com> wrote:
>
> I am trying to reach a remote mysql database from a vps using python. The destination server has created an opening in the firewall for the request, but I am still getting network unreachable errors. I gave the ip of the proxy for the vps, obtained with ping, as the expected ip. I now tried accessing a web site I control from the same vps also in python using requests.get, and in the web server log I see the request coming from what looks to be a gateway (xxx.xxx.xxx.1, rather than the proxy xxx.xxx.xxx.nnn) Is this to be expected and can I rely on the address for such requests?

Please, please, please do not expose MySQL/MariaDB to the general
internet. Instead I would suggest that you use an ssh tunnel to
connect your workstation with the remote instance. See
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#Connecting_to_the_database_replicas_from_your_own_computer
for how this can be done in a specific case. For your case really just
the target host (login.toolforge.org -> your instance) and database
server should need to change.

Bryan
--
Bryan Davis Wikimedia Foundation
Principal Software Engineer Boise, ID USA
[[m:User:BDavis_(WMF)]] irc: bd808
_______________________________________________
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

On Mon, Mar 4, 2024 at 2:16?PM Bryan Davis <bd808@wikimedia.org> wrote:
>
> On Mon, Mar 4, 2024 at 12:43?PM Tim Moody <tim@timmoody.com> wrote:
> >
> > I am trying to reach a remote mysql database from a vps using python. The destination server has created an opening in the firewall for the request, but I am still getting network unreachable errors. I gave the ip of the proxy for the vps, obtained with ping, as the expected ip. I now tried accessing a web site I control from the same vps also in python using requests.get, and in the web server log I see the request coming from what looks to be a gateway (xxx.xxx.xxx.1, rather than the proxy xxx.xxx.xxx.nnn) Is this to be expected and can I rely on the address for such requests?
>
> Please, please, please do not expose MySQL/MariaDB to the general
> internet. Instead I would suggest that you use an ssh tunnel to
> connect your workstation with the remote instance. See
> https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#Connecting_to_the_database_replicas_from_your_own_computer
> for how this can be done in a specific case. For your case really just
> the target host (login.toolforge.org -> your instance) and database
> server should need to change.

I completely misread the direction of our connection. I apologize.

Yes, outbound connections from a Cloud VPS instance to the internet
will show as coming from the shared outbound NAT gateway IP at the
remote. The only change from this is if the Cloud VPS instance has a
"floating IP" that gives it a direct route to the Internet.

Bryan
--
Bryan Davis Wikimedia Foundation
Principal Software Engineer Boise, ID USA
[[m:User:BDavis_(WMF)]] irc: bd808
_______________________________________________
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

The mysql host ISP does not wish to open a port to an entire gateway, so I
have two choices, install tailscale or request a floating IP. Is tailscale
permissible and possible on vps?

On Mon, Mar 4, 2024 at 4:19?PM Bryan Davis <bd808@wikimedia.org> wrote:

> On Mon, Mar 4, 2024 at 2:16?PM Bryan Davis <bd808@wikimedia.org> wrote:
> >
> > On Mon, Mar 4, 2024 at 12:43?PM Tim Moody <tim@timmoody.com> wrote:
> > >
> > > I am trying to reach a remote mysql database from a vps using python.
> The destination server has created an opening in the firewall for the
> request, but I am still getting network unreachable errors. I gave the ip
> of the proxy for the vps, obtained with ping, as the expected ip. I now
> tried accessing a web site I control from the same vps also in python using
> requests.get, and in the web server log I see the request coming from what
> looks to be a gateway (xxx.xxx.xxx.1, rather than the proxy
> xxx.xxx.xxx.nnn) Is this to be expected and can I rely on the address for
> such requests?
> >
> > Please, please, please do not expose MySQL/MariaDB to the general
> > internet. Instead I would suggest that you use an ssh tunnel to
> > connect your workstation with the remote instance. See
> >
> https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#Connecting_to_the_database_replicas_from_your_own_computer
> > for how this can be done in a specific case. For your case really just
> > the target host (login.toolforge.org -> your instance) and database
> > server should need to change.
>
> I completely misread the direction of our connection. I apologize.
>
> Yes, outbound connections from a Cloud VPS instance to the internet
> will show as coming from the shared outbound NAT gateway IP at the
> remote. The only change from this is if the Cloud VPS instance has a
> "floating IP" that gives it a direct route to the Internet.
>
> Bryan
> --
> Bryan Davis Wikimedia Foundation
> Principal Software Engineer Boise, ID USA
> [[m:User:BDavis_(WMF)]] irc: bd808
> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

On Mon, Mar 4, 2024 at 3:53?PM Tim Moody <tim@timmoody.com> wrote:
>
> The mysql host ISP does not wish to open a port to an entire gateway, so I have two choices, install tailscale or request a floating IP. Is tailscale permissible and possible on vps?

If there is a client with an OSI approved Open Source license, then
yes it would be permissible. If not, then likely no per the TOU. [0]
It does appear that https://github.com/tailscale/tailscale is licensed
under a 3-clause BSD license, so that is helpful. [1]

It does not at this point sound like your use case would be a
violation of the prohibition on network proxying [2] as the VPN would
be for traffic originating from your Cloud VPS instance and not a
generally open proxy for others or inbound traffic.

I wonder if there is an XY problem [3] here as well. What is the
underlying use case that requires you to connect to an off-premise
database to operate a Wikimedia focused tool in Cloud VPS? Is this
somehow related to your new project to generate ZIM files of MDWiki?
[4]

[0]: https://wikitech.wikimedia.org/wiki/Wikitech:Cloud_Services_Terms_of_use#4.3_Open_Source_and_proprietary_software
[1]: https://github.com/tailscale/tailscale/blob/main/LICENSE
[2]: https://wikitech.wikimedia.org/wiki/Wikitech:Cloud_Services_Terms_of_use#4.5_Using_WMCS_as_a_network_proxy
[3]: https://en.wikipedia.org/wiki/XY_problem
[4]: https://phabricator.wikimedia.org/T358023

Bryan
--
Bryan Davis Wikimedia Foundation
Principal Software Engineer Boise, ID USA
[[m:User:BDavis_(WMF)]] irc: bd808
_______________________________________________
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

re: [3] and [4], in order to support the creation a ZIM of MDWiki, I need a
list of all of its redirects, and I find the fastest means of obtaining
this is to query the database directly. This is the only info taken
directly from the database.

On Mon, Mar 4, 2024 at 8:42?PM Bryan Davis <bd808@wikimedia.org> wrote:

> On Mon, Mar 4, 2024 at 3:53?PM Tim Moody <tim@timmoody.com> wrote:
> >
> > The mysql host ISP does not wish to open a port to an entire gateway, so
> I have two choices, install tailscale or request a floating IP. Is
> tailscale permissible and possible on vps?
>
> If there is a client with an OSI approved Open Source license, then
> yes it would be permissible. If not, then likely no per the TOU. [0]
> It does appear that https://github.com/tailscale/tailscale is licensed
> under a 3-clause BSD license, so that is helpful. [1]
>
> It does not at this point sound like your use case would be a
> violation of the prohibition on network proxying [2] as the VPN would
> be for traffic originating from your Cloud VPS instance and not a
> generally open proxy for others or inbound traffic.
>
> I wonder if there is an XY problem [3] here as well. What is the
> underlying use case that requires you to connect to an off-premise
> database to operate a Wikimedia focused tool in Cloud VPS? Is this
> somehow related to your new project to generate ZIM files of MDWiki?
> [4]
>
> [0]:
> https://wikitech.wikimedia.org/wiki/Wikitech:Cloud_Services_Terms_of_use#4.3_Open_Source_and_proprietary_software
> [1]: https://github.com/tailscale/tailscale/blob/main/LICENSE
> [2]:
> https://wikitech.wikimedia.org/wiki/Wikitech:Cloud_Services_Terms_of_use#4.5_Using_WMCS_as_a_network_proxy
> [3]: https://en.wikipedia.org/wiki/XY_problem
> [4]: https://phabricator.wikimedia.org/T358023
>
> Bryan
> --
> Bryan Davis Wikimedia Foundation
> Principal Software Engineer Boise, ID USA
> [[m:User:BDavis_(WMF)]] irc: bd808
> _______________________________________________
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

On Mon, Mar 4, 2024 at 8:02?PM Tim Moody <tim@timmoody.com> wrote:
>
> re: [3] and [4], in order to support the creation a ZIM of MDWiki, I need a list of all of its redirects, and I find the fastest means of obtaining this is to query the database directly. This is the only info taken directly from the database.

You are probably already aware of this, but there is an Action API
endpoint to enumerate redirects:
* https://www.mediawiki.org/wiki/API:Allredirects
* https://mdwiki.org/wiki/Special:ApiSandbox#action=query&format=json&list=allredirects

Direct database access is likely faster than enumerating results 500
at a time via the Action API, but it also may reduce the complexity
and fragility of your operational deployment.

Bryan
--
Bryan Davis Wikimedia Foundation
Principal Software Engineer Boise, ID USA
[[m:User:BDavis_(WMF)]] irc: bd808
_______________________________________________
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/