Mailing List Archive

On Thu, 19 Oct 2023 at 10:07, Janis Papanagnou via Python-list
<python-list@python.org> wrote:
>
> I am pondering about writing a client/server software with
> websockets as communication protocol. The clients will run
> in browser as Javascript programs and the server may be in
> any (any sensible) programming language running standalone
> to be connected remotely by the browser-based JS clients.
>
> I found a Python sample[*] but I am neither familiar with
> Python nor with the 'simple_websocket_server' package that
> is used in that sample code. But the code looks so simple
> that I'm considering to learn and use Python for the task.
>
> The requirements I have are quite simple; I want to get the
> client "address"/identifier from an incoming message, store
> it in a list, and send responses to all active clients for
> which addresses have been stored.
>
> Can anyone tell me whether a simple extension of that "echo
> incoming message" sample[*] would be easily possible with
> Python and with that 'simple_websocket_server' package used?
>
> Thanks for any hints (or search keywords, or code samples)!

Oooh you've touched on one of my favourite topics. I *love* networking
and communication, and websockets are one of my well-used
technologies. Let's do this!!

I've never used the "simple_websocket_server" you mentioned, but I've
used this one in a few projects:

https://pypi.org/project/websockets/

Be aware that it is designed with asyncio in mind, so if you prefer
different concurrency models, you may need to look elsewhere. But I've
had good success with this one.

Broadly speaking, your ideas are great. Any programming language CAN
be used for the server (and I've used several, not just Python).

My personal preference is to build a protocol on top of websockets, for example:

* All messages are "text", and are JSON-encoded
* All messages represent objects (in Python, dictionaries) with a
"cmd" attribute
* The first message sent by the client has cmd "init" and specifies a
"type" and "group".
* The server tracks all connected clients by their groups, and can
broadcast messages to everyone in a group.

Here's one example, actually one of my brother's projects but I
contributed to the websocket aspects:

https://github.com/stephenangelico/BioBox/blob/master/browser.py

And here's one that doesn't actually use Python, but uses all the same
ideas; this is the JS end:

https://github.com/Rosuav/StilleBot/blob/master/httpstatic/ws_sync.js

(It has quite a bit more sophistication than you'll need to get
started with, but shows how the protocol can expand as needed.)

So! Tying this back in with your goals:

> The requirements I have are quite simple; I want to get the
> client "address"/identifier from an incoming message, store
> it in a list, and send responses to all active clients for
> which addresses have been stored.

Sounds to me like the best way would be for the socket group to be the
identifier of the client. You could augment the "init" message to
include some sort of authentication, or alternatively, rely on other
forms of authentication; a websocket established to the same origin as
the page itself can take advantage of regular browser credentials.

Once that's established, you can have a message from one client result
in the server sending out that message to all clients for the
recipient's address.

The reason I'm talking about "groups" here instead of simply having
one client per address is that it scales well to one person having
multiple tabs open, or having the app on their phone as well as their
computer, or anything like that. Messages will arrive on all of them.

Hope that's enough to get you started! I'd be delighted to help
further if you run into difficulties.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On 19.10.2023 01:23, Chris Angelico wrote:
> [snip]
>
> Hope that's enough to get you started! I'd be delighted to help
> further if you run into difficulties.

Thanks for your quick reply, Chris! This is already great information!
I'll dive into your resources soon, and I also appreciate your offer
and will probably come back soon with a question... - Thanks again!

Janis

--
https://mail.python.org/mailman/listinfo/python-list

On 19.10.2023 01:23, Chris Angelico wrote:
>
> Broadly speaking, your ideas are great. Any programming language CAN
> be used for the server (and I've used several, not just Python).

Out of curiosity; what where these languages? - If there's one I
already know I might save some time implementing the server. :-)

Janis

--
https://mail.python.org/mailman/listinfo/python-list

On Fri, 20 Oct 2023 at 22:31, Janis Papanagnou via Python-list
<python-list@python.org> wrote:
>
> On 19.10.2023 01:23, Chris Angelico wrote:
> >
> > Broadly speaking, your ideas are great. Any programming language CAN
> > be used for the server (and I've used several, not just Python).
>
> Out of curiosity; what where these languages? - If there's one I
> already know I might save some time implementing the server. :-)
>

I've done websocket servers in Python, Node.js, and Pike, and possibly
others but I can't recall at the moment. Might have done one in Ruby,
but that would have just been part of playing around and comparing
features ("how easy is it to do <X> in Ruby").

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On 2023-10-20, Chris Angelico <rosuav@gmail.com> wrote:
> On Fri, 20 Oct 2023 at 22:31, Janis Papanagnou via Python-list
><python-list@python.org> wrote:
>>
>> On 19.10.2023 01:23, Chris Angelico wrote:
>> >
>> > Broadly speaking, your ideas are great. Any programming language CAN
>> > be used for the server (and I've used several, not just Python).
>>
>> Out of curiosity; what where these languages? - If there's one I
>> already know I might save some time implementing the server. :-)
>>
>
> I've done websocket servers in Python, Node.js, and Pike, and possibly
> others but I can't recall at the moment. Might have done one in Ruby,
> but that would have just been part of playing around and comparing
> features ("how easy is it to do <X> in Ruby").
>
> ChrisA

*Big list of http static server one-liners*

Each of these commands will run an ad hoc http static server
in your current (or specified) directory, available at
http://localhost:8000. Use this power wisely.

<https://gist.github.com/willurd/5720255>
--
https://mail.python.org/mailman/listinfo/python-list

On 20.10.2023 23:05, Paul Rubin wrote:
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
>> I found a Python sample[*] but I am neither familiar with
>> Python nor with the 'simple_websocket_server' package that
>> is used in that sample code. But the code looks so simple
>> that I'm considering to learn and use Python for the task.
>
> I've generally used ThreadingServer(SocketServer) for this purpose
> and I think threads are less confusing than async, and performance is
> fine if the concurrency level is not too high. But, trying to write a
> web server in Python if you don't know Python doesn't seem like a great
> idea, except as a learning project.

I have a couple decades experience with about a dozen programming
languages (not counting assemblers). Asynchronous processing, IPC,
multi-processing, client/server architectures, multi-threading,
semaphores, etc. etc. are concepts that are not new to me.

I'm not, literally, intending to write a web-server. It's a JS
application that is running in (browser based) clients, and the
server is just centrally coordinating the client applications.

My expectation would be that any sophistically designed socket/
web-socket library would not impose any risk. And the intended
server by itself has only very limited requirements; listening to
incoming request, storing some client information, broadcasting
to the attached clients. Basically just (informally written):

init server
forever:
wait for request(s) -> queue
handle requests from queue (sequentially):
store specific information from new registered clients
broadcast some information to all registered clients

It seems to me that multi-threading or async I/O aren't necessary.

I'd like to ask; where do you see the specific risks with Python
(as language per se) and it's (web-socket-)libraries here?

If the web-socket IPC is well supported the algorithmic parts in
Python seem trivial to learn and implement. - Or am I missing
something?

(A brief search gave me the impression that for JS communication
web-sockets would be the method to use. Otherwise I'd just use
basic Unix domain sockets for the purpose and write it, say, in
C or C++ that I already know. But I don't know whether (or how)
plain sockets are used from JS running in a browser. Here I'm
lacking experience. And that lead me to have a look at Python,
since the web-sockets/server examples that I found looked simple.)

Janis

--
https://mail.python.org/mailman/listinfo/python-list

On Sun, 22 Oct 2023 at 04:13, Janis Papanagnou via Python-list
<python-list@python.org> wrote:
> I have a couple decades experience with about a dozen programming
> languages (not counting assemblers). Asynchronous processing, IPC,
> multi-processing, client/server architectures, multi-threading,
> semaphores, etc. etc. are concepts that are not new to me.

Oh, sweet, sweet, then you should be fine with the library I
suggested. It's certainly served me well (and I have similar
experience, having learned networking mainly on OS/2 in the 1990s).

> My expectation would be that any sophistically designed socket/
> web-socket library would not impose any risk. And the intended
> server by itself has only very limited requirements; listening to
> incoming request, storing some client information, broadcasting
> to the attached clients. Basically just (informally written):
>
> init server
> forever:
> wait for request(s) -> queue
> handle requests from queue (sequentially):
> store specific information from new registered clients
> broadcast some information to all registered clients
>
> It seems to me that multi-threading or async I/O aren't necessary.

Technically that's true, but "wait for request(s)" has to handle (a)
new incoming sockets, (b) messages from currently-connected sockets,
and possibly (c) sockets now being writable when previously they
blocked. So you have most of the work of async I/O. Since the
library's been built specifically for asyncio, that's the easiest.

> I'd like to ask; where do you see the specific risks with Python
> (as language per se) and it's (web-socket-)libraries here?
>
> If the web-socket IPC is well supported the algorithmic parts in
> Python seem trivial to learn and implement. - Or am I missing
> something?

Pretty trivial, yeah. You shouldn't have too much trouble here I expect.

> (A brief search gave me the impression that for JS communication
> web-sockets would be the method to use. Otherwise I'd just use
> basic Unix domain sockets for the purpose and write it, say, in
> C or C++ that I already know. But I don't know whether (or how)
> plain sockets are used from JS running in a browser. Here I'm
> lacking experience. And that lead me to have a look at Python,
> since the web-sockets/server examples that I found looked simple.)

Yes, that's correct. You can't use plain sockets from inside a web
browser, mainly because they offer way way too much flexibility (JS
code is untrusted and is now running on your computer, do you really
want that to be able to telnet to anything on your LAN?). So
websockets are the way to go. There are other similar technologies,
but for this sort of "broadcast to connected clients" messaging
system, websockets rule.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Janis Papanagnou wrote at 2023-10-21 04:03 +0200:
> ...
>I'd like to ask; where do you see the specific risks with Python
>(as language per se) and it's (web-socket-)libraries here?

The web server in Python's runtime library is fairly simple,
focusing only on the HTTP requirements.

You might want additional things for an HTTP server
exposed on the internet which should potentially handle high trafic:
e.g.

* detection of and (partial) protection against denial of service attacks,
* load balancing,
* virtual hosting
* proxing
* URL rewriting
* high throughput, low latency

Depending on your requirements, other web servers might be preferable.
--
https://mail.python.org/mailman/listinfo/python-list

Il 19/10/2023 00:09, Janis Papanagnou ha scritto:
> I am pondering about writing a client/server software with
> websockets as communication protocol. The clients will run
> in browser as Javascript programs and the server may be in
> any (any sensible) programming language running standalone
> to be connected remotely by the browser-based JS clients.
>
> I found a Python sample[*] but I am neither familiar with
> Python nor with the 'simple_websocket_server' package that
> is used in that sample code. But the code looks so simple
> that I'm considering to learn and use Python for the task.
>
> The requirements I have are quite simple; I want to get the
> client "address"/identifier from an incoming message, store
> it in a list, and send responses to all active clients for
> which addresses have been stored.
>
> Can anyone tell me whether a simple extension of that "echo
> incoming message" sample[*] would be easily possible with
> Python and with that 'simple_websocket_server' package used?
>
> Thanks for any hints (or search keywords, or code samples)!
>
> Janis
>
> [*] https://pypi.org/project/simple-websocket-server/

I'm not sure, but MQTT protocol could help for this application.
--
https://mail.python.org/mailman/listinfo/python-list

On 2023-10-22 7:35 PM, Dieter Maurer via Python-list wrote:
>
> The web server in Python's runtime library is fairly simple,
> focusing only on the HTTP requirements.
>
> You might want additional things for an HTTP server
> exposed on the internet which should potentially handle high trafic:
> e.g.
>
> * detection of and (partial) protection against denial of service attacks,
> * load balancing,
> * virtual hosting
> * proxing
> * URL rewriting
> * high throughput, low latency
>
> Depending on your requirements, other web servers might be preferable.

Dieter's response was very timely for me, as it provides some answers to
a question that I was thinking of posting. My use-case is reasonably
on-topic for this thread, so I won't start a new one, if that is ok.

I am writing a business/accounting application. The server uses Python
and asyncio, the client is written in Javascript. The project is inching
towards a point where I may consider releasing it. My concern was
whether my home-grown HTTP server was too simplistic for production, and
if so, whether I should be looking into using one of the more
established frameworks. After some brief investigation into Dieter's
list of additional requirements, here are my initial thoughts. Any
comments will be welcome.

I skimmed through the documentation for flask, Django, and FastAPI. As
far as I can tell, none of them address the points listed above
directly. Instead, they position themselves as one layer in a stack of
technologies, and rely on other layers to provide additional
functionality. If I read this correctly, there is nothing to stop me
doing the same.

Based on this, I am considering the following -

1. Replace my HTTP handler with Uvicorn. Functionality should be the
same, but performance should be improved.

2. Instead of running as a stand-alone server, run my app as a
reverse-proxy using Nginx. I tested this a few years ago using Apache,
and it 'just worked', so I am fairly sure that it will work with Nginx
as well. Nginx can then provide the additional functionality that Dieter
has mentioned.

My main concern is that, if I do release my app, I want it to be taken
seriously and not dismissed as 'Mickey Mouse'. Do you think the above
changes would assist with that?

When I talk about releasing it, it is already available on Github here -
https://github.com/FrankMillman/AccInABox.

You are welcome to look at it, but it needs a lot of tidying up before
it will be ready for a wider audience.

Frank Millman

--
https://mail.python.org/mailman/listinfo/python-list

On Wed, 25 Oct 2023 at 19:00, Frank Millman via Python-list
<python-list@python.org> wrote:
> 2. Instead of running as a stand-alone server, run my app as a
> reverse-proxy using Nginx. I tested this a few years ago using Apache,
> and it 'just worked', so I am fairly sure that it will work with Nginx
> as well. Nginx can then provide the additional functionality that Dieter
> has mentioned.

This, I would recommend. In general, tools like Django and Flask and
such are aimed at the creation of web apps, but not necessarily web
servers; it's convenient to have a high-performance web server like
Nginx or Apache, and then it passes along requests to the application
server. This provides a number of benefits:

1. Static files can be served directly, without involving the app
2. Need to scale horizontally? Run two or more copies of your app and
have the web server share requests among them.
3. App crashed? The web server can return an HTTP 503 error so end
users aren't left wondering what's going on.
4. DOS protection can be done in the web server (although it could
also be done in a firewall, or any other level that's appropriate)

> My main concern is that, if I do release my app, I want it to be taken
> seriously and not dismissed as 'Mickey Mouse'. Do you think the above
> changes would assist with that?

You shouldn't be dismissed as Still-In-Copyright-Decades-Old-Rodent
even if you don't make those changes. I've been hosting a number of
servers, some fairly home-grade, and nobody's ever told me that it
looks bad. If you want to be respected, the main thing is to have
something that people find interesting - everything else is secondary.
But there are a few points to consider:

* Performance. People respect something that's snappy and responsive
more than they respect something where a single HTTP request takes
anywhere from 2.89 seconds to nearly 30 seconds. And no, I'm totally
not still mindblown at having seen this kind of appalling performance
from a published and very expensive API.
* Have your own domain name. https://rosuav.github.io/AntiSocialMedia/
is a toy; https://devicat.art/ is a professional web site. (They're
not related, and the first one really is just a toy that I whipped up
in like half an hour.)
* Have a proper SSL certificate. It looks *really bad* to have a
broken or outdated certificate (or none at all, these days).
LetsEncrypt can do that for you, no charge.
* Put some effort into styling. Yeah, I know, most of my web sites are
ugly, so I shouldn't talk. But things definitely look more
professional if you take the time to style them up a bit.

> When I talk about releasing it, it is already available on Github here -
> https://github.com/FrankMillman/AccInABox.
>
> You are welcome to look at it, but it needs a lot of tidying up before
> it will be ready for a wider audience.

Cool! This is particularly of note to me personally. Back in the 90s,
I was working in the family business, and we used a fairly clunky
piece of professionally-written software (and if you want stories, ask
me about overnight runs of report generation, or Ctrl-Alt-Shift and
old mouse pedal importing, or 32-bit installers for 16-bit
applications, or a bunch of other things). There was, for a while, a
theory of us designing our own accounting system, but it turns out,
that's a really REALLY big job, and it's less effort to keep using the
thing you already have.

Your README, dating from 9 years ago, says that you support/require
Python 3.4 - that's kinda ancient now. If you want to take advantage
of newer features, I think you should be safe bumping that up a long
way. :)

I wouldn't recommend sqlite3 for any production work here, but it's
good for a demo. Postgres is a far better choice if you're going to be
storing your vital information in this.

You can list vital packages in a file called requirements.txt - this
will be recognized by both people and automated tooling.

.... huh. I'm listed as a contributor. I'll be quite honest, I do not
remember this, but presumably you shared this previously! Looks like
all I contributed was a minor suggestion and commit, but still, I have
absolutely no memory. LOL.

Looks pretty good there. I don't have time right now to download and
install it for a proper test, but based on flipping through the code,
looks like you have something decent going on.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Frank Millman wrote at 2023-10-25 09:57 +0200:
> ...
>Based on this, I am considering the following -
>
>1. Replace my HTTP handler with Uvicorn. Functionality should be the
>same, but performance should be improved.
>
>2. Instead of running as a stand-alone server, run my app as a
>reverse-proxy using Nginx. I tested this a few years ago using Apache,
>and it 'just worked', so I am fairly sure that it will work with Nginx
>as well. Nginx can then provide the additional functionality that Dieter
>has mentioned.

Good ideas.
--
https://mail.python.org/mailman/listinfo/python-list