I need to develop a Python application that is a sort of gateway between
to networks: a "serial bus" network (think of a serial port or a USB
connection) and a MQTT connection (in my case, it's AWS IoT service).
On the bus, a proprietary protocol is implemented. From the bus, the app
knows the status of the system (think of a simple ON/OFF status).
The user can retrieve the status of the system through MQTT: it sends a
message to read/status MQTT topic and receive back a message with the
current status on read/status/reply MQTT topic. Of course, they are just
examples.
On the contrary, when the app detects a status change reported from the
serial bus (think of a transition from ON to OFF), it sends a MQTT message.
I'm thinking to split the application in three classes: Bus, App and
IoT. Bus and IoT are also threads.
The thread of Bus manages the serial protocol, while the thread of IoT
manages MQTT connection with the broker (the server).
However I don't know if it could be a good architecture. Suppone Bus
thread receives a new status from the system. In the context of
ThreadBus, the object Bus could call a method of App object:
app.set_status(new_status_from_the_bus)
In the App I have:
class App():
..
def set_status(new_status): # Could be called from ThreadBus
if new_status != self.new_status:
self.new_status = new_status
# Do some actions on status change
def get_status(): # Could be called from ThreadIoT
return self.status
Of course, IoT object needs to know the current status of the system
when a message is received from MQTT. So ThreadIoT could call
app.get_status().
I think this architecture has some problems with race conditions or
threads synchronization. What happens if IoT calls get_status() exactly
when set_status() called by ThreadBus is executing? If status is a big
data structure, set_status() could be interrupted by get_status() that
could get a completely corrupted status, because it was only partly
updated by set_status().
I know I can use locks or semaphores in get_status() and set_status(),
but I don't know if this is a good approach. Consider that the system is
complex, it isn't composed by a simple single status. It has many many
parameters that are collected from the serial bus. Should I use a lock
for every [get|set]_status(), [get|set]_dimensions(),
[get|set]_battery_level(), [get|set]_mains_present(), and so on?
Another possibility is to use a Queue for Bus and a Queue for IoT. So
the set_status(new_status) called from Bus object will be transformed in
a put in the queue:
app_queue.put({"type": "new_status", "data": ...})
However how could be transformed the get_status() from IoT? How the
return value (the current status) is real retrieved?
class IoT():
..
def status_request_from_MQTT():
app_queue.put({"type": "get_status"})
# How to get the status to return?
return current_status
Should the app put the status on the same queue and should IoT waits for
a new message in the Queue?
def status_request_from_MQTT():
app_queue.put({"type": "get_status"})
try:
current_status = app_queue.get(timeout=10)
except Empty:
# What to do?
return current_status
Again another approach is to avoid multi-threading at all and create a
single "main loop" function that waits at the same time for incoming
events on the serial bus and MQTT (how?). I don't know if this could be
done in my case, because I'm using awscrt Python module and it works
through callbacks that I think is called from another thread.
Any suggestions on this architecture?
--
https://mail.python.org/mailman/listinfo/python-list
to networks: a "serial bus" network (think of a serial port or a USB
connection) and a MQTT connection (in my case, it's AWS IoT service).
On the bus, a proprietary protocol is implemented. From the bus, the app
knows the status of the system (think of a simple ON/OFF status).
The user can retrieve the status of the system through MQTT: it sends a
message to read/status MQTT topic and receive back a message with the
current status on read/status/reply MQTT topic. Of course, they are just
examples.
On the contrary, when the app detects a status change reported from the
serial bus (think of a transition from ON to OFF), it sends a MQTT message.
I'm thinking to split the application in three classes: Bus, App and
IoT. Bus and IoT are also threads.
The thread of Bus manages the serial protocol, while the thread of IoT
manages MQTT connection with the broker (the server).
However I don't know if it could be a good architecture. Suppone Bus
thread receives a new status from the system. In the context of
ThreadBus, the object Bus could call a method of App object:
app.set_status(new_status_from_the_bus)
In the App I have:
class App():
..
def set_status(new_status): # Could be called from ThreadBus
if new_status != self.new_status:
self.new_status = new_status
# Do some actions on status change
def get_status(): # Could be called from ThreadIoT
return self.status
Of course, IoT object needs to know the current status of the system
when a message is received from MQTT. So ThreadIoT could call
app.get_status().
I think this architecture has some problems with race conditions or
threads synchronization. What happens if IoT calls get_status() exactly
when set_status() called by ThreadBus is executing? If status is a big
data structure, set_status() could be interrupted by get_status() that
could get a completely corrupted status, because it was only partly
updated by set_status().
I know I can use locks or semaphores in get_status() and set_status(),
but I don't know if this is a good approach. Consider that the system is
complex, it isn't composed by a simple single status. It has many many
parameters that are collected from the serial bus. Should I use a lock
for every [get|set]_status(), [get|set]_dimensions(),
[get|set]_battery_level(), [get|set]_mains_present(), and so on?
Another possibility is to use a Queue for Bus and a Queue for IoT. So
the set_status(new_status) called from Bus object will be transformed in
a put in the queue:
app_queue.put({"type": "new_status", "data": ...})
However how could be transformed the get_status() from IoT? How the
return value (the current status) is real retrieved?
class IoT():
..
def status_request_from_MQTT():
app_queue.put({"type": "get_status"})
# How to get the status to return?
return current_status
Should the app put the status on the same queue and should IoT waits for
a new message in the Queue?
def status_request_from_MQTT():
app_queue.put({"type": "get_status"})
try:
current_status = app_queue.get(timeout=10)
except Empty:
# What to do?
return current_status
Again another approach is to avoid multi-threading at all and create a
single "main loop" function that waits at the same time for incoming
events on the serial bus and MQTT (how?). I don't know if this could be
done in my case, because I'm using awscrt Python module and it works
through callbacks that I think is called from another thread.
Any suggestions on this architecture?
--
https://mail.python.org/mailman/listinfo/python-list