Serial Port Monitoring

The monitoring module provides a class structure for interacting with a single serial port. Multiple class instances can be made for interacting with multiple ports at the same time and aggregating their data on a single plot. Synchronization is handled through Queue instances that allow multi-thread access to putting and getting data.

class liveserial.monitor.ComMonitorThread(data_q, error_q, port, port_baud, port_stopbits=1, port_parity='N', port_timeout=0.01, listener=False, virtual=False, infer_limit=15, encoding='UTF-8', delimiter='\s')

A thread for monitoring a COM port. The COM port is

opened when the thread is started.

Parameters:

• data_q (multiprocessing.Queue) – Queue for received data. Items in the queue are (data, timestamp) pairs, where data is a binary string representing the received data, and timestamp is the time elapsed from the thread’s start (in seconds).
• error_q (multiprocessing.Queue) – Queue for error messages. In particular, if the serial port fails to open for some reason, an error is placed into this queue.
• port (str) – The COM port to open. Must be recognized by the system.
• port_baud (int) – Rate at which information is transferred in a communication channel (in bits/second).
• stopbits (float) – Serial communication parameter; one of (1, 1.5, 2).
• parity – (str): Serial communication parameter; one of [‘N’, ‘E’, ‘O’, ‘M’, ‘S’].
• port_timeout (float) – The timeout used for reading the COM port. If this value is low, the thread will return data in finer grained chunks, with more accurate timestamps, but it will also consume more CPU.
• listener (bool) – specifies that this COMThread is a listener, which prints out the raw stream in real-time, but doesn’t analyze it.
• infer_limit (int) – number of raw lines to consider before reaching consensus on the format of the data (when inferring). If None, then a format inferrer is not created.
• virtual (bool) – when True, additional serial port parameters are set so that the monitor can work with socat or other virtual ports.
• encoding (str) – encoding type used to decode the byte stream from the serial port.
• delimiter (str) – regex describing the sequence of characters used to separate columns of values in a single line from the serial port.

alive

threading.Event – event for asynchronously handling the reads from the serial port.

serial_arg

dict – arguments used to contstruct the serial.Serial.

serial_port

serial.Serial – serial instance for communication.

sensors

dict – keys are sensor names; values are liveserial.config.Sensor instances used to parse raw lines read from the serial port.

inferrer

FormatInferrer – for inferring the format in the absence of configured sensor structure.

add_sensor(name, sensor)

Adds a sensor to the COM monitors active list. Sensors in the active list have their data parsed and pushed to the data queue. If no sensors are added to the list, the monitor will try to infer the sensor configuration using the first few raw data samples.

Parameters:sensor (liveserial.config.Sensor) – instance with parsed configuration values.

static from_config(config, port, dataq=None, errorq=None, listener=False, sfilter=None)

Returns a COMMonitor instance from the specified configuration parser.

Parameters:

• config (ConfigParser) – instance from which to extract the sensor list and port information. str is also allowed, in which case it should be the path to the config file to load.
• port (str) – name of the port to configure for.
• data_q (multiprocessing.Queue) – Queue for received data. Items in the queue are (data, timestamp) pairs, where data is a binary string representing the received data, and timestamp is the time elapsed from the thread’s start (in seconds).
• error_q (multiprocessing.Queue) – Queue for error messages. In particular, if the serial port fails to open for some reason, an error is placed into this queue.
• listener (bool) – specifies that this COMThread is a listener, which prints out the raw stream in real-time, but doesn’t analyze it.
• sfilter (list) – of sensor names that should be included in the monitor. By default, all sensors in the config are included that match the port.

Returns:

instance created using the configuration parameters.

Return type:

ComMonitorThread

static from_port(port, port_baud=9600, virtual=False)

Returns a COMMonitor instance for the specified port using the default configurati of port parameters and with inferrence for the structure of the data.

Parameters:

• port (str) – The COM port to open. Must be recognized by the system.
• port_baud (int) – Rate at which information is transferred in a communication channel (in bits/second).

join(timeout=None, terminate=True)

Tells the thread monitoring the COM port to clean up and return.

Parameters:

• timeout (float) – number of seconds (or fractions of seconds) to wait until returning. If None, then the operation will block until the thread terminates. See also threading.Thread.join().
• terminate (bool) – when True, the data collection is told to stop before trying to join the underlying thread; otherwise, the thread will keep processing data until join is called with terminate=True.

run()

Starts the COM monitoring thread. If an existing serial connection is open, it will be closed and a new one will be created. The monitoring will continue indefinitely until join() is called.

Inferring Raw Data Format

When sensors are not pre-configured, the package provides options for inferring the format of data by looking for string-valued columns that may represent sensor keys and then trying int and float parsing on the remaining columns. The inferrence is handled on a per-port basis using a class instance that looks at the first 15 raw lines and attempts to guess what kinds of sensors are present.

class liveserial.monitor.FormatInferrer(infer_limit=15)

Class that can infer the data types of an unknown sensor stream, provided they are consistent between calls.

Parameters:infer_limit (int) – number of raw lines to consider before reaching consensus on the format of the data (when inferring).

inferred = None

dict – keys are inferred sensor ids, values are lists of casting types as for the implementation in liveserial.config.Sensor. This dict only gets populated when no configuration is available and data structure has to be inferred from the incoming stream.

parse(raw)

Attempts to parse the specified string values into one of the formats that has been inferred by this instance.

Parameters:raw (list) – of str values read from the line.

Live Feed for Data Aggregation

Serial port data is read off by the ComMonitorThread instances. However, the amount of data generated on the serial stream exceeds what we need to generate a useful plot, and sometimes even exceeds our needs for logging. The Logger class instance monitors the data queues and periodically aggregates that data to form a single value, representative of the interval between checks (see Data Logging). These single point values are stored in LiveDataFeed.

class liveserial.monitor.LiveDataFeed

A simple “live data feed” abstraction that allows a reader to read the most recent data and find out whether it was updated since the last read.

has_new_data

dict – A boolean attribute telling the reader whether the data was updated since the last read; keyed by sensor identifier.

cur_data

dict – most recent data point placed on the feed; keyed by the sensor identifier.

add_data(sensor, data)

Add new data to the feed.

Args: sensor (str): sensor identifier for the data point.

read_data(sensor)

Returns the most recent data.

Useful Utility Functions

The monitoring module also exposes some useful functions for interacting with the data and error queues, and for listing available serial ports on a machine.

Class for monitoring a COM port using pyserial on a separate thread so that that the main UI thread is not blocked or slowed. Code adapted from https://github.com/mba7/SerialPort-RealTime-Data-Plotter/blob/master/com_monitor.py

liveserial.monitor.enumerate_serial_ports()

Scans for available serial ports.

Returns: list: of str with the availables port names.

liveserial.monitor.get_all_from_queue(Q)

Generator to yield one after the others all items currently in the queue Q,

without any waiting.

Parameters:Q (Queue.Queue) – queue to empty items from.

liveserial.monitor.get_item_from_queue(Q, timeout=0.01)

Attempts to retrieve an item from the queue Q. If Q is

empty, None is returned.

Parameters:

• Q (Queue.Queue) – queue to get an item from.
• timeout (float) – Blocks for ‘timeout’ seconds in case the queue is empty, so don’t use this method for speedy retrieval of multiple items (use get_all_from_queue for that).