Implementation

Asynchronous Session Module (pymanip.asyncsession)

This module defines two classes, pymanip.asyncsession.AsyncSession and pymanip.asyncsession.RemoteObserver. The former is used to manage an experimental session, the latter to access its live data from a remote computer.

class pymanip.asyncsession.AsyncSession(session_name=None, verbose=True, delay_save=False)

This class represents an asynchronous experiment session. It is the main tool that we use to set up monitoring of experimental systems. It will manage the storage for the data, as well as several asynchronous functions for use during the monitoring of the experimental system such as live plot of monitored data, regular control email, and remote HTTP access to the live data by human (connection from a web browser), or by a machine using the pymanip.asyncsession.RemoteObserver class. It has methods to access the data for processing during the experiment, or post-processing after the experiment is finished.

Parameters

• session_name (str, optional) – the name of the session, defaults to None. It will be used as the filename of the sqlite3 database file stored on disk. If None, then no file is created, and data will be temporarily stored in memory, but will be lost when the object is released.

• verbose (bool, optional) – sets the session verbosity, defaults to True

• delay_save (bool, optional) – if True, the data is stored in memory during the duration of the session, and is saved to disk only at the end of the session. It is not recommanded, but useful in cases where fast operation requires to avoid disk access during the session.

add_dataset(*args, **kwargs)

This method adds arrays, or other pickable objects, as “datasets” into the database. They will hold a timestamp corresponding to the time at which the method has been called.

Parameters

• *args (dict, optional) – dictionnaries with name-value to be added in the database

• **kwargs (object, optional) – name-value to be added in the database

add_entry(*args, **kwargs)

This methods adds scalar values into the database. Each entry value will hold a timestamp corresponding to the time at which this method has been called. Variables are passed in dictionnaries or as keyword-arguments. If several variables are passed, then they all hold the same timestamps.

For parameters which consists in only one scalar value, and for which timestamps are not necessary, use pymanip.asyncsession.AsyncSession.save_parameter() instead.

Parameters

• *args (dict, optional) – dictionnaries with name-value to be added in the database

• **kwargs (float, optional) – name-value to be added in the database

ask_exit(*args, **kwargs)

This methods informs all tasks that the monitoring session should stop. Call this method if you wish to cleanly stop the monitoring session. It is also automatically called if the interrupt signal is received. It essentially sets the running attribute to False. Long user-defined tasks should check the running attribute, and abort if set to False. All other AsyncSession tasks check the attribute and will stop. The sleep() method also aborts sleeping.

dataset(name, ts=None, n=None)

This method returns the dataset recorded at the specified timestamp, and under the specified name.

Parameters

• name (str) – name of the dataset to retrieve

• ts (float, optional) – timestamp at which the dataset was stored, defaults to the timestamp of the last recorded dataset under that name

• n (int, optional) – select the nth dataset (in dataset timestamp chronological order)

Returns

the value of the recorded dataset

Return type

object

dataset_last_data(name)

This method returns the last recorded dataset under the specified name.

Parameters

name (str) – name of the dataset to retrieve

Returns

dataset value

Return type

object

dataset_names()

This method returns the names of the datasets currently stored in the session database.

Returns

names of datasets

Return type

set

dataset_times(name)

This method returns the timestamp of the recorded dataset under the specified name.

Parameters

name (str) – name of the dataset to retrieve

Returns

array of timestamps

Return type

numpy.ndarray

datasets(name)

This method returns a generator which will yield all timestamps and datasets recorded under the specified name. The rationale for returning a generator instead of a list, is that each individual dataset may be large.

Parameters

name (str) – name of the dataset to retrieve

Exemple

• To plot all the recorded datasets named ‘toto’

>>> for timestamp, data in sesn.datasets('toto'):
>>>    plt.plot(data, label=f'ts = {timestamp-sesn.t0:.1f}')

• To retrieve a list of all the recorded datasets named ‘toto’

>>> datas = [d for ts, d in sesn.datasets('toto')]

async figure_gui_update()

This method returns an asynchronous task which updates the figures created by the pymanip.asyncsession.AsyncSession.plot() tasks. This task is added automatically, and should not be used manually.

get_version()

Returns current version of the database layout.

has_parameter(name)

This method returns True if specified parameter exists in the session database.

Parameters

name (str) – name of the parameter to retrieve

Returns

True if parameter exists, False if it does not

Return type

bool

property initial_timestamp

Session creation timestamp, identical to pymanip.asyncsession.AsyncSession.t0

property last_timestamp

Timestamp of the last recorded value

logged_data()

This method returns a name-value dictionnary containing all scalar variables currently stored in the session database.

Returns

all scalar variable values

Return type

dict

logged_data_fromtimestamp(name, timestamp)

This method returns the timestamps and values of a given scalar variable, recorded after the specified timestamp.

Parameters

• name (str) – name of the scalar variable to be retrieved.

• timestamp (float) – timestamp from which to recover values

Returns

the timestamps, and values of the specified variable

Return type

tuple of two numpy arrays

logged_first_values()

This method returns a dictionnary holding the first logged value of all scalar variables stored in the session database.

Returns

first values

Return type

dict

logged_last_values()

This method returns a dictionnary holding the last logged value of all scalar variables stored in the session database.

Returns

last logged values

Return type

dict

logged_variable(varname)

This method retrieve the timestamps and values of a specified scalar variable. It is possible to use the sesn[varname] syntax as a shortcut.

Parameters

varname (str) – name of the scalar variable to retrieve

Returns

timestamps and values

Return type

tuple (timestamps, values) of numpy arrays.

Exemple

>>> ts, val = sesn.logged_variable('T_Pt_bas')

logged_variables()

This method returns a set of the names of the scalar variables currently stored in the session database.

Returns

names of scalar variables

Return type

set

async monitor(*tasks, server_port=6913, custom_routes=None, custom_figures=None)

This method runs the specified tasks, and opens a web-server for remote access and set up the tasks to run matplotlib event loops if necessary. This is the main method that the main function of user program should await for. It is also responsible for setting up the signal handling and binding it to the ask_exit method.

It defines a running attribute, which is finally set to False when the monitoring must stop. User can use the ask_exit() method to stop the monitoring. Time consuming user-defined task should check the running and abort if set to False.

Parameters

• *tasks (co-routine function or awaitable) – asynchronous tasks to run: if the task is a co-routine function, it will be called repeatedly until ask_exit is called. If task is an awaitable it is called only once. Such an awaitable is responsible to check that ask_exit has not been called. Several such awaitables are provided: pymanip.asyncsession.AsyncSession.send_email(), pymanip.asyncsession.AsyncSession.plot() and pymanip.asyncsession.AsyncSession.sweep().

• server_port (int, optional) – the network port to open for remote HTTP connection, defaults to 6913. If None, no server is created.

• custom_routes (co-routine function, optional) – additionnal aiohttp routes for the web-server, defaults to None

• custom_figures (matplotlib.pyplot.Figure, optional) – additional matplotlib figure object that needs to run the matplotlib event loop

async mytask(corofunc)

This method repeatedly awaits the given co-routine function, as long as pymanip.asyncsession.AsyncSession.ask_exit() has not been called. Should not be called manually.

parameter(name)

This method retrieve the value of the specified parameter.

Parameters

name (str) – name of the parameter to retrieve

Returns

value of the parameter

Return type

float

parameters()

This method returns all parameter name and values.

Returns

parameters

Return type

dict

async plot(varnames=None, maxvalues=1000, yscale=None, *, x=None, y=None, fixed_ylim=None, fixed_xlim=None)

This method returns an asynchronous task which creates and regularly updates a plot for the specified scalar variables. Such a task should be passed to pymanip.asyncsession.AsyncSession.monitor() or pymanip.asyncsession.AsyncSession.run(), and does not have to be awaited manually.

If varnames is specified, the variables are plotted against time. If x and y are specified, then one is plotted against the other.

Parameters

• varnames (list or str, optional) – names of the scalar variables to plot

• maxvalues (int, optional) – number of values to plot, defaults to 1000

• yscale (tuple or list, optional) – fixed yscale for temporal plot, defaults to automatic ylim

• x (str, optional) – name of the scalar variable to use on the x axis

• y (str, optional) – name of the scalar variable to use on the y axis

• fixed_xlim (tuple or list, optional) – fixed xscale for x-y plots, defaults to automatic xlim

• fixed_ylim (tuple or list, optional) – fixed yscale for x-y plots, defaults to automatic ylim

print_description()

Prints the list of parameters, logged variables and datasets.

print_welcome()

Prints informative start date/end date message. If verbose is True, this method is called by the constructor.

run(*tasks, server_port=6913, custom_routes=None, custom_figures=None)

Synchronous call to pymanip.asyncsession.AsyncSession.monitor().

save_database()

This method is useful only if delay_save = True. Then, the database is kept in-memory for the duration of the session. This method saves the database on the disk. A new database file will be created with the content of the current in-memory database.

This method is automatically called at the exit of the context manager.

save_parameter(*args, **kwargs)

This methods saves a scalar parameter into the database. Unlike scalar values saved by the pymanip.asyncsession.AsyncSession.add_entry() method, such parameter can only hold one value, and does not have an associated timestamp. Parameters can be passed as dictionnaries, or keyword arguments.

Parameters

• *args (dict, optional) – dictionnaries with name-value to be added in the database

• **kwargs (float, optional) – name-value to be added in the database

save_remote_data(data)

This method saves the data returned by a pymanip.asyncsession.RemoteObserver object into the current session database, as datasets and parameters.

Parameters

data (dict) – data returned by the pymanip.asyncsession.RemoteObserver object

async send_email(from_addr, to_addrs, host, port=25, subject=None, delay_hours=6, initial_delay_hours=None)

This method returns an asynchronous task which sends an email at regular intervals. Such a task should be passed to pymanip.asyncsession.AsyncSession.monitor() or pymanip.asyncsession.AsyncSession.run(), and does not have to be awaited manually.

Parameters

• from_addr (str) – email address of the sender

• to_addrs (list of str) – email addresses of the recipients

• host (str) – hostname of the SMTP server

• port (int, optional) – port number of the SMTP server, defaults to 25

• delay_hours (float, optional) – interval between emails, defaults to 6 hours

• initial_delay_hours (float, optional) – time interval before the first email is sent, default to None (immediately)

async server_current_ts(request)

This asynchronous method returns the HTTP response to a request for JSON with the current server time. Should not be called manually.

async server_data_from_ts(request)

This asynchronous method returns the HTTP response to a request for JSON with all data after the specified timestamp. Should not be called manually.

async server_get_parameters(request)

This asynchronous method returns the HTTP response to a request for JSON data of the session parameters. Should not be called manually.

async server_logged_last_values(request)

This asynchronous method returns the HTTP response to a request for JSON data of the last logged values. Should not be called manually.

async server_main_page(request)

This asynchronous method returns the HTTP response to a request for the main html web page. Should not be called manually.

async server_plot_page(request)

This asynchronous method returns the HTTP response to a request for the HTML plot page. Should not be called manually.

async sleep(duration, verbose=True)

This method returns an asynchronous task which waits the specified amount of time and prints a countdown. This should be called with verbose=True by only one of the tasks. The other tasks should call with verbose=False. This method should be preferred over asyncio.sleep because it checks that pymanip.asyncsession.AsyncSession.ask_exit() has not been called, and stops waiting if it has. This is useful to allow rapid abortion of the monitoring session.

Parameters

• duration (float) – time to wait

• verbose (bool, optional) – whether to print the countdown, defaults to True

async sweep(task, iterable)

This methods returns an asynchronous task which repeatedly awaits a given co-routine by iterating the specified iterable. The returned asynchronous task should be passed to pymanip.asyncsession.AsyncSession.monitor() or pymanip.asyncsession.AsyncSession.run(), and does not have to be awaited manually.

This should be used when the main task of the asynchronous session is to sweep some value. The asynchronous session will exit when all values have been iterated. This is similar to running a script which consists in a synchronous for-loop, except that other tasks, such as remote access, live-plot and emails can be run concurrently.

Parameters

• task (function) – the co-routine function to repeatedly call and await

• iterable (list) – values to pass when calling the function

Example

>>> async def balayage(sesn, voltage):
>>>     await sesn.generator.vdc.aset(voltage)
>>>     await asyncio.sleep(5)
>>>     response = await sesn.multimeter.aget(channel)
>>>     sesn.add_entry(voltage=voltage, response=response)
>>>
>>> async def main(sesn):
>>>     await sesn.monitor(sesn.sweep(balayage, [0.0, 1.0, 2.0]))
>>>
>>> asyncio.run(main(sesn))

property t0

Session creation timestamp

class pymanip.asyncsession.RemoteObserver(host, port=6913)

This class represents remote observers of a monitoring session. It connects to the server opened on a remote computer by pymanip.asyncsession.AsyncSession.monitor(). The aim of an instance of RemoteObserver is to retrieve the data saved into the remote computer session database.

Parameters

• host (str) – hostname of the remote compute to connect to

• port (int, optional) – port number to connect to, defaults to 6913

_get_request(apiname)

Private method to send a GET request for the specified API name

_post_request(apiname, params)

Private method to send a POST request for the specified API name and params

get_last_values()

This method retrieve the last set of values from the remote monitoring session.

Returns

scalar variable last recorded values

Return type

dict

start_recording()

This method establishes the connection to the remote computer, and sets the start time for the current observation session.

stop_recording(reduce_time=True, force_reduce_time=True)

This method retrieves all scalar variable data recorded saved by the remote computer since pymanip.asyncsession.RemoteObserver.start_recording() established the connection.

Parameters

• reduce_time (bool, optional) – if True, try to collapse all timestamp arrays into a unique timestamp array. This is useful if the remote computer program only has one call to add_entry. Defaults to True.

• force_reduce_time (bool, optional) – bypass checks that all scalar values indeed have the same timestamps.

Returns

timestamps and values of all data saved in the remote computed database since the call to pymanip.asyncsession.RemoteObserver.start_recording()

Return type

dict