Learning models without inter-individual variability

Consider the nonlinear transmission line circuit example of Sec. 4.2 of [1]. The learning pipeline consists in the steps listed below. This tutorial allows to follow the pipeline step by step, but the full list of commands are collected in the unique file \examples\tutorial\NTL1_tutorial.m.

Definig the problem

A problem mainly consists in the definition of the input-output variables (how many they are, which are the bounds for each variable). Notice that a problem does not contain any information about the map linking inputs to outputs: such map is rather defined by a model, which is a different concept, treated in the next section. Indeed, for a given problem (in this case, the problem is linking the input current of the circuit to the output voltage) one may have several models.

Problems are defined through .ini files. The full list of tags to define a problem is available in \examples\problem_example.ini.

The problem of the current example is defined in \examples\tutorial\NTL1.ini. A problem may contain a handle to generate the high-fidelity model associated to the problem, as we will see in the next section.

Definig the high-fidelity model

A model can be regarded as a map from a time-dependent input to a time-dependent output, according to the definition of Sec. 2.1 of [1]. Each model is associated to a problem. Notice that a model comprises both the mathematical model and its discretization: it corresponds thus to the concept of numerical model.

Models are represented by MATLAB structs, that are typically generated by ad-hoc functions, for better versatility. In this example, the high-fidelity model is generated by the function \examples\tutorial\NTL1_getmodel.m. The model struct should contain the following fields:

Basic fields

  • model.nX: number of internal states

  • model.x0: initial state

  • model.dt: integration time step

Model dynamics

The model dynamics can be defined in different ways (field model.advance_type), depending on the features of the model (linear, nonlinear, ecc.). In this example, the model is defined through its right-hand side and it is solved by Explicit Euler scheme (model.advance_type = 'nonlinear_explicit'). The right-hand side is defined in the field model.f, which implements Eq. (32) of [1].

Model output

Also the output of the model can be defined in different ways (field model.output_type). In this case, it is of type 'insidestate', which means that the output is given by the first model.nY internal states (this corresponds to the input-inside-the-state approach of [1]).

Note

In alternative to the implementation contained in \examples\tutorial\NTL1_getmodel.m, models can be defined as black-boxes (rather than defining the number of variables, initial state, dynamics, etc.). This is the only possibility when the model is defined by an external software. In this case, the user only needs to write a wrapper for the external call, and bind inputs and outputs. An example is contained in \examples\tutorial\NTL1_getmodel_blackbox.m. Notice that all you need to specify is the flag model.blackbox = 1 and the handler function output = model.forward_function(test, options) (see the examples for more details).

Once the problem and the model have been defined, we can load them with the following commands:

problem = problem_get('tutorial','NTL1.ini');
HFmod = problem.get_model(problem);

To employ the model to perform simulations, first define a test struct:

test_solve.tt = [0 10];
test_solve.uu = @(t) .5*(1+cos(2*pi*t/10));

Then, solve the model as follows:

figure();
output = model_solve(test_solve,HFmod,struct('do_plot',1));

The struct output contains the output test struct of the simulation.

Generating training datasets

With the following lines, we use the high-fidelity model HFmod to generate two training datasets. With the option do_save = 1 the datasets are stored into an automatically generated path inside the data folder defined in options.ini (see Installation), according the example name and problem name.

rng('default') % for reproducibility

opt_gen.do_plot = 1;
opt_gen.do_save = 1;
opt_gen.optRandomU.time_scale = .02;

opt_gen.outFile = 'samples_rnd.mat';
dataset_generate_random(HFmod,100,opt_gen);

opt_gen.constant = 1;
opt_gen.wait_init = 1;
opt_gen.wait_init_time_wait = .2;
opt_gen.wait_init_time_raise = 0;
opt_gen.outFile = 'samples_step.mat';
dataset_generate_random(HFmod,50,opt_gen);

The first dataset contains 100 tests associated to random inputs. The second one contains 50 tests associated to step inputs.

It is possible to extract subsets from datasets and combine them. With the following code, for instance, we create a dataset by combining the first three tests of the step responses generated before with the first 8 random responses.:

dataset_def.problem = problem;
dataset_def.type = 'file';
dataset_def.source = 'samples_step.mat;1:3|samples_rnd.mat;1:8';
train_dataset = dataset_get(dataset_def);

To plot the dataset, type:

dataset_plot(train_dataset,problem)

Training the ANN

Training specifications are defined into an option file. The full list of available options, with relative documentation, can be found in \mor_ANN_blackbox\opt_example.ini.

The option file for the current example is contained into \examples\tutorial\NTL1_opt.ini. We now comment the main fields:

  • Problem\dataset_source_train and Problem\dataset_source_tests contain the specifications for the training and the test datasets, with the same sitax used before

  • Model\N: number of states in the learned model

  • ANN\layF: number of neurons in the hidden layers of the ANN

To train the network, run the following command, which will stop afer 100 training epochs:

model_learn('NTL1_opt.ini')

The trained model is stored in an automatically generated path. A name is automatically assigned to the trained model: it appears at the beginning of training but it is also copied to the clipboard just after it appears in the MATLAB console. Paste it somewhere to be able to load the trained model later. The name is generated according to the main settings of learning plus a time stamp (e.g. ‘test_int_N2_hlayF5_dof32_2019-02-28_11-07-14’)

Using the trained model

To load the learned model, run (after replacing the learned model name with the one you stored before):

ANNmod = read_model_fromfile(problem,'test_int_N2_hlayF5_dof32_2019-02-28_11-07-14');

The struct ANNmod is a model struct, as much as HFmod: it can indeed be employed to perform the same actions as the high-fidelity model:

figure();
output = model_solve(test_solve,ANNmod,struct('do_plot',1));

Moreover, being a spacial type of model (a model whose right-hand side is defined by an ANN), it has additional features. For instance, the ANN can be visualized, by running ANNmod.visualize().

To evaluate the level of approximation of the learned model, first load a test dataset:

dataset_def.problem = problem;
dataset_def.type = 'file';
dataset_def.source = 'samples_step.mat;11:50|samples_rnd.mat;21:100';
test_dataset = dataset_get(dataset_def);

Then, evaluate the error with the high-fidelity model (it must be zero):

model_compute_error(HFmod, test_dataset);

and with the learned model:

model_compute_error(ANNmod, test_dataset);