Implement decision trees ************************** When designing IoT applications, Artificial Intelligence approaches are often employed to extract meaningful data from raw signals. For this purpose, the Huxon language natively supports **decision trees** as special **processing nodes**. A decision tree is a popular tool in machine learning used to classify data according to one or more features extracted from raw data. It is a supervised learning technique, meaning that a decision tree must be trained from a labeled feature dataset. One interesting aspects of decision trees is that they belong to explainable artificial intelligence approaches. indeed, once trained, the resulting tree can be easily inspected to see which decisions lead to a given classification. The steps to implement a decision tree are as follows: 1. collect raw data and label each dataset 2. compute one or more features from the raw data 3. train the decision tree from the features data 4. implement and test the decision tree To show how to perform this process within Huxon, we consider all such steps for a simple use case. We want to classify whether an :ref:`ISM330DHCX` sensor attached to a surface is oriented up, down or it is shaking. The example is adapted from the `Machine Learning Core (MLC) `_ examples presented by STMicroelectronics. The MLC is a hardware processing engine available on several STMicroelectronics products, such as the :ref:`ISM330DHCX` sensor. The MLC allows to extract features from raw data and run decision trees on such features directly inside the sensor silicon. This allows to offload processing from the application processor to the sensor, hence reducing overall performance requirements and power consumption. The `Huxon platform `_ Smart Workload Distribution logic is capable of detecting **decision trees** within an Huxon application and, if possible, to deploy them on MLC-enabled sensors automatically. When a decision tree cannot be offloaded to a sensor, the Smart Workload Distribution implements it on a general-purpose processor. The code generation for decision trees is performed leveraging on the `Entree design flow `_. Data collection and labeling ------------------------------ To collect raw data, the simplest solution is to create a Huxon application that reads data from sensors and sends them as output. When doing so, it is important to specify a fixed configuration for the sensor to be used both for data collection and in the final application implementation. This ensures that the decision tree is trained for the same sensor configuration that is used in the final deployment. The following is an example code to collect data from a :ref:`ISM330DHCX` sensor: .. code:: cpp #include #include namespace STM = hux::sensors::STMicroelectronics; HUX_DECLARE_SENSOR_CONFIGURATION(config, STM::ISM330DHCX, .acc_odr = STM::ISM330DHCX::configs::acc_odr_12_5, .acc_fs = STM::ISM330DHCX::configs::acc_fs_2, .gyr_odr = STM::ISM330DHCX::configs::gyr_odr_12_5, .gyr_fs = STM::ISM330DHCX::configs::gyr_fs_2000, .ws = 25 /* size of non-overlapping windows over which advanced sources are computed */ ); HUX_DECLARE_SENSOR(sensor, STM::ISM330DHCX, {}, config); HUX_DECLARE_CHANNEL(sensor_data_ch, zip_latest, sensor.get_accX(), sensor.get_accY(), sensor.get_accZ(), sensor.get_gyrX(), sensor.get_gyrY(), sensor.get_gyrZ(), sensor.get_timestamp() ); HUX_DECLARE_PROCESSING(output, sensor_data_ch, { HUX_DECLARE_OUTPUT_VALUE(acc_x, Float, "acc_x", hux::get<0>(hux_input)); HUX_DECLARE_OUTPUT_VALUE(acc_y, Float, "acc_y", hux::get<1>(hux_input)); HUX_DECLARE_OUTPUT_VALUE(acc_z, Float, "acc_y", hux::get<2>(hux_input)); HUX_DECLARE_OUTPUT_VALUE(gyr_x, Float, "gyr_x", hux::get<3>(hux_input)); HUX_DECLARE_OUTPUT_VALUE(gyr_y, Float, "gyr_y", hux::get<4>(hux_input)); HUX_DECLARE_OUTPUT_VALUE(gyr_z, Float, "gyr_z", hux::get<5>(hux_input)); HUX_DECLARE_OUTPUT_VALUE(ts, Long, "timestamp", hux::get<6>(hux_input)); HUX_DECLARE_OUTPUT_VALUE(out, Object, "output", acc_x, acc_y, acc_z, gyr_x, gyr_y, gyr_z, ts ); return out; }); HUX_REGISTER_OUTPUT(output); The application can then by deployed on the physical devices through the `Huxon platform `_. At this stage it is possible to physically act on the sensor and record at least one dataset for every scenario to be labeled: sensor up, sensor down and sensor shaking. .. note:: An alternative to raw data collection is to use synthetic data generated, for instance, using a physical model of the system. Features extraction -------------------- Once the datasets are recorded, each with its label, they can be used as simulation datasets for the Huxon application. Here we assume to have recorded exactly 3 simulation datasets for the :ref:`ISM330DHCX` sensor, one corresponding to each scenario: - up.csv - down.csv - shaking.csv At this stage, using the Huxon language it is possible to compute one or more features (such as average acceleration over Z, acceleration over Z variance, etc.) from the recorded data. Such features can be computed either with custom **processing nodes**, or using the advanced sensor sources available for MLC-capable sensors. In the following example, we consider as features the average acceleration over Z and variance of the acceleration norm over non-overlapping windows of 25 samples. .. code:: cpp #include #include namespace STM = hux::sensors::STMicroelectronics; /* Uncomment one line per execution to get the features data for each dataset */ #define DATASET_NAME "up.csv" // #define DATASET_NAME "down.csv" // #define DATASET_NAME "shaking.csv" HUX_DECLARE_SIMULATION_DATA(dataset, hux::simulation::load_csv( DATASET_NAME, ";") ); HUX_DECLARE_SENSOR_CONFIGURATION(config, STM::ISM330DHCX, .acc_odr = STM::ISM330DHCX::configs::acc_odr_12_5, .acc_fs = STM::ISM330DHCX::configs::acc_fs_2, .gyr_odr = STM::ISM330DHCX::configs::gyr_odr_12_5, .gyr_fs = STM::ISM330DHCX::configs::gyr_fs_2000, .ws = 25 /* size of non-overlapping windows over which advanced sources are computed */ ); HUX_DECLARE_SENSOR(sensor, STM::ISM330DHCX, dataset, config); HUX_DECLARE_CHANNEL(features_data_ch, zip_latest, sensor.get_mean_accZ(), sensor.get_var_accV() ); HUX_DECLARE_PROCESSING(output, features_data_ch, { HUX_DECLARE_OUTPUT_VALUE(acc_z_mean, Float, "acc_z_mean", hux::get<0>(hux_input)); HUX_DECLARE_OUTPUT_VALUE(acc_var, Float, "acc_variance", hux::get<1>(hux_input)); HUX_DECLARE_OUTPUT_VALUE(out, Object, "features", acc_z_mean, acc_var); return out; }); HUX_REGISTER_OUTPUT(output); .. note:: Inside the code, we have used directly the advanced sensor sources offered by the MLC-enabled :ref:`ISM330DHCX` sensor. By running a simulation using the *up.csv* dataset we get the following result: .. code:: json { "features":{ "acc_z_mean":1.01465, "acc_variance":0 } } { "features":{ "acc_z_mean":1.0166, "acc_variance":0 } } { "features":{ "acc_z_mean":1.0166, "acc_variance":0.00195313 } } The process should be repeated also for the *down.csv* and *shaking.csv* datasets. The feature values together with their labels should then be formatted as a single `ARFF file `_. An example of an ARFF file obtained from the values of the features for each dataset is the following: .. code:: @relation 'Orientation' @attribute acc_z_mean numeric @attribute acc_z_variance numeric @attribute class {up, down, shaking} @data 1.01465, 0, up 1.0166, 0, up 1.0166, 0.00195313, up -1.03027, 0, down -0.98291, 0, down -0.972656, 0, down 0.961426, 0.416992, shaking 1.0127, 0.0683594, shaking 1.02246, 0.143555, shaking Decision tree training ------------------------ Once we have an ARFF file containing data for all the features data and the associated classification labels, we can train a decision tree from such data. For this purpose, the Huxon language provides the **arff2hpp** command from the **huxc** tool that can be used to both train a decision tree from an ARFF file and generate a corresponding C++ header file ready to be included within a Huxon application. The following command trains a decision tree from the **orientation.arff** file and generates a corresponding decision tree named **orientation** inside the **orientation.hpp** file: .. code-block:: bash huxc arff2hpp -o orientation.hpp -n orientation orientation.arff .. note:: For a description of **arff2hpp** options run: *huxc arff2hpp -h* Internally the command uses `sklearn.tree.DecisionTreeClassifier `_ from Scikit Learn to train the decision tree. If you need to tune the training process, the **arff2hpp** command allows passing parameters directly to `sklearn.tree.DecisionTreeClassifier `_. As an example, to specify a maximum depth of 2 levels for the decision tree, we can run: .. code-block:: bash huxc arff2hpp -o orientation.hpp -n orientation orientation.arff -clf-cfg int:max_depth:2 .. note:: For all the available training options refer to `sklearn.tree.DecisionTreeClassifier documentation `_. The generated decision tree file *orientation.hpp* will look similar to the following: .. code:: cpp #ifndef ORIENTATION_HPP__ #define ORIENTATION_HPP__ #include namespace { namespace orientation_description { static constexpr const std::string_view node_label = "orientation"; static const int n_total_nodes = 5; static const int n_trees = 1; static const int n_features = 2; enum Feature { acc_z_mean, acc_z_variance }; #ifdef ENTREE_REFLECTION_ENABLE #ifndef ENTREE_REFLECTION_LEVEL #define ENTREE_REFLECTION_LEVEL 1 #endif #if (ENTREE_REFLECTION_LEVEL > 0) static const std::string_view feature_labels[] = { "acc_z_mean", "acc_z_variance" }; #endif #endif static const int n_classes = 3; enum Class { down, shaking, up }; #ifdef ENTREE_REFLECTION_ENABLE #ifndef ENTREE_REFLECTION_LEVEL #define ENTREE_REFLECTION_LEVEL 1 #endif #if (ENTREE_REFLECTION_LEVEL > 0) static const std::string_view class_labels[] = { "down", "shaking", "up" }; #endif #endif typedef double input_t; typedef input_t input_arr_t[n_features]; typedef int score_t; typedef score_t score_arr_t[n_classes]; typedef input_t threshold_t; #ifdef ENTREE_REFLECTION_ENABLE #ifndef ENTREE_REFLECTION_LEVEL #define ENTREE_REFLECTION_LEVEL 1 #endif #if (ENTREE_REFLECTION_LEVEL > 1) static const std::string_view rules = "\n\ FEATURES: ['acc_z_mean','acc_z_variance']\n\ CLASSES: ['down','shaking','up']\n\ TREE #0-0:\n\ \n\ acc_z_variance <= 0.035\n\ | acc_z_mean <= 0.021: down\n\ | acc_z_mean > 0.021: up\n\ acc_z_variance > 0.035: shaking\n\ \n\ Number of Leaves : 3\n\ \n\ Size of the Tree : 5\n\ "; #endif #endif static const entree::DTEnsemble bdt = { // The struct 1, // The normalisation {0,0,0}, // The init_predict {{5}}, {1, 0, -2, -2, -2}, {0.035156263620592654, 0.02099698781967163, -2.0, -2.0, -2.0}, {0, 0, 0, 2, 1}, {1, 2, -1, -1, -1}, {4, 3, -1, -1, -1}, {-1, 0, 1, 1, 0} }; } } #endif Note that the generated file embeds a visual representation of the decision tree that has been learned: .. code:: txt acc_z_variance <= 0.035 | acc_z_mean <= 0.021: down | acc_z_mean > 0.021: up acc_z_variance > 0.035: shaking Use and test a decision tree ------------------------------ To use the generated decision tree inside an Huxon application we need to take note of a few things from the C++ header file. The most important is the **namespace** of the decision tree which, in this case, is: **orientation_description**. This namespace will be needed when declaring a decision tree of this type within the Huxon application. .. note:: The namespace of the decision tree is always *NAME_description* where *NAME* is the name specified with option *-n* when running the **arff2hpp** command. The second aspect is the enum **orientation_description::Features**. This enum lists all the features that this decision tree expects as input. The name of the features listed in the file corresponds to the names used within the ARFF file. The third and last aspect is the enum **orientation_description::Class**. The result of the decision tree will be an integer of type :cpp:member:`hux::int8_t` whose value corresponds to one of the values of the **orientation_description::Class** enum. In this example: - 0 corresponds to *orientation_description::Class::down* - 1 corresponds to *orientation_description::Class::shaking* - 2 corresponds to *orientation_description::Class::up* To use a decision tree within a Huxon application we need to include the corresponding C++ header file: .. code-block:: cpp #include "orientation.hpp" Then we can create a custom processing node that runs the decision tree logic with the :c:macro:`HUX_DECLARE_DT_ENSEMBLE` construct: .. code:: cpp /* * orientation : the label of the processing node * orientation_description : the label of the namespace of the decision tree * features_data_ch : the input channel for the decision tree */ HUX_DECLARE_DT_ENSEMBLE(orientation, orientation_description, features_data_ch); The following code provides a complete example using our *orientation* decision tree. The code can also be used to test the quality of the decision tree by simulating it using one of the 3 datasets: *up.csv*, *down.csv* and *shaking.csv*. .. code:: cpp #include #include /* Include the orientation decision tree */ #include "orientation.hpp" /* Uncomment one line per execution to test each dataset against the decision tree */ #define DATASET_NAME "up.csv" // #define DATASET_NAME "down.csv" // #define DATASET_NAME "shaking.csv" namespace STM = hux::sensors::STMicroelectronics; HUX_DECLARE_SIMULATION_DATA(dataset, hux::simulation::load_csv( DATASET_NAME, ";") ); HUX_DECLARE_SENSOR_CONFIGURATION(config, STM::ISM330DHCX, .acc_odr = STM::ISM330DHCX::configs::acc_odr_12_5, .acc_fs = STM::ISM330DHCX::configs::acc_fs_2, .gyr_odr = STM::ISM330DHCX::configs::gyr_odr_12_5, .gyr_fs = STM::ISM330DHCX::configs::gyr_fs_2000, .ws = 25 ); HUX_DECLARE_SENSOR(sensor, STM::ISM330DHCX, dataset, config); HUX_DECLARE_CHANNEL(features_data_ch, zip_latest, sensor.get_mean_accZ(), sensor.get_var_accV() ); /* * Implement a decision tree of type "orientation_description" to classify * the data coming from features_data_ch channel. * * NOTE1: the decision tree expects as input a channel producing a hux::tuple * with size equal to the number of features of the decision tree. Each element * of the hux::tuple corresponds to a feature value following the order of the enum: * orientation_description::Features (defined in orientation.hpp) * * NOTE2: the result of the decision tree is a hux::int8_t. each number corresponds * to a value of the enum: * orientation_description::Class (defined in orientation.hpp) */ HUX_DECLARE_DT_ENSEMBLE(orientation, orientation_description, features_data_ch); /* Combine the features data with the classification result in a single channel */ HUX_DECLARE_CHANNEL(orientation_ch, zip_latest, features_data_ch, orientation); /* Produce as output both the features data and the corresponding classification label */ HUX_DECLARE_PROCESSING(output, orientation_ch, { auto &features_data = hux::get<0>(hux_input); hux::int8_t classification = hux::get<1>(hux_input); /* NOTE: in more complex codes, to perform custom logic based on the * classification result one could use the following syntax: * switch(classification) { * case orientation_description::Class::down: * //... * break; * case orientation_description::Class::shaking: * //... * break; * case orientation_description::Class::up: * //... * break; * } */ HUX_DECLARE_OUTPUT_VALUE(acc_z_mean, Float, "acc_z_mean", hux::get<0>(features_data)); HUX_DECLARE_OUTPUT_VALUE(acc_var, Float, "acc_variance", hux::get<1>(features_data)); HUX_DECLARE_OUTPUT_VALUE(label, Float, "label", classification); HUX_DECLARE_OUTPUT_VALUE(out, Object, "test", acc_z_mean, acc_var, label); return out; }); HUX_REGISTER_OUTPUT(output); When testing this code in simulation with the *up.csv* dataset, we get the following result: .. code:: json { "test":{ "acc_z_mean":1.01465, "acc_variance":0, "label":2 } } { "test":{ "acc_z_mean":1.0166, "acc_variance":0, "label":2 } } { "test":{ "acc_z_mean":1.0166, "acc_variance":0.00195313, "label":2 } } All samples have been labeled as 2, which corresponds to the enum value: **orientation_description::Class::up**. Hence the decision tree correctly classified all the samples from the *up.csv dataset. Similarly, we can run a test for the datasets *down.csv* and *shaking.csv*.