Implement decision trees
When designing IoT applications, Artificial Intelligence approaches are often employed to extract meaningful data from raw signals. For this purpose, the Huxon language natively supports decision trees as special processing nodes.
A decision tree is a popular tool in machine learning used to classify data according to one or more features extracted from raw data. It is a supervised learning technique, meaning that a decision tree must be trained from a labeled feature dataset. One interesting aspects of decision trees is that they belong to explainable artificial intelligence approaches. indeed, once trained, the resulting tree can be easily inspected to see which decisions lead to a given classification.
- The steps to implement a decision tree are as follows:
collect raw data and label each dataset
compute one or more features from the raw data
train the decision tree from the features data
implement and test the decision tree
To show how to perform this process within Huxon, we consider all such steps for a simple use case. We want to classify whether an ISM330DHCX sensor attached to a surface is oriented up, down or it is shaking.
The example is adapted from the Machine Learning Core (MLC) examples presented by STMicroelectronics. The MLC is a hardware processing engine available on several STMicroelectronics products, such as the ISM330DHCX sensor. The MLC allows to extract features from raw data and run decision trees on such features directly inside the sensor silicon. This allows to offload processing from the application processor to the sensor, hence reducing overall performance requirements and power consumption.
The Huxon platform Smart Workload Distribution logic is capable of detecting decision trees within an Huxon application and, if possible, to deploy them on MLC-enabled sensors automatically. When a decision tree cannot be offloaded to a sensor, the Smart Workload Distribution implements it on a general-purpose processor. The code generation for decision trees is performed leveraging on the Entree design flow.
Data collection and labeling
To collect raw data, the simplest solution is to create a Huxon application that reads data from sensors and sends them as output. When doing so, it is important to specify a fixed configuration for the sensor to be used both for data collection and in the final application implementation. This ensures that the decision tree is trained for the same sensor configuration that is used in the final deployment.
The following is an example code to collect data from a ISM330DHCX sensor:
#include <huxon/lang.hpp>
#include <huxon/sensors/STMicroelectronics.hpp>
namespace STM = hux::sensors::STMicroelectronics;
HUX_DECLARE_SENSOR_CONFIGURATION(config, STM::ISM330DHCX,
.acc_odr = STM::ISM330DHCX::configs::acc_odr_12_5,
.acc_fs = STM::ISM330DHCX::configs::acc_fs_2,
.gyr_odr = STM::ISM330DHCX::configs::gyr_odr_12_5,
.gyr_fs = STM::ISM330DHCX::configs::gyr_fs_2000,
.ws = 25 /* size of non-overlapping windows over which advanced sources are computed */
);
HUX_DECLARE_SENSOR(sensor, STM::ISM330DHCX, {}, config);
HUX_DECLARE_CHANNEL(sensor_data_ch, zip_latest,
sensor.get_accX(),
sensor.get_accY(),
sensor.get_accZ(),
sensor.get_gyrX(),
sensor.get_gyrY(),
sensor.get_gyrZ(),
sensor.get_timestamp()
);
HUX_DECLARE_PROCESSING(output, sensor_data_ch, {
HUX_DECLARE_OUTPUT_VALUE(acc_x, Float, "acc_x", hux::get<0>(hux_input));
HUX_DECLARE_OUTPUT_VALUE(acc_y, Float, "acc_y", hux::get<1>(hux_input));
HUX_DECLARE_OUTPUT_VALUE(acc_z, Float, "acc_y", hux::get<2>(hux_input));
HUX_DECLARE_OUTPUT_VALUE(gyr_x, Float, "gyr_x", hux::get<3>(hux_input));
HUX_DECLARE_OUTPUT_VALUE(gyr_y, Float, "gyr_y", hux::get<4>(hux_input));
HUX_DECLARE_OUTPUT_VALUE(gyr_z, Float, "gyr_z", hux::get<5>(hux_input));
HUX_DECLARE_OUTPUT_VALUE(ts, Long, "timestamp", hux::get<6>(hux_input));
HUX_DECLARE_OUTPUT_VALUE(out, Object, "output",
acc_x, acc_y, acc_z, gyr_x, gyr_y, gyr_z, ts
);
return out;
});
HUX_REGISTER_OUTPUT(output);
The application can then by deployed on the physical devices through the Huxon platform. At this stage it is possible to physically act on the sensor and record at least one dataset for every scenario to be labeled: sensor up, sensor down and sensor shaking.
Note
An alternative to raw data collection is to use synthetic data generated, for instance, using a physical model of the system.
Features extraction
Once the datasets are recorded, each with its label, they can be used as simulation datasets for the Huxon application. Here we assume to have recorded exactly 3 simulation datasets for the ISM330DHCX sensor, one corresponding to each scenario:
up.csv
down.csv
shaking.csv
At this stage, using the Huxon language it is possible to compute one or more features (such as average acceleration over Z, acceleration over Z variance, etc.) from the recorded data. Such features can be computed either with custom processing nodes, or using the advanced sensor sources available for MLC-capable sensors.
In the following example, we consider as features the average acceleration over Z and variance of the acceleration norm over non-overlapping windows of 25 samples.
#include <huxon/lang.hpp>
#include <huxon/sensors/STMicroelectronics.hpp>
namespace STM = hux::sensors::STMicroelectronics;
/* Uncomment one line per execution to get the features data for each dataset */
#define DATASET_NAME "up.csv"
// #define DATASET_NAME "down.csv"
// #define DATASET_NAME "shaking.csv"
HUX_DECLARE_SIMULATION_DATA(dataset,
hux::simulation::load_csv<float, float, float, float, float, float, hux::uint64_t>(
DATASET_NAME, ";")
);
HUX_DECLARE_SENSOR_CONFIGURATION(config, STM::ISM330DHCX,
.acc_odr = STM::ISM330DHCX::configs::acc_odr_12_5,
.acc_fs = STM::ISM330DHCX::configs::acc_fs_2,
.gyr_odr = STM::ISM330DHCX::configs::gyr_odr_12_5,
.gyr_fs = STM::ISM330DHCX::configs::gyr_fs_2000,
.ws = 25 /* size of non-overlapping windows over which advanced sources are computed */
);
HUX_DECLARE_SENSOR(sensor, STM::ISM330DHCX, dataset, config);
HUX_DECLARE_CHANNEL(features_data_ch, zip_latest,
sensor.get_mean_accZ(),
sensor.get_var_accV()
);
HUX_DECLARE_PROCESSING(output, features_data_ch, {
HUX_DECLARE_OUTPUT_VALUE(acc_z_mean, Float, "acc_z_mean", hux::get<0>(hux_input));
HUX_DECLARE_OUTPUT_VALUE(acc_var, Float, "acc_variance", hux::get<1>(hux_input));
HUX_DECLARE_OUTPUT_VALUE(out, Object, "features", acc_z_mean, acc_var);
return out;
});
HUX_REGISTER_OUTPUT(output);
Note
Inside the code, we have used directly the advanced sensor sources offered by the MLC-enabled ISM330DHCX sensor.
By running a simulation using the up.csv dataset we get the following result:
{
"features":{
"acc_z_mean":1.01465,
"acc_variance":0
}
}
{
"features":{
"acc_z_mean":1.0166,
"acc_variance":0
}
}
{
"features":{
"acc_z_mean":1.0166,
"acc_variance":0.00195313
}
}
The process should be repeated also for the down.csv and shaking.csv datasets. The feature values together with their labels should then be formatted as a single ARFF file. An example of an ARFF file obtained from the values of the features for each dataset is the following:
@relation 'Orientation'
@attribute acc_z_mean numeric
@attribute acc_z_variance numeric
@attribute class {up, down, shaking}
@data
1.01465, 0, up
1.0166, 0, up
1.0166, 0.00195313, up
-1.03027, 0, down
-0.98291, 0, down
-0.972656, 0, down
0.961426, 0.416992, shaking
1.0127, 0.0683594, shaking
1.02246, 0.143555, shaking
Decision tree training
Once we have an ARFF file containing data for all the features data and the associated classification labels, we can train a decision tree from such data.
For this purpose, the Huxon language provides the arff2hpp command from the huxc tool that can be used to both train a decision tree from an ARFF file and generate a corresponding C++ header file ready to be included within a Huxon application.
The following command trains a decision tree from the orientation.arff file and generates a corresponding decision tree named orientation inside the orientation.hpp file:
huxc arff2hpp -o orientation.hpp -n orientation orientation.arff
Note
For a description of arff2hpp options run: huxc arff2hpp -h
Internally the command uses sklearn.tree.DecisionTreeClassifier from Scikit Learn to train the decision tree.
If you need to tune the training process, the arff2hpp command allows passing parameters directly to sklearn.tree.DecisionTreeClassifier. As an example, to specify a maximum depth of 2 levels for the decision tree, we can run:
huxc arff2hpp -o orientation.hpp -n orientation orientation.arff -clf-cfg int:max_depth:2
Note
For all the available training options refer to sklearn.tree.DecisionTreeClassifier documentation.
The generated decision tree file orientation.hpp will look similar to the following:
#ifndef ORIENTATION_HPP__
#define ORIENTATION_HPP__
#include <string_view>
namespace {
namespace orientation_description {
static constexpr const std::string_view node_label = "orientation";
static const int n_total_nodes = 5;
static const int n_trees = 1;
static const int n_features = 2;
enum Feature {
acc_z_mean,
acc_z_variance
};
#ifdef ENTREE_REFLECTION_ENABLE
#ifndef ENTREE_REFLECTION_LEVEL
#define ENTREE_REFLECTION_LEVEL 1
#endif
#if (ENTREE_REFLECTION_LEVEL > 0)
static const std::string_view feature_labels[] = {
"acc_z_mean",
"acc_z_variance"
};
#endif
#endif
static const int n_classes = 3;
enum Class {
down,
shaking,
up
};
#ifdef ENTREE_REFLECTION_ENABLE
#ifndef ENTREE_REFLECTION_LEVEL
#define ENTREE_REFLECTION_LEVEL 1
#endif
#if (ENTREE_REFLECTION_LEVEL > 0)
static const std::string_view class_labels[] = {
"down",
"shaking",
"up"
};
#endif
#endif
typedef double input_t;
typedef input_t input_arr_t[n_features];
typedef int score_t;
typedef score_t score_arr_t[n_classes];
typedef input_t threshold_t;
#ifdef ENTREE_REFLECTION_ENABLE
#ifndef ENTREE_REFLECTION_LEVEL
#define ENTREE_REFLECTION_LEVEL 1
#endif
#if (ENTREE_REFLECTION_LEVEL > 1)
static const std::string_view rules = "\n\
FEATURES: ['acc_z_mean','acc_z_variance']\n\
CLASSES: ['down','shaking','up']\n\
TREE #0-0:\n\
\n\
acc_z_variance <= 0.035\n\
| acc_z_mean <= 0.021: down\n\
| acc_z_mean > 0.021: up\n\
acc_z_variance > 0.035: shaking\n\
\n\
Number of Leaves : 3\n\
\n\
Size of the Tree : 5\n\
";
#endif
#endif
static const entree::DTEnsemble<node_label, n_total_nodes, n_trees, n_classes,
input_arr_t, score_t, threshold_t, true> bdt =
{ // The struct
1, // The normalisation
{0,0,0}, // The init_predict
{{5}},
{1, 0, -2, -2, -2},
{0.035156263620592654, 0.02099698781967163, -2.0, -2.0, -2.0},
{0, 0, 0, 2, 1},
{1, 2, -1, -1, -1},
{4, 3, -1, -1, -1},
{-1, 0, 1, 1, 0}
};
}
}
#endif
Note that the generated file embeds a visual representation of the decision tree that has been learned:
acc_z_variance <= 0.035
| acc_z_mean <= 0.021: down
| acc_z_mean > 0.021: up
acc_z_variance > 0.035: shaking
Use and test a decision tree
To use the generated decision tree inside an Huxon application we need to take note of a few things from the C++ header file. The most important is the namespace of the decision tree which, in this case, is: orientation_description. This namespace will be needed when declaring a decision tree of this type within the Huxon application.
Note
The namespace of the decision tree is always NAME_description where NAME is the name specified with option -n when running the arff2hpp command.
The second aspect is the enum orientation_description::Features. This enum lists all the features that this decision tree expects as input. The name of the features listed in the file corresponds to the names used within the ARFF file.
The third and last aspect is the enum orientation_description::Class. The result of the decision tree will be an integer of type hux::int8_t
whose value corresponds to one of the values of the orientation_description::Class enum.
In this example:
0 corresponds to orientation_description::Class::down
1 corresponds to orientation_description::Class::shaking
2 corresponds to orientation_description::Class::up
To use a decision tree within a Huxon application we need to include the corresponding C++ header file:
#include "orientation.hpp"
Then we can create a custom processing node that runs the decision tree logic with the HUX_DECLARE_DT_ENSEMBLE construct:
/*
* orientation : the label of the processing node
* orientation_description : the label of the namespace of the decision tree
* features_data_ch : the input channel for the decision tree
*/
HUX_DECLARE_DT_ENSEMBLE(orientation, orientation_description, features_data_ch);
The following code provides a complete example using our orientation decision tree. The code can also be used to test the quality of the decision tree by simulating it using one of the 3 datasets: up.csv, down.csv and shaking.csv.
#include <huxon/lang.hpp>
#include <huxon/sensors/STMicroelectronics.hpp>
/* Include the orientation decision tree */
#include "orientation.hpp"
/* Uncomment one line per execution to test each dataset against the decision tree */
#define DATASET_NAME "up.csv"
// #define DATASET_NAME "down.csv"
// #define DATASET_NAME "shaking.csv"
namespace STM = hux::sensors::STMicroelectronics;
HUX_DECLARE_SIMULATION_DATA(dataset,
hux::simulation::load_csv<float, float, float, float, float, float, hux::uint64_t>(
DATASET_NAME, ";")
);
HUX_DECLARE_SENSOR_CONFIGURATION(config, STM::ISM330DHCX,
.acc_odr = STM::ISM330DHCX::configs::acc_odr_12_5,
.acc_fs = STM::ISM330DHCX::configs::acc_fs_2,
.gyr_odr = STM::ISM330DHCX::configs::gyr_odr_12_5,
.gyr_fs = STM::ISM330DHCX::configs::gyr_fs_2000,
.ws = 25
);
HUX_DECLARE_SENSOR(sensor, STM::ISM330DHCX, dataset, config);
HUX_DECLARE_CHANNEL(features_data_ch, zip_latest,
sensor.get_mean_accZ(),
sensor.get_var_accV()
);
/*
* Implement a decision tree of type "orientation_description" to classify
* the data coming from features_data_ch channel.
*
* NOTE1: the decision tree expects as input a channel producing a hux::tuple
* with size equal to the number of features of the decision tree. Each element
* of the hux::tuple corresponds to a feature value following the order of the enum:
* orientation_description::Features (defined in orientation.hpp)
*
* NOTE2: the result of the decision tree is a hux::int8_t. each number corresponds
* to a value of the enum:
* orientation_description::Class (defined in orientation.hpp)
*/
HUX_DECLARE_DT_ENSEMBLE(orientation, orientation_description, features_data_ch);
/* Combine the features data with the classification result in a single channel */
HUX_DECLARE_CHANNEL(orientation_ch, zip_latest, features_data_ch, orientation);
/* Produce as output both the features data and the corresponding classification label */
HUX_DECLARE_PROCESSING(output, orientation_ch, {
auto &features_data = hux::get<0>(hux_input);
hux::int8_t classification = hux::get<1>(hux_input);
/* NOTE: in more complex codes, to perform custom logic based on the
* classification result one could use the following syntax:
* switch(classification) {
* case orientation_description::Class::down:
* //...
* break;
* case orientation_description::Class::shaking:
* //...
* break;
* case orientation_description::Class::up:
* //...
* break;
* }
*/
HUX_DECLARE_OUTPUT_VALUE(acc_z_mean, Float, "acc_z_mean", hux::get<0>(features_data));
HUX_DECLARE_OUTPUT_VALUE(acc_var, Float, "acc_variance", hux::get<1>(features_data));
HUX_DECLARE_OUTPUT_VALUE(label, Float, "label", classification);
HUX_DECLARE_OUTPUT_VALUE(out, Object, "test", acc_z_mean, acc_var, label);
return out;
});
HUX_REGISTER_OUTPUT(output);
When testing this code in simulation with the up.csv dataset, we get the following result:
{
"test":{
"acc_z_mean":1.01465,
"acc_variance":0,
"label":2
}
}
{
"test":{
"acc_z_mean":1.0166,
"acc_variance":0,
"label":2
}
}
{
"test":{
"acc_z_mean":1.0166,
"acc_variance":0.00195313,
"label":2
}
}
All samples have been labeled as 2, which corresponds to the enum value: orientation_description::Class::up. Hence the decision tree correctly classified all the samples from the up.csv dataset. Similarly, we can run a test for the datasets *down.csv and shaking.csv.