Machine Learning Partners Plugin User's Guide
Objective
This guide will teach you how to use the Machine Learning Partners plugin to capture and transmit your data to our Machine Learning (ML) partner platforms where you can develop machine learning solutions that can be deployed in your embedded application. The ML Partners plugin works within MPLAB® Data Visualizer so that you can capture a live data stream from your target device, select the desired data region, and then upload it to the platform of your choice or log the data to a file for later use.
Collecting and curating data is one of the most critical steps in developing an ML solution. It is generally the most time-consuming activity in the design cycle, and the performance of the resultant model is highly dependent on the quality of the data. This is why it is crucial to have effective tools and processes for gathering data. The goal of the ML Partners plugin is to simplify the data collection process and to enable the rapid development of embedded ML solutions.
Microchip's ML Design Partners
Microchip has partnered with experts in embedded ML in order to empower our developers with the latest tools and technologies in the field. We have multiple partners because they each have their own unique strengths and areas of expertise.
With Edge Impulse and SensiMLTM, developers can create solutions for a wide array of use-cases utilizing various sensor types. Our partners provide the necessary tools within their development platform to validate and optimize model performance before deployment.
- Solution Type: Classification, Regression, and Anomaly Detection
- Dev Platform: Edge Impulse Studio
- Output Format: C++ Inferencing SDK (Open Source C/C++)
Upload your data to the Edge Impulse Studio and start creating the next generation of intelligent devices with embedded machine learning. Edge Impulse is an open-source TensorFlow Lite-based framework for classification, regression, and anomaly detection. To learn more visit Edge Impulse's website.
- Solution Type: Classification, and Anomaly Detection
- Dev Platform: SensiML Analytics Toolkit
- Output Format: Knowledge Pack (Static C Library or Source Code)
Log your data to a file for import into SensiML's Analytics Toolkit to get started developing classification and anomaly detection solutions that can be deployed in your embedded application. The Analytics Toolkit is great for beginners and experts alike as it provides AutoML tools as well as fully customizable ML pipelines. To learn more visit SensiML's website.
Materials
Hardware Tools
SAMD21 Machine Learning Evaluation Kit with BOSCH IMU or with TDK IMU
This guide covers using the SAMD21 ML Kit for data collection as an example, however, any time-series data can be used for building ML solutions with Edge Impulse and SensiML.
Software Tools
- MPLAB X Integrated Development Environment (IDE)
- XC32 Compiler
- MPLAB Data Visualizer
- ML Partners Plugin
The ML Partners plugin and MPLAB Data Visualizer can be installed as plugins to MPLAB X IDE via the plugins manager or the ML Partners plugin as a plugin to MPLAB Data Visualizer Standalone.
Example Firmware
SAMD21 ML Kit Data Logger Firmware - 6-axis Internal Measurement Unit (IMU) data for vibration, rotation, and motion-based solutions
Procedure
To use the ML Partners plugin, MPLAB Data Visualizer must first be configured to receive data from the desired target device. This involves the configuration of the serial connection as well as the variable streamer, which will parse variables from the serial stream. Once data sources are plotted in the Time pane, click the Mark button to tag the visible data for use in the ML Partners plugin. The Mark button places the cursors (A & B) at the bounds of the visible graph so that the viewable data can then be used within the ML Partners plugin.
Figure 1: MPLAB Data Visualizer with 3-axis IMU data plotted in the Time Window
To select a new region of data, scroll, and zoom as needed in the Time pane and then click Mark when the desired segment is precisely within the viewable window. Alternatively, the bounds of the time window can be set manually in the Time Axis menu on the right, before pressing the Mark button.
Capturing sensor data with MPLAB Data Visualizer
Program the Kit with Data Logger Firmware
Use MPLAB X IDE to program the SAMD21 ML Kit with the provided example project. Be sure to select the correct project configuration in MPLAB X before programming the device. There are two configurations to support both versions of the SAMD21 ML Kit.
- IMU2 (Bosch): SAMD21_IOT_WG_BMI160
- IMU14 (TDK): SAMD21_IOT_WG_ICM42688
Figure 2: MPLAB X project configuration
If you are not familiar with MPLAB X IDE, please visit the "MPLAB® X IDE" Developer Help page.
The general application settings for the SAMD21 Data Logger, such as the sensor sampling rate and the data logging format, can be found in app_config.h. This is also where the individual axes of the IMU can be enabled or disabled. This guide will use only the three axes from the accelerometer, however, the logged axes can be reconfigured as needed based on the application.
Figure 3: Data logger application settings in app_config.h
Once the kit is programmed with the desired configuration, you are ready to move on to collecting the serial data stream with MPLAB Data Visualizer.
Configure MPLAB Data Visualizer
Leave the board connected to the computer and open MPLAB Data Visualizer. Load the Data Visualizer workspace file 3dof-imu-acc.dvws found in the example firmware repository. This workspace already contains the variable streamer required to parse the IMU data, and it will plot each variable once the serial port is configured.
Figure 4: Load Data Visualizer workspace
After loading the Data Visualizer workspace file, select the Serial/CDC Connection that corresponds to the SAMD21 ML Kit. Adjust the baud rate to 115200 and click Apply. The DGI connection can also be disabled since we will not use any debug data.
Figure 5: Serial port configuration
Use the play button on the Serial/CDC Connection to start data collection from the kit. Once data is streaming, it is available for use with the variable streamer.
Figure 6: Start streaming selected serial connection
Now select the same Serial/CDC Connection as the input data source for the IMU variable streamer, so that the data axes can be parsed from the stream.
Figure 7: Variable streamer data source selection
The IMU data should now be available in the time plot. Double-click anywhere within the time plot to start/stop scrolling of the time axis.
Figure 8: Data Visualizer Time Plot
Select Data Region and Mark Time Window
Once the desired data sources are plotted in the graph, select a region of interest in the data by focusing the Time Plot on that region. You can drag the plots in the time window to the desired region of data while scrolling to zoom in or out as needed. When you are satisfied with the data viewable in the Time Plot, click the Mark button to tag this region of data for use in the ML Partners plugin.
Figure 9: Mark the data visible in the time window for use in the ML Partners plugin
Pressing Mark will place the cursors at the bounds of the visible window. First, select a new region of data, reposition the desired data within the Time Plot, and then press Mark again.
Figure 10: Time Plot with marked data ready for use in the ML Partners plugin
After marking the Time pane, the data is ready to be used within the ML Partners plugin. This general process of configuring, plotting, and marking the serial data stream can be followed for any type of time-series data that is available in Data Visualizer.
Using the ML Partners Plugin
Edge Impulse
When using the ML Partners plugin with Edge Impulse, you can upload new data to the Data Acquisition tab in the Edge Impulse Studio. This is used to collect the training and testing data that will be used to develop the model. Once you have trained a model within the Edge Impulse studio, the ML Partners plugin can be used to test model performance by uploading new data segments to the testing endpoint and then viewing the classification results within the ML Partners plugin.
For a detailed guide on the Edge Impulse functionality, see "Using the ML Partners plugin with Edge Impulse".
SensiML
When using the ML Partners plugin with SensiML, you can save data to a CSV file that is formatted for import into SensiML's Data Capture Lab. The ML Partners plugin also provides an option to generate a .dcli file (a JSON-based SensiML metadata file) that contains additional metadata about your sample.
For a detailed guide on the SensiML functionality, see "Using the ML Partners Plugin with SensiML".
Results
You should now understand the general process of collecting live sensor data from your target device with MPLAB Data Visualizer and then sending it to one of our ML partner platforms with the Machine Learning Partners plugin. Now that this is out of the way, you can get to work developing edge-optimized, ML solutions that can be deployed in your embedded application.