Digit Recognition AI/ML Application on SAM E51 IGAT Curiosity Evaluation Kit Using MPLAB® Harmony v3: Step 9
Add Application Code to the Project
Copy Files into Project Folder
The following application files are partially or fully developed and available in the folder under <your unzip folder>/digit_recognition/dev_files/sam_e51_igat:
- app.c
- app.h
- app_gfx.c
- app_ml.cpp
- app_ml.h
- main.c
- model.cpp
- model.h
Copy these files from the ../dev_files/sam_e51_igat folder and paste them into the folder <Your project folder>/digit_recognition/firmware/src.
Add Files to Project
Adding Source Files
Add the following source files from the <Your project folder>/digit_recognition/firmware/src folder:
- app_gfx.c
- app_ml.cpp
- model.cpp
In the Projects pane, right-click on the Source Files folder and click on Add Existing Item
Redirect to the src folder, select the app_ml.cpp file and click on the Select button.
This figure shows the source files added to your project.
Adding Header Files
Add the following header files from the <Your project folder>/digit_recognition/firmware/src folder:
- app_ml.h
- model.h
In the Projects pane, right-click on the Header Files folder and click on Add Existing Item
Repeat the previous two steps for the remaining header files (“model.h”).
This figure shows the header files added to your project.
Review Application Code
The main.c file calls the APP_ML_Initialize() and APP_ML_Tasks() routines in main function to initialize the TensorFlow Setup and to run the TensorFlow application tasks.
Click image to enlarge.
The model.cpp file has TensorFlow database for Digit Recognition, i.e., data model array.
Click image to enlarge.
The app_gfx.c file has the following routines to control the GFX events and display the application output.
Click image to enlarge.
- DrawSurface_filterEvent(): This routine is to capture the touchpoints and draw movement based on the graphic event IDs when the user draws on the touch surface. The following are the graphic event IDs:
- Touch Down event
- Touch Up event
- Touch Move event
- This captured touchpoints data is passed as an image to APP_ML_Tasks(). The APP_ML_Tasks task is indicated about the availability of image data through a global flag when touch data for more than 5 points are available, and the Touch-up event is received
- The following code converts the touchpoint positions to a 28 x 28 pixel size. As previously defined, configure the DrawSurface area size as 224 x 224, and the DrawSurface area start positions as 200,50 (X, Y-positions) on the entire screen when the display graphics is designed
- event_MGScrn_DrawSurfaceWidget0_OnDraw(): This routine is to display the drawn points on the display screen when the user touch and draws
- DisplayDigit(): This routine initializes the graphics interface to display the application result. This routine will read the recognized digit by calling APP_ML_GetRecognisedDigit() function
- The following are call back handlers/functions to Legato GFX display modules that will trigger from Legato Graphic routines and tasks:
- MGScrn_OnShow()
- MGScrn_OnHide()
- MGScrn_OnUpdate()
The app.c file controls the display backlight using Timer 3 peripheral instance. It gets the inputs to control the display backlight brightness from the MGScrn_OnShow() routine in the app_gfx.c file. Also the app.c routines will read the display ready status through the GFX driver interface and it will pass to the MGScrn_OnUpdate() routine in app_gfx.c file.
The app_ml.cpp has the TensorFlow database setup code for Digit Recognition. It accomplishes this through the tflite_setup() routine. Also, it pulls only required operation implementations to reduce the code memory.
The app_ml.cpp has another routine—tflite_runInference(). This is to run the Tensorflow CNN Model by passing the drawn image received. The tflite_runInference() routine recognizes the digit drawn by the user and prints the score for indexes (i.e., digits 0 to 9) on the serial terminal.
Click image to enlarge.
You are now ready to build the code and observe the results!