Skip to Content

Working with Multiple Layers and HW Overlays

Last modified by Microchip on 2024/05/28 10:34

Introduction

This section provides information on how to use Microchip Graphics Suite (MGS) Harmony Composer to take advantage of HW-accelerated compositing features of the integrated display controllers in Microchip's graphics-enabled MCUs and MPUs.

These features enable users to design complex GUIs and UI functions while minimizing the processing load on the main CPU.

Display Controllers and HW Overlays

Microchip's graphics-enabled MCUs and MPUs have integrated display controllers that process image data and send it to the display. This involves converting the digital representation of an image into signals that can be understood by the screen, such as timing signals for horizontal and vertical synchronization, and the actual pixel data.

These display controllers can have multiple HW overlays that allow an application or device to offload the rendering of certain content to dedicated hardware rather than processing it through the main CPU or even the GPU. The overlay unit is a separate hardware block that can handle tasks such as scaling, color conversion, and blending of the video or graphical content.

Below is an example of layers in the PIC32MZ DA GLCD display controller.

Display Controller Overlays/Layeres

UI Layers and Compositing

Layers in User Interface (UI) design refer to the different levels at which elements are placed on the screen. Each layer can contain multiple components that are visually stacked on top of one another, much like layers of paper on a desk. This concept is crucial for organizing content, creating depth, and managing the complexity of interactions within the interface.

One common type of layer is the Background Layer which contains the background color, patterns, or images that form the foundation of the design.

Above the Background Layer can be Layer 1, which includes elements like headers, footers, navigation bars, and sidebars that provide the framework for the UI and help users navigate the interface.

Then on top can be Layer 2, which holds the primary information and elements that users interact with, such as text, images, buttons, and form fields.

These layers are then combined through a process called Compositing to create a final, cohesive image or interface that the user interacts with. This process is akin to how layers are managed in graphic design and photo editing software, where different elements can be stacked, organized, and manipulated independently before being merged into a single visual output.

Layer Compositing

Layer compositing is a fundamental concept in modern UI frameworks and design tools. It allows designers and developers to create complex, visually rich interfaces that are modular and easy to update.

Back to Top

HW Compositing

In Microchip's graphics-enabled MCUs and MPUs, compositing is performed using the HW overlays of the integrated display controllers. The MGL renders these layers in dedicated Frame Buffers in RAM which are read and combined by the Display Controller into a single output frame.

HW Compositing

Using the HW overlays to composite these frames enables dynamic UI content with minimal load on the main CPU or GPU.

Back to Top

Multi-Layer UI Design using Microchip Graphics Suite (MGS) Harmony Composer

MGS Harmony Composer enables users to create multi-layer UI designs. The MGL works with the display controller drivers so these layers are rendered into their respective frame buffers that are then composited into a single frame.

Note that the Multi-Layer function is device-dependent. Before using multiple layers in your design, make sure that the target platform has memory and a display controller peripheral that supports HW overlays.

Multi-layers in Composer

To create a multi-layer design in MGS Harmony Composer, the following steps are performed:

  1. In the Screen Tree, click the + button to add a new layer.
  2. Add widgets to the new layer by dragging them from the Tool Box. A background panel widget was added for each layer. The background panel provides the library with a color to use for filling or clearing the background during a frame draw.
  3. Add a ClearScheme that will be used as a background scheme for layers with transparent backgrounds.
  4. This ClearScheme must be configured to have a Base color set to 0.
  5. The ClearScheme is used on the background panel for the upper layers with transparent backgrounds. This allows the lower layers to be visible in the composited frame.
  6. For the upper layers, the background panel Enabled property is unchecked. This ensures that touch input can propagate to the lower layers.

Note that for a typical MGS Harmony Composer project using multi-layers, it is required to use the same number of layers for each screen. An exception is when the Graphics Canvas is used.

For more information on MGS Harmony Composer, refer to the "Microchip Graphics Suite (MGS) Harmony Composer User Guide".

Here's a visualization of how the layers in the above design are rendered to their respective frame buffers.

Compositing behind the scenes

Back to Top

Configuring HW Overlays

HW Overlays are configured separately using the display controller driver. In MPLAB® Code Configurator (MCC) Harmony, the display controller component will have options to enable and configure each HW overlay.

By default, Composer Layer 0 is rendered to the frame buffer used by the base HW layer in the display controller. Composer Layer 1 is rendered to Overlay 1 frame buffer, and so on.

To ensure proper functionality, make sure that the correct HW overlays are enabled and configured to support the corresponding Layers in MGS Harmony Composer.

Here's an example Layer Configuration for the LCDC display controller for the Microchip graphics-enabled SAM9x60 MPU.

Layer configuration

Back to Top

General Recommendations and Best Practices

Note that using multiple layers is not beneficial for all UI designs. Designing a UI that uses only a single layer has advantages including less memory usage (for frame buffers) and reduced bus and memory bandwidth requirements needed to read the frame buffers in memory. Be aware of these tradeoffs when deciding whether to use multiple layers or not.

Here are some examples of UI designs that can benefit multiple layers and best practices to maximize those benefits:

Use Multi-layers to Overlay Dynamic Content

Use multiple layers to separate dynamic content from static backgrounds, especially for UI designs with full-screen image backgrounds. By having the dynamic content in a separate layer, the background image does not need to be decoded and redrawn when the dynamic content changes. 

Using Multiple Layers for Dynamic Content

Use Layers to Separate Dynamic Content and Overlay UI Controls

Another use of multiple layers is to overlay UI controls to the top layer and isolate it from the content layer. This has a couple of benefits:

  1. It ensures that the UI controls immediately get the touch input events, and
  2. The dynamic content can be rendered in a different format and does not cause redraw on other UI elements.

An example is a video player application where the bottom layer is used to show video frames in RGB565 format (for smaller size), and the video player controls are on the top layer.

Video Player application

Use Layers for Transparency Effects

HW overlays can be used to create transparent or semi-transparent effects on layers. Since the integrated display controller is used to alpha-blend these layers, the load on the CPU is minimal.

Transparency Effects

Back to Top

Advanced Techniques - Graphics Canvas

The Graphics Canvas is a middleware within MGS that makes it easy for users to leverage the HW features of the integrated display controllers in Microchip's graphics-enabled MCUs and MPUs. This can be used to perform complex GUI effects like sliding, fading, screen pre-rendering and animations without loading the CPU.

The Graphics Canvas middleware provides abstracted, user-friendly APIs to configure the display controller functions for these GUI effects.

Refer to "Using the Graphics Canvas" guide for more information.

Back to Top