• Published on 17 Mar 2015
• Project News

ALMARVI

The Almarvi project originated from the growing importance of image processing for embedded systems, first within healthcare, but now increasingly also in other domains. Originally, image processing dealt with image enhancement before showing it to a user. Image processing is needed to extract information from a subject of interest in the environment, and possibly enhance the image in such a way that the most interesting part is best visible for the user. Often, the extracted information has to lead to rapid reaction of the system, or the user. Therefore low latency is usually required. In many cases images are combined in image streams, and these are by far the largest data objects in the embedded systems.

For instance in healthcare, during minimal invasive intervention, X-ray is in use to support navigation of medical equipment inside the body of the patient. Both for the medical professional, and the patient, X-ray is harmful, and therefore very low dose is demanded. Image processing is used to enhance the very low contrast, high noise images into visible ones, to recognise the organs and the medical instruments in the body, and to improve visibility of the navigation path and the target. The processed images need to be displayed in a fraction of a second to ensure navigation feedback in time.  

In security and surveillance, a distributed solution with several cameras will be used to monitor a certain region. Often these cameras are not connected to a power network, and therefore low power consumption is an issue. To reduce the power consumption, in normal state, image processing has to be performed to reduce the image exchange data size. In an alarm situation, more precise images are needed, other image processing will then be used to exchange as much relevant information as possible.

In mobile devices, increasingly the users access multimedia content, or produce it themselves. For several applications, such as face recognition, images need to be enhanced, and objects of interest need to be extracted. This content needs to be processed before exchange, taking into account bandwidth and power restrictions.

All image processing actions need a large amount of computation. In combination with low latency this is solved traditionally by expensive special signal processing hardware. However, as the applicability of embedded systems that need image processing is growing, the hardware cost needs to be low. Therefore there is a growing need to do image processing on of-the-shelf multi-core hardware. The Almarvi project aims to apply this multi-core hardware for low latency embedded system processing. An additional important issue is that the number and kinds of cores, hardware lay-out and capabilities of the multi-core systems change over time. To be effective, embedded system software developers need to be able to abstract from this heterogeneous hardware, in order to easily port software to future hardware and to ensure the possibility of making trade-offs for different client demands.

Consortium

Several industries in Europe face this same set of problems, and these partner found each other to define the Almarvi project.  Several technology providers joined the consortium. April 1, 2014 we started the Almarvi project with sixteen partners in four countries. Although the general requirements are the same, these are several differences between the partners. Some only use single chip multi-core configurations, others use multi-cores in combination with FPGA. In addition, several systems are not connected to a power network, therefore power consumption is an issue for several partners in the project.

Solution

Almarvi envisions a 3-layer architecture for image processing embedded systems, see Figure 1. At the bottom there is a heterogeneous hardware. At the top the image processing applications run. An intermediate system software stack ensures communication between both layers, and also for high adaptability of the system. Continuously a resource negotiation feedback loop is active, to ensure optimal performance and power consumption. Between the top two layers, resource requests and releases are communicated in terms of the algorithms themselves. Between the bottom layers, the resources are in hardware terms. An advanced tool chain focusing on resource usage reduces the development time of the applications drastically.

Present status

After almost 1 year, several demonstrations are defined leading to the requirements on performance and power consumption. An initial heterogeneous selection of existing underlying hardware is defined, leading to specific requirements for the system software stack. In the forthcoming years, the system software stack has to be developed and the demonstrators have to be developed, using the newly developed tools.  The three layers have to be combined and integrated. This has to lead to an evaluation of the chosen solutions.

Figure 1: Three system layers