The potential computational power of today multicore processors has drastically improved compared to the single processor architecture. Since the trend of increasing the processor frequency is almost over, the competition for increased performance has moved on the number of cores. Consequently, the fundamental feature of system designs and their associated design flows and tools need to change, so that, to support the scalable parallelism and the design portability. The same feature can be exploited to design reconfigurable hardware, such as FPGAs, which leads to rethink the mapping of sequential algorithms to HDL. The sequential programming paradigm, widely used for programming single processor systems, does not naturally provide explicit or implicit forms of scalable parallelism. Conversely, dataflow programming is an approach that naturally provides parallelism and the potential to unify SW and HDL designs on heterogeneous platforms. This study describes a dataflow-based design methodology aiming at a unified co-design and co-synthesis of heterogeneous systems. Experimental results on the implementation of a JPEG codec and a MPEG 4 SP decoder on heterogeneous platforms demonstrate the flexibility and capabilities of this design approach.

The recent MPEG Reconfigurable Media Coding (RMC) standard aims at defining media processing specifications (e.g. video codecs) in a form that abstracts from the implementation platform, but at the same time is an appropriate starting point for implementation on specific targets. To this end, the RMC framework has standardized both an asynchronous dataflow model of computation and an associated specification language. Either are providing the formalism and the theoretical foundation for multimedia specifications. Even though these specifications are abstract and platform-independent the new approach of developing implementations from such initial specifications presents obvious advantages over the approaches based on classical sequential specifications. The advantages appear particularly appealing when targeting the current and emerging homogeneous and heterogeneous manycore or multicore processing platforms. These highly parallel computing machines are gradually replacing single-core processors, particularly when the system design aims at reducing power dissipation or at increasing throughput. However, a straightforward mapping of an abstract dataflow specification onto a concurrent and heterogeneous platform does often not produce an efficient result. Before an abstract specification can be translated into an efficient implementation in software and hardware, the dataflow networks need to be partitioned and then mapped to individual processing elements. Moreover, system performance requirements need to be accounted for in the design optimization process. This paper discusses the state of the art of the combinatorial problems that need to be faced at this design space exploration step. Some recent developments and experimental results for image and video coding applications are illustrated. Both well-known and novel heuristics for problems such as mapping, scheduling and buffer minimization are investigated in the specific context of exploring the design space of dataflow program implementations.

Recently, ISO/IEC standardized a dataflow-programming framework called Reconfigurable Video Coding (RVC) for the specification of video codecs. The RVC framework aims at providing the specification of a system at a high abstraction level so that the functionality (or behavior) of the system become independent of implementation details. The idea is to specify a system so that only intrinsic features of the algorithms are explicitly expressed, whereas implementation choices can then be made only once specific target platforms have been chosen. With this system design approach, one abstract design can be used to automatically create implementations towards multiple target platforms. In this paper, we report our investigations on applying the methodology standardized by the MPEG RVC framework to develop secure computing in the domains of cryptography and multimedia security, leading to the conclusion that the RVC framework can successfully be applied as a general-purpose framework to other fields beyond multimedia coding. This paper also highlights the challenges we faced in conducting our study, and how our study helped the RVC and the secure computing communities benefited from each other. Our investigations started with the development of a Crypto Tools Library (CTL) based on RVC, which covers a number of widely used ciphers and cryptographic hash functions such as AES, Triple DES, ARC4 and SHA-2. Performance benchmarking results on the RVC-based AES and SHA-2 implementations in both C and Java revealed that the automatically generated implementations can achieve a comparable performance to some manually written reference implementations. We also demonstrated that the RVC framework can easily produce implementations with multi-core support without any change to the RVC code. A security protocol for mutual authentication was also implemented to demonstrate how one can build heterogeneous systems easily with RVC. By combining CTL with Video Tool Library (a standard library defined by the RVC standard), a non-standard RVC-based H.264/AVC encoder and a non-standard RVC-based JPEG codec, we further demonstrated the benefits of using RVC to develop different kinds of multimedia security applications, which include joint multimedia encryption-compression schemes, digital watermarking and image steganography in JPEG compressed domain. Our study has shown that RVC can be used as a general-purpose implementation-independent development framework for diverse data-driven applications with different complexities.

This paper presents some strategies for design space exploration of FPGA-based signal processing systems that are specified using the CAL dataflow language. The actor-oriented, high-level of abstraction provided by CAL allows flexible exploration and consequently results in a wide range of feasible design implementations. We have applied and extended the existing techniques for refactoring and pipelining actors and actions by means of critical path analysis, and introduced some new buffering techniques based on heuristics. The combinations of these techniques have been applied on the CAL specification of the MPEG-4 video decoder, and synthesized to HDL for evaluation in the design implementation space. Results show that using our configuration for the exploration of 48 design points, a throughput range of roughly 8× has been achieved, for slice, block RAM, frequency, and latency range of 1.3×, 2.5×, 2.5×, and 2.9× respectively.

The possibility of specifying both software and hardware components from a unified high-level description of an application is a very attractive design approach. However, despite the efforts spent for implementing such an approach using general purpose programming languages, it has not yet shown to be viable and efficient for complex designs. One of the reasons is that the sequential programming model does not naturally provide explicit and scalable parallelism and composability properties that effectively permits to build portable applications that can be efficiently mapped on different kind of heterogeneous platforms. Conversely dataflow programming is an approach that naturally provides explicit parallel programs with composability properties. This paper presents a methodology for the hardware/software co-design that enables, by direct synthesis of both hardware descriptions (HDL), software components (C/C++) and mutual interfaces, to generate an implementation of the application from an unique dataflow program, running onto heterogeneous architectures composed by reconfigurable hardware and multi-core processors. Experimental results based on the implementation of a JPEG codec onto an heterogeneous platform are also provided to show the capabilities and flexibility of the implementation approach.

Motion estimation represents a key module in video compression. The Reconfigurable Video Coding context (RVC) requires proposing a flexible solution for motion estimation. The motion estimation performance should be modified to fit with the user or the environment's constraints. Depending on the required performances fixed by the application, a full search is sometimes not suitable, hence, alternative fast/reduced solutions should be considered. In this paper, an efficient Diamond Search motion estimation, described in RVC-CAL actor language, is introduced. Starting from a high level description based CAL language, an automatic translation of the proposed CAL module to HDL is performed. This is meant to give designers an edge over its market competitors by substantially reducing the Time to Market (TTM). A comparison between the generated HDL with a previous work supporting user search strategy and handwritten code is also provided. The results show substantial improvement in term of operating frequency and silicon area to previous work based CAL description.

New generations of multi-core processors and reconfigurable hardware platforms are expected to provide a dramatic increase of processing capabilities. However, one obstacle for exploiting all the promises of such new platforms is the legacy of current applications and the development methodologies used, which is deeply rooted in a sequential way of thinking. A paradigm shift is necessary at all levels of application development to yield portable and efficient implementations, capable of exploiting the full potential of such platforms. Dataflow programming is an alternative approach that address the problem of providing portable and scalable parallel applications. Dataflow programming is able to explicitly expose the intrinsic parallelism of applications. This paper presents a hardware/software co-design methodology that starting from a unique dataflow program enables, by the direct synthesis of both hardware (HDL) and software components (C/C++), to map a signal processing application onto heterogeneous systems architectures composed by reconfigurable hardware and multi-core processors. Experimental results based on the implementation of the MPEG-4 Simple Profile decoder onto an heterogeneous platform are also provided to show the capabilities and flexibility of the approach.

Image filtering is mainly used for pre-processing in many applications of image processing such as noise removal in industrial inspection, enhancement for pattern recognition, restoration of degraded images, etc. For real-time processing, it is crucial that these applications meet the desired throughput. As processing demands get higher through complex algorithms and large resolution or very high frame rate images, general purpose computers can no longer sustain the required time. Hardware implementation on the other hand, though capable of producing high performance design, is often considered difficult and time consuming to perform due to the low abstraction level. This paper presents an efficient dataflow architecture for image filtering which allows rapid design implementation with features such as the Region of Interest, Memory Cache, and Border Replication. For parallel processing, dataflow Token Splitting technique is also applied, which provides a simple way to increase data throughput. Furthermore, the capability to synthesize CAL dataflow program to both hardware and software provides a platform for co-design of complex image processing systems. Experimental results from synthesis of four different types of filters show that parallel image filtering with a factor of four is up to 3.5 times faster on FPGA compared to sequential processing on single core general purpose processor, with additional slice of only 2% from single to four parallel filters.

This paper presents a contribution to the development of rapid prototyping tools based on dataflow description. In this context, a major key goal is to propose a design flow which includes an automatic synthesis of the hardware interfaces. Therefore, this paper presents a methodology to define and generate automatically a system's interface from a data-flow description. To achieve such an ambitious goal, a generic hardware socket has been defined. This paper presents especially the serialization and de-serialization of different data flows between the hardware design and the physical interface. Different strategies are therefore compared and discussed.

This chapter presents a new development of rapid prototyping tools for system design based on data-flow specifications. In this context, the efficiency of tools for the automatic translation from the data-flow programs to C and/or HDL are assessed by means of two design cases. The chapter also introduces the new concept of the automatic synthesis of interfaces. Such generic interfaces are implemented by using an embedded microprocessor, which can support a large variety of interfaces already available as native IP libraries in the case of FPGA. The two design cases described here have been developed, tested and validated on different implementation platforms. The results of the assessment show that flexibility, genericity and generality are attractive features of the proposed interface implementation methodology approach.

Motion estimation represents a key module in video compression. The RVC context requires proposing a flexible solution for motion estimation. According to the nature of the application, a full search is sometimes not suitable, hence, alternative fast/reduced solutions should be considered. This paper proposes a model and implementation of a flexible motion estimation engine, which can be configured to support any user-defined search strategy. Typically, the computational requirements of the search strategy can be traded with the RD-performance of the obtained video encoder. A CAL dataflow description of the accelerator is proposed so that it can be easily handled in the RVC context. An automatic translation of the proposed CAL module to HDL is performed, and a comparison between the generated HDL with handwritten HDL code of the same model is performed. This helps to evaluate the influences of the CAL model refinements.

This paper presents a contribution to the development of rapid prototyping tools based on dataflow description. In this context, the efficiency of automatic translator tools from the data-flow description to C and/or HDL are presented using two design cases. Moreover, this paper presents the novel concept of the automatic synthesis of interfaces based on dataflow description. Such “generic” interfaces include an embedded microprocessor, which enables using a vide variety of interfaces already available as optimized libraries from the FPGA manufacturers. The different design cases described have been tested and validated on different platforms. The results of the work show the flexibility and generality of the proposed wrapper methodology that is described in the paper.

This paper presents the comparison of two design methodologies applied to the design of a co-processor dedicated to image processing. The first methodology is the classical development based on specifying the architecture by directly writing a HDL model using VHDL or Verilog. The second methodology is based on specifying the architecture by using a high level dataflow language followed then by direct synthesis to HDL. The priciple of developing a dataflow description consists on defining a network of autonomous entities called actors, which can communicate only by sending and receiving data tokens. Each entity in the process of consuming and generating data tokens performs completely independent and concurrent processing. A heterogeneous platform composed by a SW processor and the designed HW co-processor is used to compare the results of the designs obtained by the two different methodologies. The comparison of the results shows that the implementations based on the dataflow methodology, not only can be completed with an important reduction of design and development time, but also enable efficient re-design iterations capable of achieving performances, which are comparable in efficiency to design obtained by hand written HDL.