Multimodal-Capable Edge Computing Platform Based on an Open-Source RISC-V CPU

Driven by the demand for low-latency, privacy-preserving edge AI inference, this paper proposes a multimodal-capable edge platform based on the open-source RISC-V E203 CPU. It integrates a tightly coupled deep learning accelerator, extends a custom deep learning instruction set, and deploys TensorFlow Lite Runtime. It supports processing image, audio, and text data from peripherals such as the OV5640 camera, microphone, and SD card, enabling multimodal input. The experimental results on the Xilinx XC7A200T-2 FPGA show a speed increase of 3.29x–9.89x for different deep learning models, with a dynamic power of 1.737W and static power of 0.167W.