Written by Peter Katarzynski, Engineering Manager at Antmicro
Due to Antmicro’s focus on the development of latest generation edge AI processing technologies, many of the company’s projects involve vision, video processing, and AI/AR, i.e. applications that push the boundaries of data processing capabilities.
The work often results in and inspires open hardware video accessories such as the Lattice CrossLink FPGA-based SDI-to-MIPI Bridge, which allows you to connect SDI inputs over long distances using a single coaxial cable to the otherwise short-range (but extremely popular and widely available) MIPI CSI-2 interface. The board has enabled various customer projects where the bridge is used together with Antmicro’s MIPI CSI-2 capable open hardware platforms, including the NVIDIA Jetson Baseboard or Snapdragon 845 Baseboard.
The first version of the board has been successfully applied in a variety of sectors, most notably robotics, autonomous vehicles, and the filmmaking industry. The board’s widespread use was a definitive argument in favor of creating an upgrade path – a next generation SDI-MIPI converter, with all design files immediately available on Antmicro’s GitHub. The board includes a much more powerful Lattice CrossLink NX FPGA and RAM to enable even more use cases and opens up the ability to build much more advanced, SDI-capable customized video processing devices.
Next-gen SDI-MIPI Video Converter board
Based on the feedback from users of the first-generation design, the new board retains the basic functionality but adds more flexibility and features. On top of the same SDI-MIPI conversion channel as its predecessor, the CrossLink-NX variant now exposes two Hard D-PHY interfaces from the CrossLink-NX FPGA on two separate connectors. This allows the implementation of MIPI CSI-CSI and potentially CSI-DSI processing pipelines. The CrossLink-NX family of FPGAs supports faster D-PHY speeds, which allows it to process higher resolution footage at higher framerates. Moreover, the new board comes with RAM, which allows you to buffer video frames for more advanced analysis.
Key features of the Video Converter include:
- Lattice LIFCL-40-9BG256C CrossLink-NX FPGA
- 3G SDI input with loopback output
- 3×4-lane CSI interfaces on two Antmicro’s 50-pin FFC connectors
- 2Gb (128Mbx16) DDR3L on-board memory
The board can be powered from the on-board USB connector or Antmicro’s 50-pin FFC connector.
On-board processing with CrossLink-NX and DRAM
SDI signal conversion in the Video Converter is implemented with a Semtech GS2971A deserializer, which passes the parallel 10-bit video data to the Lattice CrossLink-NX LIFCL-40-9BG256C FPGA.
The CrossLink-NX FPGA includes dedicated MIPI D-PHY transceivers at 10 Gb/s per interface (2.5 Gbps/lane) rather than 6 Gb/s per interface and a significantly higher amount of programmable logic (40K rather than 6K LUT) as compared to the previous-gen CrossLink devices. This is a landmark change, allowing quite advanced on-board video processing in the converter itself – such as real-time image processing, e.g., edge detection, or even integration of AI accelerators for feature recognition, presence detection, or tracking.
This is made possible by another significant improvement in the board over its predecessor: the addition of an x16 DRAM interface (also compatible with RPC DRAM), enabling buffering frames for more advanced processing.
The SDI-MIPI Video Converter board is connected to the host platform over a single I2C bus which allows to transfer configuration commands to the on-board SDI deserializer as well as upload the bitstream to the CrossLink-NX FPGA, e.g., from an embedded platform (e.g., RISC-V or ARM-based) running Linux or Zephyr.
All of these improvements, along with the FPGA’s use of 28nm CMOS and an increase in Embedded Block RAM (EBR) capacity to 1.5Mb, mean that the next-gen SDI converter is more power-efficient, faster, and better suited for mobile applications like drones or autonomous vehicles.
CSI input for even more use cases
As the board’s new version also includes MIPI CSI-2 input, it covers even more usage scenarios. This input can be used to, for example, bypass SDI and use the board for processing MIPI CSI-2 pipelines. This opens a whole new realm of possibilities as it can now be used to combine the two MIPI CSI-2 video inputs and stitch them together in a heterogenous camera system, potentially writing additional information into the frames using the CrossLink-NX. This functionality is widely used in, e.g., drone applications which need to augment video data with information like timestamps and extra readouts from IMUs, temperature sensors, range finders, or GPS coordinates. It is especially prevalent in drone photography, where you often need to stitch together several frames to create a 360-degree view or gather position data such as camera angle.
All in all, the greatly enhanced versatility of the new board makes it a great prototyping platform for a variety of video use cases, and the board can be treated as a reference design which Antmicro can later incorporate into a small-footprint embedded product.
Open source FPGA toolchain
Perhaps even more importantly, CrossLink-NX is supported in the open source FPGA toolchain that Antmicro is involved with as part of the FOSS flows for the FPGA, or F4PGA, effort within CHIPS Alliance which aims to build a fully open toolchain for developing with FPGAs from multiple vendors. Having the ability to modify the tools down to the last bit helps Antmicro iterate faster and employ more software and test-driven approaches to FPGA development as described in the previous low-latency smart video solutions for Lattice CrossLink-NX blog note.
The open toolchain makes the board even easier to work with and share fully reproducible and testable development artifacts.
Build your next hardware project with Antmicro
Antmicro provides its customers with extensive FPGA, software, and, hardware development services to help you build and implement complex video processing devices which integrate functionalities such as SDI-MIPI conversion or data pre-processing in FPGA.
The company’s experience spans many sectors, industries, and projects at any stage of development and builds on an array of open hardware boards and accessories, which can be used to rapidly prototype and then industrialize customized hardware, FPGA code, and software. If this sounds relevant to your product development needs, reach out to us at contact@antmicro.com to see how we can help.
If you have any questions or comments, please reach out to the Zephyr Technical community on the Zephyr Discord Channel.