The goal of this project was to develop and deploy a software platform that will provide user access to the event stream with graphical & memory-mapped interface.
Project Description
If you are reading this post, this and this might be useful.
Link to GitHub
More code, Vivado Project and Altium files can be requested via email.
The Project provides user access to the event stream and records the event stream. FPGA design interfaces the event camera generates the timestamp for individual events and saves data to the DDR memory. jAER software (provided by the camera manufacturer) enables the parallel output. PCB connector up translates the signals from the ULTRA96 board (1.8V -> 3.3V).
| № | Item | Description |
| 1 | Vivado Project | It generates the FPGA design (.tcl file and .bit file). |
| 2 | Python Script & Library | Programs the FPGA. Reads event stream data from the memory. Displays event stream data. |
| 3 | C++ Script | Reads the event stream from the memory. Deletes repeated events. Writes the event stream (~10 sec) to the .csv file. |
| 4 | PCB Connector | PCB connector up translates the signals from the ULTRA96 board (1.8V -> 3.3V). Altium project is provided. |
| 5 | Matlab Script | It generates a video from the recorded events. NOTE: DEVELOPED BY YON NG. |
How does it work?
The FPGA design block diagram is shown below.
FPGA design building blocks are described in the table below. DAVIS240C processing block will be described in a separate section.
| № | Block | Description |
| 1 | Data Packager | Data packager acts as a data multiplexer.The Switch is flipped at the negative edge of “Write Ready” and the positive edge of “New Transmission”.Switch = 0 -> Data Stream = EventsSwitch = 1 -> Data Stream = Timestamp There is a counter inside this block. The counter is triggered at the negative edge of “Write Ready” and the positive edge of “New Transmission”.If counter = 1021 -> Data Stream = TX Counter (Switch don’t care). |
| 2 | Transmission Init. | After FPGA is programmed and the clock is running the block will count 2 seconds and initiate the first transmission. |
| 3 | Transmission Generator | This block will generate a “New transmission” pulse at the clock edge of the “Start TX” signal or “Transmission Done” signal. NOTE: for successful “New Transmission” signal detection by the DDR Memory writer, it should be extended by 4 clock cycles. |
| 4 | Transmission Counter | 32-bit counter. Used to count transmissions. |
| 5 | DDR Memory Writer | State Machine that performs memory writing. |
On the CPU side the event stream can be read from the set memory location with a small Python or C script. The event stream can be visualised with a MATLAB script.
DAVIS240C Processing Block
The processing block is the updated version of this.
NOTE: Single Y address can be followed by multiple X addresses.
DAVIS240C Processing Block is implemented as a clock driven state machine .
There are 3 configurable parameters in this design:
1. X Delay = 20 (in clock cycles).
2. Y Delay = 20 (in clock cycles).
3. ACK Hold time 4 (in clock cycles).
X&Y delays are introduced to make sure that the bus is stable and the data is ready. Extra ACK hold time is introduced to make sure that the camera captures the acknowledge signal every time (200Mhz single clock pulse is too short for the camera to latch).
NOTE: These parameters were defined from the experiment. Delays were set to the smallest possible number with the design being able to work.
How to use it?
Connecting Hardware
- Connect your ULTRA96 board to the PC with a micro USB cable.
- Turn on the ULTRA96 board. Wait for the blue LED to light up.
- Connect your DAVIS240C to the PC with a micro USB cable.
- Open jAER -> HW Configuration -> DVS Config.
- Disable “DVS.Run”. Enable “DVS.ExternalAERControl”
- Attach the PCB Connector to the camera (Figure 3).
- Attach the PCB connector to the board (Figure 3)
. Make sure that the blue LED is still on.
Running Software
Camera Simulator
Camera Simulator is implemented as a Vivado project. There are 2 blocks in the simulation: Event Stream Processing Block, Camera Simulator.
To run the simulation open Vivado project and press “Run Behavioral Simulation” Right-click on “clk” and set 5ns period.
NOTE: Running the simulation under different scenarios and with different delays and clocks can help to understand the operation of the camera interface.
Event Stream Access with Jupiter Notebooks
If you have never worked with PYNQ -> http://www.pynq.io
This software product should be used only for FPGA programming and checking that the camera read is established.
NOTE: Program execution in “Jupiter Notebooks” is slower than in the Terminal. Do not attempt to record the event stream with “Jupiter Notebooks”.
To access the event stream with the Python Shell:
1. Connect the Hardware.
2. To open “Jupiter Notebooks” type “http://192.168.3.1” in your web browser.
3. Go to -> Software_Platform -> Event_Camera -> Event_Camera.
The first block of code will use the Python library (DVS_LIB) to run the automated events quality
1. “TX” (Transmission counter) is incrementing.
2. The timestamp is incrementing.
3.
Event Stream Access with Terminal
This method allows reading the event stream at high speed and recording data to the .csv file. To access the event stream with the C++ script:
1. Connect the hardware.
2. Program FPGA with the Jupiter Notebooks.
3. Open terminal with PuTTY (COM6 -> 115200).
4. Go to /home/Xilinx/C_CODE/1_EVENT_PRINT_FSM.
5. Run “g++ main.
6. Run “
Copy generated “data.csv” file on your PC for further processing
Event Stream Visualisation
NOTE: DEVELOPED BY YON NG.
Link to MATLAB Script
To generate a video from the events that were recorded with the C++ script.
1. Save .csv in the same folder with the MATLAB script.
2. Open Matlab and run: “fpga_loaddata” & “test_delay_USB_plugin”.
3. Wait for the plot to open.
4. Close Matlab.
5. You can find the video in the folder with the script.
NOTE: “test_delay_USB_plugin” can produce an error. Ignore it.