rFpro has integrated Solectrix’s SXIVE image signal processing (ISP) technology into its AV elevate simulation platform. The system allows engineers to develop and optimise camera systems before physical hardware is available.
The integration brings a software-based ISP into rFpro’s camera model, enabling engineers to simulate and tune processing stages including debayering, colour correction, tone mapping, denoising, lens distortion and lens shading correction.
The approach is intended to support earlier development of camera-based advanced driver assistance systems (ADAS) and automated driving systems, while reducing reliance on physical hardware for testing and data collection.
With ISP parameters adjustable within the simulation environment, engineers can assess camera performance across repeatable driving scenarios and edge cases. These include conditions such as low sun, reflective roads, bright headlights and underground or covered parking.
The system can also generate synthetic video data processed through the simulated ISP for use in training perception models. This allows different image-processing parameters to be assessed across large datasets without having to collect each dataset using physical camera hardware.
Matt Daley, Technical Director at rFpro said:With camera-based perception being central to most ADAS and autonomous vehicle performance, ISP optimisation is a key competitive differentiator. Advantages can be gained by enhancing the images for machine rather than human vision and ultimately improving the perception model performance and overall safety of the system. Our combined simulation environment and ISP model create a far faster and safer way for engineers to experience tuning set options in dynamic driving conditions, rather than heading out onto public roads and finding these edge cases conditions.
The integration allows engineers to vary both the driving environment and the camera processing configuration. Scenarios can be changed for factors such as weather, location and lighting, while image-processing modules or sensor configurations can also be modified within the simulation.
For example, engineers can compare how different colour-balancing settings affect pedestrian detection in a dark, wet urban environment with their performance in daylight conditions.
Solectrix’s SXIVE (Simplified eXtensive Image and Video Engine) comprises image-processing software, hardware accelerators, applications and plugins. The technology can be configured for applications including ADAS and automated driving cameras, driver monitoring systems and remote driving.
The companies also point to applications beyond automated driving. Digital mirrors and surround-view camera systems are increasingly being used in passenger cars, buses and heavy goods vehicles, where image processing must accommodate a range of challenging lighting conditions.
The integration is also aimed at addressing growing requirements for digital representations of vehicle sensor systems during development.
Dr. Roman Tzschoppe, R&D Manager at Solectrix said:Providing sufficient amounts of high-quality video data is a real challenge for the development of any modern AD or ADAS system. Our collaboration with rFpro enables engineers to design and execute hundreds of thousands of test scenarios and generate a wide range of highly-specific training data for camera-based perception models. It enables critical image quality ISP parameters to be varied through the dataset production, this simply isn’t feasible when relying solely on physical hardware image collection.
The integration forms part of rFpro’s AV elevate platform, which supports sensor tuning, perception and control-system training, closed-loop testing and the generation of synthetic training data within a simulation environment.
