When discussing autonomous vehicles, public attention is often focused on artificial intelligence. Yet, during his keynote at AutoSens USA, Simon Verghese, Director of Sensors at Waymo, emphasised that autonomous driving remains fundamentally dependent on the quality and diversity of the data collected from the physical world.
Speaking about sensor systems and their use in autonomous vehicle fleets, Verghese argued that robust sensing remains central to handling the edge cases that continue to challenge automated driving systems. While advances in AI have attracted considerable attention, he suggested that the performance of any autonomous system ultimately depends on its ability to perceive its surroundings accurately in changing and unpredictable conditions.
Waymo’s experience provides a substantial dataset to draw conclusions from. The company is now conducting around half a million commercial trips each week and has accumulated more than 200 million commercial autonomous miles. This operational scale exposes vehicles to a wide range of real-world scenarios, from red-light violations and collisions to cyclists, pedestrians, emergency vehicles and roadworks managed by human traffic controllers.
Many of these situations fall outside normal driving expectations. Verghese illustrated this with examples, such as motorcyclists falling in front of vehicles, pedestrians crossing motorways at night, cyclists travelling on freeways and traffic being directed manually when signals fail. In these situations, an autonomous system must detect objects, understand context, and predict behaviour.
To do so, autonomous vehicles continue to rely on multiple sensing modalities rather than a single sensor type. Cameras, lidar, radar and microphones each provide different information about the environment, and each experiences different limitations.
For example, adverse weather affects sensing technologies in different ways. Dust storms can reduce camera effectiveness through glare and contrast loss, while lidar may still detect objects beyond the visual obstruction. However, dense fog can significantly reduce lidar performance over long distances, whereas radar may continue to track vehicles ahead.
This complementary behaviour explains why sensor fusion remains a key design principle for many Level 4 autonomous driving systems. Instead of seeking a single perfect sensor, developers are increasingly combining technologies whose strengths and weaknesses differ.
Simon Verghese, Director of Sensors at Waymo said:We have a lot of sensing on the car to try and get a robust assessment of what's going on. We have many, many gigabits per second of data of different formats, of different perspectives — full 3D versus top-down radar looks versus perspective-view high-resolution cameras — and somehow combining that and making sense of it.
Verghese’s presentation also offered insight into the relationship between autonomous vehicle development and the wider automotive sector. Verghese noted that advances originally driven by the advanced driver assistance systems (ADAS) market are helping reduce the cost of autonomous vehicle sensing hardware. Improvements in semiconductor technologies, imaging sensors, radar components and lidar detectors are making higher-performance sensing increasingly accessible.
This trend could have wider implications beyond robotaxi fleets. As component costs decline and performance improves, technologies previously associated with highly automated vehicles may become more practical for broader deployment across passenger vehicles.
Notably, artificial intelligence also featured prominently throughout the presentation as a tool for interpreting sensor data. Waymo is increasingly using machine learning models to process complex scenes, identify human intent and improve perception capabilities. Examples included recognising traffic controllers’ gestures and understanding unusual situations that conventional rule-based systems might struggle to interpret.
However, despite the excitement surrounding generative AI, autonomous vehicles still face strict requirements for latency, reliability and safety validation. Systems must process vast quantities of sensor data and react within fractions of a second, leaving little room for computationally intensive models that cannot operate predictably in real-world conditions.
Verghese said:Understanding the context comes from good sensing first and then a lot of training on examples to make sure you do the right thing.
Overall, improvements in autonomous driving continue to arise from advances across sensing hardware, semiconductor technology, machine learning, simulation, validation and operational experience.
Autonomous vehicles still depend on the fundamental ability to observe the world accurately, and Verghese’s presentation emphasised that safe automation begins with seeing clearly.
