A Few Layers To The Self-Driving Onion

The development of self-driving cars faces significant liability and regulatory challenges, unlike the unfulfilled promise of flying cars. Nvidia's Xinzhou Wu detailed the evolution of autonomous EVs, emphasizing that modern vehicles are "software-defined." Nvidia provides a "full vehicle stack" via its DRIVE Hyperion platform, offering OS, safety software, and AI models. Wu highlighted the necessity of redundant systems, including parallel processing stacks for L2++ ADAS and redundant sensors like GPS, cameras, and radar, to ensure Level 4 autonomy safety. He also stressed the importance of synthetic data and "neuro reconstruct data" (NuRec) for training AI, generating diverse data variants to bridge data gaps. Wu concluded by noting that intense demand for microprocessors will likely drive up chip prices.

We’ve been hearing about self-driving cars for years.

The thing is, we’ve been hearing about flying cars for even longer – much longer – and those, the imagination-fruit of the Jetsons cartoon in the 1980s, failed to ever materialize.

I chalk most of that up to red tape in the world of personal travel. There’s simply too much risk in letting humans zoom around in the air, piloting the airborne version of a car. We have enough trouble regulating motorists on land.

As for self-driving, any change that replaces human responsibility with software is going to have to meet a very high bar in terms of liability. And again, right now, we have a strained system of insurance to cover the costs of accidents. If self-driving cars are involved in accidents, that could be a problem.

In a recent interview, Nilay Patel of the Verge and Decoder talked to Xinzhou Wu of Nvidia’s auto division, about chips, self-driving vehicles, and much more. The result was illuminating. I want to cover a few of the aspects of autonomous EV evolution that the two discussed.

Generally, software does much more in the modern car than you might think. There’s infotainment, navigation, tracking, and, increasingly, a bevy of assistive driving functions, like lane departure warning, assisted braking, balance sensing, etc.

All of that drives Wu and others to think of today’s car as a “software-defined vehicle,” which always personally makes me think of the “software-defined network” of the aughts.

Patel and Wu also went over the effort to aggregate a set of technologies that might constitute a “full stack” for autonomous vehicle design. Wu explained how Nvidia offers automakers operating systems, safety software, simulation tools, and open AI models through its DRIVE Hyperion platform, but he also referenced other feature stacks for various aspects of vehicle control and design, for instance:

“We actually have a redundant stack even for our L2++ or ADAS function,” Wu said. “(One) is the end-to-end model, which is basically pixel-in, trajectory-out. We also have a classical stack which is more developed based on this safety standard as we know it. It’s a component, basically. It’s a stack with many components, and each component can be verified using this known standard. That’s what I refer to as a classical stack. And when you have two stacks running in parallel, the classical stack acts like what we sometimes call Big Brother, but essentially, it’s a safety guardrail. It tries to verify all the trajectories from the end-to-end model and use the known safety standard to verify it’s safe at every frame.”

In any case, the overall vehicle stack is a significant consideration for this kind of business. In fact, the market mechanics was also something that Patel and Wu addressed, in the midst of describing self-driving systems that may very well represent the vehicles of tomorrow.

Going into detail about how engineers are working on the self-driving vehicle problem, Wu promoted the idea of redundancy, acknowledging some of the risks in making mission-critical systems fully autonomous:

“One of the core concepts of developing Level 4 technology is you have sensor redundancy,” Wu said. “That’s not only for GPS, but also for camera, radar, everything you see. For every single point of failure, the car has to be able to drive safely.

That, he suggested, leads into testing strategy.

“We are doing really massive validation to make sure in all these scenarios, the model is generating the right trajectory,” he continued. “That’s also super-critical for us. So this is what we do to make sure our product is safe.”

Wu also talked about the kinds of data that will drive the self-driving systems (no pun intended). Along with fully synthetic data, which he said is valuable, he mentioned something called neuro reconstruct data or “NuRec”:

“This is a very important piece of technology and simulation where we collect the data from the field, but we can use neuro reconstruction sometimes to fudge the data to change the background, or change the car trajectory,” Wu said. “We can generate a lot of variants of the same data. All this data needs a computer to generate these tens of millions of data points. We can share with everybody who’s engaged in our ecosystem. In this way, collectively from all the players that engaged with the Drive ecosystem, we can catch up on the data gap, which is very important.”

So those are four components of a pretty complex recipe for systems that presumably will outpace Tesla’s autopilot, an early foray into autonomous vehicles that didn’t pan out exactly as planned.

Toward the end of the interview, Wu also acknowledged the hunger for microprocessors that has had car makers competing with gaming and computing verticals for years, as in the dealer shortages of the 2010s.

“Most likely, even the chip price will need to go up, because of this intense demand for every chip everybody can grab onto,” he said.

That’s a little bit of concrete detail about what’s happening with self-driving cars right now. What’s your outlook? Drop me a comment, and let me know.