Why driving is hard—even for AIs

I have a couple of kids of learner’s permit age, and it’s my fatherly duty to give them some driving tips so they won’t be a menace to themselves and to everyone else. So I’ve been analyzing the way I drive: How did I know that the other driver was going to turn left ahead of me? Why am I paying attention to the unleashed dog on the sidewalk but not the branches of the trees overhead? What subconscious cues tell me that a light is about to change to red or that the door of a parked car is about to open?

This exercise has given me a renewed appreciation for the terrible complexity of driving—and that’s just the stuff I know to think about. The car itself already takes care of a million details that make the car go, stop, and steer, and that process was complex enough when I was young and cars were essentially mechanical and electric. Now, cars have become rolling computers, with humans controlling (at most) speed, direction, and comfort.

For a vehicle to even approach autonomy, it has to understand the instant-to-instant changes in its immediate environment and what they mean. It has to know how to react. And it has to know the important nearby things that don’t change—like where houses and trees are.