Autonomous F-150 Raptor learned in 38 km with no RL or labels

Hassan@HassanRIsmail

🚨 WE JUST BROKE A RECORD ON VERIFIABLE PHYSICAL AI 🚨

Physical AI (especially vehicle autonomy) has been historically dominated by black-box E2E Deep Networks.

This has proven to be problematic for a few reasons, including but not limited to:
1. Physical systems typically come with behavioral guarantee requirements (no one wants to 'just trust a robot')
2. Training networks requires access to unreasonably large quantities of high quality data; often economically in-viable quantities.
3. Networks do not reason and often memorize patterns in datasets that hinder generalization (e.g sim-to-real is a byproduct of this)

Enter Programmatic AI - a paradigm we've been cultivating for near a year now; built on the principles of neurosymbolic approaches pioneered by those like @GaryMarcus among others.

The clip you see below is an F150 Raptor driving around a forest, using only RGB vision and inertial telemetry; capable of segmenting objects, and explaining its decisions and understanding of its environment in real time.

Most impressively, this was done:
- In less than 38km of random driving (NO teacher, random driving, from scratch)
- WITHOUT the use of reinforcement learning, or supervised data (NO labels, or pretraining on driving)
- Using ONLY passive sensing i.e cameras, IMU (no ground truth, LiDAR, or other active sensors)
- A novel programmatic AI low SWaP model, small enough to be run on an iPhone, while being human inspectable (NOT blackblox, and unlike Waymo - does not require Cloud access at all)
- WITHOUT maps, on the fly long range route planning was also achieved

To our knowledge, this represents the first time a self driving system has not only been completely deterministic, but has achieved a 0% crash rate, transferred to unseen terrain - in less than 50km of driving; all with self provided labels, no RL, nor pretraining.

We believe that the future of AI is one in which humans interact with superintelligence in a meaningful manner in the real world. We also appreciate that the current approach of pure end-to-end neural networks won't cut it. We're betting that approaches like, and including ours - will be the 3rd wave of AI that takes AI from science fiction, to science fact.

If this is a mission that resonates with you, and you find yourself a contrarian among 'contrarians' - join us! Our team consists of chess international masters, ARCT 9 pianists, debate champions, ex-pro gamers, and more.

More to come soon.