Last year, Hugging Face, an AI DEV platform, launched Lerobot, a collection of open AI models, datasets and tools that help build real robotics systems. On Tuesday, Hugging Face collaborated with AI startup Yaak to expand Lerobot with robots and car training sets that allow them to autonomously navigate street and other environments.
The new set, called Learning To Drive (L2D), is more than petabytes in size and contains data from sensors installed in cars at German driving schools. The L2D captures cameras, GPS, and “vehicle dynamics” data from driving instructors and students navigating streets at construction zones, intersections, highways and more.
There are many open self-driving training sets from companies including Alphabet’s Waymo and Comma AI. However, many of these, according to the creators of L2D, focuses on planning tasks such as discovering and tracking objects that require high-quality annotations, making them difficult to scale.
In contrast, L2D is designed to support the development of “end-to-end” learning. Its creators can help predict actions (for example, if a pedestrian could cross the street) directly from sensor inputs (such as camera footage).
“The AI community is now able to build end-to-end autonomous driving models,” Yaak co-founder Harsimrat Sandhawalia and Remi Cadene, a member of the Robotics team at Hugging Face, wrote in a blog post. “L2D aims to be the largest open-source, autonomous driving dataset that empowers the AI community with unique and diverse ‘episodes’ for training end-to-end spatial intelligence. ”
This summer, the embrace plan of faces and Yaak to conduct actual “closed loop” tests of models trained using L2D and Lerobot. Companies are urging the AI community to submit models and tasks that evaluate the model, such as roundabouts and navigating parking spaces.