Recently I read that the Figure system 0 trained on 1,000 hours worth of data. Nvidia’s open source robot trained on 40k hours (40x). In terms of raw data this is a lot! But these amounts are tiny in terms of the relative human experience.
Think about language models. Models are trained on trillions of tokens. Assuming a rough amount of 1.3 tokens per word and 300 wpm (decent adult) then this is 100 million hours or 20,000 years of reading data. That’s over 1000x the largest robot training dataset, not to mention that human language is often very densely represented.
It’s no wonder that these systems are harder to build with today’s methods!