Today’s vision-language-action systems are largely imitation-driven: they inherit the coverage of their datasets and often fall back to broad trial and error outside it.
Choosing what to experience
A more capable agent should choose interactions because they reveal reusable structure, distinguish several plausible behaviors, or reduce uncertainty about the physical world. Exploration should improve both the data an agent collects and the representation it learns.
I see three complementary loops: external feedback from people or model critics; online reinforcement learning for internal policy improvement; and behavioral abstraction through unsupervised, regret-aware skill induction and multimodal action modeling.
Physical exploration has a cost
Unlike exploration in a simulator, robot interaction consumes time, energy, and hardware life, and can create safety risks. Information gain alone is therefore insufficient. The objective has to account for physical cost and the future utility of what is learned.
Open question. What exploration objective can jointly balance information gain, skill coverage, regret, physical cost, and safety?
This is a working note. Discuss the direction or return to the writing archive.