Ideas & Notes

Questions worth thinking about in public.

This page collects evolving ideas about embodied intelligence, robot learning, and the broader trajectory of AI. They are working hypotheses—not finished claims—and I update them as experiments and conversations change my mind.

If one of these questions overlaps with your work, I would be glad to discuss it. Write to zhanghe18@tsinghua.org.cn.

Research ideas

Algorithmic hypotheses I am developing around learning and control in the physical world.

Working frameworkBehavioral science × embodied AI

A Three-Level Architecture for Embodied Behavior

I organize embodied behavior by both output duration and functional role. The three levels are not a claim about a fixed biological hierarchy; they are an engineering framework for locating what an algorithm learns, reuses, and controls.

  1. Level 1 Action generation

    BC, diffusion policies, and VLAs map observations and instructions to short-horizon actions or action chunks.

  2. Level 2 Skills & habits

    Stable, temporally extended units become reusable, interruptible, and inexpensive through practice.

  3. Level 3 Goal-directed composition

    Policies select and reorganize behavior according to predicted outcomes and their current value.

Behavioral science contributes more than terminology here: outcome devaluation, contingency changes, and transition revaluation can become interventions that test whether an agent truly updates consequences, values, and plans.

Open question. Can a robot preserve fast, habit-like execution inside a skill while retaining value-sensitive override at the boundaries?

Research questionWorld models

What Should a World Model Mean?

Visual fidelity alone is an incomplete criterion for an embodied world model. For control, the useful object is an action-conditioned, updateable representation that predicts task-relevant consequences of interventions and supports planning, revaluation, and transfer.

One direction is to separate latent state into context that should remain stable within a trajectory and factors that change over time. The dynamic component should preserve temporal and controllable structure; the combined representation should retain enough information for reliable action, without requiring pixel-perfect prediction.

Open question. How can invariant context and controllable dynamics emerge through self-supervision, without goals that narrow generalization?

Research directionExploration & robot learning

Active Exploration in the Physical World

Today’s VLA systems are largely imitation-driven: they inherit the coverage of their datasets and often fall back to broad trial and error. A more capable agent should choose interactions because they are informative, expose reusable skills, or resolve uncertainty about several plausible behaviors.

I see three complementary loops: external feedback from people or model critics; online reinforcement learning for internal policy improvement; and behavioral abstraction through unsupervised, regret-aware skill induction and multimodal action modeling. Exploration should improve both the data an agent collects and the structure it learns.

Open question. What exploration objective can jointly balance information gain, skill coverage, regret, physical cost, and safety?

Talk notes

Compact versions of arguments first developed for talks and group discussions.

Talk notesAI evolution · May 2026

AI Evolution: Where Might the Next Discontinuities Appear?

I use evolution as an organizing metaphor rather than a literal forecast. The question is where recombination and pressure may force nonlinear change: the computational unit, learning architecture, hardware substrate, interpretability mechanism, and industrial ecosystem. Each frontier couples a scientific bottleneck with a deployment constraint.

Beyond capability, increasing AI agency creates a governance question. Systems operating in physical and high-risk environments need human authority, explicit safety boundaries, graceful degradation, and responsibility that is defined before autonomy is scaled.

Discussion prompt. Which boundary is most likely to trigger the next qualitative transition—and which institutions should prepare for it now?

Talk notesEmbodied intelligence · June 2026

Human-Like Behavior Is a Learning Pattern

“Human-like” should not mean copying the surface of human motion. The stronger target is a learning pattern: purposeful exploration, efficient use of experience, abstraction of causal structure, and adaptation when the environment changes.

Embodied intelligence sits at the intersection of computer science, psychology, neuroscience, control, and philosophy. Current systems have advanced rapidly along the computer-science axis; progress on perception–cognition–action may require returning to the experimental questions and mechanisms developed in those neighboring disciplines.

Discussion prompt. Which behavioral capability should count as convincing evidence that a robot understands a task rather than reproducing a pattern?

Reading notes

Personal takeaways from books that reshape how I frame research questions.

Reading note刘云浩《具身智能》 · CITIC Press · 2025

Reading Embodied Intelligence by Yunhao Liu

My main takeaway is to view embodied intelligence as a transition from describing the world to acting within it. The historical arc—from symbolic and connectionist traditions to foundation models and physical agents—helps separate enduring questions about intelligence from temporary implementation choices.

I read the field through four coupled systems: the “brain” of algorithms and architectures, the “body” of robot hardware and supply chains, the data flywheel created by real interaction, and the standards that make systems comparable and deployable. A breakthrough in only one axis is unlikely to produce robust, scalable embodied intelligence.

Question after reading. How can this system-level view be translated into research benchmarks that reward long-term learning, safe interaction, and real-world value—not only task success?