Building Machines that Learn and Think Like People

MIT Prof. Josh Tenenbaum gave a talk on Building Machines that Learn and Think Like People at the ICML 2018. His insight is that it is possible to teach a machine to learn like a child by using:

  • Game engine intuitive physics
  • Intuitive psychology
  • Probabilistic programs
  • Program induction
  • Program synthesis

This agenda is more ambitious that the current state of machine learning though it resembles more old-style machine learning an there is no guarantee that it will succeed. Still it is refreshing that we can learn from young humans to teach machines.

Imitation Learning

At the ICML 2018 conference there was a very interesting tutorial on Imitation Learning by Yisong Yue and Hoang Le from CalTech. It is quite similar to Reinforcement Learning but with an expert that the machine wants to imitate by inferring a policy that links states to actions. Imitation Learning can be applied to sequential decision making problem made by humans or other algorithms.

There are different categories of Imitation Learning:

  • Behavioral Cloning which is supervised learning on the state-action pairs of the expert
  • Direct Policy Learning (Interactive Imitation Learning) with interaction with an expert
  • Inverse Reinforcement Learning which is reinforcement learning applied to an inferred reward function from demonstrations

Direct Policy Learning can use Sequential Learning Reduction algorithms such as Data Aggregations (DAgger) and Policy Aggregations (SEARN & SMILe).

According to the presenters Imitation Learning seems to be easier to implement that Reinforcement Learning. A limitation is that the machine cannot do better than the expert. The talk is here: