At the ICML 2018 conference there was a very interesting tutorial on Imitation Learning by Yisong Yue and Hoang Le from CalTech. It is quite similar to Reinforcement Learning but with an expert that the machine wants to imitate by inferring a policy that links states to actions. Imitation Learning can be applied to sequential decision making problem made by humans or other algorithms.
There are different categories of Imitation Learning:
- Behavioral Cloning which is supervised learning on the state-action pairs of the expert
- Direct Policy Learning (Interactive Imitation Learning) with interaction with an expert
- Inverse Reinforcement Learning which is reinforcement learning applied to an inferred reward function from demonstrations
Direct Policy Learning can use Sequential Learning Reduction algorithms such as Data Aggregations (DAgger) and Policy Aggregations (SEARN & SMILe).
According to the presenters Imitation Learning seems to be easier to implement that Reinforcement Learning. A limitation is that the machine cannot do better than the expert. The talk is here: