Abstract

Most manipulation tasks can be decomposed into
a sequence of phases, where the robot’s actions have different
effects in each phase. The robot can perform actions to
transition between phases and, thus, alter the effects of its
actions, e.g. grasp an object in order to then lift it. The robot
can thus reach a phase that affords the desired manipulation.
In this paper, we present an approach for exploiting the
phase structure of tasks in order to learn manipulation skills
more efficiently. Starting with human demonstrations, the robot
learns a probabilistic model of the phases and the phase
transitions. The robot then employs model-based reinforcement
learning to create a library of motor primitives for transitioning
between phases. The learned motor primitives generalize to new
situations and tasks. Given this library, the robot uses a value
function approach to learn a high-level policy for sequencing
the motor primitives. The proposed method was successfully
evaluated on a real robot performing a bimanual grasping task.