Dynamic Movement Primitve - My Superficial Review

Let’s talk about the Dynamic Movement Primitive (DMP) for robots learning from demonstration. In this article, we make an assumption that you readers all have the background of control theory and robotics. (Updating…)

文章目录

• The Basics about DMP
• Learn a DMP: LWR

The Basics about DMP

Dynamic movement primitives (DMPs) are a method of trajectory control / planning. It was motivated by the desire to find a way to represent complex motor actions that can be flexibly adjusted without manual parameter tuning or having to worry about instability.

To begin with, we take some time to discuss 2nd order dynamic system as follows:
$\tau\dot{y}=z$τy˙​=z $\tau\dot{z}=\alpha(\beta(g-y)-z)$τz˙=α(β(g−y)−z) For the sake of simplicity, we take $\tau = 1$τ=1. Then the equations above reduces to a time-invariant linear system which has been deeply studied in linear control theory. The only equilibrium of the system above is $z=0,y=g$z=0,y=g. Here we take $g$g as the goal position, $\alpha,\beta$α,β are constant parameters to be chosen such that the system is strictly damping (e.g. $\alpha > 4\beta > 0$α>4β>0). This is why the DMP is always stable at the goal position — the dynamics of the system is always dominated by a stable linear system.

To make the DMP fit any trajectory, it is necessary to add some terms in the equations above without any effects on its stability:
$\tau\dot{y}=z$τy˙​=z $\tau\dot{z}=\alpha(\beta(g-y)-z) + f(x)$τz˙=α(β(g−y)−z)+f(x) where $\tau \dot{x} = -\alpha_{x}x$τx˙=−αx​x $\alpha_x>0$αx​>0 is a constant parameter of the dynamics of $x$x that is also called a cannonical system. As we can see, $x$x will inevitably reduces to zero with time elapsing. If $f(x)$f(x) is smooth and satisfies $f(0) = 0$f(0)=0, the stability of the DMP would not be damaged by the added term $f(x)$f(x). We can define the nonlinear function $f$f (also called ‘forceing function’) as:
$f(x) = \frac{\sum_{i=1}^{N}\psi_{i}w_{i}}{\sum_{i=1}^{N}\psi_{i}}x(g-y(0))$f(x)=∑i=1N​ψi​∑i=1N​ψi​wi​​x(g−y(0)) where $\psi_{i} = \exp(-h_{i}(x-c_{i})^{2})$ψi​=exp(−hi​(x−ci​)2)
$w_{i}$wi​ is a weighting for a given basis function $\psi_{i}$ψi​. You may recognize that the $\psi_i$ψi​ equation above defines a Gaussian centered at $c_i$ci​, where $h_i$hi​ is the variance. So our forcing function is a set of Gaussians that are ‘activated’ as the canonical system $x$x converges to its target.

The paramter $\tau$τ can be used as a temporal scaling term. To slow the system down you set $\tau$τ greater than 1 while set it between 0 and 1 to speed the dynamics up.

To examplify the basic DMP, we show a diagram when $w=0$w=0:

Learn a DMP: LWR

Now we have a forcing term that can make the system take a weird trajectory as it converges to a target point, and temporal and spatial scalability. How do we set up the system to follow a trajectory that we specify? If we are given a trajectory: $\{y_d,\dot{y}_d,\ddot{y}_d\}${yd​,y˙​d​,y¨​d​}, $f_d = \tau^2\ddot{y}_d-\alpha(\beta(g-y_d)-\tau\dot{y}_d$fd​=τ2y¨​d​−α(β(g−yd​)−τy˙​d​ And we know that the forcing term is comprised of a weighted summation of basis functions which are activated through time, so we can use an optimization technique like locally weighted regression to choose the weights over our basis functions such that the forcing function matches the desired trajectory $f_{d}$fd​. In locally weighted regression sets up to minimize:
$J_i=\sum_{t=0}^{T}\psi_i(t)(f_d(t)-w_{i}\xi(t))^2$Ji​=t=0∑T​ψi​(t)(fd​(t)−wi​ξ(t))2 where $\xi(t) =x(t)(g-y_d(0))$ξ(t)=x(t)(g−yd​(0)) Let
$S = (\xi(0),\xi(1),\dots,\xi(T))^{T}, \Tau_i = \mathrm{diag}(\psi_{i}(0),\psi_{i}(1),\dots,\psi_{i}(T))$S=(ξ(0),ξ(1),…,ξ(T))T,Ti​=diag(ψi​(0),ψi​(1),…,ψi​(T)) and $F_d = (f_d(0),f_d(1),\dots,f_d(T))^{T}$Fd​=(fd​(0),fd​(1),…,fd​(T))T Then, in a compact form: $J_{i} = (F_d-w_{i}S)^{T}\Tau_i(F_d-w_iS)$Ji​=(Fd​−wi​S)TTi​(Fd​−wi​S) that can be solved as $w_i = (S^T\Tau_iS)^{-1}S^{T}\Tau_iF_d$wi​=(STTi​S)−1STTi​Fd​
The performance is illustrated by the following diagrams: ($t$t is the desired trajectory, $y$y is generated by DMP)


That’s for exactly following a given trajectory, which is often not the case. The strength of the DMP framework is that the trajectory is a dynamical system. This lets us do simple things to get really neat performance, like scale the trajectory spatially on the fly simply by changing the goal, rather than rescaling the entire trajectory

---

Thanks

• https://studywolf.wordpress.com/2013/11/16/dynamic-movement-primitives-part-1-the-basics/
• Ijspeert A J, Nakanishi J, Hoffmann H, et al. Dynamical movement primitives: learning attractor models for motor behaviors[J]. Neural computation, 2013, 25(2): 328-373.
• Schaal S, Mohajerian P, Ijspeert A. Dynamics systems vs. optimal control—a unifying view[J]. Progress in brain research, 2007, 165: 425-445.