Miscellanous questions about learned contraction metric

[dtch1997]

I have a couple questions about learned contraction metrics in train_cm.py:

1. Is it correct that, in order for the contraction metric to be fully valid, M_loss must converge to 0?
   a. Corollary, if M_loss > 0, does it imply that there exists some x, x*, u* in the dataset on which the learned metric is not a valid contraction metric?
   b. Do you observe empirically that the M_loss eventually goes to 0?
2. Is the setting used in the script the same as that used in the paper?
3. (Edit) Another question based on the paper: At the end of section V-E, it is said that "it is possible to deploy a non-verified Lyapunov or barrier certificate safely using a real-time safety monitor" where the corresponding conditions are continuously checked online. Is it possible to do the same thing for a learned contraction metric?

Thanks!

[dawsonc]

1. This is essentially true. Technically, M_loss computes the matrix on the LHS of eq 16 in the paper and makes sure that it's maximum eigenvalue is less than -0.1, so if your maximum eigenvalue is -0.01 then the metric should still be valid but will have positive M_loss (but empirically, when the maximum eigenvalue is close to 0 then the contraction starts getting unacceptably slow in the direction of the corresponding eigenvector).
   a. Yes, with the same -0.1 caveat from above.
   b. For simple dynamics (like the ones we use), I would expect to see M_loss go to zero. If it doesn't, then I would check the empirical performance of the controller to see if it's "good enough" (the code here should log plots of controller performance to tensorboard throughout training). Sometimes you get situations where the metric isn't valid near the boundary of the training space but it's valid on the interior.
2. The experiments in our survey paper were run using the train_cm.py script.
3. The safety monitors for Lyapunov/barrier functions are pretty trivial: you approximate the derivative of V (or h) along the current trajectory using sequential samples, and if the derivative stops satisfying the relevant condition (eq 4c or 12) then you flag a safety violation. A contraction metric-based monitor would be less trivial since you would need to check that the distance between nearby trajectories is shrinking, so you can't do it using just the trajectory that you're currently experiencing, you need some way of "imagining" (or simulating) the behavior of nearby trajectories.

[dawsonc]

I'm happy to answer these more research-y questions, but let's move them over to the (newly activated) discussion section of this repo and reserve issues for bugs/feature requests for the code itself.

[dtch1997]

Thanks @dawsonc ! That helps me understand the behaviour of the contraction metric a bit more - even if it is not globally valid, it seems like we can still hope to have it valid for reference trajectories in desired subsets of the state space. Using that insight, I would like to implement some functionality to visualize valid / invalid regions of the learned contraction metric based on some online criterion, defined below:

The metric proves that $x$ is converging to $x^*$ if for all $x_p$ in a small ball $B_\epsilon(x)$ we have that $\frac{d}{dt} d(x_p, x^*) < 0$, where $d(a, b) = (a-b)^TM(a)(a-b)$.

To evaluate this, I'm thinking of using an empirical approach where I sample multiple points in the surrounding ball, evaluate the learned controller at all of them, and check that the distance is decreasing in all cases to prove that the metric is valid at a single point $x$. By repeating this many times for many points $x$ across the state space, I can obtain the valid and invalid sets of the metric w.r.t a given trajectory $x^*$.

Would the approach described above be correct?