Alternative implementation of TV norm

[bwohlberg]

Alternative implementation of TV norm.

[shnaqvi]

Thanks @bwohlberg , I love the simplicity and generalization of this implementation over arbitrary inputs. I was confused in 2 places where I've added comment. I'd appreciate further clarification there.

[shnaqvi]

That's a clever way of doing addition and difference operations over arbitrary num. of dimensions. But does it also handle the shifting by 1 which we were doing with np.roll operator in the earlier implementation?

[bwohlberg]

We don't need it here because the shift-invariant version of the highest-resolution Haar decomposition is computed by convolution, without the downsampling that would follow for the standard (shift-variant) Haar decomposition.

[shnaqvi]

I don't see how the proximal of the L1 Norm equivalent to the shrinkage operator we had defined in earlier implementation, which was taken the maximum over the difference from zero and the threshold?

[bwohlberg]

According to my reading of the paper, the recommended shrinkage function is the prox of the $\ell_1$ norm. (A more standard form would be to replace $y / |y|$ with $\text{sign}(y)$.)

[shnaqvi]

Yes that's correct. My question was more about the implementation of prox of the $\ell_1$ norm in scico. How is that equivalent to $\max \left( \left| {y} \right| - \lambda, 0 \right)\ \text{sign} \left( {y} \right) $? In particular, how is line 101 (tmp = 0.5 * (tmp + snp.abs(tmp)), equivalent to tmp = snp.max(tmp, 0)? That's what I'm not seeing. Thanks!

[bwohlberg]

Yes, that is rather opaque. It works because $(z + |z|)/2 = \text{max}(z, 0) = (z)_+$. If $z \geq 0$ then $|z| = z$ so $(z + |z|)/2 = z$, and if $z < 0$ then $|z| = -z$ so $(z + |z|)/2 = 0$.

[bwohlberg]

OK to merge?

[shnaqvi]

Yes just verified that it gives the same result on our test example. I have a question about how convolution is being carried out rather effectively using the VerticalStack operators in scico, but we’re ok to merge. 😊