More autograd helpers

[yaugenst-flex]

• Removes DeprecationWarning of DataArray.values.
• Adds smooth_min, smooth_max, and least_squares implementations to plugins.autograd.functions.
• Adds @scalar_objective decorator that extracts DataArray.data and performs some additional checks on the output data of objective functions. If an objective function is decorated with this, calling DataArray.values within the objective function is not necessary anymore. Our grad and value_and_grad implementations in plugins.autograd wrap objective functions with this decorator automatically.
• Some internal refactoring of the pad function.
• Some test refactoring to reduce the no. of function calls for grad checks.

[tylerflex]

Looks great! think at some point we should include all of these in our demo notebooks. Just a few minor comments.

[e-g-melo]

Hi @yaugenst-flex! I wonder if we can serialize the softmin and softmax functions. It would be nice to include them in GUI.

[yaugenst-flex]

Hey @e-g-melo, the thing is that we currently do not have support for n-ary operators. Currently, every metric is restricted to be scalar. I will look into it though, this is on the todo list.

[e-g-melo]

Hey @e-g-melo, the thing is that we currently do not have support for n-ary operators. Currently, every metric is restricted to be scalar. I will look into it though, this is on the todo list.

I've created a GUI version of this notebook, which has a different softmin function.

In GUI, we can make it like this:

3*(m1*exp(-m1)/(exp(-m1)+exp(-m2)+exp(-m3)) + m2*exp(-m2)/(exp(-m1)+exp(-m2)+exp(-m3)) + m3*exp(-m3)/(exp(-m1)+exp(-m2)+exp(-m3)))

I wonder if we could have something simple, like softmin([m1, m2, m3])

[e-g-melo]

Oh.. I saw your comment on Tom's notebook. Anyway, we can wait, no worries.

[yaugenst-flex]

Yeah we can definitely add that, and it's also the goal to have these more generic functions. But I still have to figure out a good interface for them..