Versioning changes

[jasongrout]

The model specification for a widget (i.e., the list of model traits) is implemented at least two places: the kernel and the frontend. Because we want different frontend and kernel-side implementations to exist, the model state specification should probably not be tied to any particular package name or version. Also, I think the view should be defined by the model version it supports and the name/module. (i.e., the version should essentially be '*'). In order for that to happen, we probably need to get rid of custom messages (which are sent from the view to the kernel) and instead route all of that information through the state.

The model version should only support the major part of the version number (i.e., essentially be an integer). Any change in the actual attributes of the models (whether adding, deleting, or even changing intent of the attribute) should be a breaking change in the model specification.

CC @SylvainCorlay

[SylvainCorlay]

Thanks for opening this. I want to give this the attention it deserves but cannot give a thorough response today. Will get back to this tomorrow.

[jasongrout]

One problem we have the current versioning system is that we really have at least three different things we need to version in the jupyter-js-widgets package:

1. The version of the widget protocol that the widget manager speaks.
2. The version of the model specification for the core widgets (i.e., the attributes and what they mean).
3. The version of the views (this also impacts the models, since the views can send a custom message that will be received on the python side).

We could simplify the view version to just be the model version the view understands. In order to do that, we really need the model to encapsulate all messages that can go between the kernel and frontend, which we could do by making everything go through the state attributes (for example, for button clicks, a new button state attribute that is number of times clicked).

[jasongrout]

See #1235 for more comments.

[SylvainCorlay]

I am ready to discuss this now!

[SylvainCorlay]

So from the in-person discussions with @jasongrout about the protocol specifications

Asymmetry

There are a few intrinsic reasons why the widget protocol will remain somewhat asymmetrical.

• The back-end can be long-running, while the front-end is transient, and may connect to an already existing kernel. This is why we have the request_state message in the protocol, which is only sent by the front-end to the back-end.

Overriding core model modules and view modules

• We have a necessity of language agnosticism in the back end. Hence, for the widget creation message from the backend, the class_name has be be meaningful for any language. In order to not depend on any specific language syntax for addressing modules, we must have an registry of widget types on the backend. On the other side, we have not met a use case for another front end than a web front end.

• Still if, for example, @rgbkrk works on an alternative implementation of views for the core widgets so that they have a more native feel for nteract, he would probably still want to support the loading of jupyter-js-widgets because js views and models of most third-party widget libraries (bqplot / pythreejs / ipyleaflet / ipyvolume / nglview) all inherit from jupyter-js-widgets bases. I wonder what would be the cleanest way to both hijack the current mechanism for core widgets and reuse the third-party widgets that inherit from core widget views in nteract.

The @register decorator

• I am not sold on the removal of the python decorator @register which was the means to register a new widget class in the back end registry. @jasongrout already met a case where this removal was causing an issue in terms of disambiguation of two python widgets having the same view and models on the js side.

[jasongrout]

Here's a case where we want a model spec version number to have a minor version bump: if we add a widget to a package, the _model_module_version increments in a backwards-compatible way.

[jasongrout]

@jasongrout already met a case where this removal was causing an issue in terms of disambiguation of two python widgets having the same view and models on the js side.

And I think that is a problem that needs to be solved in the widgets themselves (i.e., disambiguating the models), not in the registration string.

[jasongrout]

I should add that the model state specification for a given model must include the default values for each field - since those default values are not stored in the persisted serialization in the notebook, for example, they should be specified as part of the model name/module/version metadata. This means we can't have two widgets that have the same model/view metadata, but different defaults for the model fields.

[SylvainCorlay]

If no default is specified, no value will be dropped upon serialization.

[jasongrout]

I think the two fundamental tensions going on in the *_module_version numbers is that they are used to determine both compatibility with a spec and also give information about what package to dynamically load if the package is not registered. If we support dynamic loading of modules, we need a package version number, so perhaps we leave *_module_version for that, and we add a new field for spec version number (_(view/model)_spec_version?), which is used preferentially for registered modules.

Then if the kernel-side implementation specifies a _model_spec_version, then we first look in the registered models for that _model_name, _model_module, _model_spec_version. That way no matter what package/version actually loaded the model, we load a compatible model spec. If we don't find a registered model, we can either throw an error (if we don't support dynamic loading), or we can load the package given by _model_module_name, _model_module_version, which guaranteed does have the appropriate model spec number (and we verify this).

(Also, we should really call it *_package_version instead of *_module_version...)

[jasongrout]

Okay, a couple more alternatives:

(a) new more backwards compatible thought: introduce a new optional _model/view_package_version and _model/view_package that (a) defaults to the _*_module_* attributes, but lets us specify the package if it is different from the 'module' name/version. This means that basically existing packages are unchanged (if their models/view names/versions are basically package names/versions), but lets us be a little more fine-grained for things like ipywidgets.

(b) split up jupyter-js-widgets into a base package (responsible for the widget protocol via the base widget manager), a models package (implementing the model specs), and a views package (implementing the view specs). And maybe for backwards compatibility, leave jupyter-js-widgets as a package that depends on the three and exports them all. Then the models, views, and widget protocol can advance separately.

[SylvainCorlay]

I prefer (a). There is a limit to the splitting of the packages for maintainability. It also provides a clear path forward for third-party widgets.

[jasongrout]

One thing about (a) is that, since the _*_package_* will be specified on the ipywidgets repo, it will be inherited by everyone's widgets, which means everyone will need to add these fields so they aren't wrong.

[jasongrout]

Fixed in #1313.