feat(anywidget): Introduce front-end widget lifecycle methods

[manzt]

EDIT: 2024-01-26

Warning

This PR adds a deprecation notice for existing named render exports (i.e., all existing widget implementations). I'm not sure we will deprecate in the future, but migrating away from this would allow us more flexibility in loading widget ESM in the future. Migration:

Remove

export function render({ model, el }) { ... }
^^^^^^

Use

function render({ model, el }) { ... }
         ^^^^^^
export default { render }
                 ^^^^^^

TL;DR- The preferred way to define a widget front-end code is with a default object export. This object may include render function or new initialize function.

export default {
  initialize({ model }) {
    model.on("change:value", () => console.log(model.get("value")));
  },
  render({ model, el }) {
    model.on("change:value", () => el.innerText = model.get("value"))
  }
}

Combined, these methods introduce lifecycle "hooks" for widget developers when implementing front-end code.

• initialize: is executed once when the widget front end first loads. The function has access to the model (no el), which can be set to setup event handlers / state to share across views created in render.
• render is executed per view, or each time an output cell is rendered in a notebook. This function has access to the model and a unique el for the output area. This method should be familiar, and can setup event handlers / access state specific to that view.

The default export can be an object, or a function that returns an object. This can be useful for setting up some initial state for the widget:

export default function () {
  const localState = {};
  return {
    instantiate({ model }) { /* ... */ },
    render({ model, el }) { /* ... */ },
}

Towards:

• Lifecycle methods or non-view methods #266
• Support non-DOM widgets? #388

Motivation

In the MVC framework used by Jupyter Widgets, right now anywidget treats Python as the sole "source of truth" for defining the model and only supports rendering views of that model in JS (view render). We don't currently have an API to initialize some initial front-end model listeners/state (or allow for a DOM-less use of anywidget).

Surveying existing Jupyter Widgets in the wild, it seems this type of model initialization is often defined in the WidgetModel.initialize method. This PR introduces a new lifecycle method which maybe be implemented for an anywidget front-end module: initialize. Initialize has the same signature as render, except only model is provided in the context.

The semantic difference is that initialize is executed once per model (regardless of whether anything is displayed) and render is executed once per view.

Examples

Example 1: Model-level "logger" for state changes

Here, initialize adds an event listener to console.log any time value changes, and then each view adds their own event listener for updating the text node in the DOM. If the console.log were to be setup per-view, then the console would print "Value is X" for every view. Adding the handler in initialize ensures this logic is only handled once per instance.

import anywidget
import traitlets

class Widget(anywidget.AnyWidget):
    _esm = """
   async function initialize({ model }) {
        model.on("change:value", () => {
            console.log(`Value is ${model.get("value")}!`);
        });
    }
    function render({ model, el }) {
        el.innerText = `Value is ${model.get("value")}!`;
        model.on("change:value", () => {
            el.innerText = `Value is ${model.get("value")}!`;
        })
    }   
    export default { initialize, render }
    """
    value = traitlets.Int(0).tag(sync=True)

widget = Widget()
widget

widget

widget.value = 10 # logs once to the browser console and updates both DOM views

Example 2: DOM-less widget

import anywidget
import traitlets

class Widget(anywidget.AnyWidget):
    _view_name = traitlets.Any(None).tag(sync=True) # TODO: nicer way to do this?
    _esm = """
   async function initialize({ model }) {
        model.on("change:value", () => {
            console.log(`Value is ${model.get("value")}!`);
        });
    }
    export default { initialize }
    """
    value = traitlets.Int(0).tag(sync=True)

widget = Widget()
widget.value = 1 # logs in the browser console
widget # displays a plain/text repr
# Widget()

[netlify]

✅ Deploy Preview for anywidget canceled.

Name	Link
🔨 Latest commit	49c50bb
🔍 Latest deploy log	https://app.netlify.com/sites/anywidget/deploys/65b3d19ad6d0350008aa17eb

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (eadc3b0) 98.40% compared to head (a3a9008) 98.43%.
Report is 3 commits behind head on main.

❗ Current head a3a9008 differs from pull request most recent head 49c50bb. Consider uploading reports for the commit 49c50bb to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #395      +/-   ##
==========================================
+ Coverage   98.40%   98.43%   +0.02%     
==========================================
  Files           8        8              
  Lines         440      447       +7     
==========================================
+ Hits          433      440       +7     
  Misses          7        7              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[manzt]

Persisting state among views:

const value_history = [];

export async function unstable_initialize({ model }) {
	model.on("change:value", () => {
		value_history.push(model.get("value"));
	});
}
export function render({ model, el }) {
	let button = document.createElement("button");
	button.innerHTML = `Log history`;
	button.addEventListener("click", () => {
		console.log(`the count history is ${value_history}`);
	});
	el.appendChild(button);
}

This PR changes how ESM is loaded such that widgets can share state within the same module across views. Previously, we freshly re-imported the widget for each render, so each render was "isolated" from each other, now the ESM is loaded once per widget instance.

[manzt]

@davidbrochart & @MarcSkovMadsen, this PR also has the side effect of allowing communication with the widget front end before it is displayed (since setup is run when the widget is first instantiated). See Example 2 above.

• Does not work with Panel/ ipywidgets_bokeh davidbrochart/ipyscore#2

[manzt]

I have renamed this method from setup to initialize to mirror the method it represents in traditional Jupyter Widgets (WidgetModel.initialize), much like how render reflects DOMWidgetModel.render.

[benbovy]

That's great @manzt! Thanks for working on this.

[maartenbreddels]

On my first review this was the only comment I had :)

setup to initialize to mirror the method it represents in traditional Jupyter Widgets

Awesome work @manzt !

[benbovy]

Just wanted to report here that I've been experimenting with this PR and it is working great so far!

[manzt]

That's great to hear!

I’m curious about what you think of allowing (encouraging?) global (module-level state) being shared between the initialize and render hooks. We can do this because we re-load the ESM for each widget instance, so each instance is its own isolated module.

An alternative would be to have a function (like Rollup/Vite plugins) which can create some initial isolated state.

export default function () {
  let foo; // setup some initial shared state (scoped to this function)
  return {
    initialize({ model }) {
      foo = "bar";
    },
    render({ model, el }) {
      console.log(foo);
    },
  }
}

Although I like the named exports, I think this would give us more flexibility going forward to change how the ESM is loaded by the anywidget "runtime" since we could load the ESM once for a python class. Just an idea (and why I've been dragging my feet on merging!).

[benbovy]

Hmm I like the isolated state option but I'm afraid I'm not familiar enough with the front-end JS ecosystem to have a proper opinion.

How this approach vs. global module-level state would work with the multiple widgets per ESM workaround described in #382 (comment)?

[manzt]

How this approach vs. global module-level state would work with the multiple widgets per ESM workaround described in #382 (comment)?

I suppose you could delegate the widget from initialize in the default export:

function tableWidget() { /* ... */ }

function chartWidget() { /* ... */ }

const widgetRegistry = { tableWidget, chartWidget };

export default function () {
  let widget;
  return {
    initialize({ model }) {
      const createWidget = widgetRegistry[model.get("component")];
      widget = createWidget();
      return widget.initialize({ model });
    },
    render({ model, el }) {
      return widget.render({ model, el });
    }
  }
}

A little bit of boilerplate, but fairly minimal.

[keller-mark]

I prefer the simplicity of the named render and initialize functions (especially for users who are unfamiliar to JS, they may get tripped up by the need to define a function that exports an object; there are many curly brackets to keep track of). I would imagine that users will copy/paste the boilerplate from the README/docs and modify since it may be tricky to remember.

I am not sure I see the benefit of defining shared state within vs. outside of the function - does it have to do with HMR?
Would this not work?

function tableWidget() { /* ... */ }

function chartWidget() { /* ... */ }

const widgetRegistry = { tableWidget, chartWidget };

let widget;

export function initialize({ model }) {
  const createWidget = widgetRegistry[model.get("component")];
  widget = createWidget();
  return widget.initialize({ model });
}
export function render({ model, el }) {
  return widget.render({ model, el });
}

I would not be opposed to the default function export way if there are benefits. It would probably be confusing to support both options (named function exports and default function that returns an object), so if the default function export way provides benefits I would go all-in on that mechanism rather than supporting both.

[keller-mark]

I thought more about this, and I think the recommended upgrade approach would be for "average" users (who do not need the function-scoped state), they would just need to add an extra line

function render({ model, el }) {
  
}
export default () => ({ render });

or with an initializer:

function initialize({ model }) {
  
}
function render({ model, el }) {
  
}
export default () => ({ initialize, render });

which seems pretty concise.

[manzt]

Thanks for the feedback everyone! I plan to merge this week, just need to write up some release notes, documentation, and consider how to deprecate the render API.

I am not sure I see the benefit of defining shared state within vs. outside of the function - does it have to do with HMR?

Currently there isn't really a clear benefit to either choice. However, the function would be a future design consideration, which I think would allow us to avoid potential future breaking changes.

For example, the anywidget "runtime" currently imports a fresh, isolated module per widget instance (i.e., Python class instantiation). This means that things like importing dependencies is duplicated each time a widget in instantiated.

Having the shared state within the function would allow us experiment with changing the runtime, and avoid breaking changes for end users. For example, we could execute this top-level logic once per widget defintion, meaning that the ESM could define local state that could be shared across all instances.

With top-level await, this could include additional setup/logic that might be useful:

(This code is not real and doesn't work)

import * as d3 from "https://esm.sh/d3";

const globalSharedData = await fetch('https://jsonplaceholder.typicode.com/todos/1').then(res => res.json());
const globalSharedState = {};

export default function () {
  const localState = {};
  return {
    instantiate({ model }) {},
    render({ model, el }) {},
}

I thought more about this, and I think the recommended upgrade approach would be for "average" users (who do not need the function-scoped state), they would just need to add an extra line.

Ah, I didn't even realize you could use arrow functions like that. I guess we could also try something akin to Vite's configuration where you can export an object or a function that returns an object (or a promise for an object):

export default {
   plugins: [],
}

export default () => {
  return {
    plugins: []
  }
}

for the runtime, its easy enough to have:

const mod = await import(esm);
const widget = typeof mod.default === "function" ? (await mod.default()) : mod.default;

[manzt]

Thanks for all the feedback here! I have updated the PR description and implementation. Will merge and then move to updating the documentation for the preferred API.

[manzt]

Want to ping some of the existing widget implementations with anticipated deprecation notice. To be clear, I'm not sure we will deprecate but this provides a more flexible interface for future changes.

cc: @domoritz @juba @jonmmease

Warning

This PR adds a deprecation notice for existing named render exports (i.e., all existing widget implementations). I'm not sure we will deprecate in the future, but migrating away from this would allow us more flexibility in loading widget ESM in the future. Migration:

Remove

export function render({ model, el }) { ... }
^^^^^^

Use

function render({ model, el }) { ... }
         ^^^^^^
export default { render }
                 ^^^^^^

[domoritz]

Thanks for the ping. I'll update mosaic when there is a release.

[Judekeyser]

Hello, Just to confirm, what is the least minimal version where this change is not breaking existing code?
In other words: if I migrate right now my widgets to this newer API, what is the least version I can claim to support (assuming I only use this feature, for simplicity) ?

Thank you in advance!

[manzt]

what is the least minimal version where this change is not breaking existing code?

Great question!

The API is of 0.9.0. You can pin anywidget>=0.9.0 in your pyproject.toml etc. I hope that answers your question.

[flekschas]

The API is of 0.9.0. You can pin anywidget>=0.9.0 in your pyproject.toml etc.

@manzt If possible, it'd be nice to include this in the deprecation warning :)