Async data

[Rich-Harris]

Been refactoring https://svelte.technology — inching towards a more declarative/universal way of handling routing etc.

One thing that stands out is that there's really no good way for components to declare their own data dependencies in a universal way. So you have to have a dedicated handler per route, in case you need to do something like fetch a blog post's JSON.

It would be nice if that stuff could sit in the component itself. You'd still have to handle data fetching differently between server and client, but at least you'd be able to declare all your routes declaratively (up to a point, anyway).

Maybe something like this:

<script>
  export default {
    preload (params) {
      if (isServer) { // TODO figure out what this actually looks like...
        return require(`../public/blog-posts/${params.slug}.json`);
      } else {
        return fetch(`/blog-posts/${params.slug}.json`).then(r => r.json());
      }
    },

    data (preloaded) {
      return {
        post: preloaded
      };
    }
  };
</script>

This only really works for top-level components — it could get pretty crazy if you had something like this:

{{#if foo}}
  <ComponentThatPreloadsStuff/>
{{/if}}

Thoughts?

[PaulBGD]

How I handle it in a large project is I generate the state that the route uses on the server (from redis, MySQL, other APIs, etc) then pass it as JSON through a script element. That ensures that both SSR and the client are receiving the same state.

[Rich-Harris]

That makes sense for the initial load — how do you handle client-side route changes?

Not sure I got the API quite right above, I don't think there's a great way to instantiate the component if it's set up like that, we probably don't want new Component to behave any differently. Something like this could work:

BlogPost.preload({ slug: 'components-with-async-data' }).then(post => {
  new BlogPost({
    target,
    data: { post }
  });
});

// generalised
Promise.resolve(Component.preload ? Component.preload(params) : {}).then(data => {
  new Component({ target, data });
});

This only really works for top-level components — it could get pretty crazy if you had something like this:

We might actually need to solve this problem, now that I think about it. For example, the component that defines the /blog/:slug route on https://svelte.technology contains a <Page> with a nested <BlogPost> component. It would be nice if <Page> was responsible for getting the value of hashed (the hashed filenames of bundle.js and main.css) and <BlogPost> was responsible for turning the slug into a post — the component that contains both shouldn't have to worry about either.

Any good examples of how other frameworks have solved this?

[PaulBGD]

Honestly, is there anything preventing us from using redux + redux thunk? Seems to handle that case quite nicely. In theory if #469 ever gets implemented we could have svelte-redux or something to have first class support for redux as a library.

[Rich-Harris]

What would that look like in practice (i.e. from the point of view of writing the code that lives inside your route handlers on both server and client)? I'd be wary of hitching our wagon to a specific state management tool, that's a holy war I don't really want to pick a side in!

Besides, it seems that the hardest problem here isn't implementation so much as design.

[PaulBGD]

What this issue seems to be is a way for a component to load data before it's mounted. A clean way to solve that is #469 but the current way to solve it is to use any state management library + just a if check if the data is loaded.

[Rich-Harris]

#469 is purely about element-level lifecycle hooks, do you mean a different issue?

By 'if check' do you mean in the template, as in {{#if foo}}...{{/if}}? That can work in the client (though it's not ideal in all circumstances — you might want to preload data for the new view before destroying the old one) but less so on the server. I might have misunderstood you?

[PaulBGD]

I might be confused what this is suggesting. Isn't it passing data to a component to load before the component is mounted?

[Rich-Harris]

At a high level it's just saying that there needs to be an idiomatic way within the component definition to express asynchronous data dependencies, that applies in both server- and client-side contexts

[briancray]

Is this just as simple as checking if data returns a promise? If so wait for it to resolve before mounting? Perhaps combine that with a mountRequiresData boolean so the user can return a promise for async data but still have it mount if desired?

[TehShrike]

In my mind this is very strongly something that should be implemented with dynamic components (#640).

As long as the dynamic component API is expressive enough, people can pick from a few general-case async-loading components, or make one that fits their specific case.

[Rich-Harris]

@TehShrike was that a response to this comment on Gitter? I think it's a different problem to that of async data. I should make a separate issue for <:Async>, Gitter not the right place for it

[TehShrike]

Oh, sure, that's another case that I think should be handled entirely by dynamic components instead of adding new syntax.

But no, I really do think that async loading of data is also definitely something that should be achievable/achieved with dynamic components.

[aubergene]

Just an idea, can data (and computed) accept a promise and then add a way of switching based on the state of the promise to the templating?

{{#pending foo}}
Loading foo...
{{#then foo}}
Foo has a value of {{foo}}
{{#catch foo}}
No foo this time
{{/foo}}

<script>
  export default {
    data () {
      return {
        foo: () => fetch(url).then(d => d.json())
      };
    }
  };
</script>

and then call it like

<ComponentThatLoadsStuff url="//foo.com?post=123"/>

[Rich-Harris]

@aubergene Dude! I think you're onto something here.

Elsewhere in Svelte (and Handlebars generally), # and / (opening and closing blocks) are symmetrical (see #if...elseif...else.../if), so it would probably be more like this...

{{#pending foo}}
  Loading foo...
{{then value}}
  Foo has a value of {{value}}
{{catch err}}
  No foo this time (because {{err.message}})
{{/pending}}

...but I think the idea of solving this problem with syntax is brilliant. Maybe pending is the wrong name, since that's just one possible state. Maybe await?

Could also have a shorthand like this, if you didn't care about pending/error states:

{{#await foo then value}}
  Foo has a value of {{value}}
{{/await}}

One thing to think about: often, you might want to preserve the previous value until the new one has resolved, for example a list of autocomplete suggestions (you don't want them to disappear each time you type a character):

<input list='suggestions' bind:search>

{{#await suggestions}}
  <span>loading...</span>
{{then value}}
  <datalist id='suggestions'>
    {{#each value as suggestion}}
      <option>{{suggestion}}</option>
    {{/each}}
  </datalist>
{{catch err}}
  <span class='error'>could not get suggestions!</span>
{{/await}}

<script>
  export default {
    computed: {
      suggestions: search => fetch(`/suggestions.json?q=${search}`).then(r => r.json())
    }
  };
</script>

How might we express that idea — that we want the 'loading...' text initially, but thereafter we should preserve the resolved value until another one resolves?

[Rich-Harris]

One other thing to ponder: how would this work with SSR? If we had an async SSR renderer it could stream HTML to the client (i.e. it sends what it has until it hits the #await section, then gets the data and carries on streaming) that contained all the relevant data.

Then, when the client-side JavaScript kicks in without a promise representing the data, it would presumably have no choice but to display the pending state when it hydrated the markup. I'm not sure what a good solution to that would be.

Probably getting ahead of ourselves, just something to be mindful of.

[Rich-Harris]

Then, when the client-side JavaScript kicks in without a promise representing the data, it would presumably have no choice but to display the pending state when it hydrated the markup.

One possible solution: if the awaited value isn't a promise (i.e. isn't 'thenable'), skip to the then section. That way you can wait for the data before hydrating, and all will be well.

[TehShrike]

Reminds me of this excellent post on different UI states: http://scotthurff.com/posts/why-your-user-interface-is-awkward-youre-ignoring-the-ui-stack

First-class promise support would make handling three of those states way easier.

I would prefer the words used being then/catch or resolved/rejected, either one of the two pairs of words you use when interacting with promise objects.

{{#then foo as value}}
  Foo has a value of {{value}}
{{#catch err}}
  No foo this time (because {{err.message}})
{{#else}}
  Loading foo...
{{/then}}

There'd be stuff to talk about (If there's no catch section, should the else block be used for both error and pending states?), but as long as the syntax uses one of those two pairs of words, I should remember it pretty easily.

[Rich-Harris]

Good read, thanks. Yeah, 'first-class' is a good phrase to use here.

Interesting. I always use 'fulfil' rather than 'resolve' in my Promise callbacks — I always considered resolution as a term that described either fulfilment or rejection (also, it's less easy to trip up when you tab-complete from re...).

For that reason I think then/catch make more sense than resolved/rejected — no ambiguity, and easier on the fingers. (Also, resolved maybe doesn't have the same strong async connotations.)

To me it makes sense to think about the loading state first, then you think about what to do with the eventual value (or error). The developer's chronology matches the user's. I think an else section for the loading state makes it feel a bit more optional — literally an afterthought — whereas with the #await keyword your first consideration is what needs to happen while we're awaiting the value.

If there's no catch section, should the else block be used for both error and pending states?

Good question. I think probably yes (whether it's else or #await) but there's no 'right' answer here, it's just a bug that the developer needs to fix. If they didn't, then a perpetual loading state would arguably feel less broken to an end user than chunks of the UI just disappearing altogether.

Plenty of room to bikeshed that stuff anyway! I've been thinking about this question...

How might we express that idea — that we want the 'loading...' text initially, but thereafter we should preserve the resolved value until another one resolves?

...and I think I have an answer that I like — #await once:

{{#await once suggestions}}
  <span>loading...</span>
{{then value}}
  <datalist id='suggestions'>
    {{#each value as suggestion}}
      <option>{{suggestion}}</option>
    {{/each}}
  </datalist>
{{catch err}}
  <span class='error'>could not get suggestions!</span>
{{/await}}

In that case, we could add a second argument to the then block indicating whether a new value is pending:

{{#await once suggestions}}
  <span>loading...</span>
{{then value, pending}}
  <datalist id='suggestions' style='opacity: {{pending ? 0.5 : 1}}'>
    {{#each value as suggestion}}
      <option>{{suggestion}}</option>
    {{/each}}
  </datalist>

  {{#if pending}}
    <span>updating...</span>
  {{/if}}
{{catch err}}
  <span class='error'>could not get suggestions!</span>
{{/await}}

Feels that would solve a lot of fairly common (but non-trivial) UI problems that would normally have you reaching for RxJS.

[TehShrike]

#await once seems a bit magical to bake into the language to me, but maybe the use cases are there to justify it?

I love the idea of the templating language being aware of promise state, but as soon as it's juggling old versions of state and new versions of state with the same name, my mental model gets sticky.

What happens to {{#await once foo}} if I set foo to a promise, that promise resolves, and then later I do this?

set({ foo: startAsyncJob('A') })
set({ foo: startAsyncJob('B') })
set({ foo: startAsyncJob('C') })

We'll say that first, job C resolves, then A rejects, and then later B resolves.

If I understand the {{#then}} functionality to just be giving me useful if/then checks around promise state, then it's simple, foo is the last promise it was set to, and the view will reflect that promise's state.

But if I understand the once functionality to mean "after the first time a loading state is shown, it will never be shown again", I need to come up with a mental model for how old values of foo will be kept around.

For my own sanity, if I had a view that actually allowed for those sorts of launching-async-jobs-arbitrarily flows, I'd do it in some sort of state-management safe zone outside of Svelte anyway.

I'm a bit sleepy so that may not all make sense :-x feel free to question my statements here.

At the very least, this seems like the sort of thing that could be added later if that remained a pain point for folks using whatever the initial await/then/catch syntax is.

[TehShrike]

To me it makes sense to think about the loading state first, then you think about what to do with the eventual value (or error)

That makes sense, but I've found that in my code, I end up caring less about the order of the code inside my if/else blocks, and more about the conditions being clear/obvious (e.g. avoiding double negatives like if (!notCool).

So my instinct is to just push for each {{#condition}} block to read as nicely as possible and not put too much importance on the order, but I can see how other folks could prefer specific orderings.

[aubergene]

Glad you like the idea. I actually expected that Svelte already worked like this, seemed fairly intuitive and so I was rummaging around wondering why passing a promise wasn't working for me.

I like your await, then, catch syntax, I knew you'd think of something better.

I'm not sure about the once idea. In your example, as a end user I think I'd prefer it to remove the dataList and go back to loading, as that bit of UI is quite probably invalid at that point. On a slow connection if I searched sh and then change to st then it's going to get confusing if I can still select one of the previous options before the fetch returns. Also once isn't a keyword and so it feels a bit odd being an optional floaty magic word in the template.

I'm not the greatest expert on promises but couldn't something like this be achieved in the JS with a helper which wraps the initial promise and you can pass in a new one but it returns the result of the current one until the new one resolves? That seems like a more generally helpful thing.

[Rich-Harris]

Fair enough, maybe once is a step too far into magical territory. I don't think you would want the existing UI to vanish if you were building something like an autocomplete — if you go to google.com and start typing 'why do birds suddenly appear' the list is updated in place with a slight lag, rather than removed and re-added between 'why do' and 'why do birds'. The equivalent without once would be something like this:

<input list='suggestions' bind:search>

{{#if error}}
  <span class='error'>could not get suggestions!</span>
{{elseif suggestions}}
  <datalist id='suggestions'>
    {{#each value as suggestion}}
      <option>{{suggestion}}</option>
    {{/each}}
  </datalist>
{{else}}
  <span>loading...</span>
{{/if}}

<script>
  export default {
    oncreate() {
      let token;
      this.observe('search', search => {
        const currentToken = token = {};
        fetch(`/suggestions.json?q=${search}`)
          .then(r => r.json())
          .then(suggestions => {
            if (currentToken !== token) return; // responses arrived out of order
            this.set({ suggestions, error: null });
          }, err => {
            this.set({ error, suggestions: null });
          });
      });
    }
  };
</script>

What happens to {{#await once foo}} if I set foo to a promise, that promise resolves, and then later I do this?

set({ foo: startAsyncJob('A') })
set({ foo: startAsyncJob('B') })
set({ foo: startAsyncJob('C') })

We'll say that first, job C resolves, then A rejects, and then later B resolves.

In that case, you would go straight from awaiting to resolved with C. A and B would be discarded, because as of the third line they're no longer valid values for foo. That would be true with or without once.

At the very least, this seems like the sort of thing that could be added later if that remained a pain point for folks using whatever the initial await/then/catch syntax is.

Yep!

[aubergene]

The more I think about it, the more I like your earlier suggestion, including the once idea.

[paulocoghi]

I make the @aubergene's words mine. I like the await, then, catch syntax and I understand that once can be an optional feature, so that each developer can follow the path he prefers.

[arxpoetica]

I don't say this lightly. +1 😂

[Rich-Harris]

{{#await}} is available in 1.44. Docs coming soon, but in summary:

{{#await thePromise}}
  <p>loading...</p>
{{then theData}}
  <p>got data: {{theData.value}}</p>
{{catch theError}}
  <p>whoops! {{theError.message}}</p>
{{/await}}

[arxpoetica]

This is AMAZING. I'm not sure I know of anything like this in any other framework I've used.

[PaulMaly]

@arxpoetica RactiveJS, but we've stolen it from here )))))

[jackyef]

{{#await thePromise}}
  <p>loading...</p>
{{then theData}}
  <p>got data: {{theData.value}}</p>
{{catch theError}}
  <p>whoops! {{theError.message}}</p>
{{/await}}

@Rich-Harris Is it currently possible for the server renderer to try to resolve the promise during SSR so that the client can receive the HTML already filled with the data from the promise? Like what react-apollo can do with React.

[arggh]

@jackyef I was asking the exact same thing here: #3853

[AlbertMarashi]

@jackyef we need this!

[duttaoindril]

One thing to think about: often, you might want to preserve the previous value until the new one has resolved, for example a list of autocomplete suggestions (you don't want them to disappear each time you type a character):

<input list='suggestions' bind:search>

{{#await suggestions}}
  <span>loading...</span>
{{then value}}
  <datalist id='suggestions'>
    {{#each value as suggestion}}
      <option>{{suggestion}}</option>
    {{/each}}
  </datalist>
{{catch err}}
  <span class='error'>could not get suggestions!</span>
{{/await}}

<script>
  export default {
    computed: {
      suggestions: search => fetch(`/suggestions.json?q=${search}`).then(r => r.json())
    }
  };
</script>

How might we express that idea — that we want the 'loading...' text initially, but thereafter we should preserve the resolved value until another one resolves?

@Rich-Harris what happened to the once idea?

I'm stuck now wondering how to keep the data of the previous promise without moving on to the loading state again.

[Conduitry]

It lives on in the also-old issue #955.