Refactor ScopeProvider

[dmaskasky]

Overview of Behavior

atoms: a, b, c, d, A(c + d), B(a + b + A)

S1[c]: a0, b0, c1, d0, A0(c1 + d0), B0(a0 + b0 + A0(c1 + d0))
S2[B]: a0, b0, c1, d0, A0(c1 + d0), B2(a2 + b2 + A2(c2 + d2))
S3[a]: a3, b0, c1, d0, A0(c1 + d0), B2(a3 + b2 + A2(c2 + d2))
S4[A]: a3, b0, c1, d0, A4(c4 + d4), B2(a3 + b2 + A4(c4 + d4))
S5[d]: a3, b0, c1, d5, A4(c4 + d5), B2(a3 + b2 + A4(c4 + d5))
-------------------------------------------------------------
S1:
  explicit: [c1]
  implicit: []
  inherited: [0 => [a0, b0, d0, B0, A0]]
S2:
  explicit: [B2]
  implicit: [a2, b2, c2, d2, A2],
  inherited: [0 => [a0, b0, d0, A0], 1 => [c1]]
S3:
  explicit: [a3]
  implicit: []
  inherited: [0 => [b0, d0, A0], 1 => [c1], 2 => [b2, c2, d2, A2, B2]]
S4:
  explicit: [A4]
  implicit: [c4, d4]
  inherited: [0 => [b0, d0], 1 => [c1], 2 => [b2, c2, d2, B2], 3 => [a3]]
S5:
  explicit: [d5]
  implicit: []
  inherited: [0 => [b0], 1 => [c1], 2 => [b2, B2], 3 => [a3],  4 => [c4, A4]]

Let us assume we have base atoms a, b, c, d and derived atoms A and B; { where A depends on c and d, and B depends on a, b, and A }

Scope S0 is the global scope under the nearest jotai Provider or defaultStore.
Scopes S1-Sn are nested where S1 is the first level and S5 is a descendant of S1 at the fifth level.

In scope S1, primitive atom c is explicitly scoped. This means that c1 corresponding to c in S1 holds an independent value. All derived atoms in this scope will use c1. All descendant scopes will use c1 unless c is explicitly defined in a nearer ancestor.

In scope S2, derived atom B is explicitly scoped. This means that all of its dependents are implicitly scoped. Explicitly and implicitly scoped atoms are copied and do not inherit their value from the parent scope. This is why all dependents of B2 are denoted with a 2 suffix.

In scope S3, primitive atom a is explicitly scoped. All other atoms are inherited from S2 where B is explicitly scoped. Even though B2 is inherited, it still will use a3 in S3. This is a similar behavior to B0 using c1 in S1 with the only difference being B0 is unscoped vs B2 is inherited. To me, there is no difference between inherited and unscoped.

And so on... All subsequent scopes feature similar behavior as described above.

Summary

1. fixes atom derived from a scoped atom can't read it's own value #24

2. fixes Nested Scope Consistency Question #32

3. fixes Idea: lazy initialization for primitive atoms pmndrs/jotai#2458 (comment)

4. fixes: inherited atoms that can hold a value (Primitive, Writable), are reused in the nested scopes

5. fixes: ScopeProvider shouldn't read parent scope past a Jotai Provider.

Implementation

Scope

createScope handles everything dealing with scope and is moved to a separate file.

WeakMaps

1. explicit - atoms added to ScopeProvider
2. implicit - dependencies of explicit and implicit atoms are scoped
3. unscoped / inherited

• scoped atoms of the parent scope (explicit, implicit, inherited)
• derived readable and derived writeable atoms, needed for enabling access to scoped atoms

getAtom

from the originalAtom, gets the explicit, implicit, inherited, unscoped derived, and unscoped primitive atoms. As necessary, creates scoped copies (one-time) and saves them in their appropriate weakmap.

Inheriting Atoms

Atoms are inherited from the ancestor scopes. The top ancestor is the global scope where the original atom is used.

Inherited Derived Atoms

To inherit atoms that have a custom read, they are copied so that the custom read and write functions can use that scope's getAtom function.

Inheriting Primitive Atoms

To inherit primitive atoms, those atoms are stored directly and not copied. Since the atom itself is the key to its value, we need to store the original in the weakmap.

Inherited Write Only Atoms

const valuedWritableAtom = atom(0, (get, set) => {
  set(valuedWritableAtom, get(someAtom))
})

Where someAtom could be scoped.

⚠️ Dirty Hack
For write-only atoms, since these atoms also hold a value, we also want to preserve the originals. However the write function may access atoms in the current scope. To work around this, we modify the original write method synchronous before baseStore.set and restore the original write method synchronous after in a finally block.

What I'd love is a utility to clone an atom but have the clone also resolve the same value as the original. That way I can copy atoms like below and wrap their write method to resolve atoms in the current scope.

Tests

adds tests for:

1. scoped derived uses nested scope dependency
2. derived dependency scope is preserved in self reference
3. ScopeProvider provides isolation for scoped primitive atoms
4. unscoped derived can read and write to scoped primitive atoms
5. unscoped derived can read both scoped and unscoped atoms
6. dependencies of scoped derived are implicitly scoped
7. scoped derived atoms can share implicitly scoped dependencies
8. nested scopes provide isolation for primitive atoms at every level
9. unscoped derived atoms in nested scoped can read and write to scoped primitive atoms at every level
10. inherited scoped derived atoms can read and write to scoped primitive atoms at every nested level
11. implicit parent does not affect unscoped
12. scoped writable atoms can read scoped atoms and themselves

[codesandbox-ci]

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

[dmaskasky]

In order of priority, descending:

1. explicit - map of <originalAtom, scopedCopy>. atoms passed to ScopeProvider are explicit.
2. implicit - map of <originalAtom, scopedCopy>. Dependencies of explicit scoped atoms are implicitly scoped.
3. inherited/unscoped

• map of <originalAtom, scopedCopy>. Scoped atoms in parent scopes are copied to this scope so they can access scoped atoms in this scope.
• map of <originalAtom, scopedCopy>. Unscoped derived atoms are copied so they can access scoped atoms in this scope.

[dmaskasky]

defaultRead and defaultWrite are methods on primitive atoms

[dmaskasky]

@yf-yang Check this out https://codesandbox.io/p/sandbox/elated-hellman-forked-z3dkvl?file=%2Fsrc%2FApp.tsx%3A25%2C39

Try now increase base2 at layer2, as expected it is globally shared. 😊

[dai-shi]

I don't think we need this file. Just import it directly from the library index. Though, it's just a preference.

[dmaskasky]

done.

I also added exporting INTERNAL_ScopeContext from src/ScopeProvider/ScopeProvider.tsx, but am intentionally excluding this export from the package. I hope this will be a good compromise.

[dai-shi]

Thanks.

As the current config in this repo bundles dist, exporting only in the source file doesn't work.
Anyway, let's drop it: #33 (comment)

[yf-yang]

So coming back to the example:

d(a1 + a2)

<ScopeProvider atoms={[derived]}> // S1
    <ScopeProvider atoms={[atom1]}> // S2
      <Counter />
    </ScopeProvider>
</ScopeProvider>

in S2, d will access S2 a1 and no a2, how do we explain this phenomenon?

[yf-yang]

https://codesandbox.io/p/sandbox/festive-dust-5hnqmx?file=%2Fsrc%2Findex.tsx

Layer 1, write is broken

[yf-yang]

Anyway, that's a write problem.

I am still trying to understand why/how read works. Seems scopeKey is only used to distinguish direct store.get and hooked atom.read.

[dmaskasky]

Anyway, that's a write problem.

I am still trying to understand why/how read works. Seems scopeKey is only used to distinguish direct store.get and hooked atom.read.

GlobalScopeKey represents the scope key for unscoped atoms. Explicitly scoped atoms are stored with their level's scope.

Custom Read and write fns of explicitly scoped atoms pass the explicit atom's scope as the implicit scope to getAtom. This is how implicit scope works for their atom dependencies.

[dmaskasky]

https://codesandbox.io/p/sandbox/festive-dust-5hnqmx?file=%2Fsrc%2Findex.tsx

Layer 1, write is broken

I think you forgot to copy ScopeProvider code over to this example. The scope.ts code looks current, but ScopeProvider.tsx code looks stale.

After updating the ScopeProvider code, it was working correctly. I took your example and added extra debug info to make it a little easier what's going on. The example is behaving correctly.

https://codesandbox.io/p/sandbox/festive-dust-forked-8t5k5x

[yf-yang]

I now understand how it works, nice catch!

Generally speaking, we are following this pattern.

One drawback of this pattern is, it cannot deal with scoped derived atom. If an atom is scoped, then when traversing its dependency, we don't preserve any information about the dependent's scope, so we will get the global atom.

So, when hitting the derived scoped atom, its scope is recorded (the green line). When visiting its dependency, if its dependency hits the green line, the dependency would be resolved to the scope atom in the green line. Alternatively, if the dependency hits a deeper scoped atom before the green line, it would still use the deeper scoped atom.

[yf-yang]

The core mechanism can be simplified, but let's stick to this pr for now.

[yf-yang]

Have you checked #24? I think it could be solved now?

[dmaskasky]

One drawback of this pattern is, it cannot deal with scoped derived atom. If an atom is scoped, then when traversing its dependency, we don't preserve any information about the dependent's scope, so we will get the global atom.

This behavior was intentional actually. Say we had the following,

const base = atom(0)
const derived = atom((get => get(base))
const App = () => {
  return (
    <ScopeProvider atoms={[derived]} debugName="S1">
      <ScopeProvider atom={[base]} debugName="S2">
        <Counter />
      </ScopeProvider>
    </ScopeProvider>
  )
}

It makes sense that derived in S2 would read base scoped to S2. This is consistent with:

const base = atom(0)
const derived = atom((get => get(base))
const App = () => {
  return (
    <ScopeProvider atom={[base]} debugName="S1">
      <Counter />
    </ScopeProvider>
  )
}

Where now derived is not scoped and base is scoped to S1. Existing behavior is that derived would use base from S1 not base from global scope.

So to make these two cases consistent, we must follow the rule that nested scopes override ancestor scopes. This is the original motivation of this PR and addresses #32.

[dmaskasky]

Yes #24 is fixed.
https://codesandbox.io/p/sandbox/jotai-forked-fc7kns

[dmaskasky]

May I please have write access?

[dai-shi]

May I please have write access?

I can do that if @yf-yang is okay.

[dai-shi]

Well, I don't understand the entire logic at all but it's fine. I don't want to block this pr (or this repo) anyways.
@yf-yang Feel free to merge and publish a new version.

My interest is how to support jotai-scope with jotai in an abstract way. (and our current idea is unstable_is.)

Left some minor comments.

[dai-shi]

This is a nice hack, but ideally it would be nice if we could avoid this with unstable_is.

[dmaskasky]

defaultRead and defaultWrite are used to detect when an atom has a custom read or a custom write.

1. This is important when determining whether to copy the atom or use the original. unstable_is will handle this case in the future.
2. It is also a performance enhancement since defaultRead's this is faster than using getAtom to resolve the atom.

[yf-yang]

Maybe we can avoid checking it later, but in the implementation of this PR, there are some special branches using that comparison, like inheritAtom

[dai-shi]

just curious how useful is this.

[dmaskasky]

for debugging, it is useful

[yf-yang]

May I please have write access?

I'm OK with that. Do I have the team manage access? Didn't find it.

[dai-shi]

May I please have write access?

Done!

[dmaskasky]

I added a test for scoped writable and replaced the patched store symbol with instanceof.

[dmaskasky]

Need one more approval please 🙏