Introducing JSONL Store

[taras]

Motivation

It's useful to organize data into JSONL files on the file system. I'm introducing JSONL Store, which allows me to store the content from external sources as files on the file system.

[cowboyd]

I would consider making the json cache implementation a resource rather than having every operation go from a cold start. It's probably less important for something on the file system, but for remote caches, or DB caches, or redis caches, initializing once and then having instance access to the hot connection for every operation is a good practice. The way you can say:

let impl = yield* initMyCacheImpl() // <== impl is bound to this scope.
CacheContext.with(impl, function*() {
  // do stuff
});

[taras]

I would consider making the json cache implementation a resource rather than having every operation go from a cold start. It's probably less important for something on the file system, but for remote caches, or DB caches, or redis caches, initializing once and then having instance access to the hot connection for every operation is a good practice. The way you can say:

let impl = yield* initMyCacheImpl() // <== impl is bound to this scope.
CacheContext.with(impl, function*() {
  // do stuff
});

@cowboyd is this something that I need to provide now, or can we wait until we have a cache adapter that requires a hot connection?

[cowboyd]

Are you sure you don't need a hot connection in this case? What is the expectation around concurrent reads and writes.

[taras]

Are you sure you don't need a hot connection in this case? What is the expectation around concurrent reads and writes.

I'm really not sure. Currently, I don't have any use cases where multiple things are reading/writing the cache simultaneously. Most of it is very serial. Most of the time, it's essentially just reading data over the network, caching it to the file system so it can be read from the file system instead of going over the network again.

[cowboyd]

Approving. I'm still a little bit worried about synchronization between reads and writes. I.e. if you have multiple reads going, what happens if a write happens in parallel, or should there be a queue so that writes cause things to happen in parallel.

[cowboyd]

Approving. I'm still a little bit worried about synchronization between reads and writes. I.e. if you have multiple reads going, what happens if a write happens in parallel, or should there be a queue so that writes cause things to happen in parallel.

@taras You may want to put a heads up in the readme that it has not been tested with concurrent reads and writes.