Feature Request: Remote Indexing

[jacobranson]

Having all the Markdown files be available locally isn't always going to be feasible. Being able to index files on some remote, either via a URL, or ideally via an S3-compatible API would be highly beneficial.

[rufuspollock]

@jacobranson great suggestion. This would be very easy to implement here. I think it could start as an example.

My suggestion is you'd start with a copy paste of processFile called e.g. processRemoteFile https://github.com/datopian/markdowndb/blob/main/src/lib/process.ts#L17

If you want an entire folder in e.g. s3 or a github repo then work on indexFolder called e.g. indexRemoteFolder` https://github.com/datopian/markdowndb/blob/main/src/lib/indexFolder.ts#L7