Bigger file handling using file system api

[jallwine]

Ace’s performance is fantastic for in memory document editing. It advertises 4 million lines as its limit. I’m trying to determine what it would take to be able to support >=12 million lines. It seems like using a filesystem would be necessary. What would it take to implement a file backed version of ace that used the experimental FileSystem API? https://developer.mozilla.org/en-US/docs/Web/API/FileSystem

I’ve barely started to look at ace’s implementation and am looking to get an idea of what work would be involved to make this happen. For example, I imagine the assumption that all data is present in memory is made throughout ace and that there would be a substantial amount of work in all facets of the library to be able to make asynchronous file API calls. Or is enough done in WebWorkers that the synchronous File API could be used? Would it be as simple as creating a FileDocument class that mimicked the Document API? Any thoughts on the matter are appreciated!

[jallwine]

Perhaps the IndexedDB API would be more suitable: https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Using_IndexedDB

Maybe with this (looks like it’s firefox only currently): https://wiki.mozilla.org/WebAPI/FileHandleAPI

[nightwing]

This depends on the type of file you want to support. If file is small enough to fit in the browser memory, then implementation of document using multiple arrays, or a buffer would work, but session and backgroundTokenizer will need some changes to not highlight the whole file.
If the file is so large, that it can't fit in the browser memory, then filesystem api probably won't help, and some server side code will be needed to load the data for visible part of the file.

[github-actions]

This issue has not received any attention in 1 year. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[jallwine]

I haven’t had a chance to look at this at all, but the idea would be to avoid needing server side support for opening portions of a file. The file system API would allow caching of the whole file and ace could open portions of it as needed.

[whazor]

On a high level design, Ace renders only what is in the screen. But I also think the code base has quite some assumptions on the amount of lines and how documents should look like.

Considering this a feature, I wonder how many people really will want to use this to read documents with >4 million lines. If you do, 👍 on the ticket, and please share information if you have.

So far I think it is lower priority compared to all the other bugs and features.