feat!: add data, cache and temp dirs to FileSystem

[genaris]

Files created by the framework have different nature, and in this PR we introduce three categories:

• data: intended to be persistent and included in OS backups. For instance, wallet contents (used by default for Askar-based wallets)
• cache: files intended to be persistent, but that can be regenerated or re-downloaded. An example of this are the tails files for AnonCreds
• temp: temporary files that can be safely deleted by the OS after usage. For instance, the genesis file created from genesisTransactions or wallet backup generated during migration

Another change from current behaviour is that all files created by the framework are contained within basePath/.afj directory. This is somewhat similar to indy-sdk, which creates a homedir/.indy_client.

Some remaining questions:

• Will it be necessary to write migration scripts to move files (if existant) to new directories? Probably not, considering that until now all files created by AFJ are either temporary or cache
• Is it OK to use user home directory and Document directory as a base for data in (node and react-native respectively)? This selection and categorization is based on iOS storage guidelines

[codecov-commenter]

Codecov Report

Merging #1306 (1f6fb6e) into main (af384e8) will increase coverage by 0.00%.
The diff coverage is 65.38%.

@@           Coverage Diff           @@
##             main    #1306   +/-   ##
=======================================
  Coverage   85.17%   85.17%           
=======================================
  Files         835      835           
  Lines       20731    20738    +7     
  Branches     3533     3535    +2     
=======================================
+ Hits        17657    17664    +7     
  Misses       3067     3067           
  Partials        7        7           

Impacted Files	Coverage Δ
.../src/modules/indy/services/IndyUtilitiesService.ts	36.36% <0.00%> (ø)
packages/indy-sdk/src/ledger/IndySdkPool.ts	63.52% <0.00%> (ø)
packages/anoncreds/src/utils/tails.ts	18.18% <33.33%> (ø)
packages/node/src/NodeFileSystem.ts	68.62% <58.33%> (-1.59%)	⬇️
packages/askar/src/utils/askarWalletConfig.ts	43.90% <100.00%> (+1.40%)	⬆️
packages/askar/src/wallet/AskarWallet.ts	60.93% <100.00%> (ø)
packages/core/src/modules/ledger/IndyPool.ts	78.82% <100.00%> (ø)
...ages/core/src/storage/migration/UpdateAssistant.ts	89.52% <100.00%> (+0.20%)	⬆️
...src/anoncreds/services/IndySdkAnonCredsRegistry.ts	60.00% <0.00%> (+1.53%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[TimoGlastra]

Nice, great improvement 👍

[TimoGlastra]

Do we want to use the temp path for this? If you start a migration and close the app in the process, it may be nice that we could recover it and thus have to manually delete the backup (this is done I think already afterwards)

[genaris]

Yes, that's not a very good idea as it is unspecified when the OS will clean up the files, especially large ones.

But if we are going to use data directory I think we should automatically delete the file once we successfully finish the migration or restore the backup from it. This is something not happening right now (at least on framework level), so I can add it (and some test for it) within this PR.

[genaris]

Just a remark regarding something I found during this update: framework produces a backup-error file when restoring a wallet. I guess the value of doing so is to be able to do some further debugging. Anyway, in this case this file will remain in the file system.

It's not supposed to be a common case, so it's fine for me but just would like to note this if you see it could represent an issue.

[TimoGlastra]

Will it be necessary to write migration scripts to move files (if existant) to new directories? Probably not, considering that until now all files created by AFJ are either temporary or cache

No I don't think so. We may want to do a delete all for the old folders. But as it's all in the temp folder the env should take care of that autmoatically

Is it OK to use user home directory and Document directory as a base for data in (node and react-native respectively)? This selection and categorization is based on iOS storage guidelines

Seems fine to me

[TimoGlastra]

@JamesKEbert knows a lot about mobile environments. Does this look good to you?

[JamesKEbert]

On Android, the TemporaryDirectoryPath will redirect to the Caching Directory, which means you should be aware you could have conflicts between cache & temp as they end up in the same space on Android.

The absolute path to the temporary directory (falls back to Caching-Directory on Android)

[genaris]

Good point! Added a temp/cache suffix when platform is android

[JamesKEbert]

Overall looks good to me though--good changes here 👍