Data loss possible when path changes while project is open

[HeyMyian]

I swear I'm not actively looking for these behaviors. And granted, one has to be especially fogged for this to occur, but alas, here I am.

1. Open a project
2. Rename any of its parent folders in the file system
3. Close the project
4. Confirm closing the project

The result, with no way back, is a barrage of "this document has been altered outside of novelWriter, overwrite file?" prompts (possibly overlaying with the autosave tries, which also fail), and no matter the combination of "yes" or "no" answers, the project is not written to its original path, any changes to the project are lost.

It was not a problem for me when I noticed this behavior because my autosave for documents and the project is every 30 seconds. I lost no data. But I gather there might be people out there who don't use the autosave feature (which will prevent this, because of frequent error messages while the project is open), and who also have a foggy enough mind to change the directory names around an open project.

And since data is holy etc., maybe novelWriter, if possible, should just re-create the complete folder structure of where the project was last located, and save the project there. Alternatively, novelWriter could give the user the option to change their mind about closing the project if there are problems writing the files.

[HeyMyian]

Clarification; I didn't go around thinking "let me open the project and change its folder name afterwards to see what happens". I opened my project at the beginning of the day like I always do, and let it sit for hours like I always do. At some point I decided to rename my cloud folder, which I did. Hours after that I decided to start working on the project and discovered the error message upon autosave. I'd say it's unusual, but for me completely natural workflow. The only thing that saved me was autosave, ironically, not autosaving.

[vkbo]

The result, with no way back, is a barrage of "this document has been altered outside of novelWriter, overwrite file?" prompts (possibly overlaying with the autosave tries, which also fail), and no matter the combination of "yes" or "no" answers, the project is not written to its original path, any changes to the project are lost.

Before writing a document, novelWriter checks the hash of the file content against the hash the file had when it was opened. I'm guessing here that the hash returns a None since the file does not exist any more, which obviously doesn't match with the hash when opened. This is clearly a misleading error and should be fixed. I'll add a file exist check here so that this scenario is handled in a better way.

It was not a problem for me when I noticed this behavior because my autosave for documents and the project is every 30 seconds. I lost no data. But I gather there might be people out there who don't use the autosave feature (which will prevent this, because of frequent error messages while the project is open), and who also have a foggy enough mind to change the directory names around an open project.

You can't really disable autosave, only adjust the time interval up to 10 minutes.

And since data is holy etc., maybe novelWriter, if possible, should just re-create the complete folder structure of where the project was last located, and save the project there.

This is not possible since novelWriter doesn't store the entire project in memory. It only holds the project structure and the document in the editor in memory. There is no way to recreate the project at the old location.

Another issue here is that novelWriter doesn't hold any file system locks on the files it has open. This could potentially have solved the issue on Windows at least as it would have blocked the file system operation you did, but Linux is not as restrictive (which is usually a good thing) so it would not help there.

Alternatively, novelWriter could give the user the option to change their mind about closing the project if there are problems writing the files.

So, in terms of actual solutions, I have some thoughts.

In 2.0 I moved all the handling of the data storage of the project and content into a special storage class. This makes it easier to add some centralised checks to the saving process that can help here. For instance, a check that the project folder still exists would be useful in this case. That means I could interrupt the saving process before it gets stuck in the error loop you encountered. This would also make it possible for the user to go back to their file system and correct the mistake, at which point novelWriter would be able to just continue as if nothing happened.

Alternatively, or perhaps additionally, an option to point novelWriter to the new project location could fix this. It is fairly trivial to replace the project path at runtime since it is only saved in one single location in memory. This is part of the storage class change because I plan to add the option to save projects as a single file between writing sessions, which means I need to be able to handle dynamic temporary project paths at runtime.

[vkbo]

After a bit of testing, I don't find a lot of issues with the handling of the path going missing. It gives a handful of error messages, which makes sense since it is trying to save a few files and gives an error for each. I can definitely improve it a bit though as it should be able to detect the difference between an altered file and a missing file.

[vkbo]

I've made a minor fix that just ensures that things are handled a bit better in the core code, but the whole issue requires a bit of refactoring of the file saving process. I've opened #1344 for that, with a potential feature in #1346.

At the end of the day, removing the save location, either by disconnecting something, or by restructuring the file system, will cause problems and potential risk of data loss. There is no way currently to rescue the open project if it is not reachable.