#10186 still a problem (Data lost during long external editing session)

[srhuston]

Operating system

macOS

Joplin version

Joplin 2.14.22

Desktop version info

Joplin 2.14.22 (prod, darwin)

Client ID: 49827ee4b66f4367a34c0f46ea1db6d3
Sync Version: 3
Profile Version: 46
Keychain Supported: Yes

Revision: e579eb9

Backup: 1.4.0
Favorites: 1.3.2

Current behaviour

#10186 (comment) is still a problem, I only missed the 'bump' because I was out on vacation. Happens almost daily when I'm using the program.

Lately I've taken to, while in vim, writing the file and going back to Joplin to see if it's updated. If not, I click the button to turn off external editing (without closing the editor), then click again which triggers vim to try to reopen it and say that the file on disk has changed. I tell vim to go ahead and load the "new" file, then press 'u' to undo the changes (going from the correct version in the buffer to the old version on disk, then back to the correct version that was in the buffer), and write again. Usually that causes Joplin to pick up the changes on disk. But if I'm not careful the changes that the editor wrote will still be lost.

Expected behaviour

No response

Logs

No response

[laurent22]

Isn't it because you need to configure your editor so that it overwrites the source file and not keep the changes in a separate temp file?

[personalizedrefrigerator]

Possible duplicate of #10647.

See #10647 (comment) for a workaround.

[srhuston]

Isn't it because you need to configure your editor so that it overwrites the source file and not keep the changes in a separate temp file?

It does overwrite the source file, vim stores the file in an internal buffer while it's running and only writes when issued the write command. If the source file on disk changes in that time, it will error and give the option to overwrite anyway or reload the source file, which will then be considered a "change" to the buffer that can be undone. For example, if the note contains the text "One", I edit it and add "Two Three" on the next line and write the file, but Joplin doesn't see the change, when I click the button to end external editing and again to edit, vim will be called with the same file as last time. vim will then say that the file (which it believes is still open) is different than what is on disk. If I tell it to load, the current buffer now just says "One". If I press 'u' to undo, it will undo that load, which restores it to "One" "Two Three". Usually at this point writing again causes Joplin to see the new updates to the file and update the note.

[srhuston]

Possible duplicate of #10647.

See #10647 (comment) for a workaround.

Interesting idea... I will try checking the mtime of the file when this happens and see if touching it solves the problem.

[srhuston]

No luck it seems, at least based on plain mtime:

-rw-r--r--@    1 huston  staff      12510 Jul  2 10:57 edit-fbc6fc19a4624860a0ef6bc5f435567b.md

At this point I saved the file in vim:

-rw-r--r--  1 huston  staff  13082 Jul  2 14:23 edit-fbc6fc19a4624860a0ef6bc5f435567b.md

Then went to Joplin and found the changes were not there. I clicked the edit button twice (once to exit external editing, once to enter it) and vim presented me with an error that the file on disk was changed from the buffer. I told vim to reload the file on disk:

-rw-r--r--@ 1 huston  staff  12510 Jul  2 14:26 edit-fbc6fc19a4624860a0ef6bc5f435567b.md

Then I pressed u to undo the last action (loading from disk) which restored the changes I had made from Joplin's last detection of changes to most recently, at 14:23, and wrote the file to disk again:

-rw-r--r--  1 huston  staff  13082 Jul  2 14:26 edit-fbc6fc19a4624860a0ef6bc5f435567b.md

As I was going back and forth to paste everything here, I realized the changes still were not in Joplin, so I wrote the file yet again:

-rw-r--r--  1 huston  staff  13082 Jul  2 14:30 edit-fbc6fc19a4624860a0ef6bc5f435567b.md

This time the changes showed up in the note in Joplin.

Next time I see this is triggered I'll attempt a 'touch' anyway to see if that makes a difference.

[personalizedrefrigerator]

This issue might be related to the following lines:

joplin/packages/lib/services/ExternalEditWatcher.ts

Lines 287 to 291 in 7e4533d

	this.logger().debug(`ExternalEditWatcher: Update note file: ${note.id}`);
	// When the note file is updated programmatically, we skip the next change event to
	// avoid update loops. We only want to listen to file changes made by the user.
	this.skipNextChangeEvent_[note.id] = true;

One possibility is that multiple events are received when saving a note using Vim, and that the file only has updated content after the last event (which is currently ignored).

[srhuston]

This issue might be related to the following lines:

joplin/packages/lib/services/ExternalEditWatcher.ts

Lines 287 to 291 in 7e4533d

	this.logger().debug(`ExternalEditWatcher: Update note file: ${note.id}`);
	// When the note file is updated programmatically, we skip the next change event to
	// avoid update loops. We only want to listen to file changes made by the user.
	this.skipNextChangeEvent_[note.id] = true;

One possibility is that multiple events are received when saving a note using Vim, and that the file only has updated content after the last event (which is currently ignored).

I just had a note where I opened it with no content yet (new note), wrote into it during the course of a meeting, and at the end wrote the file out in macvim. Joplin did not pick up the changes and re-read the note, so in macvim I opened a terminal window and did a 'touch' on the file. Once before when I did that Joplin then picked up on the new content, but this time it did not. So again I told Joplin to no longer edit the note externally, then clicked the edit button again which triggered macvim to "open" the file (whereby it complained that the file is already open but has changed on disk), clicked the button to load the file from disk (which is empty), pressed 'u' to undo the load (which brought back all of the content that should have been there) and wrote it to disk again. This time Joplin picked up the changes in the note.

[personalizedrefrigerator]

Based on limited testing, removing an old workaround for another bug seems to fix the issue.