Introduce .nctext file extension and open in the text module by default

[max-nextcloud]

Start using a new file extension - for example .nctext to indicate these files are meant to be edited with text.

*.nctext is created by and opens in the text module by default
*.md has an option to use the text module if the user prefers to

This will

• Avoid data loss / changing the .md syntax without consent.
• Allow users who never heard of markdown to use the editor to format text with the default settings.
• *.md files continue to be opened in Text by default, with a user option to disable this.

Originally posted by @inthreedee in #593 (comment)

[juliusknorr]

Sounds like a feasible option, one thing we should evaluate is probably how different operating systems will handle those files. macOS would ask for an application to open with. Maybe the Nextcloud desktop client could register a handler for this new file type to open by default as well.

In addition apps that use text (like collectives and notes in the future) would need special care to work with those.

cc @nimishavijay @jancborchardt also for input here.

[max-nextcloud]

I think .text.md would be ideal and somewhat in line with what the markdown guidance rfc says about variants. But I am afraid we cannot register a different mimetype for that in nextcloud, can we?

[susnux]

Or to make it more clear: .nc.md?

[juliusknorr]

I think .text.md would be ideal and somewhat in line with what the markdown guidance rfc says about variants. But I am afraid we cannot register a different mimetype for that in nextcloud, can we?

Yes, that is currently not possible. We only extract the last part of a file name that starts with a . https://github.com/nextcloud/server/blob/6be7aa112f6174c1406cfef5449836124a7f6f50/lib/private/Files/Type/Detection.php#L210

[bronson]

I'm elated that this is finally being considered. Love it!

My only question is the default setting. Is it advisable to ship Nextcloud in a configuration that is known to cause data loss?

[max-nextcloud]

My only question is the default setting. Is it advisable to ship Nextcloud in a configuration that is known to cause data loss?

I think we'll need a migration strategy. If users who are not aware of markdown open their personal notes and out of the sudden those contain funny characters instead of formatting they will consider that data loss.

So my current take on this would be introducing the new extension in one release and start using it by default but still open .md files with text as well. Once the new extension has been established we can think about changing the default behavior for .md files. We'll want a way to configure apps for certain file types anyway - but it's a longer process.

[PhrozenByte]

Start using a new file extension - for example .nctext to indicate these files are meant to be edited with text.

Even though many of my users have compatibility issues with Text, I'd strongly oppose a unique file extension like .nctext. I very much like the WYSIWYG idea of Text, as it makes writing Markdown way easier - and thus makes apps like Pico CMS for Nextcloud - in theory - more accessible. A file with .nctext would no longer been seen as Markdown file by other software - even though the files created by Text are Markdown files without any doubt.

This doesn't mean that we shouldn't add an option for users to switch to a custom file extension. In my understanding we already support this on a global basis (#2718), correct? It shouldn't be the default though... Text is a Markdown editor after all.

I don't agree with the "Text causes data loss" argument. IMHO it is an exaggeration to sustain a maximum demand. Switching to a different file extension as an option, sure, why not, but not as the default. Switching the file extension would declare Text not being a Markdown editor - what simply isn't true. It has some compatibility issues though, like basically any Markdown parser. The correct solution to this is #3477, not switching to a different file extension.

I think .text.md would be ideal and somewhat in line with what the markdown guidance rfc says about variants. But I am afraid we cannot register a different mimetype for that in nextcloud, can we?

Yes, that is currently not possible. We only extract the last part of a file name that starts with a . https://github.com/nextcloud/server/blob/6be7aa112f6174c1406cfef5449836124a7f6f50/lib/private/Files/Type/Detection.php#L210

I'm not sure why we would even need that when optionally supporting switching to .nc.md (or similar): Text would still register as editor for .md files, Text would rather just choose the editor mode (rich text editing vs plain text editing) depending on the file's extension when opening it.

[jancborchardt]

Agree with @PhrozenByte here. The default experience works for a silent majority of people who don't know what Markdown is, and for whom a dialog like @juliushaertl mentions will already be very confusing:

macOS would ask for an application to open with.

So having a setting that people with advanced requirements can use for e.g. setting a different extension sounds ok. (Instead of making the default experience more complex.)

[inthreedee]

So my current take on this would be introducing the new extension in one release and start using it by default but still open .md files with text as well

I agree with this take. I think it solves the problems brought up in the parent issue with the least impact to everyone. I’d say it’s perfectly acceptable to expect instance administrators to notify their users of potential changes if they fiddle with any default settings.

The default experience works for a silent majority of people who don't know what Markdown is

And these people will not be affected by this change. Text will continue to function as it always has for them, except there will no longer be the possibility of data loss should they open a markdown file created in another app. Seems like a good thing all around. These people aren’t going to care what extension is being used.

[dumblob]

To keep this discussion non-emotional and to the point, let me point out that what @PhrozenByte wrote is generally true and I agree with the most important points (especially prioritizing #3477 - another advantage is it does not seem to require any migration).

Nevertheless the comment seems slightly convoluted at places if consulted with the Markdown RFC brought up by @max-nextcloud .

Even though many of my users have compatibility issues with Text, I'd strongly oppose a unique file extension like .nctext.

Therefore the RFC recommends .nc.md style. The fact it is not supported by NC by now is irrelevant in this discussion about future.

I very much like the WYSIWYG idea of Text, as it makes writing Markdown way easier

Everybody here does (many because it works on mobile devices) and that is the reason why this issue exists. If people would not like it, they would have simply disabled it and not care to come here to cry.

• and thus makes apps like Pico CMS for Nextcloud - in theory - more accessible.

Actually Pico CMS seems to rely on some features (and this feature set will probably grow over time without NC Text being able to catch up) which NC Text does not seem to support. So this might be a good counter example and a reasoning why NC Text must not rich-edit foreign .md files (unless the round-trip conversion produces identical results as proposed in #3477 ).

A file with .nctext would no longer been seen as Markdown file by other software - even though the files created by Text are Markdown files without any doubt.

That is true but kind of not the "full truth" 😉. The RFC does make it explicit that - and I quote:

Markdown's formatting syntax only addresses issues that can be conveyed in plain text.

So if NC Text addresses also other things (by editing them - in this case by silently removing them), then it is not exactly aligned with the RFC IMHO.

Simply meaning NC Text is not (only) a Markdown editor but an editor of a superset of Markdown (i.e. editor of strictly one Markdown flavor - specifically NC flavor) in which case we would be trying to use non-Markdown editor to edit both Markdown files and Markdown-flavored files (here I mean other than NC flavors).

The Markdown RFC says:

The reflexive media type text/markdown in this parameter value means that the author does not want to invoke Markdown processing at all: the receiver SHOULD present the Markdown source as is.

Unfortunately NC does not seem to support MIME types as e.g. email does. But I am 100% certain that majority if not all of existing .md files not created using NC Text assume text/markdown (i.e. no processing and thus expect them rather being edited plain as Windows Notepad does than to omit something). #3477 seems to address this.

This leads me to suggestion that if this discussion will not converge on something like .nc.md nor #3477 gets accepted, then the only alternative would be to make NC Text "tag" each .md file it edited by adding some header string to the top of the content of the .md file.

When opening .md NC Text would first look up this tag and if it were not there, NC Text would open it read-only. The user would then be presented with a button "reformat & edit" button telling the user it would lose certait formatting after clicking. Then the file would be tagged. Any newly created .md file would be automatically tagged to keep it user-friendly as it is now.

Basically the same as #3477 but with explicit tagging as part of the content of the .md file.

This doesn't mean that we shouldn't add an option for users to switch to a custom file extension. In my understanding we already support this on a global basis (#2718), correct?

Well, #2718 only allows the user to constrain herself (as temporary measure to avoid data loss - see below). But I think we want the exact opposite. We want users to create .md files and not .txt files.

It shouldn't be the default though... Text is a Markdown editor after all.

Yes and no 😉 - see above why it is not a "true" Markdown editor per RFC.

I don't agree with the "Text causes data loss" argument. IMHO it is an exaggeration to sustain a maximum demand.

Here I think RFC tends to lean towards "data loss" interpretation (e.g. by pushing the "double window side-by-side view" by describing it in detail unlike WYSIWYG which is only mentioned among several other options buried elsewhere in only one sentence from the whole RFC).

But your mileage may vary.

Switching to a different file extension as an option, sure, why not, but not as the default.

If "own extension" would be chosen over #3477 , then it would have to be the default to solve the "data loss" case forever. But I am open to not have it as default just to see how much time it will take to make it a default (which I see as inevitable sooner or later) 😉. My guess would be ~4 years from the release of the non-default option 😉.

Switching the file extension would declare Text not being a Markdown editor - what simply isn't true.

Yes and no 😉. See above.

It has some compatibility issues though, like basically any Markdown parser. The correct solution to this is #3477, not switching to a different file extension.

Yep.

I'm not sure why we would even need that when optionally supporting switching to .nc.md (or similar): Text would still register as editor for .md files, Text would rather just choose the editor mode (rich text editing vs plain text editing) depending on the file's extension when opening it.

Yep.

[PhrozenByte]

Just to clear up a few things: I strongly oppose .nctext or any other unique file extension, as this will heavily disturb the workflow of the majority of Text users, because other software would stop considering files created by Text to be Markdown files. This is unacceptable IMHO.

This critique is not true for "derived" file extensions like .nc.md, making things slightly better. Yes, we will still cause issues while migrating to .nc.md, negatively affecting the majority of Text users, but this disturbance might be acceptable. The thing is, switching to .nc.md doesn't actually solve "the issue": Even though a users has created a Markdown file with Text in the past, one might still edit it with any other editor - it's a .md file after all. These changes might then cause the exact same compatibility issues as before. So, what's the point of this? The informational guide (RFC 7764 is no standard) targets a different use case.

Improving compatibility (i.e. individual issues like #3516) and detecting edge cases (i.e. #3477) actually solves "the issue". Thus I believe that changing Text's file extension isn't feasible at all.

and thus makes apps like Pico CMS for Nextcloud - in theory - more accessible.

Actually Pico CMS seems to rely on some features (and this feature set will probably grow over time without NC Text being able to catch up) which NC Text does not seem to support. So this might be a good counter example and a reasoning why NC Text must not rich-edit foreign .md files (unless the round-trip conversion produces identical results as proposed in #3477 ).

As the main developer of Pico CMS for Nextcloud, I can assure you that the majority of flat file CMS users want a WYSIWYG editor, even though the features are limited. This is true for basically any WYSIWYG editor, just think of Word vs. LaTeX: yes, Word is limited and sometimes annoying to use, but still, the majority doesn't want to fiddle with syntax.

[inthreedee]

Improving compatibility (i.e. individual issues like #3516) and detecting edge cases (i.e. #3477) actually solves "the issue". Thus I believe that changing Text's file extension isn't feasible at all.

I don’t think we want to keep re-hashing this part of the debate in this issue, but I will kindly point you to the parent issue #593 for lengthy discussions on why many of us believe this doesn’t solve the issue. If further discussion about that is still needed, I think that issue is the right place for it?

[max-nextcloud]

I added another issue for the proposal @dumblob brought up: #3636.
Now we have issues for the different ideas to make sure text does not step on peoples feed / mess with their md syntax without their consent:

• Detect changes to markdown syntax and open in code editing mode #3477
• Introduce .nctext file extension and open in the text module by default #3630
• Add metadata to files indicating they were created / edited by text #3636

Please note that these proposals are not mutually exclusive. Let's try and focus in each issue on the proposal in question. We can explore ways to implement them, the implications and mitigations for problems they might cause.