Whitespace rules for <part-name>?

[lemzwerg]

What whitespace rules do apply to <part-name>? Are newlines preserved? And what about directly inserting 
?

Contrary to <display-text> there is no xml:space attribute, and I couldn't find the default value. Please advise.

[mscuthbert]

I'm researching this and looking around but my understanding is that xml:space and xml:lang can go on any xml tag regardless of whether it's specified in the MusicXML docs or not. It's part of the xml spec (hence the namespace) and not MusicXML spec. What MusicXML can specify is what parsers should do if omitted (for instance, treating lang as it not en).

My initial thought is the default without xml:space should be to preserve internal line breaks but trim spaces at beginning and end.

I think that we specified that MusicXML is a 0x0a (line feed) not 0x0d (carriage return) or both encoding. But I will look at that in the morning. In any case an entity can always be replaced with its literal encoding in xml without changing the meaning (I think...lots of work with Bleach in Artusi made this my understanding) so we can't say that a literal
line
feed
Is meaningless but literal
line
feed
 is meaningful.

(edited to add backticks)

[lemzwerg]

[Edited, I didn't test enough.]

Thanks in advance for checking. After reading the XML specification, as far as I understand the issue, it's the 'default' value that has to be set up by MusicXML (i.e., if xml:space is not set), and I couldn't find this value.

BTW, the behaviour of Finale regarding CR and LF in <part-name> (and <part-abbreviation>) while importing an MusicXML file is as follows, which I've found out after some tests:

• Leading and trailing spaces are taken into account (i.e., they enlarge the bounding box of the string horizontally).
• A sequence of one or more (real) CRs or LFs in a string are condensed into a single linefeed.
• Spaces in an empty line are added to the last non-empty line's data (i.e., the bounding box gets enlarged on the right) – this strange feature looks like a bug to me.
• 
 is handled identically to a real LF; this includes condensing behaviour.
• 
, however, is handled specially: it creates a newline but gets not condensed. For example, if you want to have two characters 'X' separated by two empty lines, you can write X

X. Other forms like X<CR><space><CR><space><CR>X make the bounding box wider.

[lemzwerg]

[Edited]

Except the strange handling of whitespace in empty lines, Finale's handling of CR and LF seems to be standard: musicxml2ly uses Python's standard 'minidom' library and produces similar results.

[lemzwerg]

Alas, life is not so simple: Testing <words> with both Finale and MuseScore, it doesn't make a difference whether xml:space is set to 'default' or 'preserve'. Using this small snippet

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE score-partwise PUBLIC "-//Recordare//DTD MusicXML 4.0 Partwise//EN"
                                "http://www.musicxml.org/dtds/partwise.dtd">
<score-partwise version="4.0">
  <part-list>
    <score-part id="P0"><part-name></part-name></score-part>
  </part-list>
  <part id="P0">
    <measure number="1">
      <direction placement="above"><direction-type>
        <words xml:space="preserve">Largo


  foo</words>
      </direction-type></direction>
      <direction placement="below"><direction-type>
        <words xml:space="default">Largo


  foo</words>
      </direction-type></direction>
      <note>
        <pitch><step>F</step><octave>4</octave></pitch>
        <duration>4</duration>
        <type>whole</type>
      </note>
    </measure>
  </part>
</score-partwise>

I get the following result with MuseScore (left) and Finale (right):

As can be seen, MuseScore doesn't collapse newlines or spaces, while Finale does collapse newlines but not spaces.

Sigh. What is a standard good for if applications don't follow it. 😞

[mscuthbert]

Sigh. What is a standard good for if applications don't follow it.

This is one of the main reasons why MNX has decided to switch to JSON -- the behavior of strings is so much better defined in JSON than in XML. We're going to trade off that JSON is not designed to be human readable/writeable.

I share your frustration, but just to know how much better MusicXML (even 0.6beta) was than the world that came before it. So there is still a lot more to gain with carrots than sticks -- letter people know that supporting MusicXML will be cheered rather than being chastised for small errors in making it. AFAIK this is the first discussion of what the standard for these elements should be since I've been following MusicXML (and I had to look up the XML spec to be sure as well) so we can set out documentation for what things should be and help others to support it better going forward. :-)

(Particular thanks for documenting Finale's behavior, since this is something that will not change so importers will need to be aware of that -- Michael Good's adding of the "Software" tag (and unlike window.navigator -- never having people spoof it) is one of the unsung genius moves in music notation history -- if there's a mistake in someone's interpretation, everyone else can work around it (perhaps with a sigh, but they can).

FWIW -- Music21 also does not respect xml:space="preserve" in deciding to work with whitespace. It may in the future, now that I know about it, but it wasn't something I was thinking about at the time.

[lemzwerg]

Thanks for your kind reply.

So: What is actually the default for xml:space? 🙂

[mdgood]

Answers to many of these questions can be found in the XML spec for whitespace handling and end-of-line handling. The "default" value is application-specific while the "preserve" value signals that all the whitespace should be preserved. Line breaks get handled as if CR or CRLF were encoded as LF instead. These are XML standards that MusicXML inherits as an XML language.

Elements like <part-name> that have no xml:space attribute use application-specific whitespace handling, which could involve discarding extra white space or not. You can't add attributes to a MusicXML element that are not defined in the schema and still have a valid MusicXML document. As the XML spec says about xml:space, "In valid documents, this attribute, like any other, MUST be declared if it is used."

[ahankinson]

The terminology isn't the same, but the concept is there. I just find it useful to think of textual content (and also comments) as "just another tag", albeit special ones without a name. This is particularly useful when trying to reason about "mixed content"; that is, text intermingled with tags, e.g.,

<p>This is some <b>bold</b> text</p>

Thinking about this as "plain text" and "tags" is problematic, but it becomes easier when you think of it more like:

<p>
<[[PCDATA]]>This is some</>
  <b>
    <[[PCDATA]]>bold</>
  </b>
<[[PCDATA]]>text</>
</p>

The other way to do it (and the way I did it at first) was inspired by the lxml library that conceives of it as "text" and "tail" which, while easy at first, started to get complicated with complex docs.

[p].text = "This is some"
[b].text = "bold"
[b].tail = "text"

Namely, it removes the relationship between the word "text" which, obviously, belongs to the p element, and shifts it to the b element as the "tail". But otherwise there isn't an obvious way to intersperse arbitrary tags within a text stream.

[ahankinson]

@lemzwerg You may also wish to check out the XML Schema spec, which, while not part of the XML spec either, defines some behaviours that processors adhere to.

For example, the section on whitespace handling:

https://www.w3.org/TR/2004/REC-xmlschema-1-20041028/structures.html#components

Specifically this:

These three levels of normalization correspond to the processing mandated in XML 1.0 for element content, CDATA attribute content and tokenized attributed content, respectively. See Attribute Value Normalization in [XML 1.0 (Second Edition)] for the precedent for replace and collapse for attributes. Extending this processing to element content is necessary to ensure a consistent ·validation· semantics for simple types, regardless of whether they are applied to attributes or elements.

Which, essentially, points to the XML Spec and says that the rules for Attribute normalization should also apply to elements, as far as I read it.

https://www.w3.org/TR/REC-xml/#AVNormalize

Notably:

If the attribute type is not CDATA, then the XML processor must further process the normalized attribute value by discarding any leading and trailing space (#x20) characters, and by replacing sequences of space (#x20) characters by a single space (#x20) character.

While the text of the section itself isn't really clear (at least to me) the examples show that any line break should be removed.

a="

xyz"

becomes

x y z

(I think the "a is NMTOKEN" column here is misleading, since in no way do I think that the spaces should intersperse the "x y z" characters -- I think it's just to illustrate that no "hidden" characters should be printed here.)

phew That was a deep dive. Would you believe that I don't consider myself an XML "guy"? 😀

[lemzwerg]

Thanks, I have seen that. However, this describes normalization of attributes (i.e., the "blabla" string in <bar foo="blabla">urgh</bar>) and not the contents (i.e., urgh) of an element. AFAICS, there is no normalization defined for the latter.

[Edit: I think it is also possible to define the value of an attribute to be 'CDATA'. And I still don't see any reference to the contents of an element.]

[ahankinson]

These three levels of normalization correspond to the processing mandated in XML 1.0 for element content ... Extending this processing to element content is necessary to ensure a consistent ·validation· semantics for simple types, regardless of whether they are applied to attributes or elements.

^^^ That's what says the attribute normalization should be extended to element content normalization.

[lemzwerg]

OK, thanks. I read too fast.

[mscuthbert]

What I was wrong about is that I thought that any XML tag could add xml:space="preserve|default" without it needing to be in the spec, but I was incorrect about that -- requiring it as a specification seems to go against the notion that XML can be extended by other namespaces, but it's not something I'm one of the worlds' experts on.