Space and suffix gobbled in note field with LaTeX codes

[jdutant]

In CSL styles that print out the note field, ending the field with a LaTeX command causes formatting errors. I'm not sure if that is a Citeproc or Pandoc issue. Crossposted on jgm/pandoc#9582

Pandoc version: 3.1.12.2

Here's a MWE. You need a CSL style that prints out the note BibTeX field. I pasted one below (can't be attached). Or add this to any CSL file :

<bibliography>
...
    <layout>
       ...
      <text variable="note" text-case="capitalize-first" prefix=". " suffix="."/>
    <layout>
<bibliography>

Bib file test.bib. (In our actual use case, the LaTeX commands are cite commands, but the error appears with simpler ones too like \leftarrow.)

@book{geach_pt:1980	,
    author = "Geach, Peter Thomas",	
    date = "1980",	
    title = "Reference and Generality: \leftarrow",
    address = "Ithaca, New York",	
    publisher = "Cornell University Press",	
    edition = 3,
    note = "first edition: \leftarrow"
}

Markdown source:

---
title: "Biblio note test"
bibliography: test.bib
csl: minimal.csl
---

@geach_pt:1980

Native Pandoc output (pandoc -C mwe.md -t native):

...
     [ Div
        ( "ref-geach_pt:1980" , [ "csl-entry" ] , [] )
        [ Para
            [ SmallCaps [ Str "Geach" ]
            , Str ","
            , Space
            , Str "Peter"
            , Space
            , Str "Thomas."
            , Space
            , Str "1980."
            , Space
            , Emph
                [ Str "Reference"
                , Space
                , Str "and"
                , Space
                , Str "Generality:"
                , Space
                , RawInline (Format "latex") "\\leftarrow"
                ]
            , Str "."
            , Space
            , Str "First"
            , Space
            , Str "edition:"
            , Str ""
            , RawInline (Format "latex") "\\leftarrow"
            , Str ""
            ]
        ]
    ]

As you can see, in the note field we get empty strings instead of a space separating "edition:" from \leftarrow and a period at the end. This doesn't happen in the title.

The bug disappears if the note doesn't end up with a LaTeX command. Change the note field to:

    note = "first edition: \leftarrow and"

The result is:

            , Str "edition:"
            , Space
            , RawInline (Format "latex") "\\leftarrow "
            , Str "and."
            ]

If it ends with a LaTeX command but there are several LaTeX command, the period is there but the last LaTeX command is still preceded by an empty string. Change the note field to:

    note = "first edition: \leftarrow and \rightarrow"

The result is:

           , Str "edition:"
            , Space
            , RawInline (Format "latex") "\\leftarrow "
            , Str "and"
            , Str ""
            , RawInline (Format "latex") "\\rightarrow"
            , Str "."
            ]

Now we have a period at the end, and a space before the first LaTeX command, but again an empty string before the second LaTeX command.

The colon after 'edition:' isn't the culprit, the behaviour remains when it's removed. Adding spaces at the end of the note field doesn't change anything: if the last non-space element of the note is a LaTeX command, the bug appears.

minimal.csl

<?xml version="1.0" encoding="utf-8"?>
<style xmlns="http://purl.org/net/xbiblio/csl" class="in-text" version="1.0" demote-non-dropping-particle="display-and-sort" page-range-format="expanded">
  <info>
    <title>Minimal</title>
    <id>minimal-id</id>
    <author>
      <name>Anonymous</name>
      <email>[email protected]</email>
    </author>
    <category citation-format="author-date"/>
    <category field="generic-base"/>
    <summary>Minimal</summary>
    <updated>2024-03-13T17:14:15+00:00</updated>
    <rights license="http://creativecommons.org/licenses/by-sa/3.0/">This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License</rights>
  </info>
  <locale xml:lang="en">
    <terms>
      <term name="editor" form="verb-short">ed.</term>
      <term name="container-author" form="verb">by</term>
      <term name="translator" form="verb-short">trans.</term>
      <term name="editortranslator" form="verb">edited and translated by</term>
      <term name="translator" form="short">trans.</term>
    </terms>
  </locale>
  <macro name="contributors">
    <group delimiter=". ">
      <names variable="author">
        <name and="text" name-as-sort-order="first" sort-separator=", " delimiter=", " delimiter-precedes-last="always">
          <name-part name="family" font-variant="small-caps"/>
        </name>
        <label form="short" prefix=", "/>
        <substitute>
          <names variable="editor"/>
          <names variable="translator"/>
          <names variable="director"/>
          <text macro="title"/>
        </substitute>
      </names>
    </group>
  </macro>
  <macro name="contributors-short">
    <names variable="author">
      <name form="short" and="text" delimiter=", " initialize-with=". "/>
      <substitute>
        <names variable="editor"/>
        <names variable="translator"/>
        <names variable="director"/>
        <text macro="title"/>
      </substitute>
    </names>
  </macro>
  <macro name="title">
    <choose>
      <if variable="title" match="none">
        <choose>
          <if type="personal_communication" match="none">
            <text variable="genre" text-case="capitalize-first"/>
          </if>
        </choose>
      </if>
      <else-if type="bill book graphic legislation motion_picture song" match="any">
        <text variable="title" text-case="title" font-style="italic"/>
        <group prefix=" (" suffix=")" delimiter=" ">
          <text term="version"/>
          <text variable="version"/>
        </group>
      </else-if>
      <else-if variable="reviewed-author">
        <choose>
          <if variable="reviewed-title">
            <group delimiter=". ">
              <text variable="title" text-case="title" quotes="true"/>
              <group delimiter=", ">
                <text variable="reviewed-title" text-case="title" font-style="italic" prefix="Review of "/>
                <names variable="reviewed-author">
                  <label form="verb-short" text-case="lowercase" suffix=" "/>
                  <name and="text" delimiter=", "/>
                </names>
              </group>
            </group>
          </if>
          <else>
            <group delimiter=", ">
              <text variable="title" text-case="title" font-style="italic" prefix="Review of "/>
              <names variable="reviewed-author">
                <label form="verb-short" text-case="lowercase" suffix=" "/>
                <name and="text" delimiter=", "/>
              </names>
            </group>
          </else>
        </choose>
      </else-if>
      <else-if type="legal_case interview patent" match="any">
        <text variable="title"/>
      </else-if>
      <else>
        <text variable="title" quotes="true"/>
      </else>
    </choose>
  </macro>
  <macro name="point-locators">
    <choose>
      <if variable="locator">
        <choose>
          <if locator="page" match="none">
            <choose>
              <if locator="section" match="any">
                <group>
                  <text term="section" form="short"/>
                </group>
              </if>
              <else-if type="bill book graphic legal_case legislation motion_picture report song" match="any">
                <choose>
                  <if variable="volume">
                    <group>
                      <text term="volume" form="short" suffix=" "/>
                      <number variable="volume" form="numeric"/>
                      <label variable="locator" form="short" prefix=", " suffix=" "/>
                    </group>
                  </if>
                  <else>
                    <label variable="locator" form="short" suffix=" "/>
                  </else>
                </choose>
              </else-if>
              <else>
                <label variable="locator" form="short" suffix=" "/>
              </else>
            </choose>
          </if>
          <else-if type="bill book graphic legal_case legislation motion_picture report song" match="any">
            <number variable="volume" form="numeric" suffix=":"/>
          </else-if>
        </choose>
        <text variable="locator"/>
      </if>
    </choose>
  </macro>
  <macro name="date">
    <choose>
      <if variable="issued">
        <group delimiter=" ">
          <date variable="original-date" form="text" date-parts="year" prefix="(" suffix=")"/>
          <date variable="issued">
            <date-part name="year"/>
          </date>
        </group>
      </if>
      <else-if variable="status">
        <text variable="status" text-case="capitalize-first"/>
      </else-if>
      <else>
        <text term="no date" form="short"/>
      </else>
    </choose>
  </macro>
  <macro name="date-in-text">
    <choose>
      <if variable="issued">
        <group delimiter=" ">
          <date variable="original-date" form="text" date-parts="year" prefix="[" suffix="]"/>
          <date variable="issued">
            <date-part name="year"/>
          </date>
        </group>
      </if>
      <else-if variable="status">
        <text variable="status"/>
      </else-if>
      <else>
        <text term="no date" form="short"/>
      </else>
    </choose>
  </macro>
  <macro name="day-month">
    <date variable="issued">
      <date-part name="month"/>
      <date-part name="day" prefix=" "/>
    </date>
  </macro>
  <citation et-al-min="4" et-al-use-first="1" disambiguate-add-year-suffix="true" disambiguate-add-names="true" disambiguate-add-givenname="true" givenname-disambiguation-rule="primary-name" collapse="year" after-collapse-delimiter="; ">
    <layout prefix="(" suffix=")" delimiter="; ">
      <group delimiter=", ">
        <choose>
          <if variable="issued accessed" match="any">
            <group delimiter=" ">
              <text macro="contributors-short"/>
              <text macro="date-in-text"/>
            </group>
          </if>
          <else>
            <group delimiter=", ">
              <text macro="contributors-short"/>
              <text macro="date-in-text"/>
            </group>
          </else>
        </choose>
        <text macro="point-locators"/>
      </group>
    </layout>
  </citation>
  <bibliography hanging-indent="true" et-al-min="11" et-al-use-first="7" subsequent-author-substitute="—" entry-spacing="0">
    <sort>
      <key macro="contributors"/>
      <key variable="issued"/>
      <key variable="title"/>
    </sort>
    <layout suffix=".">
      <group delimiter=". ">
        <text macro="contributors"/>
        <text macro="date"/>
        <text macro="title"/>
      </group>
      <text variable="note" text-case="capitalize-first" prefix=". " suffix="."/>
    </layout>
  </bibliography>
</style>