typesetting Asciidoc (DocBook) markup

[gour]

Hello,

It might be that Markdown markup might be too simple for my next project, so I'm considering alternatives...Asciidoc(tor) is one option, and wonder if there are any plans in providing support for typesetting that markup?

Of course, Asciidoc has a DocBook backend, so wonder what is a status of docbook.class ?

[Omikhleia]

Greetings,

The current docbook class is very limited: see for instance #1338 (forgiving my possibly harsh tone at the time I tried it, lol). It's definitively in need of care and love, if someone feels the guts to tackle with the task...

This being said, I don't know Asciidoc and wonder what minimal subset of DocBook might be necessary for it?

• An alternative, of course, is to use Markdown with Pandoc-flavored extensions; or Djot.¹
• Pandoc can convert Asciidoc to other formats, but it remains to be seen how much information is possibly lost. What are the important features from Asciidoc that wouldn't be possible with Markdown (enriched with Pandoc-style extensions) or Djot?

As for the broader question ("are there any plans in providing support for markup XXX?"), I can't speak for others, but I don't think there are such plans as part of the core distribution. This would be a nice project for 3rd-party package contributors, whatever XXX is ;)

Footnotes

1. Djot support was just added yesterday evening to my markdown.sile work in progress 😸 ↩

[jslabovitz]

Mostly I'm thinking about the ability to define "master" document which does input/include other chapter, so it's easy to get the whole book out with front-matter, chapters, appenix, index, glossary.

This is more of an FYI rather than a solution, but I've been working on a couple of books that have typical structures like this.

I've cobbled together some fairly simple code to assemble the books, coming from a large number of Markdown files, each named with a numeric prefix so that they stay in the desired order. (In my case, I'm not yet using the Markdown stuff built into either standard SILE or Resilient, but rather a custom Markdown-XML formatter written in Ruby, my preferred language.) The code also writes a 'master' XML file that sets up the document class & other options, and uses SILE's include command to add all the generated XML files. It works well enough: I can now issue a single CLI command to build each book to a single PDF file.

My XML converter is a custom converter for the Kramdown library, a Ruby Markdownesque gem, and writes SILE-compatible XML. It’s easier for me to build those with Ruby than SIL-format, and also because I seem to have fewer whitespace/newline problems (a known issue with SILE). But I am planning on messing around with pandoc and other methods mentioned in this thread.

[Omikhleia]

Mostly I'm thinking about the ability to define "master" document which does input/include other chapter, so it's easy to get the whole book out with front-matter, chapters, appenix, index, glossary.

Well, it's kind of doable with a mix of solution and languages. One can already have a master file in, say, SILE language, responsible for \include'ing several sub-documents (possibly written in SILE, Markdown, Djot, name-your-choice) and for switching styles in between important parts.

That's more or less how my (in-progress) book project will likely end up (ideally):

• titlepage in a mix of SILE language and Markdown, preface and introductory chapters all in Markdown, main matter from XML sources, backmatters in SILE.
• master SILE file including all these bits and switching things appropriately at some points, e.g. the main matter has a different page layout than introductory chapters - i.e. different running headers and different frame layouts

[gour]

This is more of an FYI rather than a solution, but I've been working on a couple of books that have typical structures like this.

Thank you.

I've cobbled together some fairly simple code to assemble the books, coming from a large number of Markdown files, each named with a numeric prefix so that they stay in the desired order. (In my case, I'm not yet using the Markdown stuff built into either standard SILE or Resilient,

I'm glad you mentioned Resilient since I was totally unaware of it. 😢

but rather a custom Markdown-XML formatter written in Ruby, my preferred language.) The code also writes a 'master' XML file that sets up the document class & other options, and uses SILE's include command to add all the generated XML files. It works well enough:

I was interested in Ruby long ago considering it fine language and (probably) better option than Python, but these days I simply don't have much time for programming. However, Lua seems as nice option if we settle on SILE. 😄

I can now issue a single CLI command to build each book to a single PDF file.

Yeah, that's what it wanted here as well. 😉

[gour]

Well, it's kind of doable with a mix of solution and languages. One can already have a master file in, say, SILE language, responsible for \include'ing several sub-documents (possibly written in SILE, Markdown, Djot, name-your-choice) and for switching styles in between important parts.

Hmm, that sounds good.

* titlepage in a mix of SILE language and Markdown, preface and introductory chapters all in Markdown, main matter from XML sources, backmatters in SILE.

How do you create "XML sources" and why?

* master SILE file including all these bits and switching things appropriately at some points, e.g. the main matter has a different page layout than introductory chapters - i.e. different running headers and different frame layouts

As far as writing content, I believe that some markup like Markdown (or reST, org-mode) is preferable (for me), but I also wonder about pro/cons in using e.g. Markdown directly instead of converting it to SILE markup via e.g. Pandoc?

[jslabovitz]

It seems there are a few of us here who are actively working on complex book typesetting & production. What if we start a wiki page for this topic, where we could describe successful workflows, techniques, technologies, issues, etc.? (Yes, I volunteer to edit it.)

[Omikhleia]

How do you create "XML sources" and why?

Because the original input I am typesetting in that part of the book exists in some XML format.¹

Since SILE can process XML natively (with an adequate package support for the corresponding XML schema), I can avoid the step that would imply converting the XML (via XSLT or whatever). I directly process the source XML and implement a package for that XML schema (well, the necessary subset at least. It's a huge schema).²

IMHO it's much simpler in that case (where the XML schema implies lot of layout decisions anyway, and I can keep all these in a single place).

But we are deviating quite a bit from the original question 😸

Footnotes

1. Without entering into the specifics, it's not DocBook (unfortunately, as of the initial topic of this discussion), but a specific subset of some other XML standard schema. ↩

2. The approach is quite similar to what I did here for another (specific subset of a) XML schema. ↩

[gour]

Because the original input I am typesetting in that part of the book exists in some XML format.

I did some research any my bet is that you speak about TEI format, right?

But we are deviating quite a bit from the original question smile_cat

Indeed. Everything is more or less clear now and I can proceed further. 😉

[Omikhleia]

but I also wonder about pro/cons in using e.g. Markdown directly instead of converting it to SILE markup via e.g. Pandoc?

It's an opinionated topic, at best. "Why would the intermediate SILE markup be needed at all?" would be my first-level answer ;-)

[gour]

It's an opinionated topic, at best. "Why would the intermediate SILE markup be needed at all?" would be my first-level answer ;-)

Well, it could be that SILE markup is not an intermediate one allowing richer expression than other markups (Markdown, reST,...) ❓

[alerque]

If you're talking about converting Markdown → SIL (via Pandoc like I do or any other transformation) then the "richer expression" doesn't really come into it since you are limited by the maximum expressiveness of Markdown and of Pandoc's understanding of it, so that really isn't an argument for using it as an intermediate format. If you wanted to use it as a source format that would be different of course.

BTW just something to throw out, one of the things my Pandoc writer does is handle RAW SIL markup during the conversion and pass it through to the output, so when Markdown reaches it's limits I can also reach for deeper tooling. Also since my sources include Markdown + a Lua module per document plus the template document that the conversion gets wrapped in is configurable, I can always reach advanced tooling while still having the bulk of my sources in Markdown.

@Omikhleia has a point though, and his markdown.sile module is arguably the most straightforward path from Markdown to typesetting, not having the intermediate format is quite an advantage for some workflows.

[gour]

If you wanted to use it as a source format that would be different of course.

I'm thinking about that...like one would write directly in LaTeX or ConTeXt...

BTW just something to throw out, one of the things my Pandoc writer does is handle RAW SIL markup during the conversion and pass it through to the output, so when Markdown reaches it's limits I can also reach for deeper tooling.

RAW SIL markup...that's interesting and my first encounter with it.

Also since my sources include Markdown + a Lua module per document plus the template document that the conversion gets wrapped in is configurable, I can always reach advanced tooling while still having the bulk of my sources in Markdown.

Having most of the sources in Markdown and able to typeset them in high quality is one of the nice features in SILE.

I also believe that SILE might win over ConTeXt due to organization of the project and docs. IIRC, the preferred LMTX distribution of ConTeXt does not ship with modules (including one for Markdown) and installing them is more difficult than it should be...

@Omikhleia has a point though, and his markdown.sile module is arguably the most straightforward path from Markdown to typesetting, not having the intermediate format is quite an advantage for some workflows.

His work is simply great, afaict. 😉

[alerque]

RAW SIL markup...that's interesting and my first encounter with it.

Pandoc has several extensions for raw inputs—by which it means one format embedded in another. You might have a Markdown document with some HTML embebed:

# Document

* With some <span class="special">special</span> markup in raw HTML

Of course that's a bad example because the span could be represented in markdown with [special]{.special}, but you get the idea. Processed with the +raw_html extension any markup that looks like HTML will get passed through to HTML on conversion instead of having all the characters converted to entities such as some <span class=.... The same is true for the +raw_tex extension where Markdown sources can name drop some special LaTeX commands that will be left in place when converting:

# Document

* With some \textsc{special} markup in raw HTML

Again not a great example because there are other ways to encode small caps in Markdown, but we're going for easy to understand examples here.

Likewise the SILE Writer I wrote for Pandoc adds an extension +raw_sil that allows embedding anything that is valid SIL syntax in Markdown or other document types.

# Document

* With some \font[features=+smcp]{special} markup in raw HTML

I used this a lot in my early projects using Markdown + SILE (via CaSILE) but have found myself using it less and less (to the point of I can't remember the last time I needed it) since Pandoc added native DIV support and I added div and span wrapper class handling to my Pandoc export handling. For example I used to need if for things like typesetting dedications or epigraphs:

\dedication{To all the book worms.}

# Predace {-}

Blah blah

But since the advent of Markdown div wrappers I can do this:

::: dedication
To all the book worms.
:::

# Predace {-}

Blah blah

This in my project module I register something to handle it:

self:registerCommand("class:dedication", function (options, content)
	options.eject = SU.boolean(options.eject, true)
	SILE.scratch.headers.skipthispage = true
	SILE.scratch.counters.folio.off = 2
	SILE.call("center", {}, function ()
		SILE.settings:set("linespacing.method", "fit-font")
		SILE.settings:set("linespacing.fit-font.extra-space", SILE.length("0.4ex plus 0.1ex minus 0.1ex"))
		SILE.call("topfill")
		SILE.call("cabook:font:dedication", {}, content)
	end)
	if options.eject then SILE.call("eject") end
end)

There are of course still use cases for RAW SIL code in my markup, but in managing dozens of publishing projects I've found it's generally better to keep the content itself as free from typesetting hacks as possible. I can fuss with the typesetting in the SILE module all I want, but I also need to be able to convert the same sources to clean epubs and other formats, so having good clean content markup makes that process easier.

[gour]

Pandoc has several extensions for raw inputs—by which it means one format embedded in another. You might have a Markdown document with some HTML embebed:

[...]

It looks that I'm a bit behind what is possible to do with Markdown. 🌞

Of course that's a bad example because the span could be represented in markdown with [special]{.special}, but you get the idea.

/me nods

Again not a great example because there are other ways to encode small caps in Markdown, but we're going for easy to understand examples here.

I appreciate it!

Likewise the SILE Writer I wrote for Pandoc adds an extension +raw_sil that allows embedding anything that is valid SIL syntax in Markdown or other document types.

Is it going to be part of the official release one day?

I used this a lot in my early projects using Markdown + SILE (via CaSILE) but have found myself using it less and less (to the point of I can't remember the last time I needed it) since Pandoc added native DIV support

Huh, that's also new for me. 😞

There are of course still use cases for RAW SIL code in my markup, but in managing dozens of publishing projects I've found it's generally better to keep the content itself as free from typesetting hacks as possible.

I'm strongly with you here...

I can fuss with the typesetting in the SILE module all I want, but I also need to be able to convert the same sources to clean epubs and other formats, so having good clean content markup makes that process easier.

What do you think about Djot ?

I also wonder whether I should keep usage of SILE for high-quality typesetting of articles, books...while using something else for the purpose of (PDF) slide-presentations?