Allow multiple/sectioned bibliographies

[adunning]

The idea of allowing for a bibliography that can be broken down into headings has come up on pandoc-discuss a couple of times (there is a very good thread on it from 2010, and I asked about it in August); it would be extremely useful for the many situations that require lists of resources on a number of topics (see e.g. a course site I made up), annotated bibliographies, or publications that ask items in a bibliography to be categorized in some way (e.g. many historical publications distinguish between primary sources and secondary works).

I hope it's not presumptuous of me to put in a request for this; there are some references to a bibsection capability in https://github.com/jgm/pandoc-citeproc/blob/master/tests/test-citeproc.hs#L34, but I assume these could be removed if there is no intent to implement this. A simpler alternative would be to implement something that works in reverse to the nocite option, omitting the specified items from the reference list, which would at least make it easier to deal with two bibliographies in a paper.

[dfaligertwood]

While I would like to see this in pandoc-citeproc proper, I have written a script (in dfaligertwood/pandoc-filters) that begins to implement this functionality. It is not really usable in its current form (see the exemplars in the readme), but if anyone knows a way to run pandoc-citeproc without it outputting a bibliography, it could be usable, albeit in a slightly hacky manner.

EDIT: The filter in question is 'pandoc-multibib.' link to repo: https://github.com/dfaligertwood/pandoc-filters

[dfaligertwood]

Finally got around to finishing pandoc-multibib. I haven't really tested it yet, but should work OK. You need to make a 'complete' bibliography, and then a number of files with the items you'd like to appear in each section. If something appears in the complete bibliography file but not in one of the sectional bibliographies, then it will still be referenced in-text but won't appear in the final reference list.

[nim65s]

It would also be great to section bibliographies, because when there is more than 3 or 4 references, they can not fit in a single slide (beamer / revealjs / whatever)

[sid-kap]

@dfaligertwood Did this ever get merged in?

[dfaligertwood]

@sid-kap It's not a patch for pandoc-citeproc itself, so wouldn't be included - it's a seperate filter which then (ab)uses pandoc-citeproc to actually do the citations. Not sure if it still works, as I don't have a use for it at the moment so haven't maintained it. Give it a go and feel free to hit me up if you have any issues: dfaligertwood/pandoc-filters

[ThomasPietrzak]

I'm trying to use it, but cannot get it to compile. I know nothing about Haskell, so I struggle a little. If you can make the required changes so that it compiles, I can test it. Thanks!

[ThomasPietrzak]

By the way, is it possible to use \nocite in some way with this filter?

[shiro]

Sorry to bring this up again, but has this been addressed?
It seems like a very common use case and a real blocker for writing papers in md (at least for me).

I believe the hacky scripts were meant to be a temporary workaround until this is supported by citerproc.

If there is a better way to achieve this please let me know. Thanks!

[ThomasPietrzak]

The workaround I use is a LateX file that I use in addition to the md document. It is sufficient for a CV, in which you have to separate journals, conferences, workshops, etc.

[shiro]

is it still possible to use citerproc in that case?
can you provide an example?
Thanks a lot!

[ThomasPietrzak]

I mean I produce the regular document with pandoc, and the bibliography part separately, then merge PDFs. This means I cannot cite the refs in the document. This is the (strong) limitation of this method.

[shiro]

The whole point is to be able to use pandoc support for citing, otherwise I'd rather use latex commands directly in my md file to have biblatex do what I want.
I assume that this is possible, still the syntax would be ugly, which kinda destroys the purpose of not writing this in latex in the first place...

Thanks anyways, hope this gets some attention in the near future.

[jgm]

I think the general approach pandoc-multibib takes is actually a pretty reasonable one, and it seems perfectly reasonable for this special case to be handled using another filter (applied after pandoc-citeproc in the filter chain). I have not tried pandoc-multibib myself. If there are problems compiling it, perhaps it needs some updating, or to make it easier for everyone, rewriting as a lua filter.

For reference, here's an example of how pandoc-multibib works. You specify multiple bibliographies, together with their titles, in addition to a main bibliography.

---
bibliographies:
  - test1.json: First Bibliography
  - test2.json: Second Bibliography
bibliography: test.json
---

[@Shepherd_1961_i3] [@Square_1884_i5]

Presumably the main bibliography would just be the concatenation of the subsidiary ones.

If this functionality were to be included in pandoc-citeproc, it could work in a similar way, but a little more conventently: if there's no bibliography field in metadata, look for bibliographies and read them all, keeping track of the sources of each item, then use this information at the end to sort the bibliography into sections.

Another approach would be to use the note field of bibliography items to specify the section; then only one bibliography would be needed. A special convention would have to be adopted, e.g. if note is section=Foo, then the item goes in the section titled Foo.

[shiro]

I agree, both suggestions sound pretty solid.

The before mentioned scripts are pretty old too. As I have no experience with Haskell, I couldn't get them to compile and wasted several hours on this (tried cabal and stack, but always ran into dependency issues).

whenever the solution is to rewrite the scripts or add it to citeproc, I would definitely appreciate some statically compiled binaries of the scripts, so people can use it now.

I just want to write my paper with markdown and be done with it 😭

Thanks!

[jgm]

pandoc-multibib compiles fine for me.
Steps:

1. clone pandoc-citeproc
2. get the Haskell 'stack' tool
3. in the pandoc-citeproc directory, do 'stack install'
4. then put the pandoc-multibib.hs source in this directory
5. do stack ghc -- --make pandoc-multibib.hs

You should have pandoc-multibib executable.

[seifferth]

I did something similar with a pandoc -> latex+biber -> pdf workflow, using a customised template for the job. Maybe implementing this feature for a pandoc+pandoc-citeproc -> anything workflow could be done in a similar manner.

I specify rules for splitting the bibliography in the metadata block as such:

---
splitbib:
    - title: Primary Sources
      rule:  keyword=primary
    - title: Secondary Sources
      rule:  notkeyword=primary
---

These rules are then handled by the following code in a latex template:

$if(bibliography)$
  $if(biblio-title)$\renewcommand{\bibname}{$biblio-title$}$endif$
  $if(splitbib)$
    \chapter{\bibname}
    $for(splitbib)$
    \section{$splitbib.title$}
    \printbibliography[heading=none,$splitbib.rule$]
    
    $endfor$
  $else$
    \chapter{\bibname}
    \printbibliography[heading=none]
  $endif$
$endif$

I believe that specifying rules explicitly in metadata is far more flexible than relying only on information contained in bibliography files (as do both approaches discussed so far), since it allows handling the same bibliography differently in different documents.

[dfaligertwood]

I've just updated the cabal file for pandoc-multibib, so it should at least now compile without too much fuss. I'm not in academia any more (unless anyone knows of a funded PhD going spare), so I have no particular need for it, but I have free time atm so may do some more work to implement the suggestions of embedded bibliographies and/or auto-concatenation.

I'm not sure how everyone else manages their citation DBs, but w.r.t. pandoc-multibib I used to keep everything in one 'global' zotero DB and then use a json filter to split out primary/secondary sources, hence the architecture.

Just for clarity, pandoc-multibib actually acts as a wrapper around pandoc-citeproc. This does result in some slightly confusing behaviour - specifically, items will not be in the bibliography unless they are cited in the text (or in the nocite key)

@shiro, just for reference - in future if you have issues compiling, just raise an issue on my repo or at me directly.

[tarleb]

Oh, I just learned about pandoc-multibib – very cool!
There is now also the multiple-bibliographies Lua filter, which I probably wouldn't have written if I had known about pandoc-multibib back then.

[dfaligertwood]

Ah, nice! I think you've used the same method I did, and being lua it doesn't require compilation so is probably the best option for non-techie users.