Biber reruncheck

[atticus-sullivan]

Supposed to fix #3 if biber is used.

To discussion:

• does original_wd.."/"..bib always work? (I'm not quite sure what kind of paths we can encounter in .bcf files, but as long as they are relative to the directory where the user invoked cluttex it should work)
• Maybe we should as well check if original_wd.."/"..bib is actually a real file (and print a warning if not), since I'm not sure if bcf:datasource is surely always a file. This way we also could somehow warn if original_wd.."/"..bib failed in some way.

@minoki Any thoughts on these points? (made this a draft as I'd like to discuss this before merging)

Still to be done is the bibtex support, but as I'm not using bibtex I'm not quite confident in testing this with a common setup. And there is no bcf file from where we'd get the .bib file(s) used (there has to be another way, as bibtex has to find these files as well somehow, but I just don't know it).

@minoki do you use BibTeX and can do this for the BibTeX option?

Also a small note: reruncheck.comparefiletime(mainauxfile, output_bbl, auxstatus) is being used in the bibtex option path (cluttex.lua:383). As far as I tested (tried to use an adapted version of this for the biber option), this will always return false as auxstatus works with absolute paths and mainauxfile appears to be relative. So in effect the bibtex option only uses the mechanism of comparing the hashes.

[atticus-sullivan]

Tested this for biblatex+biber for a while now and it works as expected (for my usecase). I added some comments about the processing.

Below is an example showing when what rebuild is triggered (and this is how I expect it)

Running some example cases

I used a long name for the .bib file because I wanted to check if in the .bcf file long lines are wrapped. And I stripped some paths (using relative paths), in the real output they are absolute of course.

# clean start (due to removing everything in the tex-aux folder)
$ rm -r tex-aux/* && cluttex --max-iterations=20 --output-directory="tex-aux" --change-directory --biber -e pdflatex "main.tex" > log
[EXEC] pdflatex -recorder -halt-on-error -interaction=nonstopmode -file-line-error -jobname='main' '\input"main.tex"'
[INFO] looooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong.bib is newer than aux
[EXEC] biber --output-directory '.' main.bcf
[INFO] New auxiliary file 'main.aux'.
[EXEC] pdflatex -recorder -halt-on-error -interaction=nonstopmode -file-line-error -jobname='main' '\input"main.tex"'
[INFO] File 'main.aux' was modified (size: 321 -> 389).
[EXEC] pdflatex -recorder -halt-on-error -interaction=nonstopmode -file-line-error -jobname='main' '\input"main.tex"'
[INFO] File 'main.run.xml' was modified (md5: ad3731e961a59b0834d1c04860e7a055 -> d473f1fbcd04cc7d2b127917cf618dc6). # here this run is not necessary, but the .run.xml changes (checked this) so we need to rerun in general
[EXEC] pdflatex -recorder -halt-on-error -interaction=nonstopmode -file-line-error -jobname='main' '\input"main.tex"'
[EXEC] cp './main.pdf' '../main.pdf'

# just change the timestamp of the bib file
$ touch looooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong.bib
$ cluttex --max-iterations=20 --output-directory="tex-aux" --change-directory --biber -e pdflatex "main.tex" > log
[EXEC] pdflatex -recorder -halt-on-error -interaction=nonstopmode -file-line-error -jobname='main' '\input"main.tex"'
[INFO] looooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong.bib is newer than aux
[EXEC] biber --output-directory '.' main.bcf
[EXEC] cp './main.pdf' '../main.pdf'
# bbl wasn't changed -> no rerun necessary

# change the contents of the bib file
$ vim looooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong.bib
$ cluttex --max-iterations=20 --output-directory="tex-aux" --change-directory --biber -e pdflatex "main.tex" > log
[EXEC] pdflatex -recorder -halt-on-error -interaction=nonstopmode -file-line-error -jobname='main' '\input"main.tex"'
[INFO] looooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong.bib is newer than aux
[EXEC] biber --output-directory '.' main.bcf
[INFO] File 'main.bbl' was modified (md5: ba1c78dc6fdfaf8c19ed24b06cdb2478 -> 3573a4eb35b598b72e8a2d1b0338386b).
[EXEC] pdflatex -recorder -halt-on-error -interaction=nonstopmode -file-line-error -jobname='main' '\input"main.tex"'
[INFO] File 'main.aux' was modified (md5: c553687ab70ef781212220a1e9a140a3 -> d658b8cebf4f8b2e89840beb41662cba).
[EXEC] pdflatex -recorder -halt-on-error -interaction=nonstopmode -file-line-error -jobname='main' '\input"main.tex"'
[EXEC] cp './main.pdf' '../main.pdf'

# change biblography unrelated things in the tex file
$ vim main.tex
$ cluttex --max-iterations=20 --output-directory="tex-aux" --change-directory --biber -e pdflatex "main.tex" > log
[EXEC] pdflatex -recorder -halt-on-error -interaction=nonstopmode -file-line-error -jobname='main' '\input"main.tex"'
# no biber rerun or so
[EXEC] cp './main.pdf' '../main.pdf'

If you've see any issue with this behavior and/or the changes, please let me know.

EDIT: And I did not work on bibtex further as I'm not that familiar with the workflow with bibtex (I only use biber)

[atticus-sullivan]

Yes this might be unstable. But if the output is different, in 99% of the cases, no match is being found. Thus nothing bad happens, only the additional rerun might not be triggered.

[atticus-sullivan]

Oh I just saw, I included the .bin file by accident in this PR (so there are changes which don't belong here in this PR).

[atticus-sullivan]

Except for BibTeX and some a bit unclear things about the bcf files (not really problematic if something is wrong here), I'm through with this.

Anything you can say about the mentioned reruncheck.comparefiletime(mainauxfile, output_bbl, auxstatus)?

[minoki]

@atticus-sullivan Thank you for contribution and sorry for late reply.

As for reruncheck.comparefiletime(mainauxfile, output_bbl, auxstatus), you are right, it seems.

I noticed update_dot_bib variable is always overwritten if there are multiple bib files. So only the last bib file would be checked for change. This point should be fixed before (or after) merge.

[atticus-sullivan]

Fixed the update_dot_bib issue.

What shall we do about the reruncheck.comparefiletime in the bibtex case? Fix the path or remove the check?

[minoki]

I would like to fix the path in the bibtex case.

[atticus-sullivan]

Done 👍

[minoki]

I'll tidy up the history a bit and merge this.