--flatten tries to expand \input commands even if they depend on argument to a user-defined command (e.g. \input{#1.tex} )

[averter]

I know this is not the first time this error appears in latexdiff but I don't understand what I am doing wrong.
Essentially I want to apply latexdiff-git between two commits of my thesis. I am using version 1.1.1 of Latexdiff. Since I have several \input{} commands I think (correct me if wrong) that the correct sintax is
latexdiff-git --flatten -r com1 -r com2 thesis.tex,
where com1 and com2 are the commit ref numbers.
Specifically the error states:
Couldn't open /tmp/vwLfCZssA7/latexdiff-vc-/picsBW/#1.pdf_tex: No such file or directory at /usr/bin/latexdiff line 1432.
Remarks

• In my thesis I have a pics/ and picsBW/ folder, specifically for pictures in color and black and white, respectively. But I don't understand how that can be affecting latexdiff.
• In older commits git was tracking files inside of those folders, but now all of them are in the .gitignore file.

Do you have any suggestions that I might try? Running latexdiff between the latest commit and the current files also didn't work.

[ftilmann]

If you use the --flatten option then to be on the safe side, latexdiff extracts a whole copy of the directory as committed in the chosen commit into a temporary directory. In your case I assume the newer commits, which were checked in after you made the change to .gitignore, did not contain the image files (at least not ones added after that change), so when latexdiff tries the flattening, it cannot find these. Try --flatten=keep-intermediate, which will result in the same error, but latexdiff-git will not delete the extracted directories. You can inspect them (in the current directory) to see what files are missing (just running latex separately in the two versions should show this without even invoking latexdiff). You can then copy the missing (image?) files from your working directory by hand, and should be good to go.
If successful please let me know, also if you solved it another way (for the benefit of future users, who might get directed here by a search engine).

[averter]

thanks @ftilmann . I've tried the keep-intermediate option and I noticed several things:

• the two extracted directories (ED) were missing figures as you predicted;
• then, I've copied the figures from the working directory (WD) into each (ED);
• I checked that I was able to compile the tex files inside of each ED;
• FInally I ran latexdiff as before in the WD and I was greeted with the same exact error. :-(. Do I need to change anything in the command syntax? or does latexdiff uses the extracted directories if they exist?

[ftilmann]

Did you run:

latexdiff --flatten ED1/master.tex ED2/master.tex > master-diff.tex

(with ED1,2 your extracted old and new directories, and master.tex your top-level tex file). If so and it still fails, I would need more information. Please use the '--debug' option and paste the output as well as the result of ls on both directories.

[ftilmann]

Is this file latexdiff-vc-c394f60/picsBW/#1.pdf_tex actually there but for some reason not found by latexdiff, or is it not needed, but latexdiff thinks it is? If the latter, it could either be because latexdiff does not recognise that a certain part is never executed, or because of some pattern matching mishap it looks for the wrong file. I am afraid I am at the end of what I can do without actual access to a failing example. From the titles of your files it looks like you are in writing-up stress but could I ask you to construct an MWE (minimal working example) , i.e. you strip your thesis down to a very simply document illustrating the problem (probably just containing the include/input/.. command for the offending figure in the body).

From the diagnosis so far the fault is with the latexdiff --flatten option, not with latexdiff-git, so it is enough to have old and new directory, no actual repository needed; it might be enough to just make one directory and supply this both as old and new). It should latex OK with all the files provided in the tar, but create equivalent error for latexdiff. I think you can attach the tar to a comment, or use email (you should be able to find it by Google - my institution is FU Berlin)

[averter]

I have a hint on what's happening. The below if-else statement of the preamble controls which directory is used to include figures (coloured or black and white). Notice the {./picsBW/#1.pdf} bit and specially the \input{} option. Latexdiff got jammed there.

\newif\iflogvar % false = color
% \logvartrue     % true = greyscale

\iflogvar % BLACK AND WHITE PICTURES
  \newcommand{\includesvg}[1]{%
    \executeiffilenewer{./picsBW/#1.svg}{./picsBW/#1.pdf}%
    {inkscape -z -D --file=./picsBW/#1.svg %
      --export-pdf=./picsBW/#1.pdf --export-latex}%
    \input{./picsBW/#1.pdf_tex}%
  }
  \graphicspath{{./picsBW/}}       %% Allows \incgraphics paths at all time
\else % COLOURED PICTURES
  \newcommand{\includesvg}[1]{%
    \executeiffilenewer{./pics/#1.svg}{./pics/#1.pdf}%
    {inkscape -z -D --file=./pics/#1.svg %
      --export-pdf=./pics/#1.pdf --export-latex}%
    \input{./pics/#1.pdf_tex}%
  }
  \graphicspath{{./pics/}}       %% Allows \incgraphics paths at all time
\fi

I've then tried to change this section into simply

  \newcommand{\includesvg}[1]{%
    \executeiffilenewer{./pics/#1.svg}{./pics/#1.pdf}%
    {inkscape -z -D --file=./pics/#1.svg %
      --export-pdf=./pics/#1.pdf --export-latex}%
    \input{./pics/#1.pdf_tex}%
  }
  \graphicspath{{./pics/}}       %% Allows \incgraphics paths at all time

submitting two minor commits and running latexdiff again. The returned error now is
Couldn't open /tmp/Ij8cJdFvLl/latexdiff-vc-0a46107/pics/#1.pdf_tex: No such file or directory at /usr/bin/latexdiff line 1432.

So it is the new includesvg command alone that is scrambling latexdiff, due to the \input{} bit!! The flatten option from latexdiff is trying to place the figures and is loosing track of what's happening.
Sorry for not noticing it immediately, despite being so obvious. I didn't realize it had an input command there.
But I don't see a workaround. I cannot work without the includesvg setup :-(

[averter]

Wouldn't be sensible if latexdiff --flatten ignored \input commands when inside a \newcommand environment? Do you think this might be a proposed enhancement?

[averter]

I was trying to experiment with the --exclude-textcmd option, like so
latexdiff-git --flatten --exclude-textcmd="includesvg" -r 0a46107 -r 73dea64 warwickthesis.tex
but without success. I will keep trying other ways

[ftilmann]

Your suggestion in the comment one up is not a bad one, but not quite trivial to implement and in some contexts one might wish for input within \newcommand to be expanded. What would be easier to implement is to never expand \input arguments containing a hash (#) symbol as they are likely arguments in a \newcommand environments. That would solve your problem but again might be undesirable in some situations (I will leave the issue open to remind me to implement this special check)

I can suggest three possible workarounds, but all require you to fiddle with the 'old' files, so you have to change history or do some pre-processing after extraction.

1. Move the \newcommand into preamble (\input in preamble is not expanded by latexdiff).
2. Put a space between \input and its arguments. I think this should be seen as identical by latex but will prevent latexdiff's pattern matching from finding it (in current version always, in future versions only if --allow-spaces option is not selected)
3. Define your own input command, e.g. \def\myinput\input and use that instead of the original input whenever you want it to be ignored by --flatten

In every case, you then have to be careful where you latex the diff.tex file, as it will need to be able to find the included figures.

PS You should be able to get rid of the 'Deprecated' warnings by upgrading to the latest version of latexdiff

[averter]

Brilliant! For me number 2 was the easiest and worked near perfectly. Thank you so much.
There is just a minor thing. I am using AUCTeX and Emacs to control the multiple files of my thesis. Each time I generate a latexdiff file I have to remove all the text that refers to the master file, i.e.

%%% Local Variables:
%%% TeX-master: "master"
%%% End:

and set it as the new masterfile. Minor thing but just wanted to mention it.
I am quite happy with this workflow and learned a lot about latexdiff. Great tool :-)

[ftilmann]

I have changed the processing now such that files not found in this phase lead to a warning rather than an error (the input command is left alone in this case). For 'filenames' contain #0-9, it is assumed that there are part of a command definition and not even a warning is issued. So the workaround should no longer be necessary Commit: 4c20a8b