Suggestions for a workflow running checks with latex-dev?

[muzimuzhi]

Running checks using the development LaTeX format (l3build check --dev) may result in testfiles different from those when using the stable LaTeX format.

For a repository hosted on GitHub and utilizes scheduled GitHub Action job runs, do you have suggestions on how to manage two copies of testfiles?

[josephwright]

I'd imagined that people testing against dev would be doing it manual and wanting to spot anything reporting to the team or noting for the future. If one wants to automate, I guess you could split tests into two

• Those that work the same for main and dev
• Those that need separate .tlg files for dev
  then use three directories: the core tests, one main-only, one dev-only, and use the cofig approach to run only the right ones.

[muzimuzhi]

Problem is that for upstream changes, it's impossible to know beforehand which testfiles (.tlg, .pdf) will vary between latex and latex-dev.

For example, latex3/latex2e@7726e8545 (ltmarks & multicol (#1548), 2024-11-18) changed mark implementations:

-...\marks1{\__mark_id:n {1}\MakeUppercase    []{Contents}}
+...\marks1{\__mark_value:nn {1}{\MakeUppercase    []{Contents}}}

So after the next latex dev release, .tlg tests in package repositories using \showoutput all need updates.

Basically more regular works (to update test list and testfiles) are needed. And I see no automatic or easier ways.

[josephwright]

Well yes, that's what I meant about manually checking and being ready - I'm not sure one can deal with this in an automated way, as the changes may be trivial or more significant.

[FrankMittelbach]

What I would do, I guess, is to add a branch 2e-dev and in that branch run the tests with the dev format and already update testfiles that fail for understandable reasons. That way you do the work that is necessary when the next LaTeX release hits the street already upfront and when the release changes you merge that branch back into your main and be done. So in total I don't think you really end up with more work, but you have the advantage that you can ensure your package in both releases.

It gets more complicated if a test file fail is due to something in your package that would break with the next release. But there, assuming you want the package to work with both releases you have to again put something into your package code to make that happen. That would be developed on the current main and then merged into both branches.

So bottom line I think it only offers you the ability to prepare for the next release before it is really there, but not adding much extra burden because what you do before doesn't have to be done when the release arrives (in haste because something is broken). The only extra is to run tests on both branches and extra merges you have to do to keep the branches working.