[Feature Request] Can we upgrade or loose the version of antlr4?

[dnnspark]

🚀 Feature Request

hydra requires antlr4-python3-runtime==4.9.*, but the environment of the mono-repo of the company I work for requires antlr4-python3-runtime==4.11.1. I saw there was a similar issue previously. Can we either upgrade or loose the version (using >= not ==) of this package?

[Jasha10]

Thanks for the feature request @dnnspark.
There is related discussion in issues #2155 and omry/omegaconf#798.
Here are a few key points:

• The antlr version used to build Hydra must match the antlr version used to build OmegaConf
• The version of antlr4-python3-runtime should match the version of antlr that was used to build the OmegaConf/Hydra wheel/sdist (or at least the minor version of antlr should match).
• I'm not sure whether changing or loosening the antlr version would be compatible with Meta's monorepo. For this reason, I don't think we can change the pin at the moment.
• If you're using a monorepo with controlled dependencies, you can build your own OmegaConf+Hydra with a different antlr version. Here is a recipe for building the packages with antlr version 4.11.1:

# download the omegaconf/hydra sources and the antlr 4.11.1 binary:
git clone https://github.com/omry/omegaconf
git clone https://github.com/facebookresearch/hydra
wget https://www.antlr.org/download/antlr-4.11.1-complete.jar
# replace the old antlr binaries:
rm omegaconf/build_helpers/bin/antlr-4.9.3-complete.jar
rm hydra/build_helpers/bin/antlr-4.9.3-complete.jar
cp antlr-4.11.1-complete.jar omegaconf/build_helpers/bin/
cp antlr-4.11.1-complete.jar hydra/build_helpers/bin/
# edit text files to replace the strings "4.9.3" and "4.9.*" with "4.11.1":
grep -ErlI '4\.9\.[\*3]' | xargs sed -E -i 's/4\.9\.[\*3]/4.11.1/'
# build an sdist / wheel:
pip install build
(cd omegaconf; python -m build;)
(cd hydra; python -m build;)
# install the wheels and verify antlr version:
pip install omegaconf/dist/omegaconf-2.4.0.dev0-py3-none-any.whl
pip install hydra/dist/hydra_core-1.4.0.dev0-py3-none-any.whl
pip list 2>/dev/null | grep antlr4  # check that installed version of antlr4-python3-runtime is 4.11.1

[dnnspark]

Thanks for quick and thorough response, @Jasha10. I will try your suggestion.

[daskol]

@Jasha10 Is it possible to provide an option to override ANTLR4 jar? Similar issue appears in Arch distribution. Maintainers mitigate it with patching build scripts for omegaconf but the issue still exists since hydra-core and fairseq are in user repository AUR (fix in acxz/pkgbuilds#223).