fix: add --forward flag to PATCH_EXECUTABLE command

[lemire]

Recent work by @ScottBailey making applying patches easy. However, I find that if the configuration is triggered more than once (necessary if the config changed), the tool may sometimes try to patch again the dependency. It then fails and the cmake —build command fails. It should be fine to just abort if the patch has already been applied, which is what --forward flag should do. We use -N as a shorthand for --forward in the code.

[ScottBailey]

This seems reasonable, but what happens if the patch is itself bad?

In my testing and development, I don't think this tried to re-apply. I wonder if that's because I was using SHA256 summing?

[lemire]

In my testing and development, I don't think this tried to re-apply. I wonder if that's because I was using SHA256 summing?

Here is the error I am getting...

I get this error if I touch a CMakeLists.txt file and hit cmake --build... then CMake automagically refresh the configuration.

My fix currently is to delete and restart but that's not ideal.

[ScottBailey]

My fix currently is to delete and restart but that's not ideal.

Not ideal at all.

[lemire]

@ScottBailey The project where I encounter this issue is open: https://github.com/simdjson/experimental_json_builder/blob/main/benchmarks/CMakeLists.txt

[ScottBailey]

https://github.com/simdjson/experimental_json_builder/blob/main/benchmarks/CMakeLists.txt

This is an interesting issue. I do NOT have this problem. What is your dev environment? What's different between our systems?

I am Debian Linux with cmake 3.30.0-rc3. Granted, I couldn't compile the project. But I don't think that should matter.

Do you have a cache path set? I do.

[lemire]

What is your dev environment?

I do not think it matters, but we do provide detailed instructions. I reproduce it here for your convenience. (It is in our README file.) Note that these steps will work. The failure only comes if you trigger a reconfiguration. Then it tries patching again, finds out that it cannot and it breaks the build. See screenshot above.

We are assuming that you are running Linux or macOS. We recommend that Windows users rely on WSL.

Follow the following two steps:

1. Clone the project

git clone https://github.com/simdjson/experimental_json_builder.git

2. Make sure that you have docker installed and running on your system. Most Linux distributions support docker though some (like RedHat) have the equivalent (Podman). Users of Apple systems may want to consider OrbStack. You do not need to familiar with docker, you just need to make sure that you are have it running.

3. Navigate to our repository experimental_json_builder and run the following bash script:

bash run_docker.sh bash

This will enter a bash shell with access to the repo directory. Note that this will take some time when running it for the first time, since the specific container image has to be built.

This step builds and executes a docker container defined by our Dockerfile which provides the necessary environment.

4. Configure the build system with cmake:

cmake -B build

This only needs to be done once.

5. Build the code...

cmake --build build

6. Run the tests...

ctest --test-dir build --output-on-failure

7. Run the benchmark.

./build/benchmarks/src/SerializationBenchmark

You can modify the source code with your favorite editor and run again steps 6 (Build the code) and 7 (Run the tests) and 8 (Run the benchmark).

Do you have a cache path set? I do.

I do not.

[ScottBailey]

Do you have a cache path set? I do.

I do not.

This. I do not know why; however, when I remove my export of CPM_SOURCE_CACHE I can repeat your defect. When I re-enable named caching, the defect goes away.

Before we commit your change, we need to write up and understand this new issue. I won't have time to dig into this for a little while. :-( Maybe not even for a couple weeks if it's a complicated problem. :-(

As a short term workaround you can set a cache directory as so: export CPM_SOURCE_CACHE=$HOME/.cache/CPM.

But here's my thoughts on how we should test this:

1. Generate a super simple test repo (or use the one for PATCHES).
2. Use CMake --trace to compare initial runs and post-touch of the list files both with and without CPM_SOURCE_CACHE set.

[ScottBailey]

I'm going to try to work on this this weekend.

[ScottBailey]

I wonder if patch --force is a better choice? --batch assumes reversed, and I'm not sure that's the right approach.

[ScottBailey]

@lemire Maybe we should convert this to a draft?

[lemire]

@ScottBailey I have updated the PR. If you use -- forward, then repeated application of the same patch should do nothing.

[lemire]

Maybe we should convert this to a draft?

I'd like you to consider it. You can reject/close the PR, of course.

[ScottBailey]

Maybe we should convert this to a draft?

I'd like you to consider it. You can reject/close the PR, of course.

No, actually I can't! I'm not a maintainer, just an interested party! :-)

Patching is a problem within CMake itself, there could be a "baked in" tool that "just works". In fact, there will be one day, but not yet. When we get that, we will rework the PATCHES option. For now, PATCHES is a bit of a work around for a broken part of CMake proper. It gives us the functionality we expect from a package manager while we wait for someone to fix this upstream. (I've looked into it a bit, but I don't have the bandwidth to implement the change CMake's maintainers want. PATCHES is the compromise.)

Regarding this PR, I have a few thoughts. I hope you'll consider them.

The problem you've written this PR to address is not with the call to patch, it's to and so this is not really a fix. See #577.

In package management, we might want to take a source (some_lib-v1.2.tgz) and apply patches (some_CVE.patch). I don't want a system that will fail to apply the patch without error and compile the library. What happens if the patch is for a critical vulnerability and it isn't applied? For this reason, if I were a maintainer here, I'd reject this PR.

I started working #577, but it's gonna take me some time to figure it out. In the meantime, obviously, this is broken for you. Well, we have the one workaround: set CPM_SOURCE_CACHE.

But that might not be enough for you. Maybe you could add a PATCHES_FLAGS option that you could -N or any flag to on an as needed basis?

[lemire]

Closing.