I introduced the update.sh script:

$ ./maintainers/scripts/update.sh

With the latest version of our update.sh script we have the following output:

framework :: Dev/nix-community/ethereum.nix ‹feat/per-pkg-update-script*› % ./maintainers/scripts/update.sh                                                                                                                 1 ↵
++ git rev-parse --show-toplevel
+ rootDir=/home/aldo/Dev/nix-community/ethereum.nix
++ nix eval --raw -f /home/aldo/Dev/nix-community/ethereum.nix inputs.nixpkgs.outPath
+ nixpkgs=/nix/store/j8han9cf3g8vba52yhiklaa6a500pcbv-source
++ nix eval --raw -f /home/aldo/Dev/nix-community/ethereum.nix
+ flake=/nix/store/xkkzpgjfzg0k9kny36lckf0ckbw7vc1h-source
+ nix-shell --show-trace /nix/store/j8han9cf3g8vba52yhiklaa6a500pcbv-source/maintainers/scripts/update.nix --arg include-overlays '[(import /home/aldo/Dev/nix-community/ethereum.nix).overlays.default]' --arg keep-going true --arg predicate '(
    let
        prefix = "/nix/store/xkkzpgjfzg0k9kny36lckf0ckbw7vc1h-source/pkgs/";
        prefixLen = builtins.stringLength prefix;
    in
        (path: pkg: (builtins.substring 0 prefixLen pkg.meta.position) == prefix)
  )'
trace: warning: libsForQt5 now uses makeScopeWithSplicing which does not have "overrideScope'", use "overrideScope".
this derivation will be built:
  /nix/store/d8q0cq5b8f82cqrdhpl6zc6qz8g2rab2-packages.json.drv
building '/nix/store/d8q0cq5b8f82cqrdhpl6zc6qz8g2rab2-packages.json.drv'...

Going to be running update for following packages:
 - charon-0.18.0
 - dirk-1.2.0
 - dreamboat-0.4.20
 - eigenlayer-0.5.0
 - erigon-2.57.0
 - eth2-testnet-genesis-0.9.0
 - eth2-val-tools-0.1.1
 - ethdo-1.35.2
 - ethereal-2.8.10
 - geth-1.13.11
 - geth-sealer-geth-sealer-v1.13.4
 - heimdall-rs-0.7.3
 - mev-boost-1.6.0
 - builder-1.11.5-0.3.0
 - mev-boost-relay-0.27.0
 - prysm-4.2.1
 - rocketpool-1.11.7
 - sedge-1.2.3
 - python3.11-slither-0.10.0
 - ssv-1.2.2
 - staking-deposit-cli-2.7.0
 - teku-23.10.0
 - tx-fuzz-1.3.2
 - vouch-1.7.6
 - python3.11-eth-wake-4.3.0
 - zcli-0.6.0

Press Enter key to continue...

More refinements are necessary, probably I'll port the logic to update.py and update.nix to our repository and add proper support to flakes.

OK, the first step of tests is looking promising as I've been able to perform updates on each package quite fast. I'll allocate whatever time I have left this week and separate it into a standalone Python binary instead.

Really cool work! for filtering releases for #476 we can use the approach with --verison-regex/--version for nix-update (Mic92/nix-update#198), so extraArgs is really useful for that purpose.

For example, for mev-boost-prysm it works fine (we are using regex here cuz it can't determine the stable release):

$ nix-update --flake mev-boost-prysm --version-regex '(v\d+\.\d+\.\d+$)'
...
Update 4.0.0-boost0.2.0 -> 3.2.0

We also had the same problem with lighthouse until v5.0.0 release, it always tried to update lighthouse to v4.6.222-exp instead of v4.6.0

For vouch it can recognize the stable release, so we can use just --version

$ nix-update --flake vouch  --version=unstable
...
Update 1.7.6 -> 1.8.0-beta.3

But small question, I thought I would use the original update.nix from nixpkgs, I just wanted to add non-interactive mode to it, but you brought them here cuz the original was too slow with integration with flake via overlay and predicate filter, and you improved them to work faster and more naturally?

It's really impressive and much more than I thought about that, we can move that login to nix-update-script to help other projects use the same approach like in nixpkgs without creating their own update scripts

Resuming work again! I was busy these past two weeks with my newborn son!

But small question, I thought I would use the original update.nix from nixpkgs, I just wanted to add non-interactive mode to it, but you brought them here cuz the original was too slow with integration with flake via overlay and predicate filter, and you improved them to work faster and more naturally?

I had different ideas and was still unsure of the best approach when I was exploring the other day. But whilst I was using the update.nix from nixpkgs and adding our flake packages in the overlay, yep, I noticed it was traversing all pkgs whilst applying the filter, which is not ideal and added extra time to parse.

My latest way of thinking is just to improve the update.nix and the update.py code and move it into a separate project (maybe add it to the nix-community). That way we can allow other people to use the same tooling as nixpkgs but for flakes. Also, as it is written in Python, we can call directly nix-update as a library, for pkgs that doesn't incorporate any updateScript attrset.

It's really impressive and much more than I thought about that, we can move that login to nix-update-script to help other projects use the same approach like in nixpkgs without creating their own update scripts

Correct. You got the idea very well :)

Rebased to main + comment some problem packages for now.

@aldoborrero what do you think if we merge it as is and will continue to improve them via other PRs? It helps us to update all packages which isn't update right now

@brianmcgee also fyi

@selfuryon Yeah go ahead. I never published the stand-alone version I was working on to separate the tool.