Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Archive: Respect
filter_size
in query for existing nodes
The `QueryParams` dataclass defines the `filter_size` attribute which is used in all queries to limit the number of parameters used in a query. This is necessary because without it, large archives would result in queries with a lot of parameters which can cause exceptions in database backends, such as SQLite, which define a limit of 1000 by default. The `aiida.tools.archive._import_nodes` function was not respecting this setting when determining the set of nodes from the archive that already exist in the target storage. This would result in an exception when trying to import a large archive into a storage using SQLite. The problem is fixed by using the `batch_iter` utility to retrieve the existing UUIDs in batches of size `filter_size`.
- Loading branch information