Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize Import for large dataset #206

Open
ponceta opened this issue Apr 15, 2024 · 2 comments
Open

Optimize Import for large dataset #206

ponceta opened this issue Apr 15, 2024 · 2 comments
Labels
enhancement New feature or request INTERLIS About INTERLIS exchange format (import / export)

Comments

@ponceta
Copy link
Member

ponceta commented Apr 15, 2024

We want the user to be able to import large datasets from INTERLIS 2020_1 to TEKSI.

@ponceta ponceta changed the title Optimize Import large dataset Optimize Import for large dataset Apr 15, 2024
@ponceta ponceta added enhancement New feature or request INTERLIS About INTERLIS exchange format (import / export) labels Apr 15, 2024
@sjib
Copy link
Contributor

sjib commented Apr 15, 2024

It takes very long when deselecting a class on the import wizard form (up to 5 or even 10 minutes). I can only deselect one class at a time
20240415_deselect_class_import

@domi4484
Copy link
Contributor

At the moment all to be imported objects are stored in a sql alchemy session which can be seen as a sort of transaction. When we deselect/select a class in the import dialog, all objects are removed/added to that session and this is a slow operation.
The solution to that in my opinion is to rewrite the dialog and import procedure so that all objects are in memory only at the beginning. By accepting the dialog only the selected object/classes are added to the session and committed to the working schema.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request INTERLIS About INTERLIS exchange format (import / export)
Projects
None yet
Development

No branches or pull requests

3 participants