Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot import data into existing collection (each time I import, a new collection with the same name is created) #5193

Closed
Aclz opened this issue Nov 13, 2024 · 4 comments · Fixed by #5200
Labels
bug Something isn't working

Comments

@Aclz
Copy link

Aclz commented Nov 13, 2024

Vaultwarden Support String

Your environment (Generated via diagnostics page)

  • Vaultwarden version: v1.32.4
  • Web-vault version: v2024.6.2c
  • OS/Arch: linux/x86_64
  • Running within a container: true (Base: Debian)
  • Environment settings overridden: true
  • Uses a reverse proxy: true
  • IP Header check: true (X-Real-IP)
  • Internet access: true
  • Internet access via a proxy: false
  • DNS Check: true
  • Browser/Server Time Check: false
  • Server/NTP Time Check: true
  • Domain Configuration Check: true
  • HTTPS Check: true
  • Database type: SQLite
  • Database version: 3.46.0
  • Clients used:
  • Reverse proxy and version:
  • Other relevant information:

Config (Generated via diagnostics page)

Show Running Config

Environment settings which are overridden: ADMIN_TOKEN

{
  "_duo_akey": null,
  "_enable_duo": false,
  "_enable_email_2fa": false,
  "_enable_smtp": true,
  "_enable_yubico": true,
  "_icon_service_csp": "",
  "_icon_service_url": "",
  "_ip_header_enabled": true,
  "_max_note_size": 10000,
  "_smtp_img_src": "cid:",
  "admin_ratelimit_max_burst": 3,
  "admin_ratelimit_seconds": 300,
  "admin_session_lifetime": 20,
  "admin_token": "***",
  "allowed_iframe_ancestors": "",
  "attachments_folder": "data/attachments",
  "auth_request_purge_schedule": "30 * * * * *",
  "authenticator_disable_time_drift": false,
  "data_folder": "data",
  "database_conn_init": "",
  "database_max_conns": 10,
  "database_timeout": 30,
  "database_url": "***************",
  "db_connection_retries": 15,
  "disable_2fa_remember": false,
  "disable_admin_token": false,
  "disable_icon_download": false,
  "domain": "*****://************************",
  "domain_origin": "*****://************************",
  "domain_path": "",
  "domain_set": true,
  "duo_context_purge_schedule": "30 * * * * *",
  "duo_host": null,
  "duo_ikey": null,
  "duo_skey": null,
  "duo_use_iframe": false,
  "email_2fa_auto_fallback": false,
  "email_2fa_enforce_on_verified_invite": false,
  "email_attempts_limit": 3,
  "email_change_allowed": true,
  "email_expiration_time": 600,
  "email_token_size": 6,
  "emergency_access_allowed": true,
  "emergency_notification_reminder_schedule": "0 3 * * * *",
  "emergency_request_timeout_schedule": "0 7 * * * *",
  "enable_db_wal": true,
  "enable_websocket": true,
  "enforce_single_org_with_reset_pw_policy": false,
  "event_cleanup_schedule": "0 10 0 * * *",
  "events_days_retain": null,
  "experimental_client_feature_flags": "fido2-vault-credentials",
  "extended_logging": true,
  "helo_name": null,
  "hibp_api_key": null,
  "http_request_block_non_global_ips": true,
  "http_request_block_regex": null,
  "icon_blacklist_non_global_ips": true,
  "icon_blacklist_regex": null,
  "icon_cache_folder": "data/icon_cache",
  "icon_cache_negttl": 259200,
  "icon_cache_ttl": 2592000,
  "icon_download_timeout": 10,
  "icon_redirect_code": 302,
  "icon_service": "internal",
  "incomplete_2fa_schedule": "30 * * * * *",
  "incomplete_2fa_time_limit": 3,
  "increase_note_size_limit": false,
  "invitation_expiration_hours": 120,
  "invitation_org_name": "Vaultwarden",
  "invitations_allowed": true,
  "ip_header": "X-Real-IP",
  "job_poll_interval_ms": 30000,
  "log_file": null,
  "log_level": "info",
  "log_timestamp_format": "%Y-%m-%d %H:%M:%S.%3f",
  "login_ratelimit_max_burst": 10,
  "login_ratelimit_seconds": 60,
  "org_attachment_limit": 5242880,
  "org_creation_users": "",
  "org_events_enabled": false,
  "org_groups_enabled": false,
  "password_hints_allowed": true,
  "password_iterations": 100000,
  "push_enabled": false,
  "push_identity_uri": "https://identity.bitwarden.com",
  "push_installation_id": "***",
  "push_installation_key": "***",
  "push_relay_uri": "https://push.bitwarden.com",
  "reload_templates": false,
  "require_device_email": false,
  "rsa_key_filename": "data/rsa_key",
  "send_purge_schedule": "0 5 * * * *",
  "sendmail_command": null,
  "sends_allowed": true,
  "sends_folder": "data/sends",
  "show_password_hint": false,
  "signups_allowed": false,
  "signups_domains_whitelist": "",
  "signups_verify": false,
  "signups_verify_resend_limit": 6,
  "signups_verify_resend_time": 3600,
  "smtp_accept_invalid_certs": false,
  "smtp_accept_invalid_hostnames": false,
  "smtp_auth_mechanism": null,
  "smtp_debug": false,
  "smtp_embed_images": true,
  "smtp_explicit_tls": true,
  "smtp_from": "************************",
  "smtp_from_name": "Bitwarden",
  "smtp_host": "************",
  "smtp_password": "***",
  "smtp_port": 465,
  "smtp_security": "force_tls",
  "smtp_ssl": true,
  "smtp_timeout": 15,
  "smtp_username": "************************",
  "templates_folder": "data/templates",
  "tmp_folder": "data/tmp",
  "trash_auto_delete_days": null,
  "trash_purge_schedule": "0 5 0 * * *",
  "use_sendmail": false,
  "use_syslog": false,
  "user_attachment_limit": 102400,
  "user_send_limit": null,
  "web_vault_enabled": true,
  "web_vault_folder": "web-vault/",
  "yubico_client_id": null,
  "yubico_secret_key": null,
  "yubico_server": null
}

Vaultwarden Build Version

1.32.4

Deployment method

Official Container Image

Custom deployment method

No response

Reverse Proxy

nginx/1.18.0 (Ubuntu)

Host/Server Operating System

Linux

Operating System Version

Ubuntu 20.04

Clients

Web Vault

Client Version

2024.6.2c Firefox 132.0.1

Steps To Reproduce

  1. Importing data via Tools -> Import data
  2. Select my organization in "Import destination" field
  3. Select one of the existing collections in "Collection" field
  4. Select "Bitwarden CSV" as a file format
  5. Paste import data

Expected Result

The data is imported into an existing collection chosen in the "Collection" field

Actual Result

The data is imported into a newly created collection with the same name as the chosen collection.

Logs

No response

Screenshots or Videos

No response

Additional Context

No response

@Aclz Aclz added the bug Something isn't working label Nov 13, 2024
@Aclz Aclz changed the title Cannot import data to existing collection Cannot import data into existing collection (a new collection is created each import attempt) Nov 13, 2024
@Aclz Aclz changed the title Cannot import data into existing collection (a new collection is created each import attempt) Cannot import data into existing collection (each time I import, a new collection with the same name is created) Nov 13, 2024
@BlackDex
Copy link
Collaborator

If you exported items from an org which are already in a collection (with the same name or not), and import that export into a different org which might have a collection with the same name but not the same uuid, then it will create a new one.

This is not something we can fix, as the server doesn't know what the names of the collections are, and we can't match them.

If you exported it from the exact same org, delete every cipher, and then import, it should use the same collections.

It might be that it has a link with #4700, fixed in #4702
In that case we might be able to fix it.

Could you provide more details like, is it from Org X to Org Y.
And i Org Y there just is a collection with the exact same name as in X, but both are created manually?

@Aclz
Copy link
Author

Aclz commented Nov 13, 2024

Could you provide more details like, is it from Org X to Org Y. And i Org Y there just is a collection with the exact same name as in X, but both are created manually?

No, I don't import anything previously exported, just import some brand new data, like generated somewhere else username-password pairs.

The pasted csv data for import is something like this:

type,name,notes,fields,reprompt,login_uri,login_username,login_password,login_totp
login,user1,,,0,,user1,pass123,
login,user2,,,0,,user2,pass456,

When I took a look an the HTTP request made by the web client when I press "Import" (the call to /api/ciphers/import-organization), I see the same collection ID passed in the request JSON as long as I choose the same collection to import into. So I assume the client passes the known collection ID to the server to import to the particular existing collection. But I always get a newly created collection.

@BlackDex
Copy link
Collaborator

Ok cool. Then it looks like it's the exact same issue as with the folders. Which should be not that hard to resolve.

Thanks for the details.

BlackDex added a commit to BlackDex/vaultwarden that referenced this issue Nov 16, 2024
This fixes an issue with collections be duplicated same as was an issue with folders.
Also made some optimizations by using HashSet where possible and device the Vec/Hash capacity.
And instead of passing objects only use the UUID which was the only value we needed.

Also found an issue with importing a personal export via the Org import where folders are used.
Since Org's do not use folder we needed to clear those out, same as Bitwarden does.

Fixes dani-garcia#5193

Signed-off-by: BlackDex <[email protected]>
@BlackDex
Copy link
Collaborator

Found and solved the issue. Thanks again for reporting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants