You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The database is initialized and the MAAS region controllers are up and running, forming an HA cluster
Actual behavior
All 3 MAAS region controllers are running the database migrations simultaneously, thus, one of them succeeds and the other two fail to be initialized.
Versions
Operating system: Jammy 22.04
Juju CLI: 3.6.2-genericlinux-amd64
Juju agent: 3.6.2
Charm revision: 74
Log output
2025-01-28 14:59:08 INFO unit.maas-region/19.juju-log server.go:326 maas-db:70: MAAS database credentials received for user 'relation-70'
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 Traceback (most recent call last):
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 82, in _execute
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 return self.cursor.execute(sql)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "pg_class_relname_nsp_index"
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 DETAIL: Key (relname, relnamespace)=(auth_permission_id_seq, 2200) already exists.
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 The above exception was the direct cause of the following exception:
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 Traceback (most recent call last):
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/bin/maas-region", line 8, in <module>
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 sys.exit(run())
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/lib/python3.10/site-packages/maasserver/region_script.py", line 81, in run
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 run_django(is_snap, is_devenv)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/lib/python3.10/site-packages/maasserver/region_script.py", line 70, in run_django
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 management.execute_from_command_line()
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 utility.execute()
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 413, in execute
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 self.fetch_command(subcommand).run_from_argv(self.argv)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/core/management/base.py", line 354, in run_from_argv
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 self.execute(*args, **cmd_options)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/core/management/base.py", line 398, in execute
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 output = self.handle(*args, **options)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/lib/python3.10/site-packages/maasserver/management/commands/dbupgrade.py", line 250, in handle
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 call_command(
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 181, in call_command
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 return command.execute(*args, **defaults)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/core/management/base.py", line 398, in execute
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 output = self.handle(*args, **options)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/core/management/base.py", line 89, in wrapped
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 res = handle_func(*args, **kwargs)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/core/management/commands/migrate.py", line 244, in handle
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 post_migrate_state = executor.migrate(
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/db/migrations/executor.py", line 117, in migrate
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/db/migrations/executor.py", line 227, in apply_migration
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 state = migration.apply(state, schema_editor)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/db/migrations/migration.py", line 126, in apply
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/db/migrations/operations/models.py", line 92, in database_forwards
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 schema_editor.create_model(model)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/db/backends/base/schema.py", line 345, in create_model
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 self.execute(sql, params or None)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/db/backends/base/schema.py", line 145, in execute
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 cursor.execute(sql, params)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 66, in execute
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 return executor(sql, params, many, context)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 79, in _execute
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 with self.db.wrap_database_errors:
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/db/utils.py", line 90, in __exit__
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 raise dj_exc_value.with_traceback(traceback) from exc_value
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 82, in _execute
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 return self.cursor.execute(sql)
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 django.db.utils.IntegrityError: duplicate key value violates unique constraint "pg_class_relname_nsp_index"
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 DETAIL: Key (relname, relnamespace)=(auth_permission_id_seq, 2200) already exists.
2025-01-28 14:59:12 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 Traceback (most recent call last):
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/bin/maas", line 8, in <module>
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 sys.exit(main())
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/lib/python3.10/site-packages/maascli/__init__.py", line 39, in main
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 options.execute(options)
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/lib/python3.10/site-packages/maascli/snap.py", line 433, in __call__
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 raise exc
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/lib/python3.10/site-packages/maascli/snap.py", line 430, in __call__
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 self.handle(options)
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/lib/python3.10/site-packages/maascli/snap.py", line 698, in handle
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 self._finalize_init(mode)
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/lib/python3.10/site-packages/maascli/snap.py", line 712, in _finalize_init
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 perform_work(
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/lib/python3.10/site-packages/maascli/snap.py", line 363, in perform_work
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 return cmd(*args, **kwargs)
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/snap/maas/38185/lib/python3.10/site-packages/maascli/snap.py", line 338, in migrate_db
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 subprocess.check_call(
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 raise CalledProcessError(retcode, cmd)
2025-01-28 14:59:13 WARNING unit.maas-region/19.maas-db-relation-changed logger.go:60 subprocess.CalledProcessError: Command '['/snap/maas/38185/bin/maas-region', 'dbupgrade']' returned non-zero exit status 1.
ubuntu@juju-76d202-36:~$ sudo maas status
Service Startup Current Since
apiserver enabled active today at 15:00 UTC
bind9 disabled active today at 15:00 UTC
http disabled active today at 15:00 UTC
ntp disabled active today at 15:00 UTC
proxy disabled active today at 15:01 UTC
regiond enabled active today at 15:00 UTC
syslog disabled active today at 15:00 UTC
temporal disabled active today at 15:00 UTC
temporal-worker disabled active today at 15:00 UTC
ubuntu@juju-76d202-35:~$ sudo maas status
MAAS is not configured
ubuntu@juju-76d202-34:~$ sudo maas status
MAAS is not configured
Additional context
The text was updated successfully, but these errors were encountered:
skatsaounis
changed the title
Failed to initilaize many node MAAS cluster simultaneously
Failed to initialize many node MAAS cluster simultaneously
Jan 30, 2025
skatsaounis
changed the title
Failed to initialize many node MAAS cluster simultaneously
Failed to initialize multi-node MAAS cluster simultaneously
Jan 30, 2025
Steps to reproduce
Expected behavior
The database is initialized and the MAAS region controllers are up and running, forming an HA cluster
Actual behavior
All 3 MAAS region controllers are running the database migrations simultaneously, thus, one of them succeeds and the other two fail to be initialized.
Versions
Operating system: Jammy 22.04
Juju CLI: 3.6.2-genericlinux-amd64
Juju agent: 3.6.2
Charm revision: 74
Log output
Additional context
The text was updated successfully, but these errors were encountered: