Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ERROR:src/app/rpmostree-libbuiltin.c:83:rpmostree_print_treepkg_diff: assertion failed: (deployments->len > 1) #981

Closed
miabbott opened this issue Sep 8, 2017 · 2 comments
Assignees

Comments

@miabbott
Copy link
Member

miabbott commented Sep 8, 2017

Using Fedora 26 AH (26.120), I was able to get rpm-ostree to dump core while doing an rpm-ostree deploy.

I don't have a solid reproducer, as the host I was working on had gone through a lot of operations. I think I was in a weird state with two initial deployments, where the booted deployment had a pending commit that matched the rollback deployment. I'll just copy a bunch of details below.

# rpm-ostree status
State: idle
Deployments:
● fedora-atomic:fedora/26/x86_64/atomic-host
                   Version: 26.120 (2017-09-05 00:05:09)
                    Commit: 0b0127864022dd6ffad1a183241fbd5482ef5a1642ff3c8751c2e6cae6070b1a
             PendingCommit: d792307b3708271c44ae5e30dfea089e15f804dc79c6069248c5f5a9c233afdf
            PendingVersion: 26.119 (2017-09-03 21:47:35)

  fedora-atomic:fedora/26/x86_64/atomic-host
                   Version: 26.119 (2017-09-03 21:47:35)
                    Commit: d792307b3708271c44ae5e30dfea089e15f804dc79c6069248c5f5a9c233afdf

# rpm-ostree install wget
Checking out tree 0b01278... done
Enabled rpm-md repositories: updates fedora epel

Updating metadata for 'updates': [==============] 100%
rpm-md repo 'updates'; generated: 2017-09-08 02:22:27


Updating metadata for 'fedora': [==============] 100%
rpm-md repo 'fedora'; generated: 2017-07-05 20:31:38


Updating metadata for 'epel': [=================] 100%
rpm-md repo 'epel'; generated: 2017-09-08 02:18:36


Importing metadata [======================] 100%
Resolving dependencies... done
Will download: 1 package (742.3 kB)

  Downloading from fedora: [===================] 100%

Importing: [====================] 100%
Applying 1 overlay... done
Running pre scripts... 0 done
Running post scripts... 4 done
Writing rpmdb... done
Writing OSTree commit... done
Copying /etc changes: 23 modified, 0 removed, 76 added
Transaction complete; bootconfig swap: yes deployment count change: 0
Added:
  wget-1.19.1-3.fc26.x86_64
Run "systemctl reboot" to start a reboot

# rpm-ostree status
State: idle
Deployments:
  fedora-atomic:fedora/26/x86_64/atomic-host
                   Version: 26.120 (2017-09-05 00:05:09)
                BaseCommit: 0b0127864022dd6ffad1a183241fbd5482ef5a1642ff3c8751c2e6cae6070b1a
           LayeredPackages: wget

● fedora-atomic:fedora/26/x86_64/atomic-host
                   Version: 26.120 (2017-09-05 00:05:09)
                    Commit: 0b0127864022dd6ffad1a183241fbd5482ef5a1642ff3c8751c2e6cae6070b1a

# rpm-ostree ex livefs
Diff Analysis: 0b0127864022dd6ffad1a183241fbd5482ef5a1642ff3c8751c2e6cae6070b1a => aeffeed4faad1e5bcaf79b3ce300add9d376af45eb217775ca7adae4a5c867c5
Files: modified: 2 removed: 0 added: 106
Packages: modified: 0 removed: 0 added: 1
* Configuration changed in /etc
Preparing new rollback matching currently booted deployment
Copying /etc changes: 23 modified, 0 removed, 76 added
Transaction complete; bootconfig swap: yes deployment count change: 1
Overlaying /usr... done
Copying new config files... 1

# rpm-ostree cleanup -rpmb
Transaction complete; bootconfig swap: yes deployment count change: -2
Freed pkgcache branches: 1 size: 2.9 MB

# rpm-ostree status
State: idle
Deployments:
● fedora-atomic:fedora/26/x86_64/atomic-host
                   Version: 26.120 (2017-09-05 00:05:09)
              BootedCommit: 0b0127864022dd6ffad1a183241fbd5482ef5a1642ff3c8751c2e6cae6070b1a
                LiveCommit: aeffeed4faad1e5bcaf79b3ce300add9d376af45eb217775ca7adae4a5c867c5
             PendingCommit: d792307b3708271c44ae5e30dfea089e15f804dc79c6069248c5f5a9c233afdf
            PendingVersion: 26.119 (2017-09-03 21:47:35)

# rpm-ostree deploy 26.120
Resolving version '26.120'
1 metadata, 0 content objects fetched; 569 B transferred in 0 seconds
1 metadata, 0 content objects fetched; 569 B transferred in 0 seconds
No change.
**
ERROR:src/app/rpmostree-libbuiltin.c:83:rpmostree_print_treepkg_diff: assertion failed: (deployments->len > 1)
Aborted (core dumped)

# coredumpctl list
TIME                            PID   UID   GID SIG COREFILE  EXE
Fri 2017-09-08 13:55:38 UTC    2753     0     0   6 present   /usr/bin/rpm-ostree

# coredumpctl info
           PID: 2753 (rpm-ostree)
           UID: 0 (root)
           GID: 0 (root)
        Signal: 6 (ABRT)
     Timestamp: Fri 2017-09-08 13:55:38 UTC (3min 58s ago)
  Command Line: rpm-ostree deploy 26.120
    Executable: /usr/bin/rpm-ostree
 Control Group: /user.slice/user-1000.slice/session-1.scope
          Unit: session-1.scope
         Slice: user-1000.slice
       Session: 1
     Owner UID: 1000 (cloud-user)
       Boot ID: 6ab4ecf676b8416fb8b7087665874b66
    Machine ID: 68f9bcff35ff43d2bbf5f1e7a0a3058e
      Hostname: micah-f26ah-vm0906c.localdomain
       Storage: /var/lib/systemd/coredump/core.rpm-ostree.0.6ab4ecf676b8416fb8b7087665874b66.2753.1504878938000000.lz4
       Message: Process 2753 (rpm-ostree) of user 0 dumped core.
                
                Stack trace of thread 2753:
                #0  0x00007f220f08069b raise (libc.so.6)
                #1  0x00007f220f0824a0 abort (libc.so.6)
                #2  0x00007f220fac670d g_assertion_message (libglib-2.0.so.0)
                #3  0x00007f220fac679a g_assertion_message_expr (libglib-2.0.so.0)
                #4  0x0000564f4c5b3c81 rpmostree_print_treepkg_diff (rpm-ostree)
                #5  0x0000564f4c5b3d7e rpmostree_print_treepkg_diff_from_sysroot_path (rpm-ostree)
                #6  0x0000564f4c5a9ead rpmostree_builtin_deploy (rpm-ostree)
                #7  0x0000564f4c59c519 main (rpm-ostree)
                #8  0x00007f220f06a50a __libc_start_main (libc.so.6)
                #9  0x0000564f4c59c73a _start (rpm-ostree)
                
                Stack trace of thread 2754:
                #0  0x00007f220f14eacd poll (libc.so.6)
                #1  0x00007f220faa0569 g_main_context_iterate.isra.25 (libglib-2.0.so.0)
                #2  0x00007f220faa067c g_main_context_iteration (libglib-2.0.so.0)
                #3  0x00007f220faa06c1 glib_worker_main (libglib-2.0.so.0)
                #4  0x00007f220fac7536 g_thread_proxy (libglib-2.0.so.0)
                #5  0x00007f220f42236d start_thread (libpthread.so.0)
                #6  0x00007f220f15abbf __clone (libc.so.6)
                Stack trace of thread 2755:
                #0  0x00007f220f14eacd poll (libc.so.6)
                #1  0x00007f220faa0569 g_main_context_iterate.isra.25 (libglib-2.0.so.0)
                #2  0x00007f220faa0902 g_main_loop_run (libglib-2.0.so.0)
                #3  0x00007f2210085cb6 gdbus_shared_thread_func (libgio-2.0.so.0)
                #4  0x00007f220fac7536 g_thread_proxy (libglib-2.0.so.0)
                #5  0x00007f220f42236d start_thread (libpthread.so.0)
                #6  0x00007f220f15abbf __clone (libc.so.6)

# rpm -q rpm-ostree
rpm-ostree-2017.8-2.fc26.x86_64
@jlebon
Copy link
Member

jlebon commented Sep 8, 2017

Hmm, interesting. Will try to reproduce this.

@jlebon jlebon self-assigned this Sep 8, 2017
@miabbott
Copy link
Member Author

miabbott commented Sep 8, 2017

Try this:

  1. Boot 26.120
  2. rpm-ostree install wget
  3. rpm-ostree ex livefs
  4. rpm-ostree deploy 26.119
  5. rpm-ostree cleanup -rpmb
  6. rpm-ostree deploy 26.120

jlebon added a commit to jlebon/rpm-ostree that referenced this issue Sep 11, 2017
Currently, when setting the `override-commit` key in the origin, the
upgrader pulls that commit checksum directly and then updates the
refspec to point to it. This behaviour was inherited from its ostree
version; at the time it was implemented, the pull code didn't support
passing a specific commit for a given refspec. However, we now have
the override-commit-ids option, which will make libostree update the ref
for us.

We change the code here to make use of it and simplify the function.
This also fixes the corner case of local branches: we shouldn't change
the ref if we're on a local branch. This is actually what drove me to
this patch as I was debugging coreos#981.

(Aside: I'm still not convinced updating the refspec is always the
correct thing to do even in the remote case, though it's a bit messy to
disentangle).
jlebon added a commit to jlebon/rpm-ostree that referenced this issue Sep 11, 2017
Also something I noticed while working on coreos#981. When sitting on a livefs
commit, once a user does `rpm-ostree cleanup --pending --rollback`, it's
impossible to redeploy the same booted commit. Let's allow users to do
this.
jlebon added a commit to jlebon/rpm-ostree that referenced this issue Sep 11, 2017
Commands like `upgrade` and `deploy` need to know if a new deployment
was actually laid down so that it may print a pkg diff if so. This is
implemented by listening for changes to the DefaultDeployment D-Bus
property. D-Bus emits a signal when the deployment variant changes
value.

However, in coreos#595, with the introduction of `pending-*` related keys, the
deployment variant no longer represents data solely tied to that
specific deployment. In this case, because `deploy` operations currently
set the ref to the resolved checksum, it can happen that deploying the
same base commit when the current refspec *isn't* pointing to that base
commit will result in the `pending-*` keys dropping out and a default
deployment change notification going out.

In this patch, we strengthen how we determine whether a new deployment
was laid down by actually looking at the deployment id, rather than just
assuming that a change to the property implies a new deployment.

Closes: coreos#981
rh-atomic-bot pushed a commit that referenced this issue Sep 12, 2017
Currently, when setting the `override-commit` key in the origin, the
upgrader pulls that commit checksum directly and then updates the
refspec to point to it. This behaviour was inherited from its ostree
version; at the time it was implemented, the pull code didn't support
passing a specific commit for a given refspec. However, we now have
the override-commit-ids option, which will make libostree update the ref
for us.

We change the code here to make use of it and simplify the function.
This also fixes the corner case of local branches: we shouldn't change
the ref if we're on a local branch. This is actually what drove me to
this patch as I was debugging #981.

(Aside: I'm still not convinced updating the refspec is always the
correct thing to do even in the remote case, though it's a bit messy to
disentangle).

Closes: #984
Approved by: cgwalters
rh-atomic-bot pushed a commit that referenced this issue Sep 12, 2017
Also something I noticed while working on #981. When sitting on a livefs
commit, once a user does `rpm-ostree cleanup --pending --rollback`, it's
impossible to redeploy the same booted commit. Let's allow users to do
this.

Closes: #984
Approved by: cgwalters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants