More type annotations and error fixing

[mgrabovsky]

Following up to #352

Resolves #347

[mgrabovsky]

Not yet thoroughly tested.

[packit-as-a-service]

Congratulations! One of the builds has completed. 🍾

You can install the built RPMs by following these steps:

• sudo yum install -y dnf-plugins-core on RHEL 8
• sudo dnf install -y dnf-plugins-core on Fedora
• dnf copr enable packit/abrt-retrace-server-356
• And now you can install the packages.

Please note that the RPMs should be used only in a testing environment.

[DaveWysochanskiRH]

Thanks for doing this series. This gets the vmware snapshot test case a bit different failure (see last line), but maybe this makes sense?

[2020-08-04 12:27:15] [I] Downloading remote resources
[2020-08-04 12:27:15] [I] Retrieving local file '/cores/bugzilla/regression-test/rhel7/vmware.vmem'
[2020-08-04 12:27:15] [D] File type: data
[2020-08-04 12:27:15] [D] unknown file type, unpacking finished
[2020-08-04 12:27:15] [D] Trying hardlink
[2020-08-04 12:27:15] [D] Succeeded
[2020-08-04 12:27:15] [I] Post-processing downloaded file
[2020-08-04 12:27:15] [D] File type: data
[2020-08-04 12:27:15] [D] unknown file type, unpacking finished
[2020-08-04 12:27:15] [D] File type: data
[2020-08-04 12:27:15] [D] unknown file type, unpacking finished
[2020-08-04 12:27:15] [I] Downloading remote resources
[2020-08-04 12:27:15] [I] Retrieving local file '/cores/bugzilla/regression-test/rhel7/vmware.vmss'
[2020-08-04 12:27:15] [D] File type: diy-thermocam raw data (lepton 2.x), scale 28263-29440, spot sensor temperature 0.000000, unit celsius, color scheme 0, calibration: offset 0.000000, slope 18179583481275050988767358746624.000000
[2020-08-04 12:27:15] [D] unknown file type, unpacking finished
[2020-08-04 12:27:15] [D] Trying hardlink
[2020-08-04 12:27:15] [D] Succeeded
[2020-08-04 12:27:15] [I] Post-processing downloaded file
[2020-08-04 12:27:15] [D] File type: diy-thermocam raw data (lepton 2.x), scale 28263-29440, spot sensor temperature 0.000000, unit celsius, color scheme 0, calibration: offset 0.000000, slope 18179583481275050988767358746624.000000
[2020-08-04 12:27:15] [D] unknown file type, unpacking finished
[2020-08-04 12:27:15] [D] File type: diy-thermocam raw data (lepton 2.x), scale 28263-29440, spot sensor temperature 0.000000, unit celsius, color scheme 0, calibration: offset 0.000000, slope 18179583481275050988767358746624.000000
[2020-08-04 12:27:15] [D] unknown file type, unpacking finished
[2020-08-04 12:27:15] [E] local variable 'largest_file' referenced before assignment

[mgrabovsky]

@DaveWysochanskiRH Oops, missed that one. I'm progressively working on sorting things out.

[DaveWysochanskiRH]

The latest fixes the vmware snapshots - wohoo!

One more thing you may want to fix up because retrace-server-interact has a Trackback due to "get_crashrc_path()" and the print_cmdline() function - patch below that fixes it for me.

$ retrace-server-interact 798404482 crash
If you want to execute the command manually, you can run
Traceback (most recent call last):
  File "/usr/bin/retrace-server-interact", line 167, in <module>
    print_cmdline(cmdline)
  File "/usr/bin/retrace-server-interact", line 28, in print_cmdline
    sys.stderr.write("$ %s\n\n" % list2cmdline(cmdline))
  File "/usr/lib64/python3.6/subprocess.py", line 481, in list2cmdline
    needquote = (" " in arg) or ("\t" in arg) or not arg
TypeError: argument of type 'PosixPath' is not iterable
$ diff -Nurp /usr/bin/retrace-server-interact /tmp/retrace-server-interact 
--- /usr/bin/retrace-server-interact	2020-08-04 12:23:38.000000000 -0400
+++ /tmp/retrace-server-interact	2020-08-05 10:16:55.656921948 -0400
@@ -154,13 +154,13 @@ if __name__ == "__main__":
                 cfgdir = Path(CONFIG["SaveDir"], "%d-kernel" % task.get_taskid())
                 if task.has_crashrc():
                     cmdline = ["/usr/bin/mock", "--configdir", str(cfgdir),
-                               "shell", "crash -i %s %s %s" % (task.get_crashrc_path(), str(vmcore_path), vmlinux)]
+                               "shell", "crash -i %s %s %s" % (str(task.get_crashrc_path()), str(vmcore_path), vmlinux)]
                 else:
                     cmdline = ["/usr/bin/mock", "--configdir", str(cfgdir),
                                "shell", "crash %s %s" % (str(vmcore_path), vmlinux)]
             else:
                 if task.has_crashrc():
-                    cmdline = task.get_crash_cmd().split() + ["-i", task.get_crashrc_path(), str(vmcore_path), vmlinux]
+                    cmdline = task.get_crash_cmd().split() + ["-i", str(task.get_crashrc_path()), str(vmcore_path), vmlinux]
                 else:
                     cmdline = task.get_crash_cmd().split() + [str(vmcore_path), vmlinux]
 
$ /tmp/retrace-server-interact 798404482 crash
If you want to execute the command manually, you can run
$ /cores/crashext/gitlab-runner/bin/crash -i /cores/retrace/tasks/798404482/crashrc /cores/retrace/tasks/798404482/crash/vmcore /cores/retrace/repos/kernel/x86_64/usr/lib/debug/lib/modules/2.6.32-642.el6.x86_64/vmlinux


crash 7.2.8++
Copyright (C) 2002-2020  Red Hat, Inc.
...

[mgrabovsky]

@DaveWysochanskiRH Thanks for pointing that out. I've just pushed some fixes for retrace-server-interact.

[DaveWysochanskiRH]

@DaveWysochanskiRH Thanks for pointing that out. I've just pushed some fixes for retrace-server-interact.

I ran my regression vmcores / tests and this set looks very good! Thanks for getting this done.

[DaveWysochanskiRH]

Ok I spoke too soon... there is a regression in dedup_vmcore:

==> /var/log/retrace-server/cleanup_error.log-20200805 <==
    total_savings += worker.dedup_vmcore(md5_tasks[md5])
  File "/usr/lib/python3.6/site-packages/retrace/retrace_worker.py", line 627, in dedup_vmcore
    v2_link.link_to(v1)
AttributeError: 'PosixPath' object has no attribute 'link_to'
Traceback (most recent call last):
  File "/usr/bin/retrace-server-cleanup", line 145, in <module>
    total_savings += worker.dedup_vmcore(md5_tasks[md5])
  File "/usr/lib/python3.6/site-packages/retrace/retrace_worker.py", line 627, in dedup_vmcore
    v2_link.link_to(v1)
AttributeError: 'PosixPath' object has no attribute 'link_to'

[DaveWysochanskiRH]

Ok I spoke too soon... there is a regression in dedup_vmcore:

==> /var/log/retrace-server/cleanup_error.log-20200805 <==
    total_savings += worker.dedup_vmcore(md5_tasks[md5])
  File "/usr/lib/python3.6/site-packages/retrace/retrace_worker.py", line 627, in dedup_vmcore
    v2_link.link_to(v1)
AttributeError: 'PosixPath' object has no attribute 'link_to'
Traceback (most recent call last):
  File "/usr/bin/retrace-server-cleanup", line 145, in <module>
    total_savings += worker.dedup_vmcore(md5_tasks[md5])
  File "/usr/lib/python3.6/site-packages/retrace/retrace_worker.py", line 627, in dedup_vmcore
    v2_link.link_to(v1)
AttributeError: 'PosixPath' object has no attribute 'link_to'

I think this may be an unrelated problem I'm only seeing now - it looks like this was changed back in April.

[mgrabovsky]

Ah, yes, the link_to method was introduced in Python 3.8. It was called symlink_to previously. (Correction: symlink_to creates a symbolic link while link_to creates a hard link. So there was no facility for hard links before 3.8.) I'll see what I can do.