Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[reboot-cause] A broken symlink of previous-reboot-cause file triggers a back trace in syslog and system-running state shows degraded #10746

Closed
mlok-nokia opened this issue May 4, 2022 · 0 comments · Fixed by #10751
Assignees
Labels
MSFT Triaged this issue has been triaged

Comments

@mlok-nokia
Copy link
Contributor

Description

When the previous-reboot-cause has a broken symlink, the following back trace is shown in the syslog and system-runing state show degraded. And rebooting the system will not be able to generated a new symlink for the new previous reboot cause.

May  3 22:00:49.145591 sonic INFO determine-reboot-cause[2200]: Traceback (most recent call last):
May  3 22:00:49.145671 sonic INFO determine-reboot-cause[2200]:   File "/usr/local/bin/determine-reboot-cause", line 228, in <module>
May  3 22:00:49.145752 sonic INFO determine-reboot-cause[2200]:     main()
May  3 22:00:49.145801 sonic INFO determine-reboot-cause[2200]:   File "/usr/local/bin/determine-reboot-cause", line 215, in main
May  3 22:00:49.145842 sonic INFO determine-reboot-cause[2200]:     os.symlink(REBOOT_CAUSE_HISTORY_FILE, PREVIOUS_REBOOT_CAUSE_FILE)
May  3 22:00:49.145877 sonic INFO determine-reboot-cause[2200]: FileExistsError: [Errno 17] File exists: '/host/reboot-cause/history/reboot-cause-2022_05_03_22_00_49.json' -> '/host/reboot-cause/previous-reboot-cause.json'
admin@sonic:~$ systemctl is-system-running 
degraded 

Steps to reproduce the issue:

  1. Manually made the broken symlink of the previous-reboot-cause file by using "ls /host/previous-reboot-cause -al" to find the soft link destination file and removing the destination file
  2. Reboot the system. the system will be in that state
  3. Check the syslog for the back trace and using "systemctl is-system-running" to check the result

Describe the results you received:

The following is shown in the syslog:

May  3 22:00:49.145591 sonic INFO determine-reboot-cause[2200]: Traceback (most recent call last):
May  3 22:00:49.145671 sonic INFO determine-reboot-cause[2200]:   File "/usr/local/bin/determine-reboot-cause", line 228, in <module>
May  3 22:00:49.145752 sonic INFO determine-reboot-cause[2200]:     main()
May  3 22:00:49.145801 sonic INFO determine-reboot-cause[2200]:   File "/usr/local/bin/determine-reboot-cause", line 215, in main
May  3 22:00:49.145842 sonic INFO determine-reboot-cause[2200]:     os.symlink(REBOOT_CAUSE_HISTORY_FILE, PREVIOUS_REBOOT_CAUSE_FILE)
May  3 22:00:49.145877 sonic INFO determine-reboot-cause[2200]: FileExistsError: [Errno 17] File exists: '/host/reboot-cause/history/reboot-cause-2022_05_03_22_00_49.json' -> '/host/reboot-cause/previous-reboot-cause.json'

check the system running state:

admin@ixre-egl-board15:~$ systemctl is-system-running 
degraded
admin@ixre-egl-board15:~$ 

Describe the results you expected:

  1. No back trace in syslog file
  2. checking the running state
    admin@sonic:~$ systemctl is-system-running
    running

Output of show version:

It is the latest code.

Output of show techsupport:

(paste your output here or download and attach the file here )

Additional information you deem important (e.g. issue happens only occasionally):

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
MSFT Triaged this issue has been triaged
Projects
None yet
3 participants