Handle broken pipes in arc_summary

[dinatale2]

Description

Using a command similar to 'arc_summary.py | head' causes
a broken pipe exception. Gracefully exit in the case of a
broken pipe in arc_summary.py.

Motivation and Context

#6965

How Has This Been Tested?

In a CentOS 7 VM.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the ZFS on Linux code style requirements.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.
• All commit messages are properly formatted and contain Signed-off-by.
• Change has been approved by a ZFS on Linux member.

[richardelling]

FWIW, a more comprehensive solution is to implement a generic signal handler that exits quietly. This allows you to catch SIGINT (aka control+c) gracefully, too.

[behlendorf]

I'm still able to cause a warning with the patch using python 2.7.12 on Ubuntu 16.04.

$ ./cmd/arc_summary/arc_summary.py  | head -1

close failed in file object destructor:
sys.excepthook is missing
lost sys.stderr

[dinatale2]

@behlendorf I'm not seeing that error doing the same test on Ubuntu 16.04 with python 2.7.12 with the current version of the patch. Can you test it one more time?

[loli10K]

I wonder if this is a bug in the Python interpreter; i can reproduce it easily running ./cmd/arc_summary/arc_summary.py -p 6 | head -n 1 in a loop:

root@linux:~/zfs-arcsum-broken-pipe# python -V
Python 2.7.12
root@linux:~/zfs-arcsum-broken-pipe# lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04 LTS
Release:	16.04
Codename:	xenial
root@linux:~/zfs-arcsum-broken-pipe# while :; do ./cmd/arc_summary/arc_summary.py -p 6 | head -n 1 ; done

close failed in file object destructor:
sys.excepthook is missing
lost sys.stderr

close failed in file object destructor:
sys.excepthook is missing
lost sys.stderr

close failed in file object destructor:
sys.excepthook is missing
lost sys.stderr

close failed in file object destructor:
sys.excepthook is missing
lost sys.stderr

close failed in file object destructor:
sys.excepthook is missing
lost sys.stderr

close failed in file object destructor:
sys.excepthook is missing
lost sys.stderr
^CTraceback (most recent call last):
  File "./cmd/arc_summary/arc_summary.py", line 1025, in <module>
    main()
  File "./cmd/arc_summary/arc_summary.py", line 1017, in main
    page(Kstat)
  File "./cmd/arc_summary/arc_summary.py", line 875, in _tunable_summary
    value = f.read()
KeyboardInterrupt
close failed in file object destructor:
sys.excepthook is missing
lost sys.stderr

However we did get a EPIPE, so why doesn't the try/except work? From sysdig proc.name contains arc_:

8762 20:22:31.755109477 0 arc_summary.py (24613) > write fd=1(<p>) size=4096 
8763 20:22:31.755118993 0 arc_summary.py (24613) < write res=4096 data=.------------------------------------------------------------------------.ZFS Su 
8786 20:22:31.755312801 0 arc_summary.py (24613) > switch next=1197(bash) pgft_maj=0 pgft_min=2072 vm_size=26944 vm_rss=9136 vm_swap=0 
8823 20:22:31.755376781 0 arc_summary.py (24613) > rt_sigaction 
8824 20:22:31.755377939 0 arc_summary.py (24613) < rt_sigaction 
10125 20:22:31.757558273 0 arc_summary.py (24613) > write fd=1(<p>) size=3037 
10126 20:22:31.757561449 0 arc_summary.py (24613) < write res=-32(EPIPE) data=b_io                                   0..zfs_no_scrub_prefetch                  
10131 20:22:31.757577359 0 arc_summary.py (24613) > write fd=2(<f>/dev/ttyS0) size=40 
10476 20:22:31.758200614 0 arc_summary.py (24613) < write res=40 data=close failed in file object destructor:. 
10482 20:22:31.758210653 0 arc_summary.py (24613) > write fd=2(<f>/dev/ttyS0) size=26 
10732 20:22:31.758641695 0 arc_summary.py (24613) < write res=26 data=sys.excepthook is missing. 
10741 20:22:31.758645224 0 arc_summary.py (24613) > write fd=2(<f>/dev/ttyS0) size=16 
10836 20:22:31.758885856 0 arc_summary.py (24613) < write res=16 data=lost sys.stderr. 
10837 20:22:31.758887710 0 arc_summary.py (24613) > switch next=7 pgft_maj=0 pgft_min=2075 vm_size=26944 vm_rss=9136 vm_swap=0 
10947 20:22:31.759070445 0 arc_summary.py (24613) > exit_group 
11157 20:22:31.759428149 0 arc_summary.py (24613) > procexit status=0 

[dinatale2]

@loli10K Thanks for looking into it. I think I might try what @richardelling suggested and implement a signal handler and see if it yields better results.

[dinatale2]

@loli10K I think the sys.excepthook implementation covers most of the cases. I occasionally see a python stack trace after performing a keyboard interrupt. The broken pipe and the lost sys.stderr message are solved though.

Thanks for your help!

[loli10K]

@dinatale2 i'm still seeing the same error message even with sys.excepthook defined (not "missing") but only on my ubuntu-16.04-amd64 builder (python-2.7.12): unfortunately it doesn't look like we are able to do much on the Python code side, https://bugs.python.org/issue11380, so this LGTM.

[codecov]

Codecov Report

Merging #6969 into master will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #6969      +/-   ##
==========================================
+ Coverage   75.25%   75.28%   +0.02%     
==========================================
  Files         296      296              
  Lines       95453    95453              
==========================================
+ Hits        71837    71864      +27     
+ Misses      23616    23589      -27

Flag	Coverage Δ
#kernel	74.36% <ø> (-0.27%)	⬇️
#user	67.47% <ø> (+0.02%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e2d936e...3c87f68. Read the comment docs.