Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert Debugging HDF5 Applications to doxygen #5139

Merged
merged 9 commits into from
Nov 27, 2024
Merged

Convert Debugging HDF5 Applications to doxygen #5139

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
82ad3aa
Convert Debugging HDF5 Applications to doxygen
byrnHDF Nov 21, 2024
431addf
Remove tag indicator on non-function
byrnHDF Nov 21, 2024
ac55497
Merge branch 'HDFGroup:develop' into develop-doxy-debugging
byrnHDF Nov 21, 2024
3e3d304
Merge branch 'HDFGroup:develop' into develop-doxy-debugging
byrnHDF Nov 22, 2024
c84a10c
Fix quote marks
byrnHDF Nov 22, 2024
75fd20e
Merge branch 'HDFGroup:develop' into develop-doxy-debugging
byrnHDF Nov 25, 2024
249b4cf
Merge branch 'HDFGroup:develop' into develop-doxy-debugging
byrnHDF Nov 26, 2024
5f294fc
Merge branch 'HDFGroup:develop' into develop-doxy-debugging
byrnHDF Nov 26, 2024
3e53b37
Remove TRACE macro section
byrnHDF Nov 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
244 changes: 243 additions & 1 deletion doxygen/dox/TechnicalNotes.dox
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1063,7 +1063,249 @@ been used at the lowest level to process the request.

/** \page APPDBG Debugging HDF5 Applications

\htmlinclude DebuggingHDF5Applications.html
\section sec_adddbg_intro Introduction
The HDF5 library contains a number of debugging features to make programmers' lives
easier including the ability to print detailed error messages, check invariant
conditions, display timings and other statistics, and trace API function calls
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which would also warrant revision here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But wait - we still have H5ARG_TRACE and bin/trace?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

|trace|Updates H5ARG_TRACE macros in H5ES_insert() calls (run by autogen.sh)|

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They trace function calls just for internal use with the Async APIs but in general the H5TRACE macros were removed from all API routines.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So safe to remove in this file - good!

and return values.

\subsection subsec_adddbg_intro_err Error Messages
Error messages are normally displayed automatically on the standard error stream and
include a stack trace of the library including file names, line numbers, and function
names. The application has complete control over how error messages are displayed and
can disable the display on a permanent or temporary basis. Refer to the documentation
for the H5E error handling package.

\subsection subsec_adddbg_intro_invar Invariant Conditions
Unless NDEBUG is defined during compiling, the library will include code to verify that
invariant conditions have the expected values. When a problem is detected the library will
display the file and line number within the library and the invariant condition that
failed. A core dump may be generated for post mortem debugging. The code to perform these
checks can be included on a per-package bases.

\subsection subsec_adddbg_intro_stats Timings and Statistics
The library can be configured to accumulate certain statistics about things like cache
performance, datatype conversion, data space conversion, and data filters. The code is
included on a per-package basis and enabled at runtime by an environment variable.

\subsection subsec_adddbg_intro_trace API Tracing
All API calls made by an application can be displayed and include formal argument names
and actual values and the function return value. This code is also conditionally included
at compile time and enabled at runtime.

The statistics and tracing can be displayed on any output stream (including streams opened by
the shell) with output from different packages even going to different streams.

\section sec_adddbg_msg Error Messages
By default any API function that fails will print an error stack to the standard error stream.
\code
HDF5-DIAG: Error detected in thread 0. Back trace follows.
#000: H5F.c line 1245 in H5Fopen(): unable to open file
major(04): File interface
minor(10): Unable to open file
#001: H5F.c line 846 in H5F_open(): file does not exist
major(04): File interface
minor(10): Unable to open file
\endcode
The error handling package (H5E) is described elsewhere.

\section sec_adddbg_invars Invariant Conditions
To include checks for invariant conditions the library should be configured
with --disable-production, the default for versions before 1.2. The library
designers have made every attempt to handle error conditions gracefully but
an invariant condition assertion may fail in certain cases. The output from
a failure usually looks something like this:
\code
Assertion failed: H5.c:123: i<NELMTS(H5_debug_g)
IOT Trap, core dumped.
\endcode

\section sec_adddbg_stats Timings and Statistics
Code to accumulate statistics is included at compile time by using the --enable-debug
configure switch. The switch can be followed by an equal sign and a comma-separated
list of package names or else a default list is used.
<table>
<tr>
<th><strong>Name</strong>
</th>
<th><strong>Default</strong>
</th>
<th><strong>Description</strong>
</th>
</tr>
<tr>
<td>a</td><td>No</td><td>Attributes</td>
</tr>
<tr>
<td>ac</td><td>Yes</td><td>Meta data cache</td>
</tr>
<tr>
<td>b</td><td>Yes</td><td>B-Trees</td>
</tr>
<tr>
<td>d</td><td>Yes</td><td>Datasets</td>
</tr>
<tr>
<td>e</td><td>Yes</td><td>Error handling</td>
</tr>
<tr>
<td>f</td><td>Yes</td><td>Files</td>
</tr>
<tr>
<td>g</td><td>Yes</td><td>Groups</td>
</tr>
<tr>
<td>hg</td><td>Yes</td><td>Global heap</td>
</tr>
<tr>
<td>hl</td><td>No</td><td>Local heaps</td>
</tr>
<tr>
<td>i</td><td>Yes</td><td>Interface abstraction</td>
</tr>
<tr>
<td>mf</td><td>No</td><td>File memory management</td>
</tr>
<tr>
<td>mm</td><td>Yes</td><td>Library memory management</td>
</tr>
<tr>
<td>o</td><td>No</td><td>Object headers and messages</td>
</tr>
<tr>
<td>p</td><td>Yes</td><td>Property lists</td>
</tr>
<tr>
<td>s</td><td>Yes</td><td>Data spaces</td>
</tr>
<tr>
<td>t</td><td>Yes</td><td>Datatypes</td>
</tr>
<tr>
<td>v</td><td>Yes</td><td>Vectors</td>
</tr>
<tr>
<td>z</td><td>Yes</td><td>Raw data filters</td>
</tr>
</table>

In addition to including the code at compile time the application must enable each package at
runtime. This is done by listing the package names in the HDF5_DEBUG environment variable. That
variable may also contain file descriptor numbers (the default is '2') which control the output
for all following packages up to the next file number. The word 'all' refers to all packages. Any
word my be preceded by a minus sign to turn debugging off for the package.

\subsection subsec_adddbg_stats_sample Sample debug specifications
<table>
<tr>
<td>all
</td>
<td>This causes debugging output from all packages to be sent to the standard error stream.
</td>
</tr>
<tr>
<td>all -t -s
</td>
<td>Debugging output for all packages except datatypes and data spaces will appear on the standard error stream.
</td>
</tr>
<tr>
<td>-all ac 255 t,s
</td>
<td>This disables all debugging even if the default was to debug something, then output
from the meta data cache is send to the standard error stream and output from data types
and spaces is sent to file descriptor 255 which should be redirected by the shell.
</td>
</tr>
</table>
The components of the HDF5_DEBUG value may be separated by any non-lowercase letter.

\section sec_adddbg_trace API Tracing
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Essentially everything from this point down can just be removed since the API tracing functionality was removed a bit ago.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is what I thought! Good so this goes away.

The HDF5 library can trace API calls by printing the function name, the argument names
and their values, and the return value. Some people like to see lots of output during
program execution instead of using a good symbolic debugger, and this feature is intended
for their consumption. For example, the output from h5ls foo after turning on tracing, includes:
\code
H5Tcopy(type=184549388) = 184549419 (type);
H5Tcopy(type=184549392) = 184549424 (type);
H5Tlock(type=184549424) = SUCCEED;
H5Tcopy(type=184549393) = 184549425 (type);
H5Tlock(type=184549425) = SUCCEED;
H5Fopen(filename="foo", flags=0, access=H5P_DEFAULT) = FAIL;
HDF5-DIAG: Error detected in thread 0. Back trace follows.
#000: H5F.c line 1245 in H5Fopen(): unable to open file
major(04): File interface
minor(10): Unable to open file
#001: H5F.c line 846 in H5F_open(): file does not exist
major(04): File interface
minor(10): Unable to open file
\endcode

The code that performs the tracing must be included in the library by specifying
the --enable-trace configuration switch (the default for versions before 1.2). Then
the word trace must appear in the value of the HDF5_DEBUG variable. The output
will appear on the last file descriptor before the word trace or two (standard error) by default.

To display the trace on the standard error stream:
\code
$ env HDF5_DEBUG=trace a.out
\endcode

To send the trace to a file:
\code
$ env HDF5_DEBUG="55 trace" a.out 55>trace-output
\endcode

\subsection subsec_adddbg_trace_perf Performance
If the library was not configured for tracing then there is no unnecessary overhead
since all tracing code is excluded. However, if tracing is enabled but not used there
is a small penalty. First, code size is larger because of extra statically-declared
character strings used to store argument types and names and extra auto variable
pointer in each function. Also, execution is slower because each function sets and
tests a local variable and each API function calls the H5_trace() function.

If tracing is enabled and turned on then the penalties from the previous paragraph
apply plus the time required to format each line of tracing information. There is
also an extra call to H5_trace() for each API function to print the return value.

\subsection subsec_adddbg_trace_safe Safety
The tracing mechanism is invoked for each API function before arguments are checked
for validity. If bad arguments are passed to an API function it could result in a
segmentation fault. However, the tracing output is line-buffered so all previous output will appear.

\subsection subsec_adddbg_trace_comp Completeness
There are two API functions that don't participate in tracing. They are #H5Eprint
and H5Eprint_cb because their participation would mess up output during automatic error reporting.

On the other hand, a number of API functions are called during library initialization
and they print tracing information.

\subsection subsec_adddbg_trace_imp Implementation
For those interested in the implementation here is a description. Each API function should
have a call to one of the H5TRACE() macros immediately after the <code>FUNC_ENTER</code> macro. The
first argument is the return type encoded as a string. The second argument is the types of
all the function arguments encoded as a string. The remaining arguments are the function
arguments. This macro was designed to be as terse and unobtrousive as possible.

In order to keep the H5TRACE() calls synchronized with the source code we've written
a perl script which gets called automatically just before Makefile dependencies are
calculated for the file. However, this only works when one is using GNU make. To
reinstrument the tracing explicitly, invoke the trace program from the hdf5 bin
directory with the names of the source files that need to be updated. If any file
needs to be modified then a backup is created by appending a tilde to the file name.

<em>Explicit Instrumentation:</em>
\code
$ ../bin/trace *.c
H5E.c: in function 'H5Ewalk_cb':
H5E.c:336: warning: trace info was not inserted
\endcode

Note: The warning message is the result of a comment of the form NO TRACE somewhere
in the function body. Tracing information will not be updated or inserted if such a comment exists.

Error messages have the same format as a compiler so that they can be parsed from program
development environments like Emacs. Any function which generates an error will not be modified.

*/

Expand Down
Loading
Loading