Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update URL documentation links to support site #4781

Merged
merged 2 commits into from
Aug 29, 2024
Merged

Update URL documentation links to support site #4781

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
66ea36a
Update URL documentation links to support site
byrnHDF Aug 28, 2024
644dd47
Correct path
byrnHDF Aug 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CITATION.cff
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ authors:
website: 'https://www.hdfgroup.org'
repository-code: 'https://github.com/HDFGroup/hdf5'
url: 'https://www.hdfgroup.org/HDF5/'
repository-artifact: 'https://www.hdfgroup.org/downloads/hdf5/'
repository-artifact: 'https://support.hdfgroup.org/downloads/HDF5'
12 changes: 6 additions & 6 deletions HDF5Examples/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,17 +48,17 @@ HDF5 SNAPSHOTS, PREVIOUS RELEASES AND SOURCE CODE
--------------------------------------------
Full Documentation and Programming Resources for this HDF5 can be found at

https://portal.hdfgroup.org/documentation/index.html
https://support.hdfgroup.org/documentation/HDF5/index.html

Periodically development code snapshots are provided at the following URL:
https://gamma.hdfgroup.org/ftp/pub/outgoing/hdf5/snapshots/

https://github.com/HDFGroup/hdf5/releases
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/HDFGroup/hdf5/releases/tag/snapshot

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

releases is the more general location - snapshot is just develop, snapshot-1.14 is for the releasable branch

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The snapshot URL is probably better as it is more closely related to the preceding sentence "Periodically development code snapshots are provided at the following URL:", and there are no develop code "releases". However, the last snapshot listed for develop is dated May 25. Should there be monthly/bi-weekly/weekly develop and 1.14 snapshots deployed, maybe with a reduced number of assets?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As examples can be used with any HDF5 binary - I think it it is better to point to more general location.
Merges always trigger builds - snapshot named binaries need to be forced. Not sure what the policy should be on forcing a snapshot build, maybe a week after a merge build?


Source packages for current and previous releases are located at:
https://portal.hdfgroup.org/downloads/

https://support.hdfgroup.org/releases/hdf5/downloads/

Development code is available at our Github location:

https://github.com/HDFGroup/hdf5.git

2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ Periodically development code snapshots are provided at the following URL:

Source packages for current and previous releases are located at:

https://portal.hdfgroup.org/Downloads
https://support.hdfgroup.org/downloads/HDF5

Development code is available at our Github location:

Expand Down
2 changes: 1 addition & 1 deletion config/cmake/README.md.cmake.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,6 @@ For more information see USING_CMake_Examples.txt in the install folder.
===========================================================================

Documentation for this release can be found at the following URL:
https://portal.hdfgroup.org/documentation/index.html#hdf5
https://support.hdfgroup.org/hdf5/@HDF5_PACKAGE_NAME@-@HDF5_PACKAGE_VERSION@/documentation/doxygen/index.html

Bugs should be reported to [email protected].
55 changes: 36 additions & 19 deletions doc/parallel-compression.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,9 @@ H5Dwrite(..., dxpl_id, ...);
The following are two simple examples of using the parallel
compression feature:

[ph5_filtered_writes.c](https://github.com/HDFGroup/hdf5/blob/develop/HDF5Examples/C/H5PAR/ph5_filtered_writes.c)
[ph5_filtered_writes.c][u1]

[ph5_filtered_writes_no_sel.c](https://github.com/HDFGroup/hdf5/blob/develop/HDF5Examples/C/H5PAR/ph5_filtered_writes_no_sel.c)
[ph5_filtered_writes_no_sel.c][u2]

The former contains simple examples of using the parallel
compression feature to write to compressed datasets, while the
Expand All @@ -79,7 +79,7 @@ participate in the collective write call.
## Multi-dataset I/O support

The parallel compression feature is supported when using the
multi-dataset I/O API routines ([H5Dwrite_multi](https://hdfgroup.github.io/hdf5/develop/group___h5_d.html#gaf6213bf3a876c1741810037ff2bb85d8)/[H5Dread_multi](https://hdfgroup.github.io/hdf5/develop/group___h5_d.html#ga8eb1c838aff79a17de385d0707709915)), but the
multi-dataset I/O API routines ([H5Dwrite_multi][u3]/[H5Dread_multi][u4]), but the
following should be kept in mind:

- Parallel writes to filtered datasets **must** still be collective,
Expand All @@ -99,7 +99,7 @@ following should be kept in mind:

## Incremental file space allocation support

HDF5's [file space allocation time](https://hdfgroup.github.io/hdf5/develop/group___d_c_p_l.html#ga85faefca58387bba409b65c470d7d851)
HDF5's [file space allocation time][u5]
is a dataset creation property that can have significant effects
on application performance, especially if the application uses
parallel HDF5. In a serial HDF5 application, the default file space
Expand All @@ -118,20 +118,20 @@ While this strategy has worked in the past, it has some noticeable
drawbacks. For one, the larger the chunked dataset being created,
the more noticeable overhead there will be during dataset creation
as all of the data chunks are being allocated in the HDF5 file.
Further, these data chunks will, by default, be [filled](https://hdfgroup.github.io/hdf5/develop/group___d_c_p_l.html#ga4335bb45b35386daa837b4ff1b9cd4a4)
Further, these data chunks will, by default, be [filled][u6]
with HDF5's default fill data value, leading to extraordinary
dataset creation overhead and resulting in pre-filling large
portions of a dataset that the application might have been planning
to overwrite anyway. Even worse, there will be more initial overhead
from compressing that fill data before writing it out, only to have
it read back in, unfiltered and modified the first time a chunk is
written to. In the past, it was typically suggested that parallel
HDF5 applications should use [H5Pset_fill_time](https://hdfgroup.github.io/hdf5/develop/group___d_c_p_l.html#ga6bd822266b31f86551a9a1d79601b6a2)
HDF5 applications should use [H5Pset_fill_time][u7]
with a value of `H5D_FILL_TIME_NEVER` in order to disable writing of
the fill value to dataset chunks, but this isn't ideal if the
application actually wishes to make use of fill values.

With [improvements made](https://www.hdfgroup.org/2022/03/parallel-compression-improvements-in-hdf5-1-13-1/)
With [improvements made][u8]
to the parallel compression feature for the HDF5 1.13.1 release,
"incremental" file space allocation is now the default for datasets
created in parallel *only if they have filters applied to them*.
Expand All @@ -154,7 +154,7 @@ optimal performance out of the parallel compression feature.

### Begin with a good chunking strategy

[Starting with a good chunking strategy](https://portal.hdfgroup.org/documentation/hdf5-docs/chunking_in_hdf5.html)
[Starting with a good chunking strategy][u9]
will generally have the largest impact on overall application
performance. The different chunking parameters can be difficult
to fine-tune, but it is essential to start with a well-performing
Expand All @@ -166,7 +166,7 @@ chosen chunk size becomes a very important factor when compression
is involved, as data chunks have to be completely read and
re-written to perform partial writes to the chunk.

[Improving I/O performance with HDF5 compressed datasets](https://docs.hdfgroup.org/archive/support/HDF5/doc/TechNotes/TechNote-HDF5-ImprovingIOPerformanceCompressedDatasets.pdf)
[Improving I/O performance with HDF5 compressed datasets][u10]
is a useful reference for more information on getting good
performance when using a chunked dataset layout.

Expand Down Expand Up @@ -220,14 +220,14 @@ chunks to end up at addresses in the file that do not align
well with the underlying file system, possibly leading to
poor performance. As an example, Lustre performance is generally
good when writes are aligned with the chosen stripe size.
The HDF5 application can use [H5Pset_alignment](https://hdfgroup.github.io/hdf5/develop/group___f_a_p_l.html#gab99d5af749aeb3896fd9e3ceb273677a)
The HDF5 application can use [H5Pset_alignment][u11]
to have a bit more control over where objects in the HDF5
file end up. However, do note that setting the alignment
of objects generally wastes space in the file and has the
potential to dramatically increase its resulting size, so
caution should be used when choosing the alignment parameters.

[H5Pset_alignment](https://hdfgroup.github.io/hdf5/develop/group___f_a_p_l.html#gab99d5af749aeb3896fd9e3ceb273677a)
[H5Pset_alignment][u11]
has two parameters that control the alignment of objects in
the HDF5 file, the "threshold" value and the alignment
value. The threshold value specifies that any object greater
Expand Down Expand Up @@ -264,19 +264,19 @@ in a file, this can create significant amounts of free space
in the file over its lifetime and eventually cause performance
issues.

An HDF5 application can use [H5Pset_file_space_strategy](https://hdfgroup.github.io/hdf5/develop/group___f_c_p_l.html#ga167ff65f392ca3b7f1933b1cee1b9f70)
An HDF5 application can use [H5Pset_file_space_strategy][u12]
with a value of `H5F_FSPACE_STRATEGY_PAGE` to enable the paged
aggregation feature, which can accumulate metadata and raw
data for dataset data chunks into well-aligned, configurably
sized "pages" for better performance. However, note that using
the paged aggregation feature will cause any setting from
[H5Pset_alignment](https://hdfgroup.github.io/hdf5/develop/group___f_a_p_l.html#gab99d5af749aeb3896fd9e3ceb273677a)
[H5Pset_alignment][u11]
to be ignored. While an application should be able to get
comparable performance effects by [setting the size of these pages](https://hdfgroup.github.io/hdf5/develop/group___f_c_p_l.html#gad012d7f3c2f1e1999eb1770aae3a4963) to be equal to the value that
would have been set for [H5Pset_alignment](https://hdfgroup.github.io/hdf5/develop/group___f_a_p_l.html#gab99d5af749aeb3896fd9e3ceb273677a),
comparable performance effects by [setting the size of these pages][u13]
to be equal to the value that would have been set for [H5Pset_alignment][u11],
this may not necessarily be the case and should be studied.

Note that [H5Pset_file_space_strategy](https://hdfgroup.github.io/hdf5/develop/group___f_c_p_l.html#ga167ff65f392ca3b7f1933b1cee1b9f70)
Note that [H5Pset_file_space_strategy][u12]
has a `persist` parameter. This determines whether or not the
file free space manager should include extra metadata in the
HDF5 file about free space sections in the file. If this
Expand All @@ -300,12 +300,12 @@ hid_t file_id = H5Fcreate("file.h5", H5F_ACC_TRUNC, fcpl_id, fapl_id);

While the parallel compression feature requires that the HDF5
application set and maintain collective I/O at the application
interface level (via [H5Pset_dxpl_mpio](https://hdfgroup.github.io/hdf5/develop/group___d_x_p_l.html#ga001a22b64f60b815abf5de8b4776f09e)),
interface level (via [H5Pset_dxpl_mpio][u14]),
it does not require that the actual MPI I/O that occurs at
the lowest layers of HDF5 be collective; independent I/O may
perform better depending on the application I/O patterns and
parallel file system performance, among other factors. The
application may use [H5Pset_dxpl_mpio_collective_opt](https://hdfgroup.github.io/hdf5/develop/group___d_x_p_l.html#gacb30d14d1791ec7ff9ee73aa148a51a3)
application may use [H5Pset_dxpl_mpio_collective_opt][u15]
to control this setting and see which I/O method provides the
best performance.

Expand All @@ -318,7 +318,7 @@ H5Dwrite(..., dxpl_id, ...);

### Runtime HDF5 Library version

An HDF5 application can use the [H5Pset_libver_bounds](https://hdfgroup.github.io/hdf5/develop/group___f_a_p_l.html#gacbe1724e7f70cd17ed687417a1d2a910)
An HDF5 application can use the [H5Pset_libver_bounds][u16]
routine to set the upper and lower bounds on library versions
to use when creating HDF5 objects. For parallel compression
specifically, setting the library version to the latest available
Expand All @@ -332,3 +332,20 @@ H5Pset_libver_bounds(fapl_id, H5F_LIBVER_LATEST, H5F_LIBVER_LATEST);
hid_t file_id = H5Fcreate("file.h5", H5F_ACC_TRUNC, H5P_DEFAULT, fapl_id);
...
```

[u1]: https://github.com/HDFGroup/hdf5/blob/develop/HDF5Examples/C/H5PAR/ph5_filtered_writes.c
[u2]: https://github.com/HDFGroup/hdf5/blob/develop/HDF5Examples/C/H5PAR/ph5_filtered_writes_no_sel.c
[u3]: https://hdfgroup.github.io/hdf5/develop/group___h5_d.html#gaf6213bf3a876c1741810037ff2bb85d8
[u4]: https://hdfgroup.github.io/hdf5/develop/group___h5_d.html#ga8eb1c838aff79a17de385d0707709915
[u5]: https://hdfgroup.github.io/hdf5/develop/group___d_c_p_l.html#ga85faefca58387bba409b65c470d7d851
[u6]: https://hdfgroup.github.io/hdf5/develop/group___d_c_p_l.html#ga4335bb45b35386daa837b4ff1b9cd4a4
[u7]: https://hdfgroup.github.io/hdf5/develop/group___d_c_p_l.html#ga6bd822266b31f86551a9a1d79601b6a2
[u8]: https://support.hdfgroup.org/documentation/HDF5/parallel-compression-improvements-in-hdf5-1-13-1
[u9]: https://support.hdfgroup.org/documentation/HDF5/chunking_in_hdf5.html
[u10]: https://support.hdfgroup.org/documentation/HDF5/technotes/TechNote-HDF5-ImprovingIOPerformanceCompressedDatasets.pdf
[u11]: https://hdfgroup.github.io/hdf5/develop/group___f_a_p_l.html#gab99d5af749aeb3896fd9e3ceb273677a
[u12]: https://hdfgroup.github.io/hdf5/develop/group___f_c_p_l.html#ga167ff65f392ca3b7f1933b1cee1b9f70
[u13]: https://hdfgroup.github.io/hdf5/develop/group___f_c_p_l.html#gad012d7f3c2f1e1999eb1770aae3a4963
[u14]: https://hdfgroup.github.io/hdf5/develop/group___d_x_p_l.html#ga001a22b64f60b815abf5de8b4776f09e
[u15]: https://hdfgroup.github.io/hdf5/develop/group___d_x_p_l.html#gacb30d14d1791ec7ff9ee73aa148a51a3
[u16]: https://hdfgroup.github.io/hdf5/develop/group___f_a_p_l.html#gacbe1724e7f70cd17ed687417a1d2a910
21 changes: 10 additions & 11 deletions doxygen/aliases
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,16 @@ ALIASES += THG="The HDF Group"
# Default URLs (Note that md files do not use any aliases)
################################################################################
# Default URL for HDF Group Files
ALIASES += HDFURL="docs.hdfgroup.org/hdf5"
ALIASES += HDFURL="support.hdfgroup.org"
# URL for archived files
ALIASES += ARCURL="docs.hdfgroup.org/archive/support/HDF5/doc"
ALIASES += ARCURL="\HDFURL/archive/support/HDF5/doc"
# URL for RFCs
ALIASES += RFCURL="docs.hdfgroup.org/hdf5/rfc"
ALIASES += RFCURL="\HDFURL/hdf5/rfc"
# URL for documentation
ALIASES += DSPURL="portal.hdfgroup.org/display/HDF5"
ALIASES += DOCURL="portal.hdfgroup.org/documentation/hdf5-docs"
ALIASES += DOCURL="\HDFURL/releases/hdf5/documentation"
# URL for downloads
ALIASES += DWNURL="portal.hdfgroup.org/downloads"
ALIASES += AEXURL="support.hdfgroup.org/ftp/HDF5/examples"
ALIASES += DWNURL="\HDFURL/releases/hdf5/downloads"
ALIASES += AEXURL="\HDFURL/archive/support/ftp/HDF5/examples"
# doxygen subdir (develop, v1_14)
ALIASES += DOXURL="hdfgroup.github.io/hdf5/develop"
#branch name (develop, hdf5_1_14)
Expand Down Expand Up @@ -259,13 +258,13 @@ ALIASES += sa_metadata_ops="\sa \li H5Pget_all_coll_metadata_ops() \li H5Pget_co

ALIASES += ref_cons_semantics="<a href=\"https://\RFCURL/RFC%20PHDF5%20Consistency%20Semantics%20MC%20120328.docx.pdf\">Enabling a Strict Consistency Semantics Model in Parallel HDF5</a>"
ALIASES += ref_file_image_ops="<a href=\"https://\RFCURL/HDF5FileImageOperations.pdf\">HDF5 File Image Operations</a>"
ALIASES += ref_filter_pipe="<a href=\"https://\DSPURL/HDF5+Data+Flow+Pipeline+for+H5Dread\">Data Flow Pipeline for H5Dread()</a>"
ALIASES += ref_filter_pipe="<a href=\"https://\DOCURL/HDF5+Data+Flow+Pipeline+for+H5Dread\">Data Flow Pipeline for H5Dread()</a>"
ALIASES += ref_group_impls="<a href=\"https://\DOXURL/group___h5_g.html\">Group implementations in HDF5</a>"
ALIASES += ref_h5lib_relver="<a href=\"https://\ARCURL/TechNotes/Version.html\">HDF5 Library Release Version Numbers</a>"
ALIASES += ref_mdc_in_hdf5="<a href=\"https://\DSPURL/Metadata+Caching+in+HDF5\">Metadata Caching in HDF5</a>"
ALIASES += ref_mdc_logging="<a href=\"https://\DSPURL/H5F_START_MDC_LOGGING\">Metadata Cache Logging</a>"
ALIASES += ref_mdc_in_hdf5="<a href=\"https://\DOCURL/Metadata+Caching+in+HDF5\">Metadata Caching in HDF5</a>"
ALIASES += ref_mdc_logging="<a href=\"https://\DOCURL/H5F_START_MDC_LOGGING\">Metadata Cache Logging</a>"
ALIASES += ref_news_112="<a href=\"https://\DOCURL/release_specifics/new_features_1_12.html\">New Features in HDF5 Release 1.12</a>"
ALIASES += ref_h5ocopy="<a href=\"https://\DSPURL/Copying+Committed+Datatypes+with+H5Ocopy\">Copying Committed Datatypes with H5Ocopy()</a>"
ALIASES += ref_h5ocopy="<a href=\"https://\DOCURL/Copying+Committed+Datatypes+with+H5Ocopy\">Copying Committed Datatypes with H5Ocopy()</a>"
ALIASES += ref_sencode_fmt_change="<a href=\"https://\RFCURL/H5Sencode_format.docx.pdf\">RFC H5Sencode() / H5Sdecode() Format Change</a>"
ALIASES += ref_vlen_strings="\Emph{Creating variable-length string datatypes}"
ALIASES += ref_vol_doc="VOL documentation"
Expand Down
2 changes: 1 addition & 1 deletion doxygen/dox/About.dox
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ as a general reference.

All custom commands for this project are located in the
<a href="https://\SRCURL/doxygen/aliases"><tt>aliases</tt></a>
file in the <a href="https://github.com/HDFGroup/hdf5/tree/develop/doxygen"><tt>doxygen</tt></a>
file in the <a href="https://\SRCURL/doxygen"><tt>doxygen</tt></a>
subdirectory of the <a href="https://github.com/HDFGroup/hdf5">main HDF5 repo</a>.

The custom commands are grouped in sections. Find a suitable section for your command or
Expand Down
2 changes: 0 additions & 2 deletions doxygen/dox/DDLBNF110.dox
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
/** \page DDLBNF110 DDL in BNF through HDF5 1.10

\todo Revise this & break it up!

\section intro110 Introduction

This document contains the data description language (DDL) for an HDF5 file. The
Expand Down
2 changes: 0 additions & 2 deletions doxygen/dox/DDLBNF112.dox
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
/** \page DDLBNF112 DDL in BNF for HDF5 1.12 through HDF5 1.14.3

\todo Revise this & break it up!

\section intro112 Introduction

This document contains the data description language (DDL) for an HDF5 file. The
Expand Down
2 changes: 0 additions & 2 deletions doxygen/dox/DDLBNF114.dox
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
/** \page DDLBNF114 DDL in BNF for HDF5 1.14.4 and above

\todo Revise this & break it up!

\section intro114 Introduction

This document contains the data description language (DDL) for an HDF5 file. The
Expand Down
Loading
Loading