h5_easy_access

Generate a simple C interface for hdf5 files

This is currently in early development. Reading is sort of complete, writing only supports group creation with attributes at this time.

Check out the example to get an idea how this works. Documentation is going to come. In short:

• write a .yaml file to define your data layout. Pass it to the generator script to get a C source code file and a header file.
• include the generated header file into your c program
• read the header file to see what you can do
• compile your program. Do not forget to link in the hdf5 library (both -lhdf5 and -lhdf5_hl are needed).

Define your data layout

The data layout of your hdf5 file is defined with YAML. Your configuration has three fields:

name

This field defines a prefix for all generated functions and data types

silent

If set to true, all output from the hdf5 library is surpressed. You will still receive error messages if functions generated by h5_easy_access fail.

groups

This field defines all types of groups that can be created (or read) in your hdf5 file. It consists of three sub-fields.

name

This field defines a name for your group type. It will be used in the names of the generated functions.

attributes

This field contains a list of attributes that will be attached to your group. Here, you can store metadata or whatever else you might need in addition to your actual data.

Each attribute consists of at least two sub-fields.

name

The name of the attribute. Must be a valid C variable name.

type

The C data type of your attribute. Not all are supported. Create an issue if your favourite data type is not supported.

shape (optional)

Attributes can be multidimensional arrays. Shape is an array of sizes for each of the dimensions of your attribute. You can use integer numbers or names of other attributes as sizes. If you use the name of another attribute here, it must be defined earlier in the list.

default (optional)

Here, you can give a default value for your attribute. For char array attributes, you can put a string here.

h5name (optional)

Here, you can put an additional name for the attribute that is only used inside the hdf5 file. This name can be any string, whereas name must be a valid C variable name.

tables

This field contains a list of table types that can be created below the parent group. Each entry consists of two sub-fields.

name

The name of the table type. Will be used in the names of the generated functions.

columns

This field contains - you guessed it - a list of columns for your table type. Each entry consists of at least two sub-fields.

name

Guess what.

type

The C data type of your field. Subject to the same constraints as attribute types.

shape (optional)

Column entries can be multidimensional arrays. Shape is an array of sizes for each of the dimensions of your column entry. You can use integer numbers or names of attributes from the parent group as sizes.

Generate your source

generate_interface.py is a Python 3 script that will generate two files: A C source file and a header. Include the header in your C program and compile it together with the C source file.

The script depends on pyyaml.

Use the generated interface

The interface makes use of quite a few structs as a sort of poor man's objects to contain the description of your hdf5 file. For every struct, there is a destructor function. Use that or you may experience memory leaks. Freeing the pointers is not enough.

The naming convention for structs is <prefix>_<type>_<type name>. <struct name>_t is always a pointer to <struct name>.

<prefix>_file

This struct contains everything concerning your hdf5 file itself. You get one by either opening an existing hdf5 file with <prefix>_open or creating a new one with <prefix>_create. <prefix>_create will fail if the file already exists.

<prefix>_group_<group type>

Contains all information for an hdf5 group. You get one by either opening an existing group with <prefix>_open_group_<group type> or creating a new one with <prefix>_create_group_<group type>. Every group struct has a member attributes, which is itself a struct containing all the group's attributes. You can change attributes by simply writing to the attribute struct. The values will be written back to the file once you call <prefix>_close_group_<group type> if the file is opened read-write.

You can get a list of existing groups with <prefix>_get_groups.

<prefix>_group_<group type>_attributes

Contains all attributes of a group. You get it as part of your group struct.

<prefix>_table_<table type>

Contains information about a table of a given type. You can get one by opening an existing table with <prefix>_open_table_<table type> or by creating one with <prefix>_create_table_<table type>. Read data from a table with <prefix>_get_records_<table type> or write data to it with <prefix>_add_records_<table type>.

<prefix>_table_<table type>_record

Represents one record (or row) of data from a table. All data is represented via pointers and memory is managed elsewhere (by you if you create the record, or inside the recordset if the record is read from a table).

<prefix>_table_<table type>_recordset

Represents a set of one or more records from a table. Get one by reading data from a table with <prefix>_get_records_<table type>. This object manages the memory needed to read from hdf5 tables. Use <prefix>_close_table_<table type>_recordset to free this memory after you are done with the records, otherwise you might leak lots of memory.