Updating ITK's HDF5

Engineering
21

What are your errors? Can you do an experimental build? The dashboard is pretty clean at the moment.

22

As I described above, it produces a broken MINC file on write, no errors.

The 1.8 -> 1.10 change is likely the culprit. I’ll take a look at whether there’s any big HDF5 changes which may break libminc for this.

1 Like
23

First note here:
https://support.hdfgroup.org/HDF5/docNewFeatures/

File Format Changes

This release includes changes in the HDF5 storage format. These changes come into play when one or more of the new features is used or when an application calls for use of the latest storage format (H5Pset_libver_bounds).

Due to the requirements of some of the new features, the format of a 1.10.x HDF5 file is likely to be different from that of a 1.8.x HDF5 file. This means that tools and applications built to read 1.10.x files will be able to read a 1.8.x file, but tools built to read 1.8.x files may not be able to read a 1.10.x file.

If an application built on HDF5 Release 1.10 avoids use of the new features and does not request use of the latest format, applications built on HDF5 Release 1.8.x will be able to read files the first application created. In addition, applications originally written for use with HDF5 Release 1.8.x can be linked against a suitably configured HDF5 Release 1.10.x library, thus taking advantage of performance improvements in 1.10.

I’m betting I’m getting bitten by this.

2 Likes
24

That might not be the reason. The function which deals with that is being called from MINC here and here with parameter H5F_LIBVER_18. Perhaps it needs to be called in a few more places?

25

Indeed, it looks like the only two instances of files being created are being versioned properly with the function, however I can’t h5dump the resulting files with an older HDF5 version.

26

This looks problematic:

1 Like
27

There were some examples on the HDF5 list on how to setup your codes to use HDF5 1.10.x but still write a 1.8.x compatible file. That bit of code is going to mess that up. If you search the HDF5 list archives you should be able to find the discussion.

1 Like
28

Thanks @imikejackson, your comment helped me google the right terms to find this:

Which describes how to get 1.10.x to generate 1.8.x compatible files.

I’m implementing that change upstream. Then we’ll need an external resync.

2 Likes
29

Looks like I’ve fixed it. I have a PR open on libminc upstream here:

2 Likes
30

Note that this on-disk format change will make .hdf5 files generated by ITKv5 non-backwards compatible with ITKv5 unless a similar change is made.

31

You said you planned to update ITK’s version of MINC library after the upstream fixes it? Will that solve the problem?

For pure HDF5, here is a patch which should enable the backwards compatible format.

2 Likes
32

@dzenanz While I have fixed it upstream, it looks like the actual writing of the file is done in such a way that the ITK takes over the write call

If I limit the HDF API to 1.8 using the cmake option, the MINC files are okay, but otherwise, still having an issue.

I’m digging into the MINC-ITK CXX glue layer now…

33

Awesome! Are you on the HDF5 transform too?

34

Not yet. And I am happy to let you do that one, if you so desire :smiley:

35

@dzenanz

It looks like I’m being bitten now by the itk header def mangling that is done HDF5. Particularly right here:

How are we supposed to handle the mangled headers?

36

The constant’s name is H5F_LIBVER_V18. I just took a look, it is not being mangled (as it is not a public symbol, I suppose). It is defined in H5Fpublic.h, which you need to include.

1 Like
37

k, so I’m a bit confused.

That snippet of code I wrote to handle building against HDF5 1.8.x.

H5F_LIBVER_18 doesn’t exist in some earlier versions of 1.8, so I want it to correspond to LATEST in that case.

If I remove this code, MINCIO produces a proper file, however with it left in as-is it looks like somehow this snippit is redefining it.

38

Oops, I see I linked the wrong version of the code, this is what I’m working with right now:


My comment above applies to this version of the code.

39

Okay, my problem is H5F_LIBVER_V18 and related are actually typedef enumn, not define:
snip from H5public.h:

/* Library's format versions */
typedef enum H5F_libver_t {
    H5F_LIBVER_ERROR = -1,
    H5F_LIBVER_EARLIEST = 0,    /* Use the earliest possible format for storing objects */
    H5F_LIBVER_V18 = 1,         /* Use the latest v18 format for storing objects */
    H5F_LIBVER_V110 = 2,        /* Use the latest v10 format for storing objects */
    H5F_LIBVER_NBOUNDS
} H5F_libver_t;

So my ifndef always defines latest version.

I need to adjust the check to use something else.

40

Compare H5F_LIBVER_NBOUNDS to H5F_LIBVER_V18? It should be equal for 1.8 and greater for 1.10.