VTK master + ITK master = HDF5 duplicate symbol link errors

Engineering
1

Hi all,

I’ve just tried building current VTK master + current ITK master then building my own app against them. The result is about 150 link errors, all along the lines of:

duplicate symbol _H5_libinit_g in:
    /path/to/ITK/lib/libitkhdf5_debug.a(H5.c.o)
    /path/to/VTK/lib/libvtkhdf5.a(H5.c.o)
duplicate symbol _H5_libterm_g in:
    /path/to/ITK/lib/libitkhdf5_debug.a(H5.c.o)
    /path/to/VTK/lib/libvtkhdf5.a(H5.c.o)

I checked a handful, and indeed they are in neither VTK’s nor ITK’s HDF mangling file. In addition, they all seem to be global variables (not functions).

This is on macOS, not sure if that matters.

If I run the command at the top of the ITK mangle file:

nm -g lib/libitkhdf5* 2> /dev/null | grep " [TDRS] " | awk '{ print substr($3, 2); }' | awk '{ sub(/itk_/, ""); print }' | sed 's/\(.*\)/#define \1\ itk_\1/' | sort -u

then my results include the “H5_libinit_g” symbol. OTOH, my results do not include other symbols like “H5AC_dxpl_id”.

My results also include other symbols, ex “H5AC_cache_is_clean”, that were removed by this commit (for some reason):

http://review.source.kitware.com/#/c/23666/1/Modules/ThirdParty/HDF5/src/itkhdf5/src/itk_hdf5_mangle.h

It seems that the proper/complete generation of this mangling file may require running on multiple platforms and taking the union of the results…

If I merge my results into what’s in ITK master, my link errors go away.

Patch:

Sean

1 Like
2

Thanks for sharing your issues and patch, Sean. The fix has been merged to master:

http://review.source.kitware.com/#/c/23711/

3

What’s not clear to me is why this was needed.

Must we indeed do this on every platform and then union the results?

Sean

4

Yes, that is a good idea.

5

Doing it on the three platforms might not be enough for ALL purposes. Depending on exact compiler versions, CMake configuration, other build settings and defines, the resulting list of symbols will probably vary. Doing all these combinations is not practical, given that there are thousands of symbols in HDF5 and they somewhat change between versions.

But generating the list for mangling on three platforms and taking the union is an excellent first step :smiley: Then make additional fixes as needed.

As I use Mac the least, I probably didn’t add its specific symbols to the mangling list.

6

@matt.mccormick given the call we had with the HDF5 group, maybe we should ask them to provide an comprehensive list somehow?

7

Ideally, upstream would provide an HDF5 CMake variable that allows one to specify a symbol name mangling prefix.

8

I am really going to show my naivety on this: Why do we have to mangle the symbols? What does this guard against? I guess so that different libraries can compiled against different versions of HDF5?

9

Hi hi Mike!

If SomeApp™ already links against HDF5 (because it already uses it), then it wouldn’t be easy for that app to start using ITK because it’d get duplicate symbol link errors.

Sean

1 Like