Failure to get a file lock reading HF5: Singularity Container: NFSv4: ANTS

I’m running the NiPreps project, FmriPrep inside of a Singularity container. There’s a lot wrappers, including Nipype, but I do get an ITKv5 failure to read an HF5 transform because of a file lock issue.

I’m wondering if NFSv4 could be the culprit ? Does anyone have experience with this ?
TIA

NOTE: I can step inside the container and run the ANTS apply transform. There is a temp file missing so I could not call the whole command string (tmp-r5u5c1tz/mat2itk_pos-003_xfm-00001.txt). However, I was able to leave out the missing file but still included the offending HF5 transform (registration/ants_t1_to_mniComposite.h5. There was no error.

RuntimeError: Command:
antsApplyTransforms --default-value 0 --float 1 --input /work/fmriprep_wf/single_subject_DPOP022_wf/func_preproc_ses_1_task_rest_run_7_wf/bold_split/vol0001.nii.gz --interpolation LanczosWindowedSinc --output /work/fmriprep_wf/single_subject_DPOP022_wf/func_preproc_ses_1_task_rest_run_7_wf/bold_std_trans_wf/_std_target_MNI152NLin2009cAsym.resnative/bold_to_std_transform/vol0001_xform-00001.nii.gz --reference-image /work/fmriprep_wf/single_subject_DPOP022_wf/func_preproc_ses_1_task_rest_run_7_wf/bold_std_trans_wf/_std_target_MNI152NLin2009cAsym.resnative/gen_ref/tpl-MNI152NLin2009cAsym_res-02_T1w_reference.nii.gz --transform /work/fmriprep_wf/single_subject_DPOP022_wf/anat_preproc_wf/anat_norm_wf/_template_MNI152NLin2009cAsym/registration/ants_t1_to_mniComposite.h5 --transform /work/fmriprep_wf/single_subject_DPOP022_wf/func_preproc_ses_1_task_rest_run_7_wf/bold_reg_wf/bbreg_wf/concat_xfm/out_fwd.tfm --transform identity --transform /work/fmriprep_wf/single_subject_DPOP022_wf/func_preproc_ses_1_task_rest_run_7_wf/bold_std_trans_wf/_std_target_MNI152NLin2009cAsym.resnative/bold_to_std_transform/tmp-r5u5c1tz/mat2itk_pos-003_xfm-00001.txt
Standard output:

Standard error:
HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 0:
  #000: /src/ants/build/ITKv5/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5F.c line 509 in itk_H5Fopen(): unable to open file
    major: File accessibilty
    minor: Unable to open file
  #001: /src/ants/build/ITKv5/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5Fint.c line 1400 in itk_H5F__open(): unable to open file
    major: File accessibilty
    minor: Unable to open file
  #002: /src/ants/build/ITKv5/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5Fint.c line 1615 in itk_H5F_open(): unable to lock the file
    major: File accessibilty
    minor: Unable to open file
  #003: /src/ants/build/ITKv5/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5FD.c line 1640 in itk_H5FD_lock(): driver lock request failed
    major: Virtual File Layer
    minor: Can't update object
  #004: /src/ants/build/ITKv5/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5FDsec2.c line 941 in H5FD_sec2_lock(): unable to lock file, errno = 5, error message = 'Input/output error'
    major: File accessibilty
    minor: Bad file ID accessed
Transform reader for /work/fmriprep_wf/single_subject_DPOP022_wf/anat_preproc_wf/anat_norm_wf/_template_MNI152NLin2009cAsym/registration/ants_t1_to_mniComposite.h5 caught an ITK exception:

itk::ExceptionObject (0x39845d0)
Location: "unknown" 
File: /src/ants/build/ITKv5/Modules/IO/TransformHDF5/src/itkHDF5TransformIO.cxx
Line: 363
Description: itk::ERROR: HDF5TransformIOTemplate(0x3979880): H5Fopen failed

Return code: 1

Hi Daniel,

Welcome to the ITK community! :sun_with_face:

The error looks like it may be related to access to the file?

For reference, @jtduda recently provided a tutorial on ANTs within singularity here:

1 Like

Thanks Matt.

What I though was strange, is that when I step inside the container, I can run the transform. However, it fails intermittently. I ran it several times from the command line and 1 of 10 I’d get an error.

I don’t expect you guys to solve a file system issue. I’m trying to find more information about such a problem. Hopefully the lecture will help.