How to avoid rebuilding ITK by azure CI at the GitHub of another project (elastix)?

(Niels Dekker) #1

Our image registration toolkit elastix is being built with a specific tagged version of ITK5 (actually v5.0.0), at https://dev.azure.com/kaspermarstal/Elastix/_build/results?buildId=458

As you see, a Windows build of elastix takes more than an hour, and more that 20 minutes thereof comes from generating (by CMake) and building ITK.

Do you have a suggestion how to avoid rebuilding ITK with each elastix build at the CI?

I did have a look at https://github.com/microsoft/azure-pipelines-artifact-caching-tasks but I don’t yet fully understand how to adjust our azure-pipeline yml file to get it working! Do you?

Another option could be to use prebuilt ITK binaries. Are these available already somewhere, at a location where accessible by azure CI?

(Matt McCormick) #2

Woohoo! :tada: :green_heart:

In addition, the Azure Pipelines team is in the process of rolling out a feature called Pipeline Caching. The are just starting to roll it out, so it may be an option, soon.

Another approach is be to use Azure Pipelines Artifacts with an ITK vcpkg (@dzenanz has contributed to this package configuration) generated binary. This example may help.

1 Like
(Niels Dekker) #3

Thanks for your help so far @matt.mccormick I do like the option to use generated binaries for VS2017 (64-bit Release) but I still don’t really understand how. Currently we build ITK as follows, at elastix/Testing/CI/Azure/ci.yml:

- script: |
    git clone https://github.com/InsightSoftwareConsortium/ITK "$(ITK_SOURCE_DIR)"
    pushd "$(ITK_SOURCE_DIR)"
    git checkout $(itk.version)
    popd
  displayName: Clone ITK
- script: |
    mkdir "$(ITK_BINARY_DIR)"
    mkdir "$(ELASTIX_BINARY_DIR)"
  displayName: Make build directories
- task: CMake@1
  displayName: 'CMake Generate ITK'
  inputs:
    cmakeArgs: -G "Visual Studio 15 2017 Win64" -T host=x64 -DBUILD_EXAMPLES=OFF -DBUILD_TESTING=OFF "$(ITK_SOURCE_DIR)"
    workingDirectory: "$(ITK_BINARY_DIR)"
- task: CMake@1
  displayName: 'CMake Build ITK'
  inputs:
    cmakeArgs: --build . --config Release -j 2
    workingDirectory: "$(ITK_BINARY_DIR)"

How could this be replaced by using generated ITK 5.0.0 binaries from vcpkg?

(Matt McCormick) #4

vcpkg has the ability to export a built package:

The package comes with a CMAKE_TOOLCHAIN_FILE. We need to use this file for the elastix build so the compiler, toolchain, and relevant build settings are used to ensure the binaries are compatible. Set ITK_DIR to the vcpkg ITKConfig.cmake file path and CMAKE_TOOLCHAIN_FILE to the vcpkg file. A related section of the example:

1 Like
(Niels Dekker) #5

Thanks Matt! I’m still studying vcpkg, but I’m making some progress! I’ll let you know when I got it working!

1 Like
(Niels Dekker) #6

The current version of elastix (git develop branch) still depends on the legacy interface of ITK 5. Unfortunately, the ITK portfile at github.com/microsoft/vcpkg removes the legacy support (ITK_LEGACY_REMOVE=ON):

Does this mean that currently, elastix cannot yet use vcpkg, on github azure CI?

Here is the azure-pipelines yml file I adapted in order to use vcpkg (elastix PR 161):

(Dženan Zukić) #7

ITK port is far from set in stone. Make a PR which flips the legacy on.

The legacy there is off because I like to turn it off in order to avoid new features depending on legacy stuff. But for general use including legacy stuff should be beneficial.

1 Like
(Niels Dekker) #8

@dzenanz Instead of explicitly turning the flag on or off, at https://github.com/microsoft/vcpkg/ports/itk/portfile.cmake what do you think of leaving it to its default? (In practice that does mean legacy support for ITK 5.0.0.)

In general I would expect ITK from vcpkg to behave very much as if it was configured with default settings. What do you think?

1 Like
(Dženan Zukić) #9

Sounds even better!

1 Like
(Niels Dekker) #10

Thanks @dzenanz! Please check:

1 Like
(Niels Dekker) #11

Suppose one would need ITK built with different CMake options than another? (For example, suppose you would really need to have ITK_LEGACY_REMOVE=ON, while elastix needs to have the option OFF.) Could that possibly be supported by vcpkg?

(Dženan Zukić) #12

I don’t think so, but this question is probably better for vcpkg people.

1 Like