ITK currently maintains ~60 separate remote module repositories. Each has its own git repo, CI configuration, Python packaging, and boilerplate. This creates significant maintenance overhead:
~10,600 lines of duplicated boilerplate across modules
CI workflow versions fragmented across 10 different action tags
A single CI infrastructure update requires 15+ coordinated PRs
Every new filter requires creating an entirely new repository
This came up in the SSIM filter PR #6034 discussion where @blowekamp, @dzenanz, and I agreed that consolidation is needed and that the 6.0 beta phase is the right time.
The idea
Mirror ITK’s internal Modules/[Core|IO|Filtering|...] group hierarchy in Remote modules. Group ~50 modules into 7 category repos by domain, bringing the total from ~60 repos down to under 10 ISC-managed repos.
New CMake sub-group options (ITKGroup_Remote_IO, ITKGroup_Remote_Filtering, etc.) control which category repos are fetched. All OFF by default. Individual Module_<name> toggles still work within an enabled sub-group.
New modules (like SSIM) would be added as a subdirectory PR to the appropriate category repo – no new git repo needed.
Key design decisions we need input on
Category groupings – are IO/Filtering/Mesh/Analysis/Registration/Microscopy/Kitware the right buckets?
Domain vs. maintainer – grouping is by domain, not who currently maintains it. Some “external” repos are artifacts of the old one-module-per-repo requirement. Maintainers may prefer to be part of a larger codebase. Each placement should be discussed.
Microscopy scope – IOOpenSlide is the only current candidate. Enough to justify a separate repo, or too small?
KitwareMedical modules – Kitware-managed repo, or merge into ISC categories by domain?
Large independents – RTK, TubeTK, WebAssemblyInterface likely stay independent. Others?
Full proposal with module assignments, CMake mechanism details, and implementation phases is in the tracking issue #6060. Please comment there for decisions that need to be recorded; use this thread for broader discussion.
This enable remote modules to be turned on in Groups? It does not consolidate the numerous repos?
I thought that much of the burden of maintenance was related the number of remote repositories that need to be maintained. I’m not sure this addresses it.
I would find the option to enable module based on the compliance level useful or similar meta-data. For example if a change is made in core ITK then it may be useful to enable remotes of level 4+ which would have a good guarantee that they would work out of the box.
I have a draft proposal to try exactly this out. I think it is the way to go to provide metadata, and guidelines of what level of quality users should expect. That will be much easier to maintain in a single repository.
For every integrated remote module, retain the following “report card” intent and inject the final “Grade Level” as a static cmake variable for selecting quality levels easily.
#-- # Grading Level Criteria Report
#-- EVALUATION DATE: 2020-03-24
#-- EVALUATORS: [Dženan Zukić]
#--
#-- ## Compliance level 5 star (AKA ITK main modules, or remote modules that could become core modules)
#-- - [ ] Widespread community dependance
#-- - [ ] Above 90% code coverage
#-- - [ ] CI dashboards and testing monitored rigorously
#-- - [X] Key API features are exposed in wrapping interface
#-- - [ ] All requirements of Levels 4,3,2,1
#--
#-- ## Compliance Level 4 star (Very high-quality code, perhaps small community dependance)
#-- - [X] Meets all ITK code style standards
#-- - [X] No external requirements beyond those needed by ITK proper
#-- - [ ] Builds and passes tests on all supported platforms within 1 month of each core tagged release
#-- - [ ] Windows Shared Library Build with Visual Studio
#-- - [ ] Mac with clang compiller
#-- - [ ] Linux with gcc compiler
#-- - [ ] Active developer community dedicated to maintaining code-base
#-- - [ ] 75% code coverage demonstrated for testing suite
#-- - [X] Continuous integration testing performed
#-- - [X] All requirements of Levels 3,2,1
#--
#-- ## Compliance Level 3 star (Quality beta code)
#-- - [X] API | executable interface is considered mostly stable and feature complete
#-- - [X] 10% C0-code coverage demonstrated for testing suite
#-- - [X] Some tests exist and pass on at least some platform
#-- - [X] All requirements of Levels 2,1
#--
#-- ## Compliance Level 2 star (Alpha code feature API development or niche community/execution environment dependance )
#-- - [X] Compiles for at least 1 niche set of execution envirionments, and perhaps others
#-- (may depend on specific external tools like a java environment, or specific external libraries to work )
#-- - [X] All requirements of Levels 1
#--
#-- ## Compliance Level 1 star (Pre-alpha features under development and code of unknown quality)
#-- - [X] Code complies on at least 1 platform
#--
#-- ## Compliance Level 0 star ( Code/Feature of known poor-quality or deprecated status )
#-- - [ ] Code reviewed and explicitly identified as not recommended for use
#--
#-- ### Please document here any justification for the criteria above
# Code style enforced by clang-format on 2020-02-19, and clang-tidy modernizations completed