Dear All,
I AM SORRY TO POST THE Dockerfile in such messy way, but i have a problem with building the SimpleITK from within the ubuntu container prepared specifically for SimpleITK image for running a container that has all the necessary packages in a little VM i built. by the way i follow the itk article about building SimpleITK for linux (Getting Started). I am trying to build a container with SimpleITK downloaded from github and then install all the build tools inside the container and eventually build it. I have gone as far as this like in the article
“Building using SuperBuild” when i do: cmake ../SimpleITK/SuperBuild
strangly cmake enter the cmake-3.10.1/Source directory and in cmOutputConverter.cxx file at method ForceToRelativePath give an assertion that localpath is empty?!
no idea why cmake source file but i have already build cmake binary and it is there in cmake directory source files, and i call the binary file to to super build…
I cannot post the image of the container i am running, but i am inside of the running contianer and with docker exec -it … command and i run the superbuild myself to debug the issue.
following is the contents of my Dockerfile… any help is appreciated.
btw: this line of Dockerfile < #— RUN cmake …/SimpleITK/SuperBuild > I commented out to run that myself from inside the running container!
# Download base image ubuntu 16.04
FROM ubuntu:16.04 AS simpleitk
# Update ubuntu software repository
RUN apt update
RUN apt install -y wget
RUN apt install -y g++
RUN apt install -y build-essential
RUN wget httpS://cmake.org/files/v3.10/cmake-3.10.1.tar.gz
RUN tar -xvzf cmake-3.10.1.tar.gz
RUN cd cmake-3.10.1/ && ./configure && make && make install && update-alternatives --install /usr/bin/cmake cmake /usr/local/bin/cmake 1 --force
# install build tools
RUN apt install -y cmake-curses-gui gcc g++ git
# build tools support binding different programming languages
RUN apt install -y monodevelop python python-dev tcl tcl-dev tk tk-dev
#get simpleitk source code
RUN cd ~
# clone the SimplITK repository to SimplITK directory
RUN git clone --recursive https://github.com/SimpleITK/SimpleITK.git
# get to SimpleITK directory
RUN cd SimpleITK
# Build SimpleITK
RUN cd ~
RUN mkdir SimpleITK-build
#--- RUN cmake ../SimpleITK/SuperBuild
# start build process
#--- RUN make
# add SimpleITK TO YOUR LD_LIBRARY-PATH
#---RUN export LD_LIBRARY_PATH=$LD_LIBRARY-PATH:~/SimplITK-build/lib
# optionally check whether the build was successful
# ----RUN cd ~/SimpleITK-build
# ----CMD [ "ctest"]
EXPOSE 8282
# ------CMD ["/bin/sh"]
ubuntu:16.04 has reached end of life. And the CMake version shipped with it is too old. And the version you manually install is one patch version short of the minimum required (3.10.2) for ITK.
I have tried to follow the getting started with SimpleITK in linux, I have now two folder SimpleITK, SimpleITK-build, I did run cmake …/SimpleITK/Superbuild/ from inside the SimpleITK-build, it is successfully downloaded a lot of files. now I don’t know how to make these files and then install SimpleITK ? i get a lot of errors if I run make from SimpleITK-build folder, I have only a Makefile there?! any help appreciated
Hi All,
SimpleITK build.ninja for VM docker container linux is stuck after downloading external data SHA512/…stuck at [898/3100]building CXX Object code/BasicFilters/src/CMakeFiles/SimpleITK_ITKImageGrid.dir/sitkTileImageFilter.cxx.o this is step [898/3100], stuck in this for couple of hours may be 4 hours. I checked the cpu activity in task manager, and it is high and I have 4 cores in this laptop with 6GB RAM and 2.5GHZ speed and 50GB HD is left. I am not sure what to do but I don’t think this has anything to do with docker image may be ninja build tool problem or bug. it has copied a lot of lib and header files before this External data downloading and building this tile image filter.cxx.o I think I have some libraries which are installed already prior to this and PCR… BUILD/INSTALL?
any idea what to do or ninja build has a bug? I saw some questions about this on the internet. by the way I used mat maccormicks Dockerfile to install Ubuntu base for this docker image. I am building a docker image to run a container off of it to do some programming inside the container and hopefully do some work with it.
My INTERNET IS CONNECTED!
EXTERNAL DATA IS COMMING FROM EVEN AMAZON AND OTHER SITES. MAY BE EXTERNAL DATA STUCK IF I EXIT THIS WHAT HAPPENS WITH THE BUILD, MY DOCKER IMAGE IS THERE, BUT INSTALLATION IS SCRWED UP.
I HAVE ENOUGH HARD DISK TO CONTAIN THE DOCKER IMAGE, NOT SURE IF THERE IS ANY LIMITATION ON DOCKER IMAGE SIZE? I READ 5-10 GB AND I DONT THINK SIMPLEITK AND EXTERNAL DATA IS MORE THAN THAT? IS IT?
ANY HELP IS GOOD.
any help appreciated.
p.s.: following is the mats’ docker file code snippet I used to install ninja build tools chain.
@sag I have consolidated your topic on difficulty with building SimpleITK in a Ubuntu container to your original post. Please keep post related to one topic or issue in the same topic.
@blowekamp understood. I had some more discoveries and observations that might be able to help you debug this issue. I make image without make/ninja because at some point I get error for output clipping warning /error. it does not exit. it seems just stuck there during image build with make/ninja, no idea why output is so limited in build vs buildx. I did build without ninja and then I ran a container out of that image in interactive mode with /bin/bash shell. I did ninja at the ubuntu command line but at some after ITK build and copy of .h and .a static libraries it goes into downloading External data SHA images from amazon and etc. I get stuck there, I check docker images in another windows terminal and I had two 1.55GB images of simpleitk + simpleitk container running + plus more stuff, that is 4GB still I must have 1GB to go until limit of 5GB from docker images. both images are the same ID meaning they are the same. Now I am stuck again in the same issue, like I explained.
I am a docker developer microservices back-end activemq, java, c++, .net mobile, python and I know there is something wrong with the build of SimpleITK inside the ubuntu container. Do you think I should turn off testing and EXTERNAL DATA download so that container image stops, what Dockerfile sample do you recommend I use I can send you mine but it is the same as the getting started instructions + some python environment create to facilitate the SimpleITK build. I am not sure yet what is happening when External data download and build of some of *.cxx.o files starts and then gets stuck there. could be the size of the image but I am not sure. I COULD DO MULTISTAGE DOCKERFILE AND THEN MAY DO SOME CLEANING OF CODE I DONT NEED because it is build and so forth. I appreciate if you could provide me with such multi-stage Dockerfile tested for a regular SimpleITK build/install better using ninja/ninja.build.
it is about 10 days I am stuck in this docker build of simpleitk. any help is appreciated.
I did wait longer than normal installation in windows, yet it get stuck the same way as before, I was forced to terminate docker image build. I wonder now if the GTest stage is causing this how to disable it in cmake stage? if I do that do I have other wrappers and simpleitk installed, I can leave examples out, but the GTest is for testing the correct installation and that I cannot leave out of the image build. I need to execute <ctest -v -j 3> command to test the installation? correct. what variables are the best to include in cmake to make this image build and I finally can do a simple c++ program to do some simple filtering and then save the image in a volume inside the container? I don’t even need the wrappers at this stage I can leave with c++ simpleitk and do my image processing and then in front end I can do the presentation.
what make variable I need to do this?
GTest and ctest are used for development of ITK. If you make a change to ITK, you should build and run tests (ctest -v -j3) to make you have not broken anything. If you just want to use ITK in your application, tests are not needed.
When making a docker image, it is advisable to disable tests: cmake -DBUILD_TESTING=OFF -G .... Without tests your docker image should build relatively fast and take a lot less disk space than with tests.
It is difficult to provide assistance as you are not providing the necessary details to reproduce. You did not provide a Dockerfile for you current approach. You also did not confirm if you are able to build and run the docker file I provided. If you are not able to build and run the provided Dockerfile, in my prior post, then you likely have a limitation with your Docker environment. It was just indicated that you are running MS Windows, this OS may be using a virtual machine to host Docker. The resource provided to the VM should be examined to see if a limitation is reached.
Testing can be disable with BUILD_TESTING being set to a false value.
All wrapping can be disable by setting WRAP_DEFAULT to a false.
I will build an image from your Dockerfile and let you know about the results. what i have is similar to your with more tools such as nano and other tools. it did get stuck as i explained below.
i am using Hyperv as host to docker desktop and i am not sure what are the limitations, for activemq and activemq-cpp i had no problem to use docker to host them plus other tools! i did disable testing and examples in one build i did and it did help to partially install simpleitk but at some point again got stuck and i waited on some IO class image reader class in step 609 out of 900. I had to terminate the build. I may have problem with the limitation of docker desktop and you may be right. i used DBUILD_TESTING:BOOL=FALSE DBUILD_EXAMPLES:BOOL=FALSE using GNinja/ninja configuration and code generation and ninja build.
I may use play with docker ecosystem to build tools on it and download simpleitk and see what happens.
i may build everything in WSL2 linux in windows to build simpleitk and then use ADD in docker to copy libraries and header files into the docker linux.
I did actually recreated the issue by using your Dockerfile. I created a one manager and worker in play with docker online. play with docker is provided by docker itself. this is a alpine linux. i tried to run your Docker file inside the linux alpine command line as commands and it is stuck after downloading external data on building cxx object sitkN4BiasFieldCorrectionImageFilter.cxx.o again just like when i tried to build image from docker file. There is something happening during downloading and building which may cause hanging on internet and i think somehow this is
related to simpleITK web site.
I also running another Dockerfile in another manager node inside play with docker, yet it is not finished yet. this one i build a docker imager out of provide Dockerfile by you. I will inform you about the results.
I observed the memory of the docker and it is only 4GB and i saw it displays the memory is consumed 100%. I think this is the issue. i dont know how to increase this? any help is appreciated in this regard. I will search docker documents to find out what is the solution.
I’m not sure if this in with the online free “play with docker” or locally on your system. Either way I don’t know off hand an you will need to search for the answer for that particular environment.
One work around is to limit the number of processes used during the build to 2 with a build commond such as ninja -j 2.
I did use play with docker online. I think it displayed the volume of memory inside the node alpine was running my cluster was one manager and one worker. I can try today to apply your recommendation to build with ninja -j 2 and see if that makes any difference.
one question do you know a simple key in docker build command line to use in order to send the image to the external hd during the image build.
I also will try to build my image in the WSL shell under windows 10 and will not use docker for build. later after build was successful I will write a Dockerfile to copy the libraries and header files to the docker container. I don’t know which directories to copy though, any help in this regard also would be nice.
I did try to build SimpleITK inside WSL2 subsystem of windows 10 and seems there is also memory limitation. My computer ram is about 6GB and docker plus the image seems to get close to 3.5GB at some point, I am checking task manager processes and in vmmim service shows very high memory activity and seems WSL is eating it up.I did build SimpleITK as release version so the size of the image is less than debug build, I used cmake -DCMAKE_BUILD_RELEASE with cmake -DBUILD_TESTING:BOOL=OFF -DBUILD_EXAMPLES.BOOL=OFF -GNinja …/SimpleITK/SuperBuild did not change anything. I think WSL2 VM has about 4GB memory and this is beyond the RAM Size of my computer.
I believe this is memory issue, yet I though may be I could disable some of the modules for one docker file. and add them to another docker file, that is, I create 2, 3, 4 docker images so that each one contains some of the modules and the second docker container has another set of modules but I don’t know how to do that I could not exactly pinpoint where in docs this information of list of modules are, any help regarding this or any document that can give me a list of modules of ITK OR SIMPLEITK I want to try that now.
would that kind of build be possible? and how I map each module to a macro in cmake command line code and then MODULE_ITK_REGISTRATION is in container 1 and hypothetically configured in such a way that independently works. on the other hand I create another docker container off of the second image which container MODULE_ITK_SEGMENTATION.
How do you think I do this Dockerfile, just a sample line of cmake code with variables to disable some module and keep some others and configure this for the container 1, and similarly do the same thing with different modules in container 2 and …
Were you able to add the “-j 2” flag to ninja? This will reduce the number of parallel processes running, and thereby reduce the amount of memory consumed. How many cores does the VM have? How many parallel build processes are running?
You could also try setting the build type to “MinSizeRel” to reduce the size of the binaries.
I did ninja -j 2 cores did stop again at the same line of code in the same module code/io/itkWriteImageFile. I have 5 cores by the way so I think my docker VM uses all of them because the whole laptop goes to very slow interaction with the mouse.
I thought okay lets see if I can build simpleITK in windows as usual I used to do and it work back then. I cloned it from : https://github.com/SimpleITK/SimpleITK.git. I as usual Launched a CMake application in windows and I created src and bin directories and I check out all the SimpleITK as source and copied it into the src, I did CMake it configure and generate code, btw: I had to install manually Lua, swig for wrappers. otherwise in the configure stage of cmake I got include directory error in cmake configure stage. I managed to generate code after all the manual installation and then I built it with vs2019 and I could not completely succeed, with wrappers I have 30 modules succeeded and 30 modules failed. it complained about some enum definitions in the common module fo simpleITK, ThreaderEnum not a type error, seems some headers not generated correctly, this was debug version, I did build for release and not much of difference
I did uncheck wrappers all of them and kept only c++ and still I failed in 8 projects. I am getting again errors like ThreaderEnum must be a type, some type conversion error in its::simple::ProcessObject::preUpdate, some headers are missing such as itsComposeScaleSkewVersion3DTransform.h so I am getting sort of different kind of errors. I believe I have copied all of the files from the repository and it should go and find all of the headers and type definitions and Enums, etc. I wonder ITK is not completely githubbed ?
need help with this one firstly.
is it possible because I choose vs2019 and it does not generate the right headers? should I use vs2017 instead? what is your verdict?
I plan to build in windows and then see what configurations I need and then I may be able to go to linux and build different modules in one docker container and another module in another VM, so I have 10 docker container of SimpleITK instead of One docker container. following questions are my concerns:
CMake 3.20.x is used for windows code configure and generate ( is it okay) or i need a lower version for windows?
is it okay to open the generated project in vs2019 or I should open it with vs2017?
Can you please tell me how to only cmake and generate for one module of SimpleITK per one docker container linux VM? and then ninja that to build the project simpleITK only for module IO, and Fiter, thus I have less memory consumption during cmake and ninja? how would you split SimpleITK into different modules so that they can containerized in different containers and then they can communicate via messaging like ActiveMQ? this sort of backend?
I am stuck now in windows build not even linux, I used to do this in couple of hours in previous version of VS2015 & VS2017 but vs2019 may be it is not supported the SimpleITK do you that?
It sounds like the host VM for the docker containers may be running out of memory. Try increasing the amount of memory allocated to you Docker host VM.