How to create an itk.js Image object?

Hello guys. I am a beginner. I hope to use itk and vtk in backend(C++) and use itk.js and vtk.js in front-end(JavaScript). I just read the docs of itk.js and want to create an itk.js Image object to show in browser. I wonder if I could obtain a json string through AJAX API and deserialize it to the itk.js Image object? In other words, how to convert an itk::Image to a json string in backend, which could be deserialized as an itk.js Image object?
My English is very poor and I don’t know if I make myself clear. Thank you for your help.

P.S. I’d like to calculate in back-end and show images in browser.Other solutions would also be welcomed. Thanks.

Hello @Mason,

Welcome to ITK!

A first step is to run your ITK C++ executable or Python script, then save the result in one of ITK’s supported file formats. MetaImage (.mha) or NRRD (.nrrd) are preferred because they support 2D, 3D, multiple pixel types, multiple pixel components, and they preserve image metadata. Then, serve the file on a web server, and create a link with the itk-vtk-viewer like this:

https://kitware.github.io/itk-vtk-viewer/app/?fileToLoad=https://data.kitware.com/api/v1/file/564a65d58d777f7522dbfb61/download/data.nrrd

To do more:

Since itk.js is an ITK External Module in addition to a NPM package, you can build an use it locally in your C++ application. After building it locally, use the provided itk::JSONImageIO class to write and read the JavaScript itk/Image data structure to a JSON file and a binary file that contains the pixel buffer data.

Build it with commands like:

git clone https://github.com/InsightSoftwareConsortium/itk-js.git
git clone https://github.com/Tencent/rapidjson.git
mkdir itk-js-build
cd itk-js-build
cmake -DITK_DIR=/path/to/your/ITK-build -DRapidJSON_INCLUDE_DIR=$PWD/../rapidjson/include ../itk-js/
make

In your program that uses ITK, require that the itk.js BridgeJavaScript module is available and its libraries linked into your program. In your CMakeLists.txt, call

find_package(ITK REQUIRED COMPONENTS BridgeJavaScript <other-components>)

where <other-components> are the output of running the ITK/Utilities/Maintenance/WhatModulesITK.py script on your source code.

Specify a filename that ends in .json to itk::ImageFileWriter, and it will produce a JSON file with the itk/Image structure and binary file for the pixel buffer data.

On the JavaScript side, libraries like axios make AJAX requests easier. Set the responseType to json or arraybuffer for the .json and binary files.

Then, create awesome things with essential tools like itk.js, vtk.js, itk-vtk-image-viewer, or paraview-glance.

2 Likes

Thanks a lot! Let me try.

1 Like

Hi Matt,
Is there any example that uses JSONImageIO class which converts from nrrd / dicom files to the JavaScript itk/Image datastructure and to JSONFile ?

I was able to use BridgeJavaScript lib in my C++ app and produce the 2 json files like this:
-rw-r–r-- 1 phongtd phongtd 504 Thg 1 16 09:52 t.json
-rw-r–r-- 1 phongtd phongtd 16777216 Thg 1 16 09:52 t.json.data

However on Javascript file, I wonder how itk js can read those json files and parse them into the Image Type ?

@Phong_Tran well done!

The next steps, to use your C++ app in the web browser or with Node.js on the server, is to build into JavaScript and WebAssembly. This Hello World example and this simple ITK pipeline demonstrate how to do this. Then, you can run the web executable with runPipelineBrowser. Please let us know how it goes and if you have any questions.

1 Like

Thanks Matt,
Yes I have read the Hello World example and successfully ran it. But our workflow model is to use the client-server architecture. When I read your statement

On the JavaScript side, libraries like axios make AJAX requests easier. Set the responseType to json or arraybuffer for the .json and binary files.

I thought that my C++ app will be on the server side which produces the json file and its data. And Axios, itk.js, vtk.js, itk-vtk-viewer will be on the client side which requests the json and its data files and process those files. I maybe misunderstood your thoughts. Can you please help to elaborate why do we need axios in this case ? Because if we build the C++ app into the Javascript and WebAssembly, we can use it directly in web browser.

If this is your use case, then this is correct:

We can create a function in the itk.js API, e.g. readImageHTTP to address this use case.

1 Like

readImageHTTP is implemented here:

1 Like

Hi Matt,
When I tested itk-vtk-viewer js with Image taken from readImageHTTP, I noticed that the Itk Image is not compatible with the existing data structure. For example, when I passed that Itk Image to function vtkITKHelper.convertItkToVtkImage like this:

test(‘readImageHTTP’, (t) => {
const testURL = ‘http://192.168.1.13:6502/v1/ivmero/api/service/out.json’;
readImageHTTP(testURL).then(function (itkImage) {
const imageData = vtkITKHelper.convertItkToVtkImage(itkImage)
})
})

It throws the exception in this line of vtk library

direction[col + idx * 3] =
itkImage.direction.data[idx + col * itkImage.imageType.dimension];

it’s because the Http ITK JS Image object does not have direction.data. Here is an example

direction: (9) [1, -6.123233996e-17, 7.88860904915783e-31, 6.123233996e-1

Do you think ITK JS need to change to datastructure to be compatible with existing VTK JS source ?

Hi @Phong_Tran,

Was the server-side file generated with ITKBridgeJavaScript 10.0.0? The direction.data was changed from 9.X.

Thank you Matt,
Yes I upgraded the ITKBridgeJavaScript 10.0.0 and it works. The ITK JS Image has now correct format.

1 Like