SImpleITK incremental loading JAVA

Hi @dchen, i’m not reading the hole series, i’m just reading the first file as i set here:

I reads the first file to get all metadata and continues reading the others. But my code now looks very different, i increased performance about 10 times.

For future readers:

the one responsible for managing the loading of the series is a class called Volume

This is Volume constructor:

public Volume(String dicomSeriesFolderPath, Window window) throws InterruptedException {
        this.window = window;
        this.dicomSeriesFolderPath = dicomSeriesFolderPath;
        this.dicomFilesNames = ImageSeriesReader.getGDCMSeriesFileNames(this.dicomSeriesFolderPath);
        if (this.dicomFilesNames.size() == 0) {
            throw new RuntimeException("pasta selecionada está vazia");
        }
        loadFirstFile();
        createMetaDataHashMap();
        loadMainMetaData();
        verifyVolumeExistence();
        loadFromLocalFolder();
        this.window.getViewportArea().getVolumes().add(this);
        this.indexID = this.window.getViewportArea().getVolumes().indexOf(this);
        this.window.getLeftArea().getSeriesTab().addSeries(this);
    }

loadFirstFile method:

private void loadFirstFile () {
        this.reader = new ImageSeriesReader();
        this.reader.loadPrivateTagsOn();
        this.reader.metaDataDictionaryArrayUpdateOn();
        this.reader.setFileNames(new VectorString(Collections.singleton(this.dicomFilesNames.get(0))));
        this.series = reader.execute();
        this.pixelID = this.series.getPixelID();
        //verifica se é scalar, vetor ou label
        if (this.series.getPixelID().toString().contains("Vector")) {
            this.isVector = true;
        } else if (this.series.getPixelID().toString().contains("Label")) {
            this.isLabel = true;
        } else {
            this.isScalar = true;
        }
        //verifica se é signed
        if (!this.series.getPixelID().toString().contains("U")) {
            this.isSigned = true;
        }
        //verifica quantos bytes possuem cada elemento
        if (this.series.getPixelID().toString().contains("8")) {
            this.pixelBytes = 8;
        } else if (this.series.getPixelID().toString().contains("16")) {
            this.pixelBytes = 16;
        } else if (this.series.getPixelID().toString().contains("32")) {
            this.pixelBytes = 32;
        } else {
            this.pixelBytes = 64;
        }
        System.out.println("pixel id = " + this.series.getPixelID());
        System.out.println("slices = " + this.dicomFilesNames.size());
        System.out.println("isSigned = " + this.isSigned + " | isScalar = " + this.isScalar + " | isVector = " + this.isVector + " | isLabel = " + this.isLabel);
    }

createMetaDataHashMap and loadMainMetaData are not important here.

loadFromLocalFolder method:

private void loadFromLocalFolder () throws InterruptedException {
        if (!Volume.incrementalLoading || this.isVector) {
            this.reader.setFileNames(this.dicomFilesNames);
        }
        this.series = reader.execute();
        if (this.dicomFilesNames.size() > 1 && Volume.incrementalLoading && !this.isVector()) {
            this.size = new VectorUInt32(new long[]{this.series.getSize().get(0), this.series.getSize().get(1), this.dicomFilesNames.size()});
            this.origin = this.series.getOrigin();
            this.direction = this.series.getDirection();
            this.spacing = this.series.getSpacing();
            this.reader.setFileNames(new VectorString(Collections.singleton(this.dicomFilesNames.get(1))));
            this.series = reader.execute();
            double mainSpacingThirdComponent = Math.abs(Math.abs(this.origin.get(2)) - Math.abs(this.series.getOrigin().get(2)));
            if (mainSpacingThirdComponent == 0) {
                mainSpacingThirdComponent = 1;
            }
            this.spacing = new VectorDouble(new double[]{this.spacing.get(0), this.spacing.get(1), mainSpacingThirdComponent});
            this.series = new Image(this.size, this.series.getPixelID());
            this.series.setOrigin(this.origin);
            this.series.setDirection(this.direction);
            this.series.setSpacing(this.spacing);
            loadLayer(0);
            int amountLayerLoaders = Math.min(Volume.amountLayerLoaders, this.dicomFilesNames.size());
            int layersPerThread = this.dicomFilesNames.size()/Volume.amountLayerLoaders;
            for (int threadIndex = 0; threadIndex < amountLayerLoaders; threadIndex++) {
                LayerLoader thread;
                if (threadIndex == 0) {
                    thread = new LayerLoader(this, threadIndex + 1, 1, (threadIndex+1)*layersPerThread);
                } else if (threadIndex == amountLayerLoaders - 1) {
                    thread = new LayerLoader(this, threadIndex + 1, threadIndex * layersPerThread + 1, this.dicomFilesNames.size() - 1);
                } else {
                    thread = new LayerLoader(this, threadIndex + 1, threadIndex*layersPerThread + 1, (threadIndex+1)*layersPerThread);
                }
                this.layerLoaders.add(thread);
                thread.start();
            }
        }
    }

i am using threads to load and concatenate the layers simultaneously, but there are some errors of stripes without pixels in some layers, probably because of the concurrency of the threads, today I’m going to implement a more efficient lock.

LayerLoader thread class:

class LayerLoader extends Thread {

    private final Volume volume;
    private final int from;
    private final int to;
    private final int id;

    public LayerLoader (Volume volume, int id, int from, int to) {
        this.volume = volume;
        this.id = id;
        this.from = from;
        this.to = to;
    }

    @Override
    public void run() {
        for (long fileIndex = this.from; fileIndex <= this.to; fileIndex++) {
            long startImage = System.currentTimeMillis();
            this.volume.loadLayer(fileIndex);
            long finishConcat = System.currentTimeMillis();
            System.out.println("THREAD " + this.id + ": finished index " + fileIndex + " of " + fileIndex + "/" + (this.to-this.from+1) + ",concatTime = " + (finishConcat - startImage) +
                    "ms");
        }
        this.volume.getLayerLoaders().remove(this);
    }
}

and to finish off the ugliest but most optimized method, loadLayer:

public void loadLayer (long fileIndex) {
        if (!this.loadedLayers.contains(fileIndex) && !this.loadingLayers.contains(fileIndex)) {
            this.loadingLayers.add(fileIndex);
            ImageSeriesReader layerReader = new ImageSeriesReader();
            layerReader.setFileNames(new VectorString(Collections.singleton(this.dicomFilesNames.get((int) fileIndex))));
            Image newLayer = layerReader.execute();
            if (newLayer.getPixelID() != this.pixelID) {
                System.out.println("image " + (fileIndex + 1) + "/" + this.dicomFilesNames.size() + " has different pixel id = " + newLayer.getPixelID());
                this.loadingLayers.remove(fileIndex);
                //in this case the series can only be rendered on the axial axis, and will be separated into "several series", since the pixel type is different. not implemented yet.
                return;
            }
            VectorUInt32 setPixelPosition = new VectorUInt32(new long[]{0, 0, (long) fileIndex});
            VectorUInt32 getPixelPosition = new VectorUInt32(new long[]{0, 0, 0L});
            PixelIDValueEnum pixelType = newLayer.getPixelID();
            if (pixelType == PixelIDValueEnum.sitkUInt8) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkInt8) {
                for (long y = 0; y < newLayer.getHeight(); y++) {
                    for (long x = 0; x < newLayer.getWidth(); x++) {
                        setPixelPosition.set(0, x);
                        setPixelPosition.set(1, y);
                        getPixelPosition.set(0, x);
                        getPixelPosition.set(1, y);
                        this.series.setPixelAsInt8(setPixelPosition, newLayer.getPixelAsInt8(getPixelPosition));
                    }
                }
            } else if (pixelType == PixelIDValueEnum.sitkUInt16) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkInt16) {
                for (long y = 0; y < newLayer.getHeight(); y++) {
                    for (long x = 0; x < newLayer.getWidth(); x++) {
                        setPixelPosition.set(0, x);
                        setPixelPosition.set(1, y);
                        getPixelPosition.set(0, x);
                        getPixelPosition.set(1, y);
                        this.series.setPixelAsInt16(setPixelPosition, newLayer.getPixelAsInt16(getPixelPosition));
                    }
                }
            } else if (pixelType == PixelIDValueEnum.sitkUInt32) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkInt32) {
                for (long y = 0; y < newLayer.getHeight(); y++) {
                    for (long x = 0; x < newLayer.getWidth(); x++) {
                        setPixelPosition.set(0, x);
                        setPixelPosition.set(1, y);
                        getPixelPosition.set(0, x);
                        getPixelPosition.set(1, y);
                        this.series.setPixelAsInt32(setPixelPosition, newLayer.getPixelAsInt32(getPixelPosition));
                    }
                }
            } else if (pixelType == PixelIDValueEnum.sitkUInt64) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkInt64) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkFloat32) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkFloat64) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkComplexFloat32) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkComplexFloat64) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkVectorUInt8) {
                for (long y = 0; y < newLayer.getHeight(); y++) {
                    for (long x = 0; x < newLayer.getWidth(); x++) {
                        setPixelPosition.set(0, x);
                        setPixelPosition.set(1, y);
                        getPixelPosition.set(0, x);
                        getPixelPosition.set(1, y);
                        this.series.setPixelAsVectorUInt8(setPixelPosition, newLayer.getPixelAsVectorUInt8(getPixelPosition));
                    }
                }
            } else if (pixelType == PixelIDValueEnum.sitkVectorInt8) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkVectorUInt16) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkVectorInt16) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkVectorUInt32) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkVectorInt32) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkVectorUInt64) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkVectorInt64) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkVectorFloat32) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkVectorFloat64) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkLabelUInt8) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkLabelUInt16) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkLabelUInt32) {
                //precisa implementar
            } else if (pixelType == PixelIDValueEnum.sitkLabelUInt64) {
                //precisa implementar
            } else {
                throw new RuntimeException("tipo de pixel não reconhecido");
            }
            this.loadedLayers.add(fileIndex);
            this.loadingLayers.remove(fileIndex);
        }
    }

OK, I see that you’re only loading the first slice at the beginning.

But it’s this loop that is problematic.

Looping through every pixel, allocating a new VectorUInt32 and VectorUInt8, calling setPixelAsVectorUInt8 every single time, and then garbage collecting those vectors.

And looking at your timings, some slices are much slower than others. I’d guess that’s when the garbage collector has to kick in and clear out memory.

At the very least you could create those vectors outside the loops and just change their values inside. So you’re not constructing and then deleting them every time.

But, as I said earlier, IMHO the best thing to do is collect all the slices into an array and then call SimpleITK’s JoinSeriesImageFilter to combine them into a 3d image.

Your are right, first time i was not realizing this problem, but it was fixed here:

I really wanted to use JoinSeries, the problem is that the result was not being concatenated on the AXIAL axis (or whatever the image direction), but as a fourth dimension, I can’t understand exactly how to fix this.
I will try again now.

OK, your difficulty with the Join is that when each slice is read, you get a 3d image of dimensions [512,512,1]. You really want a 2d image [512,512]. To get that, use the ExtractImageFilter and set the size to be [512,512,0].

1 Like

i understand,

but the problem I see in this is that I am limited to doing a single JoinSeries, not being able to do the join and use the result for the next join. This lands me with the same result as reader.execute() with all the files at once with no benefit.

My intention from the beginning was to show the first slice on the screen and load the next ones in the background just like the radiant dicom viewer software does.

How about if you pre-allocate a blank volume the size of the entire DICOM series. Then use the PasteImageFilter to paste each slice into the volume when it’s loaded?

1 Like

hmm very interesting, i gonna try.

https://simpleitk.org/doxygen/latest/html/classitk_1_1simple_1_1PasteImageFilter.html

1 Like

Hi @dchen,

i’m applying the filter but it doesn’t seem to be working properly, the final image is fully null.

newLayer has size [512, 512, 1], and this.series is [512,512,53]

        PasteImageFilter filter = new PasteImageFilter();
        filter.setSourceIndex(new VectorInt32(new int[]{0, 0, 0}));
        filter.setSourceSize(new VectorUInt32(new long[]{newLayer.getWidth(),newLayer.getHeight(),0}));
        filter.setDestinationIndex(new VectorInt32(new int[]{0, 0, (int) fileIndex}));
        filter.setDebug(true);
        filter.execute(this.series, newLayer);