How to transfert pixel buffer from Javascript to Python?

Hi,
I’m trying to send the pixel buffer of an itk image (3D images) using a websocket in javascript to a python server.
On client side (javascript), I send the image data as binary content like this:

// image is a itk/image
var byteLength = image.data.BYTES_PER_ELEMENT
var step = 1024 * 1024 / byteLength;
// Send the pixel data in multiple frame elsewhere the Browser crashed (memory issue)
for (var k = 0; k < image.data.length; k += (step + 1)) {
	ws.send(image.data.slice(k, k + step)); // ws is the websocket
}

// Send now the image informations (direction, pixel type,....)
var message = {};
message["dimension"] = image.imageType.dimension;
message["pixelType"] = image.imageType.pixelType;
message["componentType"] = image.imageType.componentType;
message["components"] = image.imageType.components;
message["origin"] = image.origin;
message["spacing"] = image.spacing;
message["direction"] = image.direction;
message["size"] = image.size;
ws.send(JSON.stringify(message));
// All data are finished to be sent, so sent a flag to tell the server to do some work on the image
d = {};
d["data_sent_completed"] = true;
ws.send(JSON.stringify(d));

On server side (python), I receive like this using tornado library:

def on_message(self, message):
        # cl is a list of client
        if self in cl:
            if not self in buffers.keys():
                buffers[self] = dict()
            # if the input data is some bytes, so the data recieved is the pixel buffer data
            if isinstance(message, bytes):
                if not "data" in buffers[self]:
                    buffers[self]["data"] = []
                buffers[self]["data"].append(message)
            # If the message is some string, so the information of the image is sent
            if isinstance(message, str):
                if not "json" in buffers[self]:
                    buffers[self]["json"] = dict()
                j = json.loads(message)
                for k in j:
                    buffers[self]["json"][k] = j[k]

            # If the full image is sent with all the information necessary
            if "data" in buffers[self] and "json" in buffers[self]:
                if 'data_sent_completed' in buffers[self]["json"]:
                    # Flatten all the sub buffer of the data into one
                    buffers[self]["data"] = np.array(
                        buffers[self]["data"]).flatten()
                    # then read the buffer inside a numpy array
                    im_array = np.frombuffer(
                        buffers[self]["data"], dtype=itk_to_numpy_type[buffers[self]["json"]["componentType"]])
                    size = buffers[self]["json"]["size"]
                    # Reshape the array to have the same size as input size. The size of the numpy array is stored like this: Slices, Rows, Column (K, J, I)
                    im_array = im_array.reshape((size[2], size[1], size[0]))
                    # Create a SimpleITK image from the numpy array
                    im = sitk.GetImageFromArray(im_array)
                    im.SetOrigin(buffers[self]["json"]["origin"])
                    im.SetSpacing(buffers[self]["json"]["spacing"])
                    im.SetDirection(buffers[self]["json"]["direction"]["data"])
                    sitk.WriteImage(im, "test.nii.gz")

The code above is just a simple usage with no verification of the input data given.

The data is transferred with no problem detected. The save the image on server side also works.
Unfortunately, when I load the server image, I have some shift image like this:
snapshot0001

I verified the orientation, spacing, and origin and I have the same inputs as the client side. The dimension of the image is also the same between the input and the output.

Even if I didn’t want to store directly an itk Image (I don’t always have the hand on client side), I tried to use writeImageArrayBuffer but I have this error message :

"TypeError: imageIO.CanWriteFile is not a function
    at _callee3$ (blob:null/6f63af5e-7c88-48be-9e86-751e9f6b63f5:1491:26)
    at tryCatch (blob:null/6f63af5e-7c88-48be-9e86-751e9f6b63f5:110:40)
    at Generator.invoke [as _invoke] (blob:null/6f63af5e-7c88-48be-9e86-751e9f6b63f5:339:22)
    at Generator.prototype.<computed> [as next] (blob:null/6f63af5e-7c88-48be-9e86-751e9f6b63f5:162:21)
    at asyncGeneratorStep (blob:null/6f63af5e-7c88-48be-9e86-751e9f6b63f5:4:24)
    at _next (blob:null/6f63af5e-7c88-48be-9e86-751e9f6b63f5:26:9)
    at blob:null/6f63af5e-7c88-48be-9e86-751e9f6b63f5:33:7
    at new Promise (<anonymous>)
    at blob:null/6f63af5e-7c88-48be-9e86-751e9f6b63f5:22:12
    at _writeImage (blob:null/6f63af5e-7c88-48be-9e86-751e9f6b63f5:1573:22)"

Does anyone understand why there is some shift on the output image?
Is it normal to have the error message using itk writeImageArrayBuffer function?

Thank you

Edit: The problem was client side (javascript) with the loop:
instead of doing this:
for (var k = 0; k < image.data.length; k += (step + 1)) {
I do this:
for (var k = 0; k < image.data.length; k += step) {
A dumb mistake due to my misunderstanding of the slice method.

1 Like