Between the Canvas and the WebWorker
A few weeks ago, I had written a pure Javascript implementation of the Seam Carving algorithm using Canvas and Web Workers. The advantage of using web workers was to allow the intensive processing of seam carving to run in the background without freezing the UI thread.
There was only one step in this process that still did not fit into the picture - passing the result from the Web Worker to the DOM due to context differences. This was slow as all pixel had to be copied into the CanvasPixelArray of the ImageData. This was a nested loop taking n x m operations.
This post is about the various alternatives I looked at to make this process better, and how the technique used here could feed into a generic canvas+webworker framework for image manipulation.
DataURIs
If I did not want to copy the n x m pixel,I needed a way to send the information from the Web Workers in a way that can be used in the DOM. DataURIs are a good alternative.I looked at a PNG library written in Javascript and it looked promising. Unfortunately, the compression and processing inside the Web Workers was taking longer than pixel copying. Also, it did not work for images wider than 325 pixels.
To avoid the overhead of compression, I tried using a Bitmap library but was having a hard time getting it to work. The dataURI for the uncompressed image that this library generated was invalid. I was losing the alpha transparency parameter anyway.
Forking off for Chrome
If I could not get a image processing library working, I had to find a way to make the pixel copy quicker. I had earlier written that in Firefox and Opera, I could create an object with the height and width as dimensions of the image and assign an array to its data property. I could draw this image using the putImageData and it worked like a charm.Ch
Chrome however, threw an exception because imageData.data was not of type CanvasPixelArray. I caught this exception and resorted to pixel copy in case of Chrome. Interestingly, Chrome traces that part of the code and runs it significantly faster than the other browsers.
Here is the commit with this implementation.
Internet Explorer?
Internet Explorer 9 Beta does not support WebWorkers yet and hence, the demo won't work. I am planning to see if I can move the SeamCarver.js to run on the server using Rhino or Nodejs.
About the Framework
When I started with project, I was also talking to the creator of Caman. It looked like a promising framework for image manipulation using Canvas and Web Workers. Unfortunately, it was plagued by the same issue of pixel copy, and the creator decided to move away from using Web Workers.
The library currently breaks the image into horizontal partitions and processes each strip with a setTimeout.
Apart from still hogging the UI thread (to a lesser extent), this also does not support from image processors like Seam Carving that require processing the whole image.
I hope to convince Caman to move back to its original model, or fork the project back to its original form and use this method of passing information between the canvas and web workers.
Watch this space for more updates.
