I had written about Seam Carving in Javascript in my earlier posts. The example that I had till now only demonstrated removing the pre-calculated seams and ran in the browser's foreground thread. This post is about SeamCarving done using Web Workers and Canvas.
A couple of week ago, I stumbled upon CanMan.js on reddit and thought that I could add seam carving as a filter to it. The filters leverage Web workers and it looked like a framework to implement image manipulation filters. Unfortunately, the API signature required for the filters did not allow manipulation based on adjacent pixel in an image. Hence, it was not suitable for Seam Carving.
Having started, I decided to enhance the existing example to use Web Workers. First, I added the missing logic to calculate seams for any image. Since this was an operation what would take a long time, I had to move this processing to web workers. Some interesting observations along the way include
Passing Image Data
The SeamCarver operation takes the image pixels as input and returns image pixels after the desired operation. The canvas' getImageData returns the type ImageData which cannot be passed around in the worker context. Hence I had to deconstruct and reconstruct this every time the processing was complete. This meant that I had to copy all the pixels from an array to the array-like CanvasPixelArray in the ImageData. Interestingly, I could simple reassign the array and Firefox/Opera seems to be fine with it. Chrome however did not accept the simple array assignment, and I had to iterate on the pixels, copying them into the CanvasPixelArray.
Another interesting observation here was that Chrome, with its tight loop tracing performed a lot better than Firefox. I am assuming that Firefox 4 would get the same speed benefits.
Debugging Workers
The hardest part during development was debugging. With so much pixel manipulation, I needed a way to break on statements. Unfortunately, neither Firebug, nor Venkman broke on web worker threads. Chrome developer tools were also not much of a use either. I am hoping that the generic script tools include worker scripts in the script tags.
UI Thread responsiveness
The idea of web workers was to demonstrate UI responsiveness. I thought of using loader gifs, rewriting the DOM with javascript, etc, but finally zeroed in on using a scrolling marquee.
Next Steps
The seams are carved vertically only, so resizing it on the y axis is still not possible.
Some of the things that I am looking at doing in the future include
- Support y-axis resizing
- Port carving to server (using node, etc) and the browser re-sizes the image.
- Try to see if I can use as an example for getting the right API signature in frameworks like CanMan.
