Between the Canvas and the WebWorker
A few weeks ago, I had written a pure Javascript implementation of the Seam Carving algorithm using Canvas and Web Workers. The advantage of using web workers was to allow the intensive processing of seam carving to run in the background without freezing the UI thread.
There was only one step in this process that still did not fit into the picture - passing the result from the Web Worker to the DOM due to context differences. This was slow as all pixel had to be copied into the CanvasPixelArray of the ImageData. This was a nested loop taking n x m operations.
This post is about the various alternatives I looked at to make this process better, and how the technique used here could feed into a generic canvas+webworker framework for image manipulation.
DataURIs
If I did not want to copy the n x m pixel,I needed a way to send the information from the Web Workers in a way that can be used in the DOM. DataURIs are a good alternative.I looked at a PNG library written in Javascript and it looked promising. Unfortunately, the compression and processing inside the Web Workers was taking longer than pixel copying. Also, it did not work for images wider than 325 pixels.
To avoid the overhead of compression, I tried using a Bitmap library but was having a hard time getting it to work. The dataURI for the uncompressed image that this library generated was invalid. I was losing the alpha transparency parameter anyway.
Forking off for Chrome
If I could not get a image processing library working, I had to find a way to make the pixel copy quicker. I had earlier written that in Firefox and Opera, I could create an object with the height and width as dimensions of the image and assign an array to its data property. I could draw this image using the putImageData and it worked like a charm.Ch
Chrome however, threw an exception because imageData.data was not of type CanvasPixelArray. I caught this exception and resorted to pixel copy in case of Chrome. Interestingly, Chrome traces that part of the code and runs it significantly faster than the other browsers.
Here is the commit with this implementation.
Internet Explorer?
Internet Explorer 9 Beta does not support WebWorkers yet and hence, the demo won't work. I am planning to see if I can move the SeamCarver.js to run on the server using Rhino or Nodejs.
About the Framework
When I started with project, I was also talking to the creator of Caman. It looked like a promising framework for image manipulation using Canvas and Web Workers. Unfortunately, it was plagued by the same issue of pixel copy, and the creator decided to move away from using Web Workers.
The library currently breaks the image into horizontal partitions and processes each strip with a setTimeout.
Apart from still hogging the UI thread (to a lesser extent), this also does not support from image processors like Seam Carving that require processing the whole image.
I hope to convince Caman to move back to its original model, or fork the project back to its original form and use this method of passing information between the canvas and web workers.
Watch this space for more updates.
Refreshing IndexedDB Examples to FF 4 B9
The IndexedDB implementation in Firefox 4 Beta 9 had some interesting changes.
Event.result change
The biggest change was that for all IDBRequests, the result was available on the result. For example, the code snippet to get the cursor would be something like
This has changed to
I modified the examples in TrialTool to start using these changes appropriately.
Indexing Bug
The second interesting issue that I stumbled upon was about Indexes. I noticed that I was able to iterate over only the data that was added to the object store after the index was created. For some reason, data added before the index was created was not available on the index's cursor.
All IndexedDB data is stored using SQLite under the %profile%/IndexedDB. If you open the file using SQLite Browser, you would notice that all Indexed data is stored under a table called index_data. Opening this table confirms the fact that older data was not indexed.
I told Ben from Mozilla, and here is the bug tracking this.
Undefined Cursor
Another fact I stumbled upon was that the cursor was "undefined" when I try to open it on an empty database. I usually associate "undefined" to javascript properties that have not been initialized. Interestingly, the cursor is set to "undefined" after the iteration is complete, causing the confusion. I had a brief conversation with Jonas Sicking and suggested that setting it to null or false may be better. This is least disruptive and not break the existing way the end-of-iteration is checked using if (!cursor) {..handle end..}.
All the changes are not available at http://nparashuram.com/trialtool/index.html#example=/ttd/IndexedDB/moz_indexedDB.html
Event.result change
The biggest change was that for all IDBRequests, the result was available on the result. For example, the code snippet to get the cursor would be something like
var request = db.openCursor();
request.onsuccess = function(){ var cursor = event.result;}
This has changed to
var request = db.openCursor();
request.onsuccess = function(){ var cursor = result.result; }
I modified the examples in TrialTool to start using these changes appropriately.
Indexing Bug
The second interesting issue that I stumbled upon was about Indexes. I noticed that I was able to iterate over only the data that was added to the object store after the index was created. For some reason, data added before the index was created was not available on the index's cursor.
All IndexedDB data is stored using SQLite under the %profile%/IndexedDB. If you open the file using SQLite Browser, you would notice that all Indexed data is stored under a table called index_data. Opening this table confirms the fact that older data was not indexed.
I told Ben from Mozilla, and here is the bug tracking this.
Undefined Cursor
Another fact I stumbled upon was that the cursor was "undefined" when I try to open it on an empty database. I usually associate "undefined" to javascript properties that have not been initialized. Interestingly, the cursor is set to "undefined" after the iteration is complete, causing the confusion. I had a brief conversation with Jonas Sicking and suggested that setting it to null or false may be better. This is least disruptive and not break the existing way the end-of-iteration is checked using if (!cursor) {..handle end..}.
All the changes are not available at http://nparashuram.com/trialtool/index.html#example=/ttd/IndexedDB/moz_indexedDB.html
DoctypeHTML5 (Hyderabad)
I was at the DoctypeHTML5 session - Hyderabad Edition.
Here are the slides from my session of Multimedia..
Here are the slides from my session of Multimedia..
Comparing IndexedDB on FireFox, Chrome and IE
I have been trying out implementations of IndexedDB on various browsers over the weekend and here are the examples using TrialTool. The browsers used were
- Internet Explorer 8 with the IndexedDB prototype.
- Firefox 4.0 Beta 8
- Google Chrome Canary build 10.0.624.0
There were some differences in the API which I hope gets ironed out before the final release. Some of the differences in the API between Internet Explorer (IE), Firefox (FF) and Chrome (CH) are as follows.
Prerequisites
In case of IE, the activeX object needs to be initialized. In case of FF and CH, the properties are moz_indexedDB (mozIndexedDB in FF4B9) and webkitIndexedDB. Some other properties like ranges and constants (like webkitIDBRange, etc) also need to be initialized to the common property.
Transactions
The IE implementation allows an explicit close method while the transaction is automatically closed in case of FF and Ch when it goes out of scope.
Creating Object Store
The specification can only be created in a version change scope. IE does not enforce this yet. In case of FF and IE, the keyPath is specified as strings, while CH requires a json literal of the form {"keyPath" : "field_name"}. FF4b9 will follow the json object for the second argument.
Object and Key Cursors
IE and FF have a method to openObjectCursor, that is openCursor in CH. On CH, the openKeyCursor is the equivalent of openCursor on FF and IE. FF4b9 will follow the standards and use the methods like CH.
Unimplemented GetAll and Clear methods
CH does not implement getAll and clear method on ObjectStore.These are FF only methods but are really useful. Hope it gets into the standard.
Transactions in setVersion
Creating and removing Indexes in the version change transaction scope is interesting. In the onsuccess(event) callback, it is available as the event.result in CH and event.transaction in FF. It will change to event.result in FF4b9.
The list of differences is not exhaustive; if you find any interesting facts, please do drop in comments.
The source is located at https://github.com/axemclion/ttd/tree/gh-pages/IndexedDB.
Thanks to Ben Turner from Mozilla for providing inputs for the indexes and changes coming in for Firefox 4, Beta 9. Can't wait for the next version !! :)
IndexedDB for Internet Explorer
Run IE in administrator mode. Select "Run ActiveX for all websites" in the security bar if prompted.
I had previously posted about TrialTool and how it was used to showcase the IndexedDB APIs in Firefox. You can find the application here. A few weeks ago, Microsoft also released their prototype of IndexedDB, and I wanted to see if I could get some examples working for IE.
To run the examples, you need to register the ActiveX plugin, as indicated in the README file. To run the examples here, you would also need to run IE in the administrator mode. IE will pop up the security ribbon at the to, asking if you would like to allow the ActiveX to run. Select the option to run the control for all websites as IE, for some reason, resets the fragment identifier. Since this example is identified by its fragment identifier, you need to select that the ActiveX runs for all websites.
To run the examples, you need to register the ActiveX plugin, as indicated in the README file. To run the examples here, you would also need to run IE in the administrator mode. IE will pop up the security ribbon at the to, asking if you would like to allow the ActiveX to run. Select the option to run the control for all websites as IE, for some reason, resets the fragment identifier. Since this example is identified by its fragment identifier, you need to select that the ActiveX runs for all websites.
You can find the code for the examples here. The pre-requisites code is picked from the bootlib.js file in the download package. It basically initializes the ActiveX control and assigns a set of constants.
The harder part was to play around with transactions, opening a new READ-WRITE transaction for every operation. Only one active transaction is possible at a time. The other examples are simple, following the API to the book.
I have not had a chance to update the documentation yet, but that is something I will work on, as I get time. There was also a change to the TrialTool, marked by the commit here.
Watch out this tag for more updates.
Subscribe to:
Posts (Atom)