Of the various specifications that are considered a part of HTML5, the IndexedDB API is one of the few APIs that are independent of the DOM and are impacted by Javascript. It would be an interesting to analyze the impact of newer Javascript features to this API set.
This post looks at current features of ECMA5 in addition to proposals in Harmony. I had previously posted slides for a talk on IndexedDB and the last slide touched briefly about this.
Modules
The module pattern has been widely accepted as a useful feature, thanks to CommonJS and specifically NodeJS implementations. Today, the IndexedDB object is a property of the global Window or WebWorker Objects. With modules, IndexedDB can be included into the code using something like require("IndexedDB") for async and require("IndexedDBSync") for the sync version of the API. With this distinction, will developers be allowed to use the sync version in non-web worker environments also?
Promises
Like modules, the promises pattern is also gaining traction. Currently, all Async operations on IndexedDB are on a IDBRequest Object. Should IDBRequest be changed to follow the Promises pattern?
Transaction Scope with Let and Var
With the changes in variable scopes using the let and var keywords, transaction scope will also be impacted. Transactions today auto-commit when the transaction variable goes out of scope and no more requests can be placed against it. With the Let statement, this could happen sooner inside blocks allowing even shorter transactions. This would also avoid closure related confusions in transaction scopes. This would also ensure that transactions variables are not leaked into the global scope.
Iterators
Iterating
over a cursor today is done using a cursorRequest. Similar syntax should be used
for iterating over cursors. The cursor.continue could be replaced with
the iterator next method and the cursor could be treated as a
collection. However, it would be interesting to see how the update and
delete method on the cursor change for the new iterator pattern.
Generator Expressions and Array Comprehensions
While
iterating over cursors, developers mostly perform some operations on
the current item.Generator expressions are based on array comprehensions
and make sense here, given the asynchronous nature of the API.
Additionally, Array functions like each, filter, some, etc could also
eliminate a lot of cursor boiler plate code.
Destructuring Assignments
Continuing
the discussion about cursors, the cursor.value and cursor.key return
the key and the value respectively. However, restructuring assignments like [key, value] = cursor.continue() would help extract the keys and values easily. Similarly, doing something like [objectValue, key] = objectStore.get(id) could also be a useful notation.
Binary Blobs
With the introduction of Binary Blobs, specially for file, WebGL and devices API, it would be interesting to see how
the databases (LevelDB or SQLite) store binary data and how they impact
the quotas. I am guessing that indexing based on Binary Data would not
be possible in the near future.
READ_WRITE/READ transaction modes and Object Freeze/Seal
Objects
can be frozen or sealed to prevent them from being manipulated. In case
of a READ transaction, the object obtained with a get or in cursors
could be sealed and frozen to indicated that it was obtained from a READ
transaction as opposed to a READ_WRITE transaction. This may however be
an extreme way to indicate the transaction type.
Object Getters and Setters
With objects now having the ability define custom getters and setters, developers can use this method to provide a transparent abstraction that persists objects when properties are changed or loaded from a database when objects are read. Combining this with a Hibernate like query language could do ORM-magic that are traditionally done only with server-side languages. However, the logic on when to do (lazy vs eager) an async write or read would require some thought.
Classes
With the introduction of classes, it would be interesting to see schema for Object Stores. This may also change the way IndexedDB implementations could possibly change the way objects are stored internally. For example, IndexedDB on Firefox stored Blob for objects in SQLite today.
I have not come across similar documentation that discusses how IndexedDB works with new constructs in Javascript and I think this is an interesting topic of discussion. Please leave your opinions in the comments.
jsFoo Bangalore - Session Videos and Slides
Designing the IndexedDB API
Developing Apps for Windows 8 - My Experiences and Opinions
Disclaimer: The views expressed in this post are my own and do not reflect any opinions of my employer. The app was written using publicly available resources as an Independent developer.
A few posts ago, I had written about the TrialTool app for Windows 8. TrialTool is a web based application that illustrates the capabilities of various Javascript APIs. It is like an API playground that lets you view, modify and run examples and this version of TrialTool specifically illustrates the various Windows 8 APIs.
1. Background
The app was developed on Windows 8 Developer Preview with Visual Studio 11 Express. The entire application was written using HTML5, CSS and Javascripht. The tool does not exhaustively cover all APIs yet, but the APIs used are a good representative of the programming model in Windows 8.
2. Development Environment
My primary IDE is Eclipse or Cloud9 and this was actually the first time I have used Visual Studio exclusively for an end to end project.
The application starts from choosing a pre-defined project type or a simple blank JS template in Visual Studio. The templates do a good job of creating the skeleton and filling in the boiler plate code. As with any skeletons, it may be hard to navigate the multiple files created as a part of the template for a newbie. Once you know your way around the files and the parts to edit, the templates are good. Unfortunately, looks like Visual Studio is the only option for now as I was not able to find any command line utilities that I could use with Eclipse or Notepad++ to build, package or deploy the project.
3. Writing Code
Starting with a blank project is similar to writing a web page from scratch. Using a template however starts us off with the Metro Style UI. TrialTool used the grid layout template that starts with an "index" page that displays a list of items. Each item can be clicked to take us to a "details" page.
The Javascript files created by the template recommend using strict mode.
3.1 Single Page application
The app created is inherently an equivalent of one-page website. Navigation from the "index" page to a "details" page results in the details being loaded as a DOM fragment. Loading and unloading events give us control over the content to be displayed on each action. The WinJS module defines events that let us access the activate or DOM fragment load events for navigation in addition to events for layout changes (portrait to landscape mode for tablets, re-size, etc. )
I noticed that the by default, files of type HTML, CSS and JS had a directory each. I would prefer to group files by modules instead of file type; file types are distinguished by their extension anyway.
3.2 CSS Grids
The use of ms-grid to define the metro UI is great. Grid layout for CSS are still in work in progress. Tiles are an prime to the Metro UI and the implementation of ms-grid makes building tiles in Metro UI is a lot easier.
Vendor prefix would be removed in the final version I guess.
3.3 Data Binding
The Templates also have support for data binding. With the data-win-* attributes in a the HTML tags, render views can be defined. These render views can define specific controls (like ListView) and options for these controls. JSON data can be loaded via AJAX and use these "itemRenderers".
The concepts here are very similar to frameworks like Backbone. However, some features like automatically changing the view when data changes, etc needs to explicitly coded unlike backbone. A way to point a itemRenderer to a URL would be even better.
Another feature that I would like to have is the ability to declare formats in the item renderers. Currently, the data from the model can be converted, but that seems to be manual. A data-win-option on individual elements to specify the format would be good.
3.4 Promises
All asynchronous operations (like AJAX, File reading, etc) seem to return promises. This is definitely better for an interactive event driven application. The "async" keywork in C#
is the equivalent.
The documentation for the promised could have been better. Looks like the documentation is written for C# and force fitted to Javascript. For example, in an operation like FilePicker, the return value is a promise. However, to see the actual arguments passed to the promise.then() method, you have to navigate to the "GetResults" method.
4. Windows 8 APIs
The main reason I would write a Windows 8 app (apart from store distribution) would be to use the Windows 8 APIs; otherwise, the app may as well be a web site viewed in the Metro IE.
Working with native Windows 8 APIs in Javascript is no different manipulating DOM using native DOM methods. Popping up a Windows 8 Toast is no different from invoking the native window.alert. Working with native File APIs, or cryptography is also very easy and straight forward.
The documentation on the Windows Runtime APIs is exhaustive but I would would have been more comfortable if they were grouped better and the first list the over high level categories (like Storage) and then you could drill down to understand a specific API (Bulk Access or File properties). This would help since a lot of APIs have common concepts before we look at specific APIs. This was probably why I wrote TrialTool in the first place.
The other issue I found was an example of APIs that require features not present in the Javascript language. For example, reading the contents of a file returns an InputStream that does not seem to show any Javascript API. It is not very clear that DataReader can in fact be used to read the data.
I wish that the APIs converge to become similar to the HTML5 File/Blob APIs.
5. Debugging
Debugging with Visual Studio is so good that you would not really miss Firebug or Web Developer toolbar. From basic features like support for logging to break points or inspecting the HTML source, the debugger is very powerful. I would recommend a dual monitor setup for for a better experience.
One feature that I found missing was a way to profile (like Speed Tracer) the Javascript app for performance issues. Adding to the wishlist would be a way to remotely debug an application (like Opera dragonfly but that is a feature I would like in IE too). Remote debugging may prove useful specially in case of tablets.
The usual annoyance of printing [object] with console.log still exists. It would be even better if we are able to integrate console.log with Windows application or system logs during a production deployment.
6. Packaging and Deployment
Though Visual Studio has a Store menu, looks like the Store is currently not available. The only option available today would be to create a local app. Packaging the app creates an appx file, a bat file and certificate file. Executing the bat file with Admin privileges install the application.
7. Conclusion
For the Web Developers, the option of using HTML/CSS/JS for writing native windows 8 applications is definitely compelling. Existing web development knowledge can be put to use here, but to leverage the full power of the operating system, it is best to familiarize with the APIs that Windows8 provides.
Overall, a great approach in bringing web development to native applications - hope writing Android or iOS apps were this simple for a Web Developer.
A few posts ago, I had written about the TrialTool app for Windows 8. TrialTool is a web based application that illustrates the capabilities of various Javascript APIs. It is like an API playground that lets you view, modify and run examples and this version of TrialTool specifically illustrates the various Windows 8 APIs.
1. Background
The app was developed on Windows 8 Developer Preview with Visual Studio 11 Express. The entire application was written using HTML5, CSS and Javascripht. The tool does not exhaustively cover all APIs yet, but the APIs used are a good representative of the programming model in Windows 8.
2. Development Environment
My primary IDE is Eclipse or Cloud9 and this was actually the first time I have used Visual Studio exclusively for an end to end project.
The application starts from choosing a pre-defined project type or a simple blank JS template in Visual Studio. The templates do a good job of creating the skeleton and filling in the boiler plate code. As with any skeletons, it may be hard to navigate the multiple files created as a part of the template for a newbie. Once you know your way around the files and the parts to edit, the templates are good. Unfortunately, looks like Visual Studio is the only option for now as I was not able to find any command line utilities that I could use with Eclipse or Notepad++ to build, package or deploy the project.
3. Writing Code
Starting with a blank project is similar to writing a web page from scratch. Using a template however starts us off with the Metro Style UI. TrialTool used the grid layout template that starts with an "index" page that displays a list of items. Each item can be clicked to take us to a "details" page.
The Javascript files created by the template recommend using strict mode.
3.1 Single Page application
The app created is inherently an equivalent of one-page website. Navigation from the "index" page to a "details" page results in the details being loaded as a DOM fragment. Loading and unloading events give us control over the content to be displayed on each action. The WinJS module defines events that let us access the activate or DOM fragment load events for navigation in addition to events for layout changes (portrait to landscape mode for tablets, re-size, etc. )
I noticed that the by default, files of type HTML, CSS and JS had a directory each. I would prefer to group files by modules instead of file type; file types are distinguished by their extension anyway.
3.2 CSS Grids
The use of ms-grid to define the metro UI is great. Grid layout for CSS are still in work in progress. Tiles are an prime to the Metro UI and the implementation of ms-grid makes building tiles in Metro UI is a lot easier.
Vendor prefix would be removed in the final version I guess.
3.3 Data Binding
The Templates also have support for data binding. With the data-win-* attributes in a the HTML tags, render views can be defined. These render views can define specific controls (like ListView) and options for these controls. JSON data can be loaded via AJAX and use these "itemRenderers".
The concepts here are very similar to frameworks like Backbone. However, some features like automatically changing the view when data changes, etc needs to explicitly coded unlike backbone. A way to point a itemRenderer to a URL would be even better.
Another feature that I would like to have is the ability to declare formats in the item renderers. Currently, the data from the model can be converted, but that seems to be manual. A data-win-option on individual elements to specify the format would be good.
3.4 Promises
All asynchronous operations (like AJAX, File reading, etc) seem to return promises. This is definitely better for an interactive event driven application. The "async" keywork in C#
is the equivalent.
The documentation for the promised could have been better. Looks like the documentation is written for C# and force fitted to Javascript. For example, in an operation like FilePicker, the return value is a promise. However, to see the actual arguments passed to the promise.then() method, you have to navigate to the "GetResults" method.
4. Windows 8 APIs
The main reason I would write a Windows 8 app (apart from store distribution) would be to use the Windows 8 APIs; otherwise, the app may as well be a web site viewed in the Metro IE.
Working with native Windows 8 APIs in Javascript is no different manipulating DOM using native DOM methods. Popping up a Windows 8 Toast is no different from invoking the native window.alert. Working with native File APIs, or cryptography is also very easy and straight forward.
The documentation on the Windows Runtime APIs is exhaustive but I would would have been more comfortable if they were grouped better and the first list the over high level categories (like Storage) and then you could drill down to understand a specific API (Bulk Access or File properties). This would help since a lot of APIs have common concepts before we look at specific APIs. This was probably why I wrote TrialTool in the first place.
The other issue I found was an example of APIs that require features not present in the Javascript language. For example, reading the contents of a file returns an InputStream that does not seem to show any Javascript API. It is not very clear that DataReader can in fact be used to read the data.
I wish that the APIs converge to become similar to the HTML5 File/Blob APIs.
5. Debugging
Debugging with Visual Studio is so good that you would not really miss Firebug or Web Developer toolbar. From basic features like support for logging to break points or inspecting the HTML source, the debugger is very powerful. I would recommend a dual monitor setup for for a better experience.
One feature that I found missing was a way to profile (like Speed Tracer) the Javascript app for performance issues. Adding to the wishlist would be a way to remotely debug an application (like Opera dragonfly but that is a feature I would like in IE too). Remote debugging may prove useful specially in case of tablets.
The usual annoyance of printing [object] with console.log still exists. It would be even better if we are able to integrate console.log with Windows application or system logs during a production deployment.
6. Packaging and Deployment
Though Visual Studio has a Store menu, looks like the Store is currently not available. The only option available today would be to create a local app. Packaging the app creates an appx file, a bat file and certificate file. Executing the bat file with Admin privileges install the application.
7. Conclusion
For the Web Developers, the option of using HTML/CSS/JS for writing native windows 8 applications is definitely compelling. Existing web development knowledge can be put to use here, but to leverage the full power of the operating system, it is best to familiarize with the APIs that Windows8 provides.
Overall, a great approach in bringing web development to native applications - hope writing Android or iOS apps were this simple for a Web Developer.
Writing Browser Extensions - Comparing Firefox, Chrome and Opera
MediaPlus is a browser extension that gives users more control over media elements like video, images, games, flash, etc by allowing them to move, resize, download, magnify and perform other actions on them.
Writing an extension for browsers is very similar to writing web pages that work across browsers - similar concepts but interesting quirks along the way; hence this post.
1. Background
MediaPlus started as a browser bookmarklet that injected some javascript into the current web page. This javascript was responsible for identifying various media elements like flash, images, etc and mark them as actionable. When the user needs to interact with a specific media content, they move the mouse over the media content and a panel with various actions shows up. This is the core functionality of MediaPlus.
In addition to the above functionality, MediaPlus also needs to provide interaction points to start, suspend or stop the plugin. All these actions are folded into the bookmarklet button on the browser.
As an browser extension for Firefox, Opera and Chrome, MediaPlus provides these extension points. This post deals with the similarities and differences in the extension system of the three browsers.
The comparison boxes below are color coded to indicate the parts I felt was best/easiest against worst/hardest.
For Firefox, the jetpack way of writing extensions was used. A browser extension for Safari is under development and I am still learning to write extensions for Internet Explorer; these would be possibly covered in future posts.
2. Parts of an extension - Concepts
The basic concepts of building an extension as very similar.
Note that all the three browsers support userscripts, thanks to their architecture.
2.1 Manifest File
All extensions start with a manifest file. This is the file that defines the basic behavior of the extension.
They are called package.json, config.xml and manifest.json in Firefox, Opera and Chrome respectively that define the name, version, description, author, icons, permissions and other meta-data about the extension. Though they are similar, Opera interestingly chooses XML over JSON.
2.2 Additions to Browser Interface
The manifest file also defines the different controls that the extension will add to to the Browser UI. This include things like Context Menu Items, Toolbar buttons and other panels.
2.3 Content Scripts
The core of MediaPlus rests in the script that is injected into the page. This is called a content script or injected script.
In all three browsers, content scripts do not share the run time with the page for security reasons. Extensions usually can do more (like access local file system, cross domain ajax etc) than a script inside a webpage and hence the security boundary.
2.4 Background Processes and Message Passing
A background page is a singleton javascript loaded once in the browser that is responsible for background operations and other house keeping activities. All the three browsers have this concept with different ways to pass messages between content scripts, browser interaction parts and the background script.
3. Coding
Any standard HTML and Javascript editor can be used for creating these extensions.
The issue with Firefox may be attributed to the fact that Jetpack extensions are actually compiled to the XPI that Firefox understands.
4. Debugging
Once the extension is written, the ability to trace through the code and see log messages helps a log in Debugging.
5. Publishing
Once the extension is ready, it needs to be published to a place where users can discover and download the extension. All the three browser have a store like interface where users can discover and download the extensions. In case of all three stores, screenshots, icons and videos have to be provided in specific dimensions as required by the stores.
Auto-updating the extension when a new version is available is also great in all three browsers. All the three also provide support for hosting the extensions at places other their store with support for auto-update URLs.
5. Analytics
Post publication, a mechanism is needed to monitor the traffic and downloads to the extension.
6. Conclusion
To conclude, all the three browsers strive to provide and most powerful yet simple platforms for developing extensions. Though there are not standards (like for web pages), each browser tries their best to infer from existing ways of doing things.
Note that this article derive from my experiences of building MediaPlus and I may not have touched upon all aspects of extension writing.
You can download MediaPlus for your browsers from the following links.
Writing an extension for browsers is very similar to writing web pages that work across browsers - similar concepts but interesting quirks along the way; hence this post.
1. Background
MediaPlus started as a browser bookmarklet that injected some javascript into the current web page. This javascript was responsible for identifying various media elements like flash, images, etc and mark them as actionable. When the user needs to interact with a specific media content, they move the mouse over the media content and a panel with various actions shows up. This is the core functionality of MediaPlus.
In addition to the above functionality, MediaPlus also needs to provide interaction points to start, suspend or stop the plugin. All these actions are folded into the bookmarklet button on the browser.
As an browser extension for Firefox, Opera and Chrome, MediaPlus provides these extension points. This post deals with the similarities and differences in the extension system of the three browsers.
The comparison boxes below are color coded to indicate the parts I felt was best/easiest against worst/hardest.
For Firefox, the jetpack way of writing extensions was used. A browser extension for Safari is under development and I am still learning to write extensions for Internet Explorer; these would be possibly covered in future posts.
2. Parts of an extension - Concepts
The basic concepts of building an extension as very similar.
| Chrome looks to be inspired by Greasemonkey way of building things - just making the thing better. | Opera is heavily influenced by W3C Widgets | Firefox Heavily borrows from commonjs, specifically the modules part. |
2.1 Manifest File
All extensions start with a manifest file. This is the file that defines the basic behavior of the extension.
They are called package.json, config.xml and manifest.json in Firefox, Opera and Chrome respectively that define the name, version, description, author, icons, permissions and other meta-data about the extension. Though they are similar, Opera interestingly chooses XML over JSON.
2.2 Additions to Browser Interface
The manifest file also defines the different controls that the extension will add to to the Browser UI. This include things like Context Menu Items, Toolbar buttons and other panels.
| For chrome, these are usually included statically in the manifest. Interface elements include context menus, URL bar plugins, menu items, etc. | Opera seems to be better as they provide flexibility of adding browser interface elements during run-time. The interface is defined in index.html along with its actions for clicking the button, etc. | Firefox is also similar to Opera where the browser interaction elements are specified in the main module. This allows more actions to be specified by the extension based on the page. |
2.3 Content Scripts
The core of MediaPlus rests in the script that is injected into the page. This is called a content script or injected script.
| Chrome is the simplest as Javascript and CSS can be inserted at any point using an API call. | Opera does not have a direct way to inject script and hence, AJAX requests are made to the extension to read the script content and then inject it dynamically as HTML content.Opera includes all scripts in the includes folder as a part of the page. This however means that injecting scripts dynamically become hard. | Firefox provides page-mod that includes scripts. Injecting scripts into specific tabs with similar URLs using a page-mod is hard that forced MediaPlus to use the same mechanism employed in Opera where scripts were read and written with HTML tags. |
In all three browsers, content scripts do not share the run time with the page for security reasons. Extensions usually can do more (like access local file system, cross domain ajax etc) than a script inside a webpage and hence the security boundary.
2.4 Background Processes and Message Passing
A background page is a singleton javascript loaded once in the browser that is responsible for background operations and other house keeping activities. All the three browsers have this concept with different ways to pass messages between content scripts, browser interaction parts and the background script.
| Chrome again provides the simplest model of message passing using simple APIs. Messages can be relayed between content scripts, background process and browser UI (panels, popups, etc.). Messages are handled using simple message listeners. There are also concepts of long and short lived connections. | Opera strictly follows the W3C message passing mechanism with Channels. Though complex, this is more powerful, providing events "onconnect", to actually start listening to a specific channel. | Firefox seemed to be the most powerful with message passing had both channels and events. The best past of Firefox was the ability to add an event listener ("onMyMsg") to specific messages and just act on it, thus avoiding switch case. |
3. Coding
Any standard HTML and Javascript editor can be used for creating these extensions.
| Chrome has a dedicated developer dashboard that can be used to point to the extension directory. Making changes to code requires hitting a reload button on the "Manage Extensions" to ensure that the latest code is running. | In case of Opera, the config.xml can simply be dropped onto the Opera Window (in developer mode) and the entire directory structure is picked up. Hitting the reload button is require here too. | Setting up the development environment for Firefox is the hardest given that Jetpack extensions are actually compiled to XPI. This also means that any change would mean compiling and reloading the extension - something not native to web developers. |
The issue with Firefox may be attributed to the fact that Jetpack extensions are actually compiled to the XPI that Firefox understands.
4. Debugging
Once the extension is written, the ability to trace through the code and see log messages helps a log in Debugging.
| Chrome here steals the show with its web developer toolbar. With a single interface, all parts of the extension including the background process, content script and browser UI can be debugged. The simple console.log prints all messages in the console. | Injected Scripts in Opera can be debugged with breakpoints using Dragonfly. However, I have noticed that tracing through background index.html or browser UIs seems hard. The opera.postError is required to post messages to the Opera Error console - and it does not live well with JSON. | Possibly a result of its development experience, debugging in Firefox can be hard. However, I ensured that I always pointed to a profile directory that had Venkman and Firebug. This way, I could actually debug all parts of the extension. |
5. Publishing
Once the extension is ready, it needs to be published to a place where users can discover and download the extension. All the three browser have a store like interface where users can discover and download the extensions. In case of all three stores, screenshots, icons and videos have to be provided in specific dimensions as required by the stores.
| The extension simply needs to be zipped and uploaded to the store. The Chrome store seems to be the most mature of them. There is also support for a payment system. You extension needs to get approved before it is actually visible on the store. The store is also one single place for apps, themes, etc. | The OEX file (zip of extension files needs to be uploaded) The Opera Store has a great/quick approval process too. I have not seen the support for payments in the store. | The packaged XPI file is uploaded to the store. Given that Firefox had extensions even before some of the other browsers existed, it has a great eco-system of extensions. Though they are no "payments", donations are supported for developers. |
Auto-updating the extension when a new version is available is also great in all three browsers. All the three also provide support for hosting the extensions at places other their store with support for auto-update URLs.
5. Analytics
Post publication, a mechanism is needed to monitor the traffic and downloads to the extension.
| The store page for Chrome is can be integrated with Google Analytics, providing it with great analytics. Apart from traffic sources, downloads and weekly installs, with user comments are also available. | Analytics in Opera store is pretty basic. Ratings, download count and user comments are the only available features. Download per | Though firefox is not integrated with any powerful analytics, it does provide details like visits and downloads per time. It seems to be an in house version of an analytics system. |
6. Conclusion
To conclude, all the three browsers strive to provide and most powerful yet simple platforms for developing extensions. Though there are not standards (like for web pages), each browser tries their best to infer from existing ways of doing things.
Note that this article derive from my experiences of building MediaPlus and I may not have touched upon all aspects of extension writing.
You can download MediaPlus for your browsers from the following links.
http://nparashuram.com/projects/flashresizer.html
Edit: As people pointed out, the article deals only with building extensions for Firefox using Jetpack. You can build extensions without Jetpack, but that cannot be compared to what Opera and Chrome offer.
TrialTool for Windows 8
TrialTool is a web based application that illustrates the capabilities of various Javascript APIs. It is like an API playground that lets you view, modify and run examples and see the output right inside the browser.
A few days back, Microsoft released the Windows 8 Developer Preview that would give developers a peek into the next version of the operating system and to start developing applications.
HTML, CSS and Javascript was one of the ways to build Windows 8 native applications. The list of APIs in the reference was huge and I wanted a way to not just read but also play around with the APIs to understand and experience them. To this end, I ported the web version of TrialTool to illustrate the various Windows 8 APIs available.
The video below is a screencast of how TrialTool looks like, on Windows 8 Developer Preview.
As show in the video above, you can browse through various sets of examples. You can click on an individual example, view the corresponding code, modify it and run it inline.
TrialTool is also integrated with Windows Search. You can hit "Windows Key" + F to bring up the search panel, type in the name of the API to search and select TrialTool from the list of applications to directly navigate to the API you are interested in.
TrialTool is also a Windows 8 Share Target. While reading MSDN documentation or looking at code sent over email or IMs, you can select the code, bring up the Start menu (usually at bottom left) and click Share. You can select TrialTool from the list of listed applications. The selected code is shown in a "code cache". You can select and add more code to the code cache and see how the snippets work. With this capability, not only can you just read documentation, you can actually see how the API works.
The project and the source code are available on codeplex. If you have an installation of Windows 8, simply download the zip file and run the bat file with Administrator privileges.
As seen in the video above, not all APIs are shown yet; I am working to make more APIs available. I am also working on adding features like syntax coloring and a better search, etc for this version of TrialTool for Windows 8.
If you are interested in helping out, please contact me.
A few days back, Microsoft released the Windows 8 Developer Preview that would give developers a peek into the next version of the operating system and to start developing applications.
HTML, CSS and Javascript was one of the ways to build Windows 8 native applications. The list of APIs in the reference was huge and I wanted a way to not just read but also play around with the APIs to understand and experience them. To this end, I ported the web version of TrialTool to illustrate the various Windows 8 APIs available.
The video below is a screencast of how TrialTool looks like, on Windows 8 Developer Preview.
As show in the video above, you can browse through various sets of examples. You can click on an individual example, view the corresponding code, modify it and run it inline.
TrialTool is also integrated with Windows Search. You can hit "Windows Key" + F to bring up the search panel, type in the name of the API to search and select TrialTool from the list of applications to directly navigate to the API you are interested in.
TrialTool is also a Windows 8 Share Target. While reading MSDN documentation or looking at code sent over email or IMs, you can select the code, bring up the Start menu (usually at bottom left) and click Share. You can select TrialTool from the list of listed applications. The selected code is shown in a "code cache". You can select and add more code to the code cache and see how the snippets work. With this capability, not only can you just read documentation, you can actually see how the API works.
The project and the source code are available on codeplex. If you have an installation of Windows 8, simply download the zip file and run the bat file with Administrator privileges.
As seen in the video above, not all APIs are shown yet; I am working to make more APIs available. I am also working on adding features like syntax coloring and a better search, etc for this version of TrialTool for Windows 8.
If you are interested in helping out, please contact me.
jsFoo - Bangalore
I was at the jsFoo Conference, Bangalore yesterday.
I had two sessions, one on IndexedDB and the other on how Javascript Engines work. Here are the attached slides. The videos of the sessions will be made available shortly.
I had two sessions, one on IndexedDB and the other on how Javascript Engines work. Here are the attached slides. The videos of the sessions will be made available shortly.
Indexed DB
View more presentations from Parashuram Narasimhan
Understanding Javascript Engines
View more presentations from Parashuram Narasimhan
MediaPlus - Firefox addon
The last two posts described MediaPlus as a Chrome and an Opera extension. To extend the browser matrix, MediaPlus is now ported to work as a Mozilla Firefox extension. As described in the previous post, the core functionality of MediaPlus is to interact and change the contents of the web page. The peripheral functionality like stopping the plugin, re-scanning the page, etc are invoked using the browser addon UI.
The MediaPlus add on for Firefox uses the new Jetpack way to write "no-restart" plugins. The concepts are very similar to writing extensions on Opera or Chrome -
- A singleton background page that runs as a background process. For firefox, this is the lib/main.js CommonJS module.
- Pop out icon and the corresponding pop out panel defined in the main.js file.
- Content Scripts, also defined in the background page.
First, there is no direct way to inject script into webpages dynamically. Though the page_mod allows script injections into the web page, the page_mod cannot be changed to add more scripts later. Unfortnately, page_mods also do not share the execution context and hence, cannot communicate in a straightforward way. Hence, scripts are added to the webpage when the pop out are clicked using the tabs.activeTab.attach method by adding a script tag to the page. The source of this script is obtained using the data.url method, relative to the extension.
To communicate with these added scripts, the (ugly) unsafeWindow method is used. Since no "trusted" addon methods are leaked while using the unsafeWindow, the security concerns should be addressed.
The scripts added in this way are inserted into the page immediatly. This turns out to be a problem if the target webpage is still loading. Once the web page finishes loading, all the injected scripts are replaced with the new content. The icon needs to be clicked again to start the add on. Though the tabs.on(ready) method exists, this is only invoked once the page has completed loading. Hence, we would need logic to re-load the scripts again, when the onReady (DOMContentLoaded) event is fired. I could also not find a way to unbind this event and hence, this seems to be fired for every navigation on the web page; another ugly solution. The onready could remember the all the tabs for which it was fired, but that it just too much housekeeping for now. For now, you would have to click on "Start MediaPlus" again, after the page has loaded to activate it.
Though the message passing was not as simple as Chrome, it looked better than the way things are done in Opera, with custom events in addition to the message channel.
The source for Firefox extension is available here and you can download the addon from here. Follow this space for updates on MediaPlus as a bookmarklet or as an extension for browsers. If you have suggestions or have found bugs, you can contact me here for help.
I have also made the statistics for the addon public and you could see the number of users, downloads, etc here.
DocType HTML5 - Chennai Edition - CSS3 and Presentations
Link - http://blip.tv/doctype-html5/css3-and-presentation-5502619
MediaPlus - Opera Extension
The previous post discussed MediaPlus as a Google Chrome Extension. MediaPlus lets users mpdify and take control of media content on web pages. It lets uses move, resize and change media content on the web page; all these tasks are about changing content on a web page. The bookmarklet version of MediaPlus inserts a javascript file in a webpage and this script is responsible for displaying the charms on images or flash videos when the mouse is moved over them.
In addition to this core functionality, MediaPlus also has additional actions like stopping it, rescanning the page for dymanically adding content or simply turning off full screen. In case of the bookmarklet, these actions are initiated by clicking the bookmarklet again. In the case of an extension, these options can be available off an icon in the browser UI, outside the web page.
The Google Chrome extension does exactly this - the core functionality is handled by a content injected into the web page while the peripheral functions are displayed in a pop out panel as a part of the browser UI. Writing an extension using Opera is very similar to writing a Chrome extension. In short, the extenion has three basic parts
- A singleton background page with scripts, included in index.html and the background.js referred in the index file.
- Pop out button and corresponding popout.html, defined in the index.html
- Content script in the includes directory that are injected into all web pages.
There are also two differences and here is how they are solved for the Opera extension. First, passing messages between the pop out and the content script is not simple and requires a background script to broker the connection the first time. This article explains how it is achieved.
The second issue is that external stylesheets scripts are not injected into the page using API calls, but by placing the script inside the includes folder. This unfortunately means that it is harder to inject dynamic scripts in a page. The first version injects scripts by referring to external URL, but this practice is frowned upon. The latest version does the following. The scripts can be injected by reading them using XHR using paths relative to the extension root and inserting them as textContent of script tags in the web page. Since the content scripts seem to have access to the global window object of the web page, they can communicate with the injected scripts. Note that Google Chrome separates the execution context of the web page and the content script.
To load images dynamically, they are added as img elements in the background page, drawn on a canvas, and the base64 data:url returned by the canvas.toDataURL is passed back to the content script. The content script sets the data:uri as the source of the image to be drawn. Background images referred using external stylesheets are still not loaded. External stylesheets (like jquery UI css) have to be changed to have data:uri before building the runtime.
The Opera extension is available for download here and source code is available here. To follow MediaPlus, check out this page.
MediaPlus as a Chrome Extension
I had written about MediaPlus, a bookmarklet that gives you more control over media contents like Flash videos, images, games, HTML5 movies, etc. in my previous posts. Though the bookmarklet was powerful, it has some shortcomings inherent to bookmarklets. Many people hide the bookmarks toolbar on their browser to get more browsing space. Additionally, though the concept of adding a bookmark is known, bookmarklets with their javascript: protocol and adding them to the toolbar is still not common. Finally, people also voiced a concern that clicking on the bookmarklet icon to perform multiple actions like toggling it, rescanning the page, etc was not intuitive.
MediaPlus only manipulates an the DOM of an existing webpage as a result of certain user actions. The MediaPlus chrome extension provides a better user interface for triggering these user actions. It adds an icon next to the URL bar that needs to be clicked to interact with the extension. The rest of the post is about the technical details of the implementation.
Extensions for chrome can inject content scripts in a running page that can interact and change the DOM. MediaPlus chooses not to inject these scripts for all pages, but does so only when the user first clicks the "Start MediaPlus" button. The scripts are loaded from the extension in a background page. Since the runtime provided to this script is isloated from the javascript environment of the page, scripts and the page can have their own versions of libraries like JQuery.
Once the script is loaded, all media elements (flash, canvas, image, video, iframe) have a class added to them. The mouseover event handler that is added as a part of injecting the script kicks in to display the possible actions on the media element. This is the exact same code path as for the bookmarklet.
The only deviation here is the way notifications are shown. Since chrome provides webkit notifications, these are used instead of manually using divs for the bookmarklets.
When the user makes a video full screen or darkens the background, they may want to undo the action. In case of the bookmarklet, the bookmarklet icon in the toolbar needs to be clicked. In the case of the extension, clicking the icon brings up a menu that lets the user restore the media element to the previous state.
Except for these changes, the code path is just like the bookmarklet. Infact, the bookmarklet and the extension share the same core codebase, as illustrated in the source repository.The only additional work is to pass message between the popup on clicking the icon and the content script injected in the page.
The extension is available on the Chrome Web Store and do give me feedback on how you find it. The support page is here, and you can report bugs here.
Watch out this page for more updates.
MediaPlus - Video and Image Enhancements
MediaPlus now supports the ability to edit/enhance HTML5 Videos and Images.
To install MediaPlus with these enhanced functions, drag and drop the following bookmarklet to your favorites/bookmarks bar.
The previous post explains how to install MediaPlus as a bookmarklet and how it can be used to move, resize or download media content on web sites. This post describes the additional functionality added to MediaPlus to perform enhancements like brightness, contrast, color adjustment, etc on images and videos.
Architecture
MediaPlus uses Pixastic for manipulating the media content. An image or a canvas can directly be passed to the Pixastic processor. Pixastic then reads the image data from the image or the canvas and manipulates it. It draws the final image on the target canvas.
In case of videos, MediaPlus passes a canvas to Pixastic instead of the actual video. MediaPlus continuously draws the latest frame of the video onto this canvas that is picked up by Pixastic for processing.
Limitations
Since pixastic relies on getImageData, it is restricted by the same domain policy. The Canvas Security policies mandate that pixel data cannot be read from a canvas if content from a different domain have been drawn on the canvas. Unfortunately, a lot of popular websites host their video content on CDN servers that has domain, different from the page. This results in the inability to manipulate HTML5 videos from youtube or facebook.
Most browsers also do not support the CORS way to reading image data yet.
Workaround for Images
The same origin restriction also apply for images. However, Images can be converted to data:uris using a web service and drawn on the canvas. Such images can then be manipulates as they are not from a different domain. However, the data:uri web service can only manipulate images that are directly addressed using a URL.
Image Zoom
Another addition specific to images is the ability to magnify images. The image zoom works just like most image zoom plugins, drawing a larger image inside the magnifying glass and moving it around with the magnifying glass is dragged over the image.
Next Steps
I am working on the ability to allow rotations and CSS3 3D transforms on videos and images. This is a little trickier as I would have to ensure that other functionlity like resizing or dragging continue to working on media content that is rotates in the z axis. Watch out the project page or this blog space for updates.
If you encounter bugs when using MediaPlus, you can report them or contact me.
To install MediaPlus with these enhanced functions, drag and drop the following bookmarklet to your favorites/bookmarks bar.
The previous post explains how to install MediaPlus as a bookmarklet and how it can be used to move, resize or download media content on web sites. This post describes the additional functionality added to MediaPlus to perform enhancements like brightness, contrast, color adjustment, etc on images and videos.
Architecture
MediaPlus uses Pixastic for manipulating the media content. An image or a canvas can directly be passed to the Pixastic processor. Pixastic then reads the image data from the image or the canvas and manipulates it. It draws the final image on the target canvas.
In case of videos, MediaPlus passes a canvas to Pixastic instead of the actual video. MediaPlus continuously draws the latest frame of the video onto this canvas that is picked up by Pixastic for processing.
Limitations
Since pixastic relies on getImageData, it is restricted by the same domain policy. The Canvas Security policies mandate that pixel data cannot be read from a canvas if content from a different domain have been drawn on the canvas. Unfortunately, a lot of popular websites host their video content on CDN servers that has domain, different from the page. This results in the inability to manipulate HTML5 videos from youtube or facebook.
Most browsers also do not support the CORS way to reading image data yet.
Workaround for Images
The same origin restriction also apply for images. However, Images can be converted to data:uris using a web service and drawn on the canvas. Such images can then be manipulates as they are not from a different domain. However, the data:uri web service can only manipulate images that are directly addressed using a URL.
Image Zoom
Another addition specific to images is the ability to magnify images. The image zoom works just like most image zoom plugins, drawing a larger image inside the magnifying glass and moving it around with the magnifying glass is dragged over the image.
Next Steps
I am working on the ability to allow rotations and CSS3 3D transforms on videos and images. This is a little trickier as I would have to ensure that other functionlity like resizing or dragging continue to working on media content that is rotates in the z axis. Watch out the project page or this blog space for updates.
If you encounter bugs when using MediaPlus, you can report them or contact me.
MediaPlus - Technical Details
In my previous post, I had written about a MediaPlus - a browser bookmarklet. MediaPlus lets you control media content on websites better. This post talks about some of the interesting technical challenges that were encountered when rewriting it from FlashPlus.
The Basics
MediaPlus can be installed by dragging the MediaPlus link to the browser bookmarks/favorites bar. When this bookmark is clicked while browsing a web page, a javascript file is inserted into that web page. This script is responsible for all the magic from drawing borders around elements to making videos full screen. Once the bookmarklet is loaded, it loads additional javascript and css modules dynamically.
Element Selectors
When the mouse is moved over supported (flash, video, canvas, iframes, silverlight) target elements on a web page, the border that lights up around the element with the toolbox at the bottom is the element selector. These element selectors are not one single div (with a border and the height and width of the target element), but four [divs] around the corners of the element (very similar to the Aardvark bookmarklet). Though a single element would do the job just as well, placing it below or above would interfere with mouse events of the target element. Additional jquery "mouseenter" events are directly added to these target elements to show up the selector. The selector element is redrawn around the target element every time the mouse enters a different target element.
Proxy Substitutes
The structure explained above seems to work for all elements except embedded flash, as some browsers draw flash above all elements to prevent clickjacking. Object and Embed tags also cannot hold data. Just for the case of Flash, a proxy <div> is placed on top of the flash. To allow the Flash element to capture its mouse events as usual, the proxy div is "sent to back" as soon as the mouse enters it, bringing the Flash Element to the top.
Moving and Resizing Elements
When an element is resized or moved from its position, three things happen .
Next Action
Some actions like full screen or switch off lights hide all other elements. To restore the page to the state prior to the action, the bookmarklet link has to be clicked again. A button on the page could have done this job, but placing one may look ugly, especially when a video is full screen. Hence, some commands like full screen or switch-off-lights define a "next-action" that is executed when the bookmarklet is clicked again.
Support for tags
This newer version has a better support for tags, and the actions that can be performed on tags. Every tag is defined by a separate object that in turn declares the commands or actions (like move, full-screen, download) that can be performed on the object. Common actions are added using a mixin like pattern.
To look at how MediaPlus works under the cover, you can check out the source code here.
The Basics
MediaPlus can be installed by dragging the MediaPlus link to the browser bookmarks/favorites bar. When this bookmark is clicked while browsing a web page, a javascript file is inserted into that web page. This script is responsible for all the magic from drawing borders around elements to making videos full screen. Once the bookmarklet is loaded, it loads additional javascript and css modules dynamically.
Element Selectors
When the mouse is moved over supported (flash, video, canvas, iframes, silverlight) target elements on a web page, the border that lights up around the element with the toolbox at the bottom is the element selector. These element selectors are not one single div (with a border and the height and width of the target element), but four [divs] around the corners of the element (very similar to the Aardvark bookmarklet). Though a single element would do the job just as well, placing it below or above would interfere with mouse events of the target element. Additional jquery "mouseenter" events are directly added to these target elements to show up the selector. The selector element is redrawn around the target element every time the mouse enters a different target element.
Proxy Substitutes
The structure explained above seems to work for all elements except embedded flash, as some browsers draw flash above all elements to prevent clickjacking. Object and Embed tags also cannot hold data. Just for the case of Flash, a proxy <div> is placed on top of the flash. To allow the Flash element to capture its mouse events as usual, the proxy div is "sent to back" as soon as the mouse enters it, bringing the Flash Element to the top.
Moving and Resizing Elements
When an element is resized or moved from its position, three things happen .
- A new proxy element <img> with the dimensions of the original target element is inserted in its place to ensure that the parent of the target element does not collapse when the target element is moved out of its child hierarchy.
- The element is given an absolute position to allow the drag.
- The element is also re-attached directly under the body to ensure that overflow:hidden in the parent element does not prevent the target element from being moved around.
Next Action
Some actions like full screen or switch off lights hide all other elements. To restore the page to the state prior to the action, the bookmarklet link has to be clicked again. A button on the page could have done this job, but placing one may look ugly, especially when a video is full screen. Hence, some commands like full screen or switch-off-lights define a "next-action" that is executed when the bookmarklet is clicked again.
Support for tags
This newer version has a better support for tags, and the actions that can be performed on tags. Every tag is defined by a separate object that in turn declares the commands or actions (like move, full-screen, download) that can be performed on the object. Common actions are added using a mixin like pattern.
To look at how MediaPlus works under the cover, you can check out the source code here.
MediaPlus 2.0
What is MediaPlus?
MediaPlus is a plugin for your web browser that makes media content on web site easier to use. It gives you control to move or resize videos, play flash games in full screen or popout content to a new window.
Media Content, huh ?
Media Content here refers to Flash videos and games, HTML5 Videos, HTML5 Canvas and iFrames. MediaPlus can manipulate the media content when they are inside almost any webpage.
What can MediaPlus do?
- Move and re-size media content like videos or games on web pages
- Make media content, maximized inside your browser. Play games in full screen.
- Pop out media content into a new browser window
- Switch off lights on the page to view media content better on the page.
- Download Media content from specific websites.
- Enhance HTML5 videos by changing contrast, brightness, etc.
- Delete annoying media like advertisements, etc.
Interesting, now how do I install it ?
Since this is a bookmarklet, no installation is required.
We are working on making this available as extensions for Firefox, Chrome and Opera.
How do I use it?
When you are browsing a page, you could click on the MediaPlus bookmarklet link and it would get activated. You can now move your mouse over media content on the page and you should be able to see a blue highlight around the content with a toolbox below it. Place your mouse over the icons inside the toolbox to see what they do, and click on them to manipulate that media content.
At any time, you can click on the MediaPlus bookmarklet for additional options. For example, when your video is full screen, or the page is dark, click on the bookmarklet to restore the content to its original size or switch the lights back on.
And how does it work?
MediaPlus is a bookmarklet. When you click the MediaPlus bookmarket in your favourites or bookmarks bar, a javascript file is injected in the page you are browser. It is this script that does all the magic. It does not transfer any content from your webpage to anywhere, so it is also safe to use.
What are the supported browsers?
MediaPlus is tested on Firefox 3.5+, IE8+, Chrome 9+ and Opera 10+.
Was this not called FlashPlus earlier?
MediaPlus is like the next version on FlashPlus. FlashPlus only allowed manipulation on Flash content on web pages. MediaPlus is a complete rewrite of FlashPlus and is more efficient and has more functionality. See this link for more technical details.
More details?
You can follow this link for more details. The blog will be update with technical details and other major releases.
At Microsoft WebCamp, Hyderabad
Attending the Microsoft Hyderabad WebCamps.
I will be speaking on Sizzlers on the Web: Web Sockets & Indexed DB. The presentation is available on Skydrive here - https://cid-88695823769e7fbc.office.live.com/fullscreen?resid=88695823769E7FBC!139&filename=Sizzlers%20on%20the%20Web%20-%20WebSockets%20and%20IndexedDB.pptx&wx=p&wv=s&wc=officeapps.live.com&wy=y