SeamCarving with Canvas and Web Workers


I had written about Seam Carving in Javascript in my earlier posts. The example that I had till now only demonstrated removing the pre-calculated seams and ran in the browser's foreground thread. This post is about SeamCarving done using Web Workers and Canvas.
A couple of week ago, I stumbled upon CanMan.js on reddit and thought that I could add seam carving as a filter to it. The filters leverage Web workers and it looked like a framework to implement image manipulation filters. Unfortunately, the API signature required for the filters did not allow manipulation based on adjacent pixel in an image. Hence, it was not suitable for Seam Carving.
Having started, I decided to enhance the existing example to use Web Workers. First, I added the missing logic to calculate seams for any image. Since this was an operation what would take a long time, I had to move this processing to web workers. Some interesting observations along the way include

Passing Image Data
The SeamCarver operation takes the image pixels as input and returns image pixels after the desired operation. The canvas' getImageData returns the type ImageData which cannot be passed around in the worker context. Hence I had to deconstruct and reconstruct this every time the processing was complete. This meant that I had to copy all the pixels from an array to the array-like CanvasPixelArray in the ImageData. Interestingly, I could simple reassign the array and Firefox/Opera seems to be fine with it. Chrome however did not accept the simple array assignment, and I had to iterate on the pixels, copying them into the CanvasPixelArray.
Another interesting observation here was that Chrome, with its tight loop tracing performed a lot better than Firefox. I am assuming that Firefox 4 would get the same speed benefits.

Debugging Workers
The hardest part during development was debugging. With so much pixel manipulation, I needed a way to break on statements. Unfortunately, neither Firebug, nor Venkman broke on web worker threads. Chrome developer tools were also not much of a use either. I am hoping that the generic script tools include worker scripts in the script tags.

UI Thread responsiveness
The idea of web workers was to demonstrate UI responsiveness. I thought of using loader gifs, rewriting the DOM with javascript, etc, but finally zeroed in on using a scrolling marquee.

Next Steps
The seams are carved vertically only, so resizing it on the y axis is still not possible.
Some of the things that I am looking at doing in the future include
  • Support y-axis resizing 
  • Port carving to server (using node, etc) and the browser re-sizes the image. 
  • Try to see if I can use as an example for getting the right API signature in frameworks like CanMan. 
You can checkout the demo at http://www.nparashuram.com/seamcarving and look at the source code at https://github.com/axemclion/seamcarving. You can follow this space for updates on my experiments with seam carving.

Porting my web site to other form factors

With the start of the holiday season, it is a time for family reunions. With relatives come their mobile phones, and with that, I get a chance to make my website look good on the extra form factor :)
I finally got my website working on the Samsung Galaxy Android phone and this post details some of the changes that went in.
It all started with media queries that activates the mobile specific stylesheet. The stylesheet is activated as long as the max-width is 800px.

Percentage widths
Since the number of mobile browsers and their viewport sizes vary, I decided to set the widths of elements in percentages. Any margins, padding or font sizes greater than 0.5 em were also in percentages instead as they were elements that I could allow scaling for. Thinner padding or borders are defined in pixels as scaling them may not make a big difference .


Hiding fancy elements
The website on the mobile phones had a lot of content that made the site look cluttered. I hid most of the elements to de-clutter the site. Given that the site has no backend except for templates, I could not use server to send lesser content. The difference in size is minimal in the case of my site and hence, I can live with it.


Undoing rollovers
The projects and articles page had some rollover effects and the absence of mouseover in case of mobile browsers make the site unusable. These effects were negated using the extra CSS.

Floats to vertical menus
In mobile browsers, horizonal space is expensive and hence, floats only make the situation worse. In the mobile stylesheet, all the floats were removed and clear:both added. This ensured that all side-by-side items were placed vertically. 

Some other recommended things apart from stylesheets were to use the right form elements, optimizing bandwidth requirements by shrinking resources, etc. Since the site does not have logic at the back end and the difference is not noticeable, I took the liberty to skip these optimizations.

You can take a look at the website at http://nparashuram.com. Check it out on your browser and do let me know how it looks.

Some good references on this topic include this and this

Time2Blog

A lot of people have been using TwitteyBot to schedule tweets easily by text uploading files.The application has turned out to be a useful tool to use twitter as a social marketing platform.

A few days ago,I got a feature request to morph the application to publish to blogger instead of twitter. Keeping the basic infrastructure the same, I could modify the Task Servlet to send an email to the secret blogger id to post using Mail2Blogger. The associated data objects were also changed. Though some data object like the blog account was not really required, I left it there as a change would make bringing changes from twitter bot in the future, harder.
The final change was for the twitter OAuth page. Instead of the OAuth page, the user is now redirected to a simple page where the blogger email id is entered.
The email sent to blogger was via the Google App Engine Mail API as text/html which allows images in the blog posts.
The code is available at http://code.google.com/p/csv2blogger/. If you would like an instance of the application to automate blog posts just like you schedule tweets, drop me a line and I could help you with a dedicated instance of the application.

My Website - Some Gotchas

My last post was about the things that I had done to get my website, http://nparashuram.com,  running. I thought that I should dedicate a post to write about the interesting problems that I encountered while building it, so here it is.

Anchors as block level elements

On firefox, the articles page looks bad at times. I noticed that the divs inside the anchor tags break out and are displayed as siblings to the anchor tag. Once the page is refreshed, the layout falls in place. Hence, I have removed all the blocks on that page as anchors and made them divs, losing the "links" in the process.

Order of audio tag source
I recorded the audio using Windows sound recorder and converted it to OGG and MP3 using Super and VLC. I noticed that the VLC-OGG did not play on Firefox, but worked well on Opera and Chrome. Though Chrome plays Oggs, Super-OGG did not work on Chrome. Hence, I added the MP3 source before the OGG for chrome to pick up the former. Firefox and Opera ignore the MP3 and play the OGG. For IE, I just opened the mp3 file in a new window that would trigger the default action for such files.

Box Shadows
Box and text shadows combined with opacity can lead to some cool effects, like the garage door on the projects page. However, I noticed that the though the animation is very smooth on Chrome, it is jittery on Firefox. For IE9, it is even worse, leaving lines as it animates. I had to add an IE specific CSS to remove box shadows for IE9.

Microdata and SEO
Though there have been promises that Microdata will show up as special snippets in search engines, I have not seen it happen. Sticking to good old title and opening lines looks like the way to go for now.

My Website - The new HTML5 look

I finally stole some time to give my website hosted at http://nparashuram.com a makeover. It may look marginally better, but the focus was more on technology this time. Some of the changes in the website include
  • Semantic Tags: HTML5 semantic tags for most parts of the web page. There are headers, footers, sections and articles in the page now. Modernizr makes it work of IE too. The modernizr script is loaded using IE conditional comments to ensure that the newer browsers do no suffer the extra download. 
  • Microdata: I added hcard and itemprops to the appropriate sections. However, I am yet to see search engines utilize this data.
  • SEO friendly: The earlier version of the site loaded content using Javascript. Though AJAX requests can be made crawlable, it it too much work given that the site has almost zero server side support. Hence, all pages are statically server and indexable.
  • Domain Name: I finally configured the GitHub to use the CNAME file and redirect http://axemclion.github.com to http://nparashuram.com. There are problems due to this in shortened URLs of TrialTool pages and looks like there is no way around it.
  • Jekyll templates: When I said that I had zero server support, I lied. I did use Github's builtin Jekyll support to add a basic skeleton (header, content, footer, etc) to all the pages. I tried creating dynamic pages with HTML templates, but more on that approach later.
  • CSS Usage: CSS transitions, gradients, box shadows, text shadows using vendor pre-fixes when generously used. For IE, the filter property equivalents were used. I ensured that the page degraded gracefully in browsers that did not support these. I still need to nail the mobile browsers and plan to use media queries for that.
  • HTML5 Video: What is a HTML5 if the videos are in flash. A lot of my projects had videos and I added iFrames from Youtube that would render HTML5 video or flash, depending on the browser capabilities.
  • HTML5 Audio: The about page had a link next to my name that runs an audio clip on pronouncing my really long name, done with the audio tag. The mp3 file for Chrome/Webkit and ogg for Firefox and Opera run well.
  • Performance: I also wanted to ensure the YSlow and PageSpeed give me decent results. All the javascript were at the bottom, but embedded in the page to remove that extra http request. Image tags have an src_defer instead of src for deferred loading. Some issues like CDN or cache could not be as I do not have control on the headers that are served.
With all these changes, the website was done. Please let me know if there are "technology" things that I could look at for the site.
I will also write a follow up post on the gotchas encountered during the entire process so follow this space.
And if you do not have a website, it may be time to make one - its easier than you think :)

Dial to find the price of a book

Dial  +91-40-44556944(pin 8586), enter ISBN and know the cost of a book on flipkart.

Scenario
You are in a book store, looking at an interesting it. Before buying it, you would like to peek at its price online and see if you can order it online for a better deal. You have a mobile phone, but no internet on it.

Use Case
Dial a number (+91-40-44556944 and pin 8586), enter the ISBN number of the book that you are holding and instantly get the price of the book.

APIs used
This is the scenario and the solution that I hacked together, during the weekend. Hacker News carried a story about kookoo.in some days ago. The phone API looked interesting and I decided to give it a try.
KooKoo.in has Java APIs that can be used to create web services that feed into the phone API they provide. I fired up an instance of Google App Engine at http://flipring.appspot.com/price.

Technical WorkFlow
  1. When the user dials the number and the PIN, the web service is called the first time and it plays a message telling the user to enter the ISBN number. A session variable stores the state.
  2. When the user enters the ISBN number, the web service is called again, something like http://flipring.appspot.com/price?data=0333933427. This time, the value in the session variable indicates that the user has entered the ISBN number and its price is required. 
  3. A crawler kicks in an gets the search site from flipkart.com with the ISBN. The product page is obtained and a naive search is done to extract the price, discount, book name and author. 
  4. This data is played back and the session variable is set to initial state again. 
The code for this servlet is available here. The entire application was put together in under an hour and the crawler code is brittle.

Next Steps
As the next steps, I am looking at
  • Improve the crawler, get prices from more sites to return the cheapest price
  • Host the code as a project on github or google code.
This is my first DTMF application and please do write to me if you have feedback. Watch out this space for updates.

Sounds of some sorts


Earlier this week, I came across this youtube video that illustrated various sorting algorithms with sound effects. Unfortunately, it was a video and not a live demo.
I picked up the baton and decided to play around with HTML5 audio and see if I could get a live demo out. Unfortunately the Audio API can only playback audio files, there is no support for generating sounds yet. I decided to pick up some audio files from the Piano and Drum beat examples. I was able to find 12 unique sound files, and associate them to each value in the sample input.
The tricky part was to keep the code for the algorithms while allowing the animations and sound effects have time delays. I wrote a couple of basic functions like greater() and swap() that were called for comparisons and swapping values in the algorithm. They also added the required animation/sound effect to a queue that played asynchronously till the queue was empty.
There you have it; the sounds of some sorting algorithms. The web page is available here.
I was also exploring the additions to the audio API that were proposed and are a part of Firefox 4. It has interesting way to not only playback audio files but also generate audio. Next step - try to see if I can have larger samples and generate sounds for them !!
Watch out this space for updates.

TwitterReplyPipe - Technical Details

Previously, I had posted about TwitterReplyPipe, a Yahoo Pipe that scanned the public twitter time line with a search criteria that you specify and reply with a message from your set of replies. This post is about some of the interesting technical problems encountered while building the application.
The pipe itself is a simple, starting off by scanning the public twitter time line using the search feed. The parameters to this URL are as specified in the twitter advanced search page. This feed is then pruned for tweets by unique author and author's URL is replaced with @usernameand the reply that the user specified is appended to this from another pipe.
This is set as the title of the feed. The GUID of every item is the author's name to prevent more than one reply to a user. This is then passed on to twitterfeed and sent as replies.
Two interesting issues deserve a mention here.

1. Combining 2 feeds into one.
The Yahoo Pipes union operator only appends one feed to the other; I could not find a way to combine the properties of an item of one feed into the items of another. That was the reason a second pipe was used to give out a random reply from a file with list of replies. A loop in the first pipe uses the second pipe as the operator to combine the feeds.

2. Generating random numbers in YAHOO pipes
I could have picked up random numbers after parsing the output from random.org, but that seems too tedious for this. I simply used the date operator and obtained the utime of the date as a random number. Interestingly, I was not able to get the %S operator working on V2 of pipes and had to resort to multiplying the "seconds" in the time with a very large number and then mod it to the total number of lines to get a random line. This however is not random when the pipe gets called within a second.

You can  read about the steps to use this pipe here and please let me know if you have any questions.

Auto Reply to tweets using Yahoo Pipes - TwitterReplyPipe

TwitterReplyPipe is a simple mash-up that can search the public twitter time line based on a search criteria you define and reply to the tweets with a message from a set of messages that you define. Some use cases include
  • Look for mentions of your product and send replies telling users to look at products that you can cross sell.
  • Search for the category of your product and tell users to look at what you have to offer.
  • Search for tweets mentioning your competition and redirect them to a comparison page.
To set it up,
  1. Visit http://pipes.yahoo.com/axemclion/twitter_replier and enter the three inputs as explained below. 
    •  Query Key is the key as specified in the twitter advanced search page. To search for a phrase, use "phrase" and to search to a list of terms, try "q" as the input.
    • The Tweet file is a simple text file hosted on the internet with a set of messages that will be sent at random. You can host your files at http://pastebin.org using the steps below
      1. Visit http://pastebin.org and type in all the messages that you would like to send, one in each line. 
      2. Enter your name in, select that you would like to save the file forever and click "send"
      3. In the next page, copy the "download" link. It will look something like http://pastebin.org/pastebin.php?dl=
    • should be the phrase or the terms that you would like to search for.
  2. Click on"Run Pipe" button and you can see a preview of what the tweets will look like. Copy the "Get RSS" link which you will need for setting up the twitter feed account.
  3. Visit twitterfeed.com and create a new feed. In the RSS feed URL, paste the Pipes URL that you just copied. 
  4. Expand "Advanced Settings" and ensure that the "Post Content" is title only, "Post Sorting" is based on GUID and "feed is sorted" is unchecked. Also, uncheck the "Post Link"
  5. Continue to next step and associate your twitter account and complete the process
Once twitterfeed is also setup, replies would automatically be sent.
This is a followup to a post that I had written about a way to use YAHOO pipes to automatically reply to tweets that mentioned certain keywords. The initial pipe was rudimentary in a way that it could only tweet one message and could not use the extensive search capabilities of twitter.
If you have questions, please write to me and I would he happy to help.  You can follow this space to check out updates to this pipe.

More APIs illustrated with TrialTool

Some of the previous posts talk about TrialTool and using it to illustrate various client side APIs. An older post also featured the procedure to create your own example. All the examples are now hosted in a github project. At present, there are three APIs that are featured.

Jquery Templates
This project is a Jquery Templating engine and is hosted here. As Rey Bango put it in his blog,
the cool thing about templates is that it lets you easily structure your content display without all the hassle of string concatenation.The first set of examples used this this are from  Rey's blog. The second set of examples uses the code listed on the github gh-pages of the project.

Firefox 4
Babu Srithar put together a set of examples showing some of the newer developer features that were released in Firefox 4 Beta. I could import only the examples for IndexedDB. This is exposed using the window.moz_indexedDB in Firefox. You can find more details about this on his blog post. This is interesting given that these APIs not documented in detailed and this would help developers get a feel of thenew APIs. 

YUI Examples 
This was a set of examples that I had written to show case YUI examples during the recent YAHOO India hack day. You can read more about this in the blog post. The examples were scraped from the index of examples from the YUI developer network and added to the project.



I am also talking to a couple of other people who said would be interested in using TrialTool to publish their examples. If you have client side API that you would like to have examples for, please do contact me, and I would be glad to help you with a version.

Using TrialTool to create your own examples

I had previously posted about TrialTool, an API viewer to showcase your Javascript APIs. This post describes the way to use the tool to showcase your own applications.
TrialTool has support to "Fork" existing examples. Reload the page with the &fork=true appended to the end of the URL and a "Fork" link would appear on the left, near the examples. Clicking on the link would start the forking process.
Moving the mouse over the example shows up a couple of icons that lets you move, rename or edit the names of the example categories or examples. The names of the categories also have additional icons to add more examples.
The following video shows the various ways to change the examples and save it.


Once the editing is complete,
  1. Download the TrialTool package to your local machine



    • Get the zip or the tarball, and extract it
    • Alternatively, fork the source from github
  2. Copy the example file that you downloaded as [TrialTool's location]/examples/default.html
  3. Use your web server to serve this folder as a web resource 
You can also take a look at the example files. It shows some more complex cases where dependencies between examples and linking to external libraries are defined.

TrialTool currently has two examples
I would be adding more examples, so watch out this space for updates.

    TrialTool for YUI Examples

    In my last post, I had mentioned about YAHOO Hack day, and the hack that won us the XBox and iPods. This post is about another hack that we had submitted.
    Some believe that the best way to start writing code is looking at examples. The YQL Console is a great example that show the various YQL queries and lets users manipulate and execute them in the console to see results.
    TrialTool is a console for Javascript APIs and I thought it would be great if we could adapt the YUI examples be shown in a console similar to YQL. There are a lot of examples and writing them in the Trial Tool Example file format would be hard. Hence, we used YQL and Yahoo pipes to automatically parse individual examples and change them to a format that TrialTool could use. If you look at the source code of individual examples, all examples start and end with comments. Yahoo pipes would visit the individual web sites, and pick up the content inside the comments.
    The content inside the comment is then split into fourparts
    • Title of the page which is all the content enclosed inside the H1 tag. This is assigned to [a.class="example-name"]'s innerHTML.
    • The content of the Script tag
    • Everything, except script tags inside [div.id="example-canvas"] , inside a [div.class="example-html", to indicate the corresponding HTML tags required for the example.
    • Everything else inside the [div.class="example-docs"] tag, for documentation.
    The four are enclosed inside an [li.class="example"] and given out. Thus, we have all the examples listed in sequential order. I then manually categorized them by forking functionality. Using the "Fork Mode", examples that were incorrectly parsed were removed. These were examples that were
    • were dependent on server
    • required to be opened in a new window
    • did not confirm to the parsing format.
    Looks like I will have to manually add the examples in. I seem to have most of the examples in and I hope people trying out YUI3 find these examples in TrialTool, a good starting point.

    Hacking for the Hack Day

    The YAHOO India Open Hack day just got over and was held in Bangalore on the 24th and 25th of July, 2010. The list of hacks submitted this time was impressive; totaled to around 140 hacks.
    Though I was not in Bangalore, this was an event I could not miss. I teamed up with Srithar, Sudeep, Santosh and Surya to work on some really interesting hacks.
    We came up with three hacks, and one of them won the best in show award.
    We worked on making the YUI examples work in Trialtool. You can see the result at http://nparashuram.com/trialtool/index.html#example=/ttd/YUI/default.html.
    The hack was called FlickrSubz which added closed captioning to videos on Flickr in real time.
    The audio output is captured and routed via Microsoft Speech API and Julius to get the captions for the audio. This was then placed as an overlay on the Flickr video page using a Greasemonkey script. We also wanted to create a Google chrome extension, but that was when we ran out of the
    24 hour time limit.

    Here is a video by Srithar explaining the hack.


    You can check out the hi-def version of the video on its youtube page.

    On this hack, we won
    • Chris Heilmann's complement:)
    • Certificate signed by David Filo !!!
    • XBOX 360 Elite for the team 
    • 3 IPod nanos 4G 8gb for each team member !! 

    Twitteybot - CSV Upload Bug Fixes

    There were a lot of customers using TwitteyBot, but only a couple are using the CSV upload facility to schedule the tweets. Both of them were are in the same timezone as I am in, and everything seemed to be in order. However, a couple of weeks ago, a user notified my of an error with the dates in CSV.
    There were two errors. The first one was regarding the time format. I had erroneously set the time format of the Java Time formatter to the 12 hr format. Hence, 12 noon was interpreted as 00:00 hrs. All it required was changing the format from hh:mm to HH:MM, as in line 57 of this diff.
    The second error was about using the timezones of the browser, when uploading the file. The application was picking up the timezone of the user from the browser and applying it before saving the dates. I was correcting the time, but the date was not corrected. Hence, for anyone to the west of GMT, the time would be corrected, but the date be for the previous day. This is also corrected in lines 282-289.
    With these errors corrected, all I have also contacted who have their own instances of the application. If you have cloned the application and are hosting your own version, please do update the application to have this bug fixed, as per this commit.

    TrialTool - Example file format

    This is a continuation on the posts about TrialTool and this post talks about the format that the example files should be, so that they can be consumed by TrialTool. Examples can be loaded by
    The example file is a simple HTML file that is to be in a specific format. You can use the template located at http://github.com/axemclion/trialtool/blob/master/examples/Template.html as a starting point. 
    • Examples can be under categories called example-sets using <li class = 'example-set'> and these categories can be expanded or collapsed using the TrialTool UI. 
    • Individual examples should be under <li class = 'example'> tags. Note the use of anchor tags to indicate the names of the examples and example-sets.
    • Eamples can have an id to identify them uniquely. 
    • Either an example set, or and example can depend on other examples using depends = "id-of-example-this-is-dependent-on"
    • Clicking on "Load Prerequisites" loads the dependencies of the parents of the current example, and then the dependencies of the example. All dependencies are loaded only once.
    • Documentation for example is specified using <div class = 'example-docs'> All HTML inside this is displayed in the Reference Tab at the bottom of TrialTool.
    • If documents are not inline, they are link to other elements inside the body using <link class = 'example-docs' href='jquery-selector-of-documentation'>
    • All scripts and stylesheets defined in the header are loaded in the console. This is done so that examples can use third party javascript or CSS libraries.
    You can start by cloning the trialtools on github and using the examples folder. Please do let me know if you are using this to show your API, or need help in cloning.

    Use the new symbol for the Indian Rupee already

    I was able to spend some time last weekend to come up with the CSS to allow me to start using start using the new symbol for the Indian Rupee on websites. There were mentions of blogs that had to font available for download, and it was not much to convert it to a form to be used by the websites.
    The project is hosted on Github and details on using it are available at http://axemclion.github.com/rupee/.

    To start using the symbol, simply include the CSS file available at http://axemclion.github.com/rupee/rupee.css. Then, add the HTML tag <span style="font-family: rupee;">R&;lt;/span>at all the places where the symbol is to be inserted. The CSS file is around 8 KB and has the font embedded inside it. For IE, the fallback URL of the EOT file is specified to render it correctly. This CSS file was generated using Font Squirrel using the font here. Font Squirrel was used with Custom Subsetting and only "R" was included in the file to reduce the size of the CSS.

    It becomes interesting to see how existing Indian sites look with the new symbol. You can find some screen shots here.
    I also created a bookmarklet and a greasemonkey script that would replace all "INR" and "Rs." on existing web pages to give a feel of how the pages will look with the new symbol.

    Tested on Firefox 3.6, IE 8 and Chrome5.

    Google Custom Search Sidebar - Greasemonkey Script Updated

    The Google Custom Search Sidebar is one of the oldest and the most frequently greasemonkey script that I have been using. I had written about it here. With Google changing the page structure, the sidebar was appearing at the wrong position. Infact, it was appearing above the actual results, floated to the right. This made the search page unusable and a lot of users had to disable the script.
    This problem is not fixed and the latest version has the following updates
    • The sidebar is absolutely positioned to the right like it was 
    • Added the Greasemonkey AutoUpdate script that would prompt for a download when a newer version of the script it detected.
    The change itself was simple, the search results are added to the body of the Google Search results page. The CSS styles make the positioning absolute and float it to the right. You can download this script and it will overwrite the older version that you may have disabled.
    You will also get updates automatically going forward.

    TrialTool - A Javascript API Viewer

    A lot of web applications are written starting with code snippets and examples, modifying them to suit the requirements. Using third party libraries becomes very easy if they are supported with lots of examples. A good example is the YUI page, where all components have examples that can be copied to applications for a start.
    However, documenting these samples is hard for the API or library developers. Tools like JSDoc or YUIDoc generate pages that are not very helpful for programmers looking to pick samples quickly. These tools also do not contain live examples. The Google API Playground is a great tool that not only has documentation, but also showcases working examples. However, using it for your libraries or APIs not very easy; you would have to set them up on Google App Engine and write tons of example files to get your sample working. This, IMHO, is an overkill for simple, CSS, HTML or Javascript APIs and libraries.
    I wanted to write something that would be easy to use, and quick to setup. I repurposed a lot of old code to come up with such a tool. The project is called TrialTool and has the following. 
    The guiding requirements for this principles would be
    • For API and Library Developers and
      • Should be easy to deploy. Backend server code should not be mandatory
      • Writing examples should be easy
      • The examples should be independent and able to run without the TrialTool
      • Should be able to re-purpose existing code to show are examples
    • For API users
      • Users should be able to see examples
      • Users should be able to see the documentation for APIs
      • Pre-requisites, dependencies should be loaded in the example itself.
    TrialTool is hosted on Github and a sample trial example can be viewed at http://axemclion.github.com/trialtool. The interface should be self explanatory.

    • Top left page shows the list of examples
      • Examples can be grouped as sets and can be dependent on other examples
      • Example groups can be expanded, etc.
    • The top right pane is the editor where code can be loaded
      • This is based on codemirror and users can edit example code here.
    • The bottom are can show the code output and the console.
    • The tool bar at the top has the following buttons
      • Run Snippet executes the code
      • Load Pre-requisites loads other examples that the current examples is dependent on
      • View Output / View Docs 
      • Clear Console
    A template file shows how an example can be created. Examples are grouped under sets and can refer to one another for dependencies. If you are interested in using this for your libraries or samples, I would be glad to help and you can contact me.
    To create your own set of examples, check this out. Follow this space for updates on the project.

    FlashPlus: Technical details of a Chrome extension converted to a Bookmarklet

    The previous post showed a video of how FlashPlus, a  Google chrome extension works for Firefox and Internet Explorer as a bookmarklet. This effectively merged the source code of the original bookmarklet called FlashResizer and the chrome extension. This post is about the differences between the two and how the chrome extension was converted into a bookmarklet.
    The key to converting it to a bookmarklet was to move all the chrome extensions specific functions to a separate file. In this case, the chrome.* API calls were moved to PageActions.js. The real logic lies in FlashPlus and Commands. FlashPlus enumerates all the flash content on the page while Commands is responsible for defining actions that can be taken over individual flash elements.
    Hence, the difference between the bookmarklet and the extension is the entry point. The Chrome extension uses PageActions.js to insert the script within the page. This is supported by the background page and the popup icon in the URL bar. On the other hand, Bookmarklet.js is a simpler entry point that simply enables the functionality for all available media content.
    This leads to slightly different capabilities between the two.
    • The bookmarklet cannot work on Media content inside iFrames as the script is inserted only for the current frame.
    • Pop out functionality does not currently work as it requires additional javascript to be executed once the new window is created. 
    • Media content may restart after starting the bookmarklet as the wmode parameters have to be re-written. 
    The most interesting part for this change was to make FlashPlus and Commands, cross browser compatible; something that you do not worry about, while writing the extension.
    The bookmarklet now lives in a different branch and the releases are tagged as bookmarklet. Regular development would continue on the extension and changes to FlashPlus and Commands would be merged back to the bookmarklet branch.
    You can watch out this and this space for updates on the bookmarklet and the extension respectively.

    FlashPlus Bookmarklet - Internet Explorer and Silverlight

    The code for FlashPlus for Chrome and the FlashResizer bookmarklet have been merged and the bookmarklet has almost all the functionality as the Google Chrome extension.
    Here is a video demonstrating it.

    ChromaHash - Demo Video

    On a webpage, password fields obscure your input with star's to mask them from shoulder surfers. At times, even the users are not sure if they have typed the password correctly. Chroma-Hash is an experiment that displays a variety of visualization like background gradients or background. After sometime your would recognize the gradient that should appear for your passwor, and when the wrong gradient appears, you know that you mis-typed the password.

    Chroma-Hash takes an MD5 hash of your input and uses that to compute the visualization.
    The MD5 hash is non-reversible, so no one could know what your password just from the colors.

    Here is a demo video of how ChromaHash works

    FlashPlus - Version 1.5.3 Released

    FlashPlus 1.5.3 was released a couple of days ago and it seems to getting decent adoption. There were some interesting bug fixes and a good amount of code re-factoring.

    One important feature that was added was support for Silverlight media,  HTML5 canvas and video tags. Silverlight was very similar to the way flash works in the extension. Just like flash has a wMode parameter, we would have to change the windowless parameter for Silverlight. HTML5 Video and canvas were even simpler. Three lines of code, and they work good.
    For canvas, you can open this or this.To check out the extension on the video tag, you could use this, this or even this youtube page. Apparently, these two additions seem to make the name of this extension obsolete. [:)]

    There were also a substantial amount of code-refactoring that went in regarding the way, position of media elements on the page are calculated. There is a whole branch dedicated to this. However, we were still not able to fix the issue that arises with this page and will continue to work on it.

    There was also refactoring to fix the way new tags were added, to make it more scalable.

    Finally, there were a couple of bug fixes also. The first bug fix was correcting the way wMode param was treated in flash. Regardless of its presence, this flag seemed to be set whenever any action was performed on the media, making the flash to play from the start. Now, we set the wMode only if the value is not Opaque.

    Watch out for the next releases here, where we plan to release better support for internationalization and the way Media content is handled in iFrames.

    Twitteybot - Demo Video

    TwitteyBot is a very simple twitter application that lets you schedule your tweets for later. All you do is upload a file with messages and your tweets are scheduled. To showcase its simplicity, I put together this video.

    Auto Reply twitter bot with YQL and Yahoo Pipes

    One of the features that I wanted to implement in TwitteyBot was the ability to automatically reply to people who have certain search term. Unlike the normal updates that only reach the people who follow you, this could be noticed by people whom you are replying to, thereby having a wider, newer audience. Here is the scenerio.
    Searching for a term like "#chromeextensions" would show all the people who are talking about Google Chrome extensions. It would make sense to reply back to them and maybe tell them about something like FlashPlus.

    You can have your own bot by changing the parameters of this pipe, copying the "Get RSS" link and configuring it in Twitterfeed.

    I created such a bot using YQL and Yahoo Pipes. Here is the pipe that spits out a feed that should be tweeted. Twitterfeed can then be used to convert this RSS feed to tweets for a specific twitter account.
    The pipe uses this YQL to find all the tweets that have a certain twitter term. It then follows through a loop where the user is extracted and the advertisement line is added. This is spit out as a feed that is configured into Twitterfeed.
    The only limitation is that this method has a limit of 5 tweets every 1/2 hour. I tried using Friendfeed to overcome this limit but friend does not seem to pick up the changes in the RSS feed.

    If we make the advertisement text dynamic, we could have a bot that sends out answers from a we service. I created a @questionbing bot that tries to fetch the instant answers results from bing for questions sent to it. Here is the pipe that does this. Try asking it questions like [http://twitter.com/home?status=@questionbing how many days in a year] and you should get an answer.

    FlashPlus - Version 1.5.2

    FlashPlus 1.5.2 has been released with a couple new features and some minor bug fixes. The adoption of this chrome extension has been growing steadily.You can read the release notes here.

    One feature that was asked for by the users was to pop out the flash content into a new window. This would give more real estate to the flash, making watching videos or playing games better. I was considering two approaches to do this.
    The first approach involved opening a new window with the current page. The element's selector would be passed to the page. The page would use the selector to pick the element and then make it full screen. The only problem with this approach is constructing the selector. I did write some code for it and it seemed to work, but it would not cater to dynamic elements on the page that change with every page load. You can check out this version here.
    An easier approach was to simply pick up the HTML of the flash object, with its params and attributes, place it on the new window with the current URL and convert it to full screen. Since the page in the new window is same as the current page, relative URLs and javascript access to the flash would work fine.

    An Options page was also added with the following configuration options.
    • Option to manipulate the flash elements when the page starts. Doing so does not require the flash to reload, and hence, when you load the layer on top of the flash, your movie does not restart. This may have to be turned in cases where the page alignment is screwed.
    •  An option to set the time till which FlashPlus should check for Flash content loaded using Javascript. 
    • Options to show logs in the console window.
    • A footer with details about the extension, author and place to ask for features or report bugs.
    There were also some bug fixes
    • Some thumbnails in the Page Action (the FlashPlus Icon in the address bar) do not show up properly. This was due to bad scaling math, which is now corrected
    • iFrame support has been enhanced. 
    • A user pointed FlashPlus was not working on certain pages. This was due to the same CSS class names used. The class names in the extension now have a prefix. 
    • There were logical bugs with permutations of actions like "FullScreen", "Block", "Unblock", "Back to Windowed Mode". This bug was also fixed.
    This was a major release with a lot of features put in. There are some more features that I would start working on, when I have time.
    The extension is now a substantial amount of code and it would be hard to work on it alone.  I would appreciate any help with coding, please do drop in a comment and we could collaborate.

    Time 2 Tweet : More updates

    It has been sometime since time2tweet was last deployed and updated. Since there, there have been a lot of users and we have finally hit our Google App Engine free users limit. I had sent out an email to all the users requesting them to host the application on their instance to continue scheduling their tweets seamlessly. Interestingly, a lot of people responded back, wanting to continue using the application to tweet their applications.
    Over the last week, I helped them host the applications on their server, and many users now are successfully running the application. During that time, I also discovered some interesting issues that I fixed. To start with, I moved the source control to Mercurial on google code.
    The first issue I encountered was asking the users to register for twitter Oauth and enter those credentials for the application to use. Since it was an admin module, I was using a URL that would set the OAuth Secrets using the GET method. Not only is this a bad design, people were confused when the URL spit out a blank page. This release has a nice looking UI around it.
    The second bug was interesting. I was reading the OAuth values in the cache once they were set. Once read, they would not be re-read till the cache flushes it out. If the OAuth values were incorrect, no tweets would be posted. It would also not be possible to update the cache as these values were never removed from cache. This was a big problem as the cache seems to maintain state even on new applications. The admin page now explicitly sets this value in the cache and the database, eliminating this problem.
    There was a Google App Engine failure some time ago and that had stopped all cron tasks. Once App engine was up, the Cron job has too much work to do and used to time out. This let to no tweets being posted. This ug was fixed but explicitly forcing the Cron job to handle only 20 tweets at a time, no matter how much backlog it has. 
    Finally, I also corrected the tasks to post have one tweet to be posted per task. This may be heavy on the task, but seemed to be the right thing to do. Given that there can be 30 tasks per minute and the Cron sets 20 tasks every minute, this seemed to be a good thing to do for the load.
    As with all systems, not all deployments are great. I still see some errors with the Tasks about "null" screen names, etc. and am still investigating. Please follow this space for update.
    I would also like to thank all the users who think this application is good and were ready to deploy it on their instance of App engine.

    FlashPlus - Version 1.3.1

    With Youtube now steaming IPL live, the new version FlashPlus is even more useful. Some of the features in this release are
    • Support for Flash Movies inside iFrames of the page
    • Corrected the z-index to show flash videos above other components of the page.
    • Support for pages that embed Flash using only the object tag.
    The Youtube IPL page has atleast three nested iFrames and the extension works like a charm. The only problem is with the boundaries of the iFrame that limit the movement of the Flash Content. This is something that will be fixed in the next release.
    The next release would mostly be
    • Add ability to pop out flash movies into a new window
    • The Page action icon shows screenshots of flash with distortion. This is to be fixed.
    • Flashplus currently replaces flash when the icon is clicked. This would cause flash to reload, restarting it. This is bad when watching streaming videos and playing games. There will be an options page for this. 
    Watch out this space for updates.

    FlashPlus - Version 1.0.2 Released

    FlashPlus 1.0.2 released yesterday with a couple of bug fixes and some new features. The features in this release first.
    1. Many people had requested for the ability to block flash. Extensions like FlashBlock prevent extensions from loading. However, you may want to see if the flash is really an advertisement that you want to block, or a genuine component in the page. The new action lets you block flash after you see it. We are also working on implementation of FlashBlock inside FlashPlus. This way, you can use just one extension. This is the fix.
    2. A lot of cosmetic changes. The icon of FlashPlus has changed; so have the icons for  "black out the page". The black out is now translucent, this lets the user still know that they are on the page.
    Some of the bugs that were fixed include
    • Flash is sometimes loaded using Javascript. Previously, the  extension used to look only for Flash embedded in the HTML file. A new check has been added to check for Flash components that are added by Javascript later. Some more work is required to check if some flash components are loaded right away, and others are loaded using Javascript.
    • Size of the pop up on clicking the button on the URL bar now changes according to the number of Flash movies on the page. It used to be small earlier, making it look ugly. Cosmetic change, fixed now.
    • Flash movies sometimes have incorrect dimensions. Hence, clicking on the pop up did not activate FlashPlus. This has now been fixed.
     I have forked a new branch for Release 1.3 and would be working on the following.
    • Issue 14 : Adding an Options page where user can configure the following parameters for Flash Plus
    • Issue 7 : Correct the problems in iFrame. Make FlashPlus work on IFrames.
    There are a lot of interesting features lined up for the next release. Watch out this space for updates.

    Phishing with Google Wave

    Phishing is usually be categorized under social engineering rather than a technical hack. It is inherently about tricking the user to click a link, or visit a web page. However, if the victim is tricked into visiting the phished page, even while they are on a genuine site should be a cause of concern. Since the victim did not initiate to move to the phished page, they are caught off guard.

    This post is about a possible attack on Google Wave which could at the least disrupt the wave experience, if not steal the credentials. This attack is similar to the one on Orkut Opensocial that I had published earlier. I am sure someone somewhere would have already figured this out, but I chose to post this anyway since I got some time off FlashPlus, my Google Chrome extension.
    The hack can be done by anyone anonymously and on public waves.
    1. Create a phished Google Login page. You could check out tackle.
    2. Search for public waves
    3. Reply to one of the messages, insert a gadget in your reply
    4. The gadget sets the top.location to the phished page.
    5. The victim now visits the wave and opens this unread wave
    6. The gadget kicks in, redirects the user to a phished page
    7. Since the victim was still inside and browsing wave, they may not suspect a phished page. They may think that they were simply logged out.
    The following video shows these steps.



    A couple of ways to anonymize the attacker could be
    • Make the gadget to set top.location after a window.setTimeout, instead of doing it immediately.
    • Do not redirect all users. Redirect them if a certain cookie is not set. If a cookie is set, they were already phished.
    • Create anonymous accounts on Gmail, host gadget and phished page using Google Gadget Editor on iGoogle. This shows the wave URL on a gmodules.com domain, something that's more believable.
    • Submit credentials from the phished page to a form created using Google spreadsheets.
    I am not sure how harmful this hack can get. I have pinged a friend a Google about this.

    FlashPlus - Now for Chromium on MAC

    I am currently working on FlashPlus, a Google Chrome Extension. I had written about the first release here that included a demo video.
    This initial release seemed to work fine with the Windows version of Chrome, but were failing for Chromium on Mac, as Aadith reported. The error was in the manifest file where absolute URLs were used to identify resources. This bug is now fixed and the extension works on Mac also. I also managed to sneak in a couple of more features over the weekend with the latest version.
    If there are more than one flash movies on the page, the popup was not always showing exact flash movie. A visual indication is now in place to "find" which flash is shown in the popup.
    The JQuery version was also upgraded to 1.4, with some changes.
    The extension file is also smaller as I realized that the initial version was packing unnecessary demo and .svn folders. Those removed, the extension is a lot smaller now.
    The final change in this release was to move from Subversion to Mercurial, specially because there would be a lot of branching and I would also be doing a lot of offline development.

    About the next release, I am looking at four major features, in order or priority.
    1. Add options to  download a Flash. This is a tricky feature as streaming videos are different from simple .swf files. However, most people have asked for downloading Videos and it may make sense to show the download link from flash video download sites selectively. I have created a branch and am working on it.
    2. Block or hide Flash Movies to save CPU/Memory usage. I am not really sure how much memory can hiding a flash save, but from the looks of this feature, it sound like it could help. This is easy to do.
    3. Configuring the flash parameters.  This is not a big ask as it is an advanced feature that lets you change the way SWF files behave.
    4. Blocking Flash as soon as it loads. This feature aims to bring FlashBlock under its hood, and should not be very hard to do. The question however is, do I need to replicate the FlashBlock functionality?
    If you think that you need a feature more than the other, or have other exiting ideas, please do drop in your comments. If you interested in helping with the code, it would be awesome :)
    Watch out this space for updates.

    FlashPlus - A Google Chrome Extension

    I have been working on the FlashResizer Bookmarklet for a while now, and I thought of converting it into a Google Chrome extension. Here is a video of the functionality of the extension.



    This extension is still a work in progress and I am working on ironing out some non-critical issues. The extension has been submitted to the Chrome extensions Gallery and is pending approval. If you would like to try it out, you can download it from here and open the .crx file in Google Chrome. Chrome will ask you for confirmation at the bottom, and once accepted, the extension should show up on all web pages containing flash movies.
    You can watch out for updates here. Please do drop in your feedback as comments or contact me. Any offering to help with the development of the extension would be great too :)

    Deploying TwitteyBot on your instance on Google App Engine

    If you would like to deploy your own instance of twitteybot on Google App Engine, here are the steps for you. The process is not wholly automatic, and I am looking at way in which I can automating deployments to Google App Engine, and then, configure them. Till then, here are the steps.
    or the deployment, would need

      1. Compiling the source
        1. Sync the source code using an SVN client from here.
        2. Open build.properties.template and change sdk.dir to point to your location of java app engine directory
        3. Run ant clean enhance compile. This should create a svn_root/trunk/war directory.
      2. Google AppEngine account
        1. Create a new app engine application. 
        2. Run /bin/dev_appserver.cmd update war to update the source on app engine.
      3. Twitter OAuth credentials
        1. Head to http://twitter.com/oauth and create a new application
        2. Set the following properties
          1. Application Type : Browser
          2. Call Back URL :[ yourapplication].appspot.com/pages/manageTwitterAccount?action=oauth
          3. Default Access Type : Read and Write
          4. Use Twitter for Login : No
        3. On saving this, you would get a consumer key and a consumer secret.
      4. Open the browser and visit the following URLs to set the consumer secret and consumer key.
        1. http://[yourapplication].appspot.com/admin?action=add&key=consumer_key&value=[consumer_key]
        2. http://[yourapplication].appspot.com/admin?action=add&key=consumer_secret&value=[consumer_value]
      5. Your application has been successfully set up.
      You can hit the home page of the application and start managing twitter accounts.
      Note: The database indexes may not be built immediately on the App Engine instance. You may have to wait a while before the indexes are built.