Hi,
We just released a new release of Sneak-O-Scope with a cleaner database. Though there were not many cosmetic changes to the applications, the code base is a lot more managable now.
The first change was the addition of the app engine console to the backend. The console is a simple python shell that executes commands at the server. This was very useful to test the database and the limitations imposed by the Google App Engine.
The second most important change was that the enter and exit time columns in the database were converted to the python datetime format. It was a textProperty earlier and that led to invalid date and times.
Another change was the introduction of a table to separately store unknown visits. This was done as querying for unknown and known visits separately is not allowed. Since we sort on enterTime in descending order, we could not have an inequality for unknown visitor. Hence, we had to separate them out into two tables and query them separately. This also helped us to pick 100 known and 100 unknown visitors, populating the list at the UI with atleast some known visitors instead of all visitors being unknown.
We also used the Preferences table to aggregate data. The table now has a Text column that stores the JSON for total visitors, known visitors, etc. The field will later used to store browser and friend statistics also.
We also included the simplejson library to parse JSON to and from the requests. This is better than parsing the strings to get data out.
There are also a couple of migration scripts that are to port data from the database at sneak-o-scope.appspot.com to orkut-scope.appspot.com. This would be the vertical partioning of the database for users from orkut ,myspace, etc.
The IP address has now been put to use with a YAHOO pipe that displays the location, ISP, etc in an iFrame. Details about this on a later post.
Last but not the least, we also included a HTML page to indicate that the application is not installed properly and has to be moved above other applications. There is also inline text next to unknown visitors about Orkut's privacy policy that does not let us display details of visitors who do no have the application installed.
