After giving it much thought, I figured out the following solution to running cron jobs in google app engine.

Step 1: Create a datastore entity that represents a task the cron job has to perform:

# Class representing tags cron job
class CronJobTask(db.Model):
# Put any properties here


Step 2: Create a class representing the cron

class CronJob(webapp.RequestHandler):
def get(self):
# Do your cron processing here by fetching the crontasks from the datastore
# Do not forget to remove completed cron tasks from the datastore (if they are not periodic)
# Output the following code as a response
self.response.out.write("< script >function reload(){ document.location = '/cron?time=' + new Date().getTime() } setTimeout('reload()', 1000); < /script >")


Step 3: Map some url to the cron job class like “/cron” below.

def main():
application = webapp.WSGIApplication([('/cron', CronJob), ('/', MainPage)], debug=True)
wsgiref.handlers.CGIHandler().run(application)


The following code snippet is the key idea:

self.response.out.write("< script >function reload(){ document.location = '/cron?time=' + new Date().getTime() } setTimeout('reload()', 1000); < /script >")


What the above does is that it causes the browser requesting this page to periodically poll the cron url. The cron can be started by pointing your browser to http://yourapp.appspot.com/cron. Opening multiple browser sessions/tabs to the same url spawns multiple cron processes which complete the cron tasks in parallel. You will always need to have atleast one browser tab pointing to this url at all times.

< META name='unavailable_after' content='Wed, 01 Aug 2007 00:00:01 GMT'>

By specifying this tag, webmasters can tell Google not to index a particular page after the specified date OR consider the given page as stale. This would be appropriate for promotional pages where the promotions expire after a given time. This would help unclog the search engine indexes of irrelevant data.

This valuable piece of information, provided by the ‘unavailable_after’ tag, would not only be used to clear up the Google’s index but could also make its way into Google’s ranking algorithms. There are two perspectives to how a search engine could use this data for ranking:

• When a page specifies its expiry/unavailability date, it implicitly tells the search engine of the period for which it would be most relevant. Hence, as the unavailability date of a page approaches it should start becoming less irrelevant to a users query. For example: The user query for ‘fedora release notes‘, currently, has the top 3 results pointing to the notes of FC7, whereas the other results have a random mix of release notes for FC3, FC2, FC4 and FC5 (with FC3 and FC2 pages being ranked higher than FC5 and FC4 respectively). Lets say that FC8 was going to be released this November [schedule]. Assume that the Release Notes page for FC7 has the unavailable_after tag set for sometime around December. Thus, as December approaches, FC7 pages would start losing their relevancy for such queries and gradually transition lower in the search rankings, making FC8 the most relevant result. This would resolve the current inaccurate ordering of results obtained for the query ‘fedora release notes‘ on Google. This could be achieved in a manner similar to proposed in the following paper: Time Damping Of Textual Relevance.
• This perspective (inverse to the above perspective) would be very specific to promotional/shopping related pages. Most shopping promotions/offers are valid for a given period. Hence, such pages should become more relevant to a users query as they approach their expiry date. For example: Consider a user query for ‘20% discount shoes‘. Lets assume that the results have pages from Zappos as well as Shoebuy both offering a 20% discount on shoes. The Shoebuy sale is going to last for about two more weeks from today (as specified by the unavailable_after tag) whereas the Zappos sale would be ending in another two days. Since both the stores are offering the same percentage discount, it would be more appropriate to rank the Zappos page higher as its offer would be ending soon. From the point of a user (shopper), he would be more interested in looking at offers ending soon, as he can always checkout the other long lasting offers at some later time.

The perspectives above represent only a few of the conceivable usages of the unavailable_after tag. There could be a numerous other perspectives to how this data could be utilized to improve search rankings.

I would lover to hear your take on the unavailable tag, particularly if you can provide another perspective to utilizing this data.

We do not have the proper event planning resources in the Bangalore office to support an event such as this. However, we are webcasting the US general session and one of the main conference tracks. We’re also video recording the event and will be posting it online after the event.

Darn! There goes my chance of tasting the sumptuous Google food.

It is not possible to be presensitized in different directions at the same time just as it would not be possible to design a golf club that was the best club for driving and at the same time the best club for putting. That is why the Six Hats method is essential. It allows the brain to maximize its sensitivity in different directions at different times….The essence of parallel thinking is that at any moment everyone is looking in the same direction — but the direction can be changed. An explorer might be asked to look north or to look east. Those are standard direction labels. So we need some direction labels for thinking. What are the different directions in which thinkers can be invited to look?

This is were the hats come in. Each hat symbolizes a thinking direction. De Bono’s six hats are:

• White Hat: is neutral and objective, concerned with objective facts and figures.
• Red Hat: relates to intuition, emotions & opinion.
• Black Hat: is gloomy, and covers the negative – why things can’t be done.
• Yellow Hat: symbolizes brightness and optimism, indicating hope and positive thinking.
• Green Hat: focuses on creativity: the possibilities, alternatives and new ideas.
• Blue Hat: is concerned with the control and organization of the thinking process.

Thats all about the Six Hats. But where does Google fit in?

Lets take a look at the Google logo:

If we apply the Six Hat approach to interpret the logo, we get:

• 2 blues: signifying the unified and organized thinking process at Google.
• 2 reds: signifies that they value intuition, emotions, opinions of the users. Outputting innovative products by not just their brains but their hearts as well. This is evident from the simplistic and intuitive UI designs of Google Search and Gmail.
• 1 yellow: symbolizes their optimistic outlook, hoping that every new product would bring them one step closer to their ultimate aim and that is to organize the worlds information.
• 1 green: new ideas, alternatives, innovation…Well, thats Google :)

Conclusion

The above interpretation of the logo actually seems to be in sync with the current image of Google and the prevalent mindset of the average googler. Did I just unearth Google’s success formula?

Hope so :)

Larry Page Sergey Brin

What if terrorists decided to terrorize the Internet? Although they have been spreading messages of hate, what if they decided to bring it down? Bring such a large cluster of machines down! Thats like waging a war against the world. But what if they decided to bring down services that seem to have become an important part of our internet life.

What If Terrorists Decide To Bring Down Google

To most of us, ‘the internet is Google‘ and even the thought of Google not being anymore gives us the chills. Bombarding the datacenters would definitely cause a considerable loss to Google. In the map below, I have tried to mark out a few of the Google datacenters.

With Google venturing into things like solar energy it should also probably think of getting itself an anti-missile shield or build its datacenters in underground bunkers :).

After all the search history I have built up over time is priceless ;)

• Blogger + Google Spreadsheets = Dynamism as in WordPress
• Google Maps + Google Spreadsheets = a whole multitude of data retaining maps
• Gmail + Google Spreadsheets = a powerful tagging system for Gmail
• Google Search + Google Spreadsheets = a search engine for my textual data (probably having a query syntax like: ‘anand insheet:sheet1 incolumn:name‘
• Google Analytics + Google Spreadsheets = custom analytic views based on what I want to track

As far as storage is concerned, which could be a bottleneck as databases continue to grow, GSpreadsheets could be linked up with Gmail to utilize the free space from the virtually unlimited mail box size.

Dabble DB is trying to do something similar but they don’t have an SQL-like query interface yet. Hopefully Google would be willing to venture out in this idea first.

In all the forums I visited I came across this one point again and again: “Google Coop is susceptible to spammers“. I don’t agree. Like any other social app which is fuelled by the people, at the first glance it does seem susceptible. But it is the social factor that seems to ward of spammers. Quoting a FAQ from Google Coop:

Who will see my labels?

Users who subscribe to you will see your labels for relevant searches. As your labels become higher quality and more comprehensive, and as more users subscribe to you, your labels may start surfacing to more Google users than just those who explicitly subscribed. A number of factors help determine how broadly your labels appear — such as the number of subscribers you have, how many websites you’ve labeled, and, most importantly, how often your labels make it easier for users to find what they’re looking for.

Google seems to have realised that it can achieve a lot more by utilizing the human intelligence, Intelligence which is the core of the Web 2.0. After all the ubiquitous search is also based on human knowledge (creation of links between pages).

Over the past two years I have grown as a blogger and so have my needs from a blogging platform. Therefore I decided to switch to a privately hosted WordPress platform (Privately? Well thats another discussion altogether). And to tell you the truth, I haven’t looked back ever since, infact I’m loving it (-:.

What does the Blogger lack?

Blogger has always been the first choice due to its affiliation with Google, but although Google may be the God in search…not quite though here.

• To speak of the UI, WordPress is fairly much cleaner and organized. Blogger is cluttered and less user friendly. With all the AJAX, movable divs, and coloured highlights thrown in WordPress definitely rocks.
• One vital feature lacking in Blogger is the ability to create categories. I just so couldn’t do without this feature that I decided to write a hack for it.
• WordPress always dynamically generates its pages, so you never have to regenerate pages unlike in Blogger.
• Blogger is not open source, WordPress is. Open source means that everyone and anyone contributes. Just to illustrate, Blogger has become stale. It has not released any new features lately, the last I remember was the blog search. With Google around this was something expected. But yet nothing worthwhile to facilitate blogging. WordPress on the other hand has countless plugins with many more under development.
• Blogger has no password protection for posts. A well thought feature of WordPress, which lets me password protect individual posts. Quite useful when you have a personal daily blog where certain posts are too personal for strangers to read.
• Switching to WordPress from another platform is easy as hell. The import blog feature is definitely cool. I don’t remember Blogger having something like this.

Every blogger has his own reasons for sticking to a platform, but for all those still stuck onto Blogger try a switch. Trust me on this.