Jeetu's shared memory

Iddo Genuth writes “Alaskan state officials have recently announced their intention to begin funding the exploration and surveying of Alaska’s largest volcanoes in hopes of utilizing these as a source of geothermal energy. They say this volcano could provide enough energy to power thousands of households, and according to some estimates, Alaska’s volcanoes and hot springs could supply up to 25% of the state’s energy needs.”

Read more of this story at Slashdot.

There was an interesting presentation at OSCON 2008 by Evan Henshaw-Plath and Kellan
Elliott-McCrea entitled Beyond
REST? Building Data Services with XMPP PubSub. The presentation is embedded below.

The core argument behind the presentation can be summarized by this
tweet from Tim O’Reilly

On monday friendfeed polled flickr nearly 3 million times for 45000 users, only
6K of whom were logged in. Architectural mismatch. #oscon08

On July 21st, FriendFeed had 45,000 users
who had associated their Flickr profiles with
their FriendFeed account. FriendFeed polls Flickr about once every 20 – 30 minutes
to see if the user has uploaded new pictures. However only about 6,000 of those users
logged into Flickr that day, let alone uploaded pictures. Thus there were literally
millions of HTTP requests made by FriendFeed that were totally unnecessary.

Evan and Kellan’s talk suggests that instead of Flickr getting almost 3 million requests
from FriendFeed, it would be a more efficient model for FriendFeed to tell Flickr
which users they are interested in and then listen for updates from Flickr when they
upload photos.

They are right. The interaction between Flickr and FriendFeed should actually be a
publish-subscribe relationship instead of a polling relationship. Polling is a good
idea for RSS/Atom for a few reasons

• there are a thousands to hundreds of thousands clients that might be interested in
  a resource so the server keeping track of subscriptions is prohibitively expensive
• a lot of these end points aren’t persistently connected (i.e. your desktop RSS reader
  isn’t always running)
• RSS/Atom publishing is as simple as plopping a file in the right directory and letting
  IIS or Apache work its magic

The situation between FriendFeed and Flickr is almost the exact opposite. Instead
of thousands of clients interested in document, we have one subscriber interested
in thousands of documents. Both end points are always on or are at least expected
to be. The cost of developing a publish-subscribe model is one that both sides can
afford.

Thus this isn’t a case of REST not scaling as implied by Evan and Kellan’s talk. This
is a case of using the wrong tool to solve your problem because it happens to work
well in a different scenario. The above talk suggests using XMPP which
is an instant messaging protocol as the publish-subscribe mechanism. In response to
the talk, Joshua Schachter (founder of del.icio.us)
suggested a less heavyweight publish-subscribe mechanism using a custom API in his
post entitled beyond
REST. My suggestion for people who believe they have this problem would be to
look at using some subset of XMPP and experimenting with off-the-shelf tools before
rolling your own solution. Of course, this is an approach that totally depends on
network effects. Today everyone has RSS/Atom feeds while very few services use XMPP.
There isn’t much point in investing in publishing as XMPP if your key subscribers
can’t consume it and vice versa. It will be interesting to see if the popular “Web
2.0″ companies can lead the way in using XMPP for publish-subscribe of activity
streams from social networks in the same way they kicked off our love affair with
RESTful Web APIs.

It should be noted that there are already some “Web 2.0″ companies using
XMPP as a way to provide a stream of updates to subscribing services to prevent the
overload that comes from polling. For example, Twitter has
confirmed that it provides an XMPP stream to FriendFeed, Summize, Zappos, Twittervision
and Gnip. However they simply dump out every update that occurs on Twitter to these
services instead of having these services subscribe to updates for specific users.
This approach is quite inefficient and brings it’s own set of scaling issues.

The interesting question is why people are just bringing this up? Shouldn’t people
have already been complaining about Web-based feed readers like Google
Reader and Bloglines for causing the same
kinds of problems? I can only imagine how many millions of times a day Google Reader
must fetch content from TypePad and WordPress.com but
I haven’t seen explicit complaints about this issue from folks like Anil
Dash or Matt Mullenweg.

Now Playing:
The
Pussycat Dolls – When
I Grow Up

The Google/Digg acquisition negotiations were in full swing as of last Tuesday, had passed the term sheet stage and the two companies were in final negotiations in the $200 million range. But sometime this last week Google decided to walk from the deal. Digg was notified on late Thursday or Friday.

Google was in the due diligence stage of the deal, where they peer deep into Digg’s technology and financial statements. Most term sheets are non binding, so anything that gives the buyer pause can be used as an excuse to walk away – but generally the buyer already has a very good idea what they are getting well before the term sheet stage.

Two sources close to the companies suggested that some issue that came up during technical due diligence was to blame. One source said that the issue was more personality driven, and that Google decided after spending more time with Digg’s top team that there just wasn’t a fit.

Either way the deal appears to dead and can be added to the long list of failed Digg acquisition deals. And when a company is “left at the altar” other buyers are usually hesitant to step in.

So what will Digg do now? We’re hearing they’ll just push through with a new round of financing. Digg hired Allen & Co. to represent them in the sale, but the investment bank is just as good at closing massive venture financings, too (they represented both Slide and Ning in their recent a half billion dollar valuation financings).

Crunch Network: CrunchBoard because it’s time for you to find a new Job2.0

Google has announced that its search index, which chronicles the vast majority of the internet, has hit a major milestone: 1 trillion unique URLS. That’s 1,000,000,000,000 pages – more than twice as many as there are stars in the Milky Way.

The index has grown exponentially since Google first launched. In 1998, the site had ranked approximately 26 million pages. The site reached the 1 billion page milestone in 2000, and has since grown by more than one thousand fold. Over the last decade Google has changed the way it indexes the web in order to accommodate its massive scale. In the early days it would sporadically recrawl the web to create a new index, which would last for a fixed period of time. Now, the site is continuously monitoring the web, updating the index multiple times a day.

Google’s post clarifies that the size of the internet really depends on what you classify as a unique webpage. In theory, there are an infinite number of pages, as many sites can automatically generate new ones (for example, a calendar could create a new page for every day in the future). The database also ignores pages that are near-exact reproductions of other sites.

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

With the news coming out of F8 this week, it was hard to not get caught up in the enthusiasm for Facebook Connect, the new authentication methodology which will allow you to login to third-party web sites using your Facebook ID and port your friend graph from Facebook with you. On the one hand, you have to admit this is revolutionary. The web will be transformed from the still (somewhat) closed system it is today, to a massively social experience – it’s the “always logged-in internet.” On the other hand, the company bringing this web to us is Facebook, the same people who had to be told by their users why Beacon was a huge mistake. Do you trust Facebook to control the next iteration of the web?

Facebook Connect

As Facebook Connect grows and is adopted by more sites, it will push your social graph to the far corners of the web, out to places where it doesn’t even exist today – that is, places like the corporate web sites whose own attempts at creating social networks of their own were a waste of money. These businesses never needed a social network – they need to tap into your social graph and Facebook Connect gives them the power to do so.

Through the seamless Facebook Connect integration, sites can access your Facebook account details and friend graph and move that data back and forth between their site and Facebook. For example, people commenting on a blog using the Moveable Type platform will be able to login via Facebook Connect. Their comment will link to their Facebook profile and the commenting activity itself will make its way back into your activity feed. On Digg, another site adopting Facebook Connect, you can login with your Facebook ID and your digging activity is returned to Facebook, too.

Why Facebook Connect Could Win This

A large majority of today’s more mainstream users have a Facebook account and still, the site grows. Although MySpace still remains king in the U.S., Facebook is quickly closing that gap, having grown 40% over the last year. But worldwide, Facebook officially caught up to MySpace in April 2008 in terms of monthly worldwide visitors – around 115 million per site per month (source: Comscore).

Although Facebook Connect is just one example of this new trend involving the portability of our social graphs, it already has a leg up on both Google’s Friend Connect and MySpace’s implementation of OpenID.

With Google’s Friend Connect – not even fully launched yet – they’re relying on the power of their brand. But although the site is a household name now, that doesn’t mean that everyone has a Google account or a friend graph there. Like Microsoft and their “Passport” (now Live ID) initiative, the largest source for collecting user accounts is via their webmail offerings. For those that don’t use Gmail or any of Google’s other customized services requiring a login, there’s no value to Google’s Friend Connect because there’s no friend graph there. You would be creating an account to have the sake of the account. This doesn’t appeal to anyone.

As for OpenID, as much as we’re thrilled to see it finally making its way onto huge web sites like MySpace, it will still have to overcome the “user education” issue. A mainstream web user will not know what an OpenID is (and will often not take the initiative to find out). But they will be able to wrap their heads around Facebook Connect. “Login with your Facebook ID” – that’s pretty much as straightforward as it comes. It’s unfortunate, but you have to acknowledge the fact that if OpenID can’t dumb down their concept enough for the “everyman,” it may not be able to reign dominant across the web. Like Dick says on the Identity 2.0 blog: “…frankly — typing in a URL is pretty geeky to most users.”

Why Facebook Connect Could Be Dangerous

Unlike with OpenID, Facebook Connect put the power of the social web into the hands of one company. One private company. Not only that, but a company that’s known for rolling out changes without so much as a warning to its users then having to react to the ensuing uproar.

Even the introduction of the Mini-Feed was protested upon launch. And Beacon – the advertisement system that sent data from external web sites back to Facebook, telling your friends about your purchases on 40+ partner sites – was literally a fiasco. It launched before there was a way to even opt-out.

In the past, user privacy on Facebook seemed always seemed to be an afterthought. Although their direction appears to be changing a bit now – recent updates to Facebook today make sure to cover how your privacy is going to be affected – it’s only because they’ve learned to cater to their users’ demands. It’s harder to believe that it’s because they genuinely care.

Facebook has always known that their value – that is, their monetary value – is selling off bits and pieces of your privacy to advertisers. The “real you” on Facebook is a holy grail for marketers. Now, with the power to spread that to sites across the entire web, Facebook will need to figure out how to cash in. In the process, they may again make another misstep. The problem is that this time it might not be something as innocuous as the video you rented at Blockbuster that finds its way back to your Facebook profile. As more of the corporate and business-oriented web adopts Friend Connect, the greater the chance for privacy intrusion.

Conclusion

What do you think about this new social web? Do you see Facebook Connect as having a chance to win this all? Or will it be Google Friend Connect or OpenID? Or perhaps all three can co-exist peacefully?

This was on the cards but is now official. The Indian creators of the popular Facebook application Scrabulous have been sued by Hasbro, the US based company that owns copyright over Scrabble. Check out reports here, here & here. The suit has been filed against RJ Softwares, the Kolkata based software development company owned by Rajat Agarwalla & Jayant Agarwalla, who together created the game. Hasbro is apparently seeking damages from RJ and asking them to stop using the name ‘Scrabulous’ for their game. Not just that, they have also sent a copyright notice to Facebook under the DMCA (Digital Millennium Copyright Act) saying that it infringes the company’s intellectual property and asked them to remove Scrabulous from its site, which Facebook has refused.

Hasbro incidentally now has its own version of Scrabble on Facebook and that has 8K users compared to over two million for Scrabulous.

Its not going to be easy for the Indian team to fight the lawsuit since it is filed in New York, while they are based in India. Apparently some talks had happened between the two parties over a possible partnership (or acquisition) but that did not materialise. Facebook’s position is precarious as well. Under the Safe harbour provisions the DMCA, they are bound to take down the application failing which they could be dragged into a bigger litigation.

While I’m not an expert on this topic, I do have a working knowledge of DMCA. I think the use of the title “Scrabulous’ might have been a mistake by the Indian team and it might prove crucial in this case. The standard procedure adopted by internet companies in DMCA infringements is to either take down the disputed content straightaway if they think it violates IPR, or suspend it temporarily till a decision has been reached.

Question to Indian readers- without being jingoistic, whats your opinion on this issue?