Jeetu's shared memory

Yesterday Patrick Galbraith asked What is the official branch of MySQL? which got a lot of attention, including on Slashdot (and the token PostgeSQL comments quickly appeared).

Here’s the funny thing. It doesn’t matter anymore. Patrick’s question is interesting in an academic sense, but it’s mainly a distraction from what really matters. (Hint: What’s the official Linux and who really cares? Ubuntu? RedHat? Debian? CentOS?)

Storage Engines

Nowadays what matters is the set of available storage engines. InnoDB, Percona’s XtraDB, PrimeBase’s PBXT, Maria, Falcon, and several others are available or will be soon. I predict that for the foreseeable future, any MySQL distribution or derivative must support the storage engine plug-in API that MySQL 5.1 defined. And since that’s the case, it largely won’t matter which flavor you using.

Protocol(s)

Look at what’s happened in the world of key/value databases in the last few years. More than a few of them speak the memcached protocol as either their native and default or an optional add-on. I suspect the same thing will be the case here. All MySQL distributions and derivatives will speak the “traditional” MySQL protocol (just like memecached has the old protocol). Some of them, notably Drizzle, will have other (newere, better) protocols available as well (much like memcached has the new binary protocol).

Summary

In summary, the choice of MySQL version or derivative won’t matter as much as you might think because they’ll have the same Storge Engine plug-ins available (thanks to the shared plugin-in API), they’ll all speak a common protocol (this may not be true for replication–watch that area closely), and will largely offer the same subset of SQL and SQL extensions.

They’ll all be supported by different groups/companies (including some “database appliance” vendors), will all be tuned differently and aimed at slightly different uses cases, and will certainly benefit from a lot of cross-pollination.

That doesn’t sound so bad to me.

The fact that nobody can point to the “real” MySQL in a few years just won’t matter. Does anyone ask (anymore) which is the “real” Linux? Nope. And for very similar reasons. Think of MySQL as “kernel” and Storage Engine as “filesystem” and you’ll realize we’ve been down this road before.

We’re looking at the upgrade from 5.0 to 5.1 soon at Craigslist and don’t know if we’ll be using InnoDB or XtraDB yet. Time will tell.

See Also: The New MySQL Landscape, which I wrote a few months back–before a good chunk of the MySQL team had left Sun.

(comments)

Most things on the web are free and in abundance. Something that bucks this trend is data about internet usage, web traffic trends, specially what’s dependable and comes from a credible source. In my day job with SlideShare, this factor has caused me so much heartburn, that it’s just not funny!

Come Akamai and their quarterly internet traffic trends are a treasure trove of insights into whats running in the world’s internet pipes. They release this every quarter and it is free. Check out the Akamai State of the Internet report for Quarter 4 (Q4) that has just been released. To read the report, use the full screen button on the bottom right or head to slideshare to download your own copy.

Much as I hate to copy/paste directly from press releases, this is one press release that needs to be quoted verbatim. Check out the India specific highlights…

* India ranked #20 globally for number of unique IP addresses seen by Akamai, with 2.63 million IP’s
- Up 42.91% year-over-year, and 1.69% from Q3 2008
- In comparison, United States was ranked #1 with 114.1 million unique IPs

* India’s average connection speed was at 772 Kbps and ranked #115 in terms of average connection speed
- Globally, the average connection speed was approximately 1.5 Mbps
- The United States ranked #17 globally, with an average connection speed of 3.9 Mbps while South Korea ranked #1 with an average connection speed of 15 Mbps

* India ranked #17 globally in terms of attack traffic, with 1.16% of observed attack traffic
- In comparison, United States was ranked #1 with 22.85% of observed attack traffic

* India ranked #148 globally for number of unique IP addresses per capita, with 0.0023

* India ranked #62 globally for high broadband adoption, with 0.56% of connections to Akamai at speeds over 5 Mbps

* India ranked #81 globally for high broadband penetration, with fewer than 0.0001 high broadband IPs per capita in Q4 2008

* India ranked #93 globally for broadband adoption, with 3.74% of connections to Akamai at speeds over 2 Mbps

* India ranked #118 globally for broadband penetration, with 0.0001broadband IPs per capita in Q4 2008

* India ranked #55 globally for narrowband adoption, with 25.8% of connections to Akamai at speeds below 256 Kbps

So what do you think of the above figures? Makes sense?

Something that seems hard to swallow is “India’s average connection speed was at 772 Kbps”. Can this be really true or it is likely to be a biased sample. For Akamai’s data is likely to be based on the CDN traffic which is flowing through their network and it can be argued that CDN usage is probably confined to high end usage (web 2.0, media sharing etc).

I posed this question to Akamai and their reply is thought provoking. Akamai says that CDN usage being top end in India is a myth for they are serving 20% of the Internet traffic, 29/30 top media companies, 40/40 top internet retailers and 6 of the largest media houses (in India). Being global leaders, they are in a unique position to track global traffic and they see every IP address in International once in a week. Since users always access the top sites atleast once in a quarter (which seems like a reasonable assumption!), they are able to track everyone and their connectivity speed. The 772 kbps figure may be in part attributed to the proliferation of most of the IP addresses from workplaces and cybercafés.

Googlers Alon Halevy, Peter Norvig, and Fernando Pereira have an article, “The Unreasonable Effectiveness of Data” (PDF), in the April 2009 IEEE Intelligent Systems on semantic interpretation using big data.

Some excerpts:

The number of grammatical English sentences is theoretically infinite … However, in practice we humans care to make only a finite number of distinctions. For many tasks, once we have a billion or so examples, we essentially have a closed set that represents (or at least approximates) what we need.

We’re left with … interpreting the content, which is mainly that of learning as much as possible about the context of the content to correctly disambiguate it …. What we need are methods to infer relationships between … entities in the world. These inferences may be incorrect at times, but if they’re done well enough we can connect disparate data collections and thereby substantially enhance our interaction with Web data.

Unlabeled data … is so much more plentiful than labeled data … With very large data sources, the data holds a lot of detail. For natural language applications, trust that human language has already evolved words for the important concepts. See how far you can go by tying together the words that are already there.

The article talks in more detail about work at Google and elsewhere on extracting relationships from massive crawls of text, tables, and the deep web.

On a related note, Google announced some new features a couple days ago, improved query suggestions and snippets, that Googler Ori Allon apparently described as scanning pages “in real-time … after a query is entered” and identifying “conceptually and contextually related sites/pages” using “an ‘understanding’ of content and context.” Many news articles are referring to this as a step toward semantic search.

Please see also my April 2008 post, “GoogleBot starts on the deep web“, which discusses related work by Alon Halevy on mining data in tables and the deep web.

Please see also my post on the WSDM 2008 keynote by Oren Etzioni on semantic interpretation. His work is mentioned a few times by Halevy et al.

[IEEE article found via the Google Research Blog]

Amazon shutters warehouses for first time since 2006

By Eric Engleman on March 26, 2009 at 12:00 PDT

Comments (0)    Permalink

Bad news  e-commerce  AMZN  Amazon.com

Amazon.com is closing a trio of U.S. distribution centers, the first time time it’s closed a warehouse since 2006. Amazon said the closures in Munster, Ind., Red Rock, Nev., and Chambersburg, Pa. are part of a reorganization of its fulfillment network — moving capacity to larger warehouses that can “better balance product mix and customer orders.” The distribution centers slated for closure employ some 215 people.

Here’s more from Amazon spokeswoman Patty Smith:

A variety of factors went into the plan to close these three buildings. We added approximately 3 million square feet of capacity in the North American fulfillment network in 2008, and will be converting a fulfillment center in Phoenix to accommodate our larger items (such as big screen TVs) and adding several hundred thousand sq feet to that facility. This additional capacity allows to the opportunity to better balance product mix and customer orders across the network year-round.

The 215 workers at the three warehouses slated for closure will receive pay through May 25, receive benefits through May 31, and get a minimum of 3 weeks severance pay, according to Smith, who said “eligible” workers will be offered the chance to transfer to other U.S. distribution centers.

Smith said Amazon now has “dozens” of fulfillment centers, but didn’t provide the exact number. Amazon on its website says it has fulfillment centers in 12 U.S. states and 8 foreign countries.

Amazon seems to be paying a lot of attention to its warehouses lately. Amazon CEO Jeff Bezos recently spent a week working in a fulfillment center in Lexington, Ky.

The closure of the Muncie warehouse was first reported by the Post-Tribune of Indiana, which noted that the facility opened just 18 months ago.

Digg needs to sell itself. Kevin Rose‘s headline-voting site is drowning; the more popular it gets, the more red ink it generates. But who needs a bunch of news stories rated? Here’s an idea: Amazon.com.

Sure, start scoffing. But Digg’s past acquisition talks with Current and News Corp. failed in part because they looked at Digg as a media play, and community-generated sites like Digg aren’t particularly attractive to advertisers. More recently, Digg and Google got close to an acquisition. That deal fell apart, according to a source familiar with the talks, because Google wanted to closely probe the quality of Digg’s engineering staff early on in the deal, and Digg did not relent until talks were well along. (Digg CEO Jay Adelson refused to comment on the company’s talks with Google.) The lesson: Digg’s not a media company, and not a technology company. It’s something else altogether.

Who makes money off of online community? The surprising answer is Amazon. One study suggests that Amazon.com makes $2.7 billion — billion! — a year in incremental sales because of its user-written reviews. Amazon uses the simple mechanism of asking shoppers if a review was helpful to rank its reviews.

It’s remarkably similar to Digg’s option of “digging” or “burying” a news story. Where might that be useful? Amazon.com’s Kindle e-book reader. In addition to selling digital books, Amazon already charges for some news feeds available for free on the Web. Magazine and newspaper editors are delusionally optimistic that they might be able to charge by the article on a device like the Kindle, through a scheme of micropayments.

Micropayments have been technically possible for more than a decade. The problem has always been consumer behavior: How do you know if an article is worth paying for? The time spent pondering that question isn’t worth the nickel people hope to charge for it.

But what if you didn’t have to ponder that question? What if you knew, through Digg’s rating system, that a large number of people had read the story and given it a thumbs-up?

An Amazon-owned Digg wouldn’t have to charge for access to its website or the stories it links to; indeed, that would be against its interests, since the rating activity on Digg requires free access to work. The Wall Street Journal even gives Digg users free access to its stories so they can read them and vote.

Instead, Digg would charge Kindle users for a new service which delivers a personalized newspaper to the device — a service far quicker and simpler than the cumbersome process of going to Digg.com and scrolling through endless lists of popular headlines. They’d only pay for the stories they read — which in turn would provide more valuable feedback on what Amazon can charge for. The payment would be essentially voluntary, since readers could always pull up publishers’ websites and read the stories for free there — but they payment would be more for the simplicity and ease of use, rather than the content itself. (Arguably, that’s why people pay for music on iTunes rather than download it from file-sharing networks.)

Is Amazon.com thinking about such a move? We haven’t heard anything about talks between Amazon.com and Digg. But, intriguingly, we heard whispers that Amazon.com is talking to Twitter. Amazon CEO Jeff Bezos is a personal investor in Twitter. Presumably, the attraction would be the same: getting some kind of real-time pulse on what people are interested in.

But Digg’s focus on headline voting and Amazon’s push into news distribution make them seem like a better match. Will Bezos dig the idea?

(Photoillustration by Richard Blakeley)

KOLKATA: Angry students atIIT-Kharagpur on Sunday forced the institute’s director to resign after athird-year student of electrical engineering

died of medical neglect.

Rohit Kumar was left unattended for three hours at the premierinstitute’s BC Roy Hospital

after he sustained head injuries from a fallduring a basketball game, the students claimed. A profusely bleeding Rohit wasfinally referred to SSKM Hospital in Kolkata – a three-hour drive away – but hedied on the way.

The institute director, Damodar Acharya, resignedafter angry students ransacked his house and refused to lift the siege till hetook responsibility for the death.

“We tried to make themunderstand that there would be a proper inquiry to ascertain the cause of deathand if there is any negligence on anyone’s part, action would be taken. Ialso told them that I would try to improve the infrastructure at BC RoyHospital. But they insisted on my resignation. So, I resigned,” Acharyatold TOI over the telephone. He has reportedly sent his resignation to the boardof governors.

Rohit suddenly fell unconscious while playingbasketball with his friends in the morning. His head hit the ground as he fell.Friends rushed him to the hospital on campus but the doctor on emergency dutywas absent. “When the doctor was finally located, he recommended a CT scanand special attention by a neurosurgeon. Unfortunately, such facilities are notavailable here,” said a student.

The hospital advisedRohit’s friends to take him to Kolkata and an ambulance was arrangedthough no doctor or nurse was available to accompany him. Two friendsvolunteered, but on the way, when Rohit started vomiting blood, they took him toMidnapore Medical College and Hospital, where he was declared dead.

As the news of Rohit’s death spread, at least 2,000 studentsgheraoed the director’s residence and ransacked it. They refused to liftthe siege till the director accepted responsibility. Initially, a small policeforce led by the deputy superintendent of police reached the campus to help liftthe agitation. Later, a bigger force was sent by SP Manoj Verma, but thestudents locked the campus gates and refused to let the cops in. “Weshowed restraint because we were dealing with students,” Vermasaid.

“Despite our repeated pleas, no measures have been takento ensure that doctors and equipment are made available at the hospital. Onweekends the situation is worse as doctors do not report for duty. Even the oneon emergency duty has to be called in if a patient is brought to thehospital,” the students complained.
They also alleged that thehospital lacked even basic medical facilities

. “If you check the stocks ofmedicine, you’ll find that most are past their expiry dates!” theyadded.

Rohit’s classmates and friends at the Lala Lajpat Raihostel claimed he had never had such fits in the past.
When contacted,Rohit’s father, R P Sahni, was on his way to Kharagpur from Darbhanga,where the family lives. He said he had received news of his son’s death at2 pm, but didn’t know the details. Rohit was the younger of his two sons.

An extremely jovial person with a large number of friends, Rohit wasalso one of the brightest in his class.
Deputy director MadhusudanChakraborty said: “There is trouble on the campus surrounding astudent’s death and we are looking into the matter. At this stage I willnot be able to comment any further.” Insiders, however, said Acharya hadgiven charge to Chakraborty.

Santa Singh died and went to heaven. 

When he got to the pearly gate
Saint Peter told him that new rules were in effect due to the advances
in education on earth.

In order to gain admittance a prospective
heavenly soul must answer two questions:

1. Name two days of the week
   that begin with “T”.
2. How many seconds are there in a year?Santa
   thought for a few minutes and answered…

1. The two days of the week that
begin with “T” are Today and Tomorrow.

2. There are 12 seconds in a
year.

Saint Peter said, “OK, I’ll buy the Today and Tomorrow, even
though it’s not the answer I expected, so your answer is correct.But
how did you get only 12 seconds in a year?”Santa replied, “Well,
January 2nd, February 2nd,March 2nd, etc….”Saint Peter lets him in
without another word”

Creating Amazon widgets just got a whole lot easier. You can now create the most popular widgets in a variety of standard IAB sizes. We’ve introduced the following new features:

1. Search for widgets by size

You can now search for widgets that fit a specific IAB ad slot on your web page or blog. Navigate to the Widgets tab in Associate Central and click on ‘Search by Size’ in the left menu. Or navigate to https://widgets.amazon.com/Widgets-By-Size/. Then choose an IAB size from the ‘Select Size’ drop down to view supported widgets.

2. Select a preset IAB size or create a custom-sized widget

For the My Favorites and Search widgets, you now have the option to select a predefined IAB size or create a custom-sized widget. Choose IAB sized widgets when you have a predefined ad slot. Experiment with custom-sized widgets when you have more control over the layout of your web site.

3. Available in additional IAB sizes

The most popular widgets are now available in additional IAB sizes:
•    My Favorites is now available in 300×250, 336×280, 180×150, 160×600.
•    The Search widget is available in 300×250, 336×280, 160×600

Get started by clicking on the Widgets tab in Associate Central or visit https://widgets.amazon.com/.

Yahoo is cutting more fat today by closing its travel bargains website FareChase, which it originally acquired back in July 2004 and re-launched two years later. The company will be announcing the shut-down later today, and will start redirecting visitors of the service to its main travel site soon.

The service let customers perform comparative searches for pricing on flights, hotels, cruises and cars, but it was apparently not enough of a strategic product enhancement for Yahoo Travel, hence the company discontinuing it altogether to tighten its focus and cut costs in these difficult times.

Sounds like a plan to me.

According to Travel Weekly, Yahoo signed a new agreement with partner Travelocity in late February 2009, ensuring that the latter could continue its role as the primary booking engine for Yahoo Travel. In the past, Travelocity sparred with Yahoo over the prominent role that Yahoo gave FareChase on Yahoo Travel.

This is the latest deadpool decision from Yahoo in a long series of announcements.

The company had previously sold off shopping engine Kelkoo and shuttered online storage service Briefcase, photo sharing service Yahoo Photos, social network Mash, live video streaming service Yahoo Live, Ads in RSS, web-based video editing service JumpCut and student community website / job board KickStart.

Which service could be next on the chopping block?

Crunch Network: CrunchBase the free database of technology companies, people, and investors