The long-suffering blog indexing service is struggling to cope with service outages, scalability problems and now an irate blogosphere. What can be done to make Technorati better?Technorati, the Weblog popularity index, received the blogosphere's equivalent of a curse last week.
"Pretty much everyone I talk to in the industry thinks the site sucks and we've just been waiting for it to get better," wrote the popular blogger Jason Kottke on his Web site. "Well, I'm tired of waiting. Goodbye Technorati
your URL will darken the door of my browser no longer."
Kottke is ranked the 15th most popular blogger on the Internet by Technorati's own scale, but with that post he became the company's most powerful detractor. His sermon generated over 32 follow-up posts and hundreds of comments from other blogs, and Technorati is still scrambling to repair the damage to its reputation and traffic.
Since David Sifry founded Technorati in 2002, millions of bloggers have depended on this constantly updated search engine that tracks blog posts, measures blog popularity and searches out new blogs.
At the same time, blogs have multiplied like radioactive rabbits, overwhelming the start-up resources at Technorati.
Is there a market for RSS beyond bloggers? Click here to read more.
Technorati's problem is one of publishing as much as of technology, since its service is largely considered to be one of the major ways for bloggers to advertise their new posts. Technorati has in effect become the clearing house for amateurand even professionalcontent producers on the Internet.
While Technorati tried to ameliorate the damage done by Kottke's post last week, concerned bloggers gave Publish.com some ideas for how to improve the beleaguered Web service.
Stop adding more data
"The obvious answer is to add more hardware," Kevin Burton said in a telephone interview. "If that's not working, they should stop loading data until they ramp up the service."
Burton has spent the last nine years in the Bay Area programming scene, writing code for the RSS aggregator at newsmonster.org and rojo.com. At both jobs, Burton struggled to create code that could sensibly cope with a growing number of users"scaling" the program to cope with an enormous amount of data.
"I'd rather have a Technorati that was fast and always worked, even if that meant only indexing 1 [million] blogs," Burton wrote on feedblog.org, responding to Kottke's post.
Does blogger influence need to be regulated? Click here to read more.
According to Technorati's most recent figures, the company indexes 1.2 million new blog posts every day, monitoring linkages between 16 million blogs. Sifry announced that Technorati now has 400 machines working in the data center, struggling to scale search results to match this overwhelming posting rate.
Keep the service consistent
Paul Scriven, founder of the Web design company 9Rules.com, summed up Technorati's most glaring problem in his Whitespace blog: "It rarely, if ever, works, and those times you are fortunate enough to get it to work, it is really slow."
Blog readership has increased dramatically over the last two years, often overloading Technorati servers. The company's "please try again later" screen infuriates users accustomed to light-speed connections.
Scriven begged the company to remedy these service disruptions, and over 10,000 impressionable Technorati users read his post. Sifry visited the 9rules.com blog in August, posting promises and his phone number to placate Scriven's readers.
Next Page: Less bling means more speed.
Less bling, more speed
"I'd recommend they drop the flashy shading, icons and curves in the site's design in favor of a more efficient, faster and less bandwidth/processor-expensive design," Web designer Nick Lewis said in an e-mail interview.
Lewis works as Webmaster at the Metropolitan Austin Interactive Network, an evolving open-source computer network in Texas that aims to unite city and civic agencies under one virtual roofanother daunting scaling task.
While Lewis railed against the "lynching of Technorati" in his blog posts, he agreed that the service needs some infrastructure work. Lewis said he felt that Technorati's site is too crammed with images, logos, lists and headlines.
Click here to read about how search engines and bloggers are fighting spam.
Instead, he urged designers to create a simpler and elegant interface that would allow Technorati to handle larger data loads, just like Google's white-patterned homepage.
Thou shall not commit Leaderboardism
Matt Mower has been working with Web design and blog searches since 2002. After developing blogging tools for Radio Userland and Evectors, Mower said, he decided that the blogosphere is threatened by "a misguided obsession with blog popularity rankings"a problem he terms Leaderboardism.
According to the Technorati Web site, the company focuses on the "perceived relevance" of blogs, a goal that can conflict with the attempt to comprehensively track a billion links.
Loaded with "Top 100" lists of books, movies, and blogs, Technorati tends to measure Web traffic in a way reminiscent of popularity contests between teenagers.
In Mower's view, he said, popularity has no relationship to the utility of a particular post, and the company should devote coding efforts and computer space to bringing exposure to unknown blogs instead of plugging the famous blogs.
Repair the URL-cosmos function
For the last three years, Daryl Sng has blogged about his life as a Harvard-educated DJ living in Singapore. At dsng.net, Sng expressed another familiar blogger gripe: "Technorati is getting worse and worse at its primary function of tracking down what bloggers are saying."
His concern revolved around the "link cosmos" function of Technorati, the tool that allows bloggers to discover how many people have linked to their blogs. This is one of Technorati's primary goals, to connect bloggers with similar interests.
As the blogosphere expanded too quickly, Technorati struggled to index all the flickering lights in the Internet heaven. Like the Hubble Telescope with a broken mirror, Technorati is full of potential, but delivering grainy pictures of the blog cosmos.
Since Kottke's dramatic post, Sifry has visited countless blogs to soothe his customers. In his personal blog, Sifry publishes periodic "performance and scalability" reports that address some of the issues mentioned here, and most bloggers respond to his messages with giddy enthusiasm.
Without addressing Kottke's blog post directly, Sifry tried to rally his troops one more time: "[Technorati] has gone through growing pains" he wrote, "[We] are working very hard to live up to your expectations."