Topsy.com Providing a Better Twitter Search

Topsy, new Twitter search site, was launched in May 2009. Rankings for social media data is purely based on the number of times the link has been tweeted, for whatever time window you select: hour, day, week, month, all-time. When grouping by URLs, they resolve compressed versions from bit.ly or tinyurl. The resolved link, along with the number of tweets, and the text of a recent tweet, are displayed. Along the right margin, you get a list of common users and the number of times they’ve contributed to the result set. You can also search within a particular user.

If you click on a tweet group, you’ll get a list of posts, and a list of “What’s Related”. The quality of the related tweet groups is a bit sketchy. Tweets are short, not much to go on. You could imagine just pulling out the most statistically overrepresented words from the a tweet group. For the baseline you’d want to calculate word frequency from a lot of tweets, not a corpus of standard written English. Tweets have their own language. Take the top few words, throw them into a full-text search engine, and group the results by their resolved URL. As I mentioned before in a post about real-time search, its not terribly complicated. The tricky part is getting a feed of tweets from Twitter. Once you have it, you pull every URL. For each one, do a HTTP HEAD request to get past bit.ly, tinyurl and the like. That will save you some time and bandwidth over downloading the entire page. Perhaps have a short list of URL-shortening services and use those to filter. Some of them, like bit.ly, also provide an API where you can expand the shortened version without doing the HEAD request.

You have the URLs, timestamps, usernames, and tweet text. Since you’re grouping by URL’s, you might also consider only indexing tweets that have links — that will reduce the size of your indicies. This is not rocket science. I was baffled by their $15MM purchase of Summize. I suppose in the case of Summize a premium was paid to get it quickly. According to CrunchBase, Topsy raised $15MM. Of that, $11MM was raised last December. I wonder why they need all that money? Right now they list 14 people on their web site, and their product feels like a $4MM product, what they have so far is not terribly complicated even when implemented for scale. Perhaps they have big plans for the future, or anticipate big hardware bills and low revenue in the short run.

So although I like Topsy very much, they are dependent on Twitter for their feed of tweets, and whatever current value you might give to their enterprise, most of that value is in their access to the complete tweet feed. Twitter controls that, and can shut it off, so they don’t seem like a good target for acquisition for anyone except Twitter. I don’t see any patent applications on file at the USPTO. Or as with Summize, they might hope to fetch another premium for getting a search product out quickly. Twitter already did that once, I doubt they’ll do it again, they already have search expertise in-house now. Perhaps they want to get a big user base, and hope that Twitter won’t have the guts to shut them down — negotiating with someone who’s holding a gun to your head, and hoping they don’t want to bloody their shirt.

You don’t raise $11MM without a plan, even with all the crazy hype over real-time search. That money probably put the finishing touches on their current search offering, but will largely be going towards new products that would be better targets for acquisition. Their investors have experience with mobile technology, perhaps they have a clever way to move this onto mobile devices.

The environment, gas prices, etc., are giving drivers more incentive than ever to consider the alternatives, it’s a good time to review all the celebrities who have already taken the plunge with the most famous alternative: the Toyota Prius Hybrid.

Celebrity Toyota Prius Drivers…

Among the first famous Prius drivers is early adopter Leonardo DiCaprio, who’s been driving one since 2001 when they were new. Then Cameron Diaz got one. Then, like, EVERYBODY got one. “Everybody” includes usual green and/or lefty celeb suspects like: Alicia Silverstone , Arianna Huffington, Brad Pitt, Julia Roberts, Bill Maher, Prince Charles, Woody Harrelson, Harrison Ford, Ed Begley, and Susan Sarandon.

But now the Prius is common enough that less-expected Prius owners in Hollywood number easily in the dozens (maybe more like hundreds), including Kevin Bacon, Jack Nicholson, Billy Crystal, Larry David , Ewan McGregor, Robin Williams, David Duchovny, Alexandra Paul, America Fererra, Billy Joel, Dr. Andrew Weil, Ted Danson, Will Ferrell, Salma Hayek, Patricia Arquette, David Hyde Pierce, Ellen DeGeneres, Tom Hanks, Rob Reiner, Hart Bochner, Donny Osmond, Kurt Russell, Jack Black, Donna Mills, and Kirk Douglas. The jury is still very much out on whether the Prius is the best (or even a good) green car option, but it is certainly the most recognized.

Celebrity Ford Fusion Drivers… American Idol runner-up Adam Lambert loves his new hybrid. Lambert says, “Hybrid all the way. Let’s protect the environment.” He may not have won first place on American Idol, but runner-up Adam Lambert is happy anyway, especially since he was given a Ford Fusion Hybrid. Lambert and winner Kris Allen each received hybrid cars from the Ford Motor Company.

Backstage after last night’s American Idol finale, Lambert was all smiles, psyched about getting to sing with Queen and KISS. Asked if he was happy with his Ford Fusion, Lambert replied,

“The Fusion is beautiful. It’s so whisper quiet. Hybrid all the way. Let’s protect the environment.”

Hey, rolling through Los Angeles in a Ford Fusion hybrid is a pretty good consolation prize!