August 2008

I decided that it’s been long overdue for us to do a comprehensive review of all the major blog search engines. Several planets aligned that made me decide this was the time.

1) The ongoing effort on the behalf of clients to feed our sentiment tagging engine with complete and accurate content as it appears. For our band monitoring projects it’s search results that get analyzed for sentiment, so we want to capture all the posts that matter.

2) To make sure we’re prepared for our big CES blogger bash, we need to identify all the bloggers who work the consumer electronics beat.

3) Splogs are threatening the social media ecosystem, and I am curious as to which engines are managing the problem well (or not so well.)

4) I just wanted to see how the MacBook Air stacks up against the new HP Voodoo Envy.

So far, 5 engines have been analyzed for the results the provide. More are being added.

We’ve categorized the results as mostly either “original content” (the good stuff) or “splogs” (the bad stuff) with a few other categories we’ll describe in a later post.

At this stage, let’s look at just the results deemed to be original content. See the graphic below. On the left you can see we have dozens of individual sites deemed “good” and what engines displayed them when I searched using HP Voodoo Envy along with Macbook Air terms.

Engadget content was captured only by Digg and Technorati, while Gear Live was captured only by Digg, FriendFeed, and Twitter.

Some quick conclusions can be reached here. One interesting one is that Technorati captured the greatest number of relevant posts. Note that since they offer the option to filter by authority, I took advantage of that and sought only sites with “a lot of authority.”

The big conclusion is for complete brand monitoring you can’t get away with using one engine alone. Even two won’t likely cut it. A bunch of sites were presented by Technorati that Google missed and vice versa. FriendFeed captured links that Technorati and Google both missed etc.

More to come…

Since we have such a uniquely flexible and accurate sentiment tracking system, we’ve been debating for some time whether or not to attach it to Twitter directly. Summize beat us to the punch with their algorithmic sentiment tracking system last year, and then Twitter acquired them — so we’ve feared competing with the “mother ship.”

Summize appeared to be great at analyzing large volumes of data, and is an impressive general twitter search engine. Unfortunately, like all other traditional text-mining systems chasing sentiment, it had sporadic but gaping accuracy holes that may have turned off those looking to do detailed brand monitoring. We’re speculating that this may be why after Twitter acquired them, the sentiment feature has been largely buried.

Joel Spolsky just posted at length today that he’s chosen to buy the Nokia E71 over the Apple iPhone 3G largely because of focused twitter searches:

When Apple’s iPhone 3G came out, I was pretty sure I’d get one. It had all the features I was waiting for. But the lines just weren’t going away.

I searched Twitter. For a week, then two, every day brought fresh reports of five-hour waits.

And then the reports of bugs started coming in. The Exchange synchronization features weren’t up to snuff, I heard. The phone crashed regularly, I heard. Basic operations were painfully slow. Battery life was abysmal.

Adam Curry suggested getting a Nokia E71. I had never heard of this thing. Nokia? Really? For years I had always thought that Nokia made chunky Europhones that were always just one button short of a usable user interface.

But, no, the more I investigated, the more it seemed that the E71 was a truly credible alternative to the iPhone 3G. The reviews coming in from Europe were stellar…

If stories like this don’t convince the luddite PR professionals that are still so prevalent, to get on the Twitter bandwagon I don’t know what will.

We’re in the middle of engineering several Sentimine tests for clients and prospects for new dashboards, services, and reports, but it seems like it’s time to put a Twitter sentiment system plan on the whiteboard. If anyone reading this has a concrete monitoring project (preferably brand, product or stock-oriented) they’d like to alpha test let me know. Steve AT parnassusgroup DOT com.

Here at Gnomedex everyone is wearing a big, colorful badge designed to match people up with like-minded others wandering through the hallway.

These badges are courtesy of Pathable, the group running the event registration and behind-the-scenes social networking that comes attached to a Web 2.0 conference nowadays. So the question is: does it work?

[click to continue...]

For years, we’ve been saying that search should largely drive the content (particularly headlines) that business bloggers choose to create.

Robert Scoble comments today on the useful Google AdWords Keyword tool that can assist in this effort. We leverage this and other free (and paid) services (WordTracker etc.) to help determine what subject lines will drive the most relevant readers to the blogs we create for clients.

One service we provide is a virtual “peek” into competitor’s server logs to see what searches are driving traffic to their site. This can tell us how successful they are, and give clients ideas for terms and phrases that may be profitable for them to post about.

As an example let’s take a peek at Robert’s blog. First, let’s use the service at tagcrowd.com to see what terms are frequently used by Robert in his content:

Now, let’s see what popular keywords are actually driving search traffic to his site:

Notice how many people come to his blog because they are searching for product/service information (Vista, iPhone, Facebook.) Also notice that Robert writes quite a bit about Techmeme and Twitter, but the resulting traffic is not proportional.

Here is an analysis that marketers might find informative. It shows what terms related to products, services, and companies are driving the most traffic to Robert:

One of the biggest issues facing the blogosphere today is how the “bad apples” in the PR industry chase after bloggers who are not relevant to their market or the products they’re pushing. We get tons of “story idea” emails that don’t remotely fit our editorial beat. If more companies leveraged services like these to better target the bloggers they pitch, there might be less vitriol sent in their direction.

Hey PR people — ping us if you want some help targeting your outreach. Leveraging this kind of analysis can help you get more traction in the blogosphere.

For marketing a political candidate this may be the case. This from the Economist (emphasis mine:)

Peter Daou, Hillary Clinton’s internet director, says that YouTube, even more than Facebook or MySpace, has had a huge impact on the campaign. Will.i.am’s ode to Mr Obama, “Yes we can”, has had nearly 9m views since it was uploaded six months ago; some 1.9m have watched the McCain Girls’“Raining McCain” over the past four months. Thousands of clips created by ordinary people have been uploaded, to be shared in blogs and often covered by traditional media too.

Both nominees have their own YouTube channels. Mr Obama’s videos have had 52m views, Mr McCain’s 9.5m. Mr Obama’s entire 37-minute speech on race in America has been viewed 4.7m times on YouTube. But the inflamatory sermons of his former pastor, Jeremiah Wright, have also been seen by millions.

There are a myriad of social media tools and services available on the web right now.

It can be tricky finding out which ones you should be paying attention to and, more importantly, how you can use them to generate measurable improvement in your bottom line.

There’s good news and there’s bad news when it comes to business in social media.

The bad news is that creating and maintaining customer relationships effectively takes time and effort, there is no magic formula that takes all the work out of it.

The good news is that it’s not rocket science. In fact, it’s a straightforward process that practically anyone can manage.

[click to continue...]

Monitor110 (now shut down) was a web monitoring service akin to our own Sentimine product, but with a Wall Street focus. They had developed an impressive prototype service that received some amazing press coverage. Several people — including one of the founders — are blogging like crazy about what went wrong, and I find it riveting.

One of my first impressions is that even with an investment of 20 million dollars, and a staff of dozens, they ran into several of the same key challenges we have, notably the difficulty of eliminating spam from results, and accurately tagging for sentiment (both issues we have been fortunate to solve quickly.)

It appears that since our engineers don’t believe that NLP is the default answer to most problems, we have avoided chasing the algorithmic chimera that others have pursued.

One thing we were sure of early on: if Google can’t solve the splog issue, it ain’t worth even trying…