I read a few blog posts lately about where the Web is going and they struck a cord. They managed to articulate more clearly some of the things that we felt intuitively when we started building ContextVoice / uberVU and were not able to clearly express.

It’s going to be a pretty long article, so you can skip to the short version/conclusion here.

Streams

The Web is definitely moving towards a stream-like, conversational structure. Twitter, Facebook, FriendFeed, all of these work as streams. More importantly, to emphasize the trend, feed readers and even Gmail are starting to look like streams/conversations. I find myself spending over 50% of my personal time online in a stream environment and, if you consider Gmail a stream-like experience, probably close to 80% of my work time inside streams.

There are two approaches to streams that, if you look at how we function as human beings in real life, are totally complementary.

Imagine meeting a friend outside a coffee shop en route to a meeting. The friend, with the context around her is an atom in a stream of inputs that demands your attention. You start talking to her about something, the conversation then suddenly focuses on something else. Your phone rings and so you focus your attention on the phone call, which is actually just another atom in the stream. Your meeting partner is going to be late. You then switch your attention to your friend and start talking about something completely different.

This is the way we operate in real life when we try to find information or socialize, yet the Web does not reflect that. Until now.

I’d associate the stream of information all around you with the way Twitter works, while the way we dig deeper into a certain subject / interact with a social object is more like FriendFeed works – or ContextVoice / uberVU for that matter.

Whatever the type of stream you experience, soon enough you realize you cannot consume it all, as it was not conceived for that. Navigation becomes more important that it has been so far with pages, because a stream changes continuously. It’s not just about finding some “relevant” information, like you do on Google, because that may not be relevant anymore in a changing stream. I’d argue that it’s not even just about finding “real time” information, like you do on Twitter search, because that only addresses the time axis. It’s a combination of the two that probably yields the best results.

How we think about streams

We started tracking comments from all over the Web around URLs because we felt a conversation does not stay trapped within a single site or domain and so you should not experience the conversation that way. The conversation flows between blogs, Twitter, FriendFeed and social networks but it’s essentially the same conversations.

Pages don’t matter anymore, it’s all about being able to tap into the right stream at the right time and filter it as you like.

What ContextVoice does is find related bits of streams (trackbacks, comments, tweets) from the larger stream, sticks them together and creates a new stream that we keep track of. We think it’s one way of navigating the larger Stream, by sticking together related pieces that are part of the same conversation.

As streams will be the fabric of the Web, being able to tap into streams to extract valuable information is going to be key. Navigating and participating in streams will be part of most web apps and will be used by both people and businesses. Everybody will be using this technology, just like e-mail and RSS. It’s already happening, with buzztracking tools and customer service on Twitter.

We can easily imagine applications tapping into streams to extract events, financial and health information, interests, relationships and a lot of other types of data, processing it and then pushing back to the Stream, to be consumed. This is where ContextVoice comes in providing applications with easily accessible ways of tapping the Stream.

Taking this to the next level, we believe search is probably going to still be the preferred way of finding information in the Stream. But I think of search not as it is today, but mostly as a combination of Tracking and Discovery. You’ll express what you want to know about, not necessarily now, but perpetually, and search system should be smart enough to pick up the appropriate pieces of the Stream, apply filters and deliver the right data to you, as a stream.

This is the approach we’re taking with uberVU. uberVU will be a search and analytics product, where you’ll be able to both search for conversations around URLs and search for keywords. The results returned will be whole conversations, not just posts or tweets containing the keyword. And, of course, you’ll be able to experience searching for a keyword as a stream of conversations updated in close to realtime.

We think this is an important distinction which ties into the role Context will play in the new Web.

The Role of Context

The Stream is made up of atomic pieces that constantly flow, with not much context to go by. Think of your Twitter stream – continuous updates on different subjects, shared links – each update does not have much context except the author and the time of posting.

People’s need for context became evident even on Twitter. That’s why we have @, RT or #. These symbols are trying to bring in a piece of context and encapsulate it in the atomic bit of information. So context is needed if we are to make sense of what information in the Stream means and where our attention should lie.

A more subtle implication of this fact is that context should be able to be incapsulated in the information itself and easily travel with the information wherever it goes. Of course, context changes over time and a piece of information can be looked at from within different contexts.

This is exactly why we built ContextVoice as a different product than uberVU.

Firstly, ContextVoice gets CONTEXT around stories from all over the Web. Instead of getting simple bits of information with no context around them, we get comments from all over the Web that are about the same story. This makes for some interesting information. You can see actual comments but also how they’re related to each other (RT, threads, comments to trackbacks), how they happened on the timeline, how fast the conversation has accelerated and decelerated, etc.

Secondly, as I said before, CONTEXT should be able to travel with the data. This can prove really tricky, as some conversations are made up of over 10,000 bits of information. In order to solve this problem, ContextVoice can return all the CONTEXT around a story in a single API call. We can’t encapsulate it in the data, as it’s not practical, but we have set it free and easily accessible.

Thirdly, we are not the only ones that will need this data. A lot of companies will use it, as we move more and more towards a more streamy Web. What we’re doing with uberVU is a single use case in a sea of possibilities.

Context can be used for many things. More context is usually better, but after a certain limit, I think context will only be useful in realtime search. People won’t be able to process it, but we’ll need it if we want to get relevant search results in realtime. As uberVU will mostly be a search product, we’re trying to get as much context around stories as we can, there no such thing as too much.

In Short

The Web is turning into a realtime Stream. We won’t be able to digest it all, so navigation will be increasingly important. One way to navigate the Stream will be through Search, but smart realtime search needs a lot of Context around the atomic bits of information if we are to find the truly relevant information we need from this everflowing, noisy Stream.

We built ContextVoice in order to get Context around stories from all over the Web. We get comments, mentions, reactions, tweets and other things, some of them in close to real time. ContextVoice is an API because of two reasons:

1. Context should be able to travel with the data. As we can’t encapsulate this much context within the data itself, providing it as an API call seems to be the easiest way.

2. As the Stream becomes the very fabric of the Web, being able to tap it will probably become like RSS or e-mail. Everyone will need this technology as part of their apps or businesses.

uberVU, the product that uses ContextVoice will try to tackle one of the problems of navigating this Stream – what you’d call Search and I’d call Track/Discovery. Finding fully contextual conversations (streams) to participate in, not just atomic bits that contain keywords.

Back to top

It’s been a long article, but if you’ve come this far, I encourage you to hang in there for a couple more seconds and share your thoughts.

Conversations are becoming the fabric of the social web. Millions of people are commenting everywhere about everything. With the rise of services like Twitter, this is happening faster and faster, in close to real time.

Tapping into these conversations and communities, although difficult technically, is not something new. There are dozens of companies out there offering buzztracking services for marketing, PR and advertising professionals based on these online conversations.

However, as social media has become mainstream and social business software has become more widespread, some interesting changes have started to happen.

1. First of all, social media tracking is becoming a commodity. There are a lot of free tools out there that are doing a good job at tracking all of this. Also, a lot of people are starting to use these tools, even outside of professional circles. People track their names, their friends, their favorite bands or companies on Twitter and on blogs. They’ve started to take social media for granted.
2. Secondly, as social business software proliferates, more people at lower levels of the organization are starting to make the use of social media central to their work. They’re talking to people inside and outside of the organization and need tools to track and manage all those conversations.
3. Thirdly, as Fred Wilson states, a new layer of social media is emerging. This new layer works on top of existing platforms (social networks, blog comments, Twitter) to provide valuable data, insights and analytics. As this new layer takes shape, the underlying data it’s based on should become easier to track in real time, making data gathering a commodity.

Right now every company that wants to tap into social media conversations either to create a new product or to use that data internally has to develop crawling software and hardware infrastructure that’s complicated to build and very costly to run. Crawling is not their core business, yet they have to do it. This is, of course, highly inefficient, like having every company building and providing its own internet connection or ERP software.

This is where ContextVoice comes in. We feel that social media conversations will become as widespread as e-mail, even inside businesses. Using social media daily in your work and personal life will take an even more central role. With thousands of conversations happening this second, crawling, analyzing, filtering and making sense of the data will not be easy, but it will be essential.

The ContextVoice API delivers close to real time conversations as a service, because crawling and filtering are just a necessary evil if they’re not the core business. We think social media conversations are a commodity that companies and individuals should be able to tap into cheaply and effectively and that’s our purpose.

Techcrunch reports that Google Docs is getting more and more like Microsoft Office. And soon it will have all the most important features of its desktop counterpart. That’s fine but I believe Google has chosen a wrong path.

You see: at uberVU we love Google Docs. We do most of our writing inside Google Docs. And we do it because of the sharing capabilities, not because of the extended set of publishing features. We don’t care about all the fancy features that Google is adding to make it look and behave more like Word. We don’t care about those because we rarely print anything that we write from inside Google Docs. However, we do publish a large percent of the documents. Online. Blogs, Twitter, Facebook, Email, Online Magazines etc. And a web page is so… so different than a printed page that I am wondering: Why doesn’t Google make a good editor … for the online world?

Most people see the distributed web as a chaotic and uncontrollable space. We see it as a big opportunity. And it is not just us. Bellow is an article by Daniel Waterhouse (sector partner at 3i Venture Capital).

One of the enduring and real elements of the web 2.0 discussion is the new way in which online businesses are reaching their audiences. This has spawned some buzzwords, such as ‘open APIs’ and ‘widgets’, and has added a layer of complexity into building an online business. Those who succeed in harnessing these techniques command a competitive advantage. As an investor I am always looking for teams which react fast to changing market dynamics – effectively utilising the distributed web is one such test of reaction speeds. But what does this all mean?

In the first iteration of the web, it was enough to publish your web site and try to drive people to it, to use your service and buy your goods. Perhaps you closed a few deals to get other sites to carry a version of your service and you certainly spent some money on advertising. All of these channels to get people to your site cost money – unless you stumbled upon the ingredients of word-of-mouth marketing which a few notable sites (e.g. Google) did.

In the past few years, however, several high-profile examples of new ways to get your user in front of your product have emerged, including open APIs and widgets. APIs are application programming interfaces that allow people to interact with your product/service/data/technology.

Opening up part of it to people outside of your organisation may sound like a recipe for losing control. However, when done successfully it can be very powerful. The best example of where this works well is Google Maps. Littered across the web now are Google Maps embedded in other web sites. The site owner has not made a deal with Google but has just utilised the open API it provides. In return, Google receives free marketing for its product on thousands of web sites (and will most likely start embedding adverts into the maps at some point and make money from this free distribution).

Widgetising your product involves allowing the mainstream internet user to embed your product into their personal page (blog, MySpace page, etc). YouTube is a great example – on every page is an ‘embed’ code which users can insert into their pages. YouTube gained huge traction early on via users viewing videos on MySpace pages which contained YouTube embeds. Several other large businesses have been built in this way, where the user interacts with their product on a blog or MySpace page.

This new form of distribution is very cheap and can be incredibly powerful. The downside is the inevitable loss of control and also dependence on the policies of the likes of MySpace which have been known to ban certain embeds. Using these channels effectively will become a core competence of many online businesses, especially as audiences fragment to more niche locations in the ever expanding world wide web. The companies that exploit this most effectively will be successful and we at 3i look forward to talking with them in the future.

The article was published in MicroScope magazine.

Perspective

Not long ago, you could hardly express yourself on the Web. You lacked the tools to do that. Blogging came along and changed all that. Then you wanted to share your newly discovered web life with your friends. Social networking came along and helped you do that. Once social networking became the norm, you wanted to be able to take advantage of the data that lied in the very social links that formed on the Web. Call it social intelligence, crowd sourcing, whatever. Some social networks opened up and let you build and use applications that used the social connections on top of which they were built to bring you more value. Poking yourself isn’t that much fun, after all.

It’s simply the way we do things in our daily live, only now it’s on the web. We talk to the people around us, now we ca express ourself just as easily on the Web. We know who our friends are and we usually meed friends of friends in real life. We can do the same on the Web. And we go shopping, we plan trips, we go to concerts, we read books. We do all of this socially, with input from our friends in our real life. And now we do the same on the Web.

But what if we could go further? What if we could use the Web in new exciting ways so that we can do things that it’s difficult for us to do in real life? Think about the next possibilities for a moment…

You have a favorite restaurant where you go to all the time. They know what you like and how you like it. But you go on vacation to some distant place. What if the restaurant of the hotel you’re staying at could know what the restaurant back home knows about you? Would they be able to serve you better? Would you have a more pleasant experience there? Probably. Or imagine that you like to order in. You have 2 or 3 dishes you’d like for dinner. But now that you’re on vacation, you can’t find the 3 dishes at a single restaurant. You find them at 3 different ones. What if there could be a delivery service that knew what dishes you liked, and could get them for you, even while you were on vacation?

What’s the next step

These are just simple examples that show one thing: We have the platform to create and deliver new, exciting services, but we lack the data about the users most of the time. That data exists somewhere in most of the cases. The problem is that it’s locked in.

So the next step in social networking, I argue, is letting go of the data. Facebook knows who my friends are (on Facebook) and what I’m doing there. LiveMocha knows who my friends are (on LiveMocha), and what I’m learning there. It would really be useful if these two social networks could communicate. And I’m not just talking about having all my friends in one place, or a single friend social stream, or a single friend request inbox. These are useful features too.

Just to make things clear, I’m not talking about Data Portability here. Data portability is just the first layer. The second layer are the applications that will be created, and the third layer are the social implications of those applications. I know people are working on data portability for quite some time, and we’ll get there soon. What I’m talking about is how we’ll be using that data, once it’s free. Portability is just the beginning of it all.

What I’m talking about is LiveMocha finding out that I’m planning a trip to Iceland this summer. I’m going with some friends on Facebook. Now LiveMocha could offer me an Icelandic 101 language course. It could recommend some people that are Iceland natives. Not just to teach me the language, but to give me some information I might need, or even to hook up while I’m there. At present this is a scenario that simply can’t happen.

Why we should do it

First of all, I believe that the data is ours. Maybe not all of it, but who my friends are, how I’m connected to them and what I’m talking to them about is mine. I should be able to do whatever I want with it.

Secondly, this is probably the most effective way to get new, great services that haven’t been possible before. It may even be the only way. Just look at Facebook Apps. There are thousands of them out there. Facebook alone would have had a really tough time creating all of them. And there are thousands more to come. Only by letting the community contribute can we get new applications that we haven’t seen before.

Thirdly, this can spark competition. A lot. It’s really hard to start a new social application that requires a lot of people to sign up. If they are already on a social network, half the problem is solved. But just think what we could do if we could take the social graphs from several social networks, combine them with some other open services (maps, search, photos, calendars, etc.) and get really cool new services? Without an open infrastructure, it’s really hard for a startup to handle all of these problems. Using networks and platforms that already exist and that are open could help a lot of new startups take off.

Fourthly, technology is getting to a point where it’s becoming transparent. We surf the web to get information, meet people and discuss things. Why should I care if some interesting people are on Facebook and if some great discussion is taking place on Yahoo! Groups? I just want to meet the people and take part in the conversation. The “location” and the technology behind it all should be of no interest to me. Opening the data could make these technologies transparent to us.

These are only a few of the reasons that support opening up the data. I’m sure you can find many others.

What do you think the challenges might be in doing this? And what cool services would you like to see, that aren’t available yet because of this problem?

Let’s face it, nobody posts content to just one site anymore. You probably have a blog, a social network profile, you post videos on YouTube and you keep your pictures organized with Flickr. The people you want to touch, your audience, don’t come to your site anymore. They hang around in communities, so you have to get to them.

That’s why you probably post videos to YouTube, then embed them on your site. You don’t post them on your site or host them there. The simple fact that your videos are on YouTube gives you access to a great publishing platform and to a great potential audience. Here’s the problem we have with this approach.Just because the content is now distributed on a variety of sites, platforms, widgets and so on, that means your attention in handling conversations also has to be distributed.

You just got back from a cool trip and you want to blog about it. You upload the pics to Picasa or Flickr, Post some cool trip videos to YouTube.
Then you embed some of them in a blog post. Now everybody knows about the crazy fun you had on the trip. But if you want to see people’s reactions to your post, to your pics and videos, you have to visit Flickr and YouTube over and over again. Although the pictures, the videos and the blog post are connected by one main meaningful theme, you can’t see them all in one place. And you can’t see the way they are related.

This means that, just to keep track of the conversations you start, your attention is all over the place. That’s not very fun, you waste a lot of time and it’s hard to keep track with what’s being said. We believe that, since you are the creator of your content, you should be able to manage it properly. It’s your digital life, and in a sense, you’re now spending as much time managing it as you spend living it. As more and more ways to create, publish and remix appear, you’ll need a good way to manage all you create more and more.

Welcome to the future, where bits and pieces of your digital self are scattered all over the place. Don’t let them get lost, they are parts of you.