Friday, December 14, 2007

Web Next & Data Ecosystems

Web Next is not some quantum leap in reality but rather the culmination of several long term trends. Web 1.0 was the translation of real world services (e.g. Amazon) onto the Internet and Web 2.0 is Darwinian evolution of UI and social media tools and technologies. Web Next is exploiting of information to achieve new products and services with no direct analog in the real world. Some will call this Web 3.0 but I prefer the simpler moniker Web Next.

Web Next is about the creation of value not through the control of information but via the creation of synergies and knowledge through combining information and functionality. There already exists the primitive examples in the Web2 world, those such as the map-based mash-ups. Essentially value is derived via the showing a spatial relationship between data. However, these mash-ups are relatively primitive. They rely on a users existing knowledge of the spatial area in question. I personally have no appreciation of the real layout of New York City having never been there. Consequently, the value I gain from viewing or using a mash-up consisting of crime statistics plotted on a map of NYC is less than someone who has visited which will be less than a resident of NYC.

The synergy of information and functionality is created through Data Ecosystems.

Data Ecosystems

A Data Ecosystem is two or more different data sets that when combined with complementary functionality produce multiplicative effect in usefulness. Or put another way, a Data Ecosystem contains more than one source of data (a data set) (e.g. temperatures and rainfall) that can be combined, analysed and processed with the overall Data Ecosystem being more valuable than data or functionality on its own. Importantly, having more than one data set is not sufficient on its own. Rather you need various tools and functions that allow the user to act on the data. An example will help to clarify.

Take an individual piece of data, say a series of temperatures measurements. On its own you can't do much with those temperature measurements but combined with rainfall measures, annual growth rates and a map, those temperature measurements suddenly have a lot of value. Now a farmer can research and plan when to plant his crops or adjust his crop forecasts based on historical growth rates versus temperature and rainfall. To be able to make the forecast of crop tonnage the farmer needs a series of tools that allow him to find correlation factors and extrapolate the growth trends based on rainfall and temperatures. Without those functions having the data is not particularly useful. There is no use in having gobs and gobs of data if there is poor functionality in the Data Ecosystem.

Data Ecosystems highlight a very interesting point about data. One that I find is continually ignored or not understood by most data companies (including web companies). Data on its own has little intrinsic value. Data only has value with what you can do with it and what you can do with it is determined by what other data you have along with the functionality you can apply to the data.

Like a biological ecosystem, a data ecosystem must mesh together. A Data Ecosystem needs to be internally consistent. If a Data Ecosystem is not consistent then it will not generate value for the user. There is no point in trying to create a Data Ecosystem that has rainfall patterns from Australia and crime statistics in New York City. Designers of Data Ecosystems must not design the systems so they become inconsistent. Consistency is crucial. But given human nature I fully expect consistency will be ignored.
"And there's the sign, Ridcully," said the Dean. "You have read it, I assume. You know? The sign which says 'Do not, under any circumstances, open this door'?"
"Of course I've read it," said Ridcully. "Why d'yer think I want it opened?"
"Er...why?" said the Lecturer in Recent Runes.
"To see why they wanted it shut, of course."
-Terry Pratchett, Hogfather

At this point I expect some readers will be thinking that Data Ecosystems is simply the Semantic Web. Data Ecosystems is not the Semantic Web. Semantic Web technologies will be a part of Data Ecosystems, but Semantic Web is neither a precondition for nor sufficient on its own in order to build Data Ecosystems. Semantic Web helps by automating building the relationships between bits of information. In the temperature example above Semantic Web would have provide information such as the lat/long of the measurements, how it was measured, the accuracy of the measurement, the date and times of the measurement etc. This would then allow a computer to match the data automatically with rainfall data from the same location and time and plot together on a map. Semantic Web makes building Data Ecosystems easier and like objects in programming will allow Data Ecosystem platforms to increase the ease the deployment of Data Ecosystems.

Data Ecosystems can also be built of other Data Ecosystems. The output of several Data Ecosystems can be used as the sources for another Data Ecosystem and so on. Each step creating more value by allowing an individual to achieve more. Data Ecosystems will in effect create an L-space

Why are Data Ecosystems Important?

Data Ecosystems are important for one very, very crucial reason. Data Ecosystems allow people to achieve things effectively. Unlike Web 1.0 which was essentially removing transaction costs from existing real-world processes, Data Ecosystems unlock the potential for new services that are impossible in the real world.

Consider a Data Ecosystem based travel service. Such a service will allow you to research a holiday; book all transport, accommodation and activities; create a comprehensive itinerary of the holiday, send alerts at key points along the trip; calculate how much money you'll spend on the holiday; help you automatically tag video, audio and photos from the holiday and create holiday memorabilia from the items you have uploaded. All through a single Data Ecosystem.

And there are hundreds, thousands, millions of probable Data Ecosystems that have no analog in today's web.

How Things are Already Changing

To close out I want to consider something that has been banging its way around the blog-sphere and offline world: the fate of journalism. Without re-hashing the debate you can read Bill Keller's speech with Jeff Jarvis's responses here and here as background.

If everyone is a citizen journalist, then what is the point of professional journalist? A seemingly valid question but one that has the implicit assumption that both are or will be doing the same process. From the perspective of Data Ecosystems the job of a professional journalist becomes very different from a citizen journalist. The citizen journalist is a source of data. They will most likely only provide a very narrow bit of data on any particular story. Put another way, the citizen journalist becomes a source like the news wires.

The professional journalist moves on from being the source of the story to gathering all the disparate bits of information about a story and then assembling into a consistent and cohesive context around the core story. Professional journalists go form being the gate keepers to information to value builders by creating context to stories. The role of a newspaper/media company is to provide or create a Data Ecosystem within which the professional journalist can assemble, create and publish the context to stories. Within Data Ecosystems, professional journalists, news agencies and citizen journalists will co-exist and combine to produce a more valuable service than either would on their own or exists today.

The media world is already going through pain as it is forced to adjust to the realities of an information economy. Data Ecosystems provide a means to effectively adapt to the information economy. But Data Ecosystems are not limited to media. Data Ecosystems will exist right across the information world. In fact they will reach into material world as L-space is linked into materials at the molecular level. This is the true revolution of Data Ecosystems, they facilitate the merger of L-space and Real Space.

Tags: Web Next, Data Ecosystems, Web 3.0, Web Services, Semantic Web, Web 2.0, L-Space


young fogey said...

"The professional journalist moves on from being the source of the story to gathering all the disparate bits of information about a story and then assembling into a consistent and cohesive context around the core story."

-- an almost perfect definition of sub-editing, which is good news as it should mean that there is more of a role for the cynical, curmudgeonly production journalists in the brave new world.

This prediction, however, seems to ignore the fact that most newspapers are cutting their production departments to the bone and most broadcast organisations are forcing reporters to self-produce to some degree. Are they missing a trick?

Simon Cast said...

I think so. Most of the media organisations are still very stuck in concept of being the harbringers and gatekeepers of news/information. Jeff Jarvis of Buzzmachine blogs on the issue of media companies missing a trick a lot.

His post on the 2nd of Jan Halfway Round the Track. Covers one such instance and what the media companies could have done and benefited from.

I should have being a little more clear in that sentence i think. The journalist will still write stories but their job goes from being that single task to the large role of bringing together a much greater context as Kara Swisher has been recently posted about when discussing her own career