Digital Small Business and the Web

A digital economy is far more than simply having lots of internet start-ups. A digital economy is as much about the use of technology as creating technology. The problem with a lot of the conversation around knowledge and digital economy is this requirement is lost as people focus on creating the next Google.
Just as much needs to be done to have small business taking advantage of technology to improve their business as trying to fund new technology companies. It is important to understand the issue here. To have a truly digital economy, small business has to use technology and particular web technology to their greatest potential.

This covers how a business interacts with their customers, with their suppliers/partners and how they manage information internally. You could split these up but to really make a difference small business needs to integrate via workflows (I elaborate on workflows here) not through insiders and outsiders paradigm.

For example, instead of plumbing company having one booking system used by employees to book appointments and another for website bookings rather use a service like BookingBug that both customers and employees can use to make bookings.

But that is only one step. Let’s look at the whole workflow. Basic plumbing workflow is:

1. appointment is made
2. plumbing task is done
3. payment is made.

A customer comes to the site and books an appointment for a plumber. The BookingBug widget shows appointments that have already been booked either by other users on the site or by staff. They pick an appointment and fill in the details. A plumber is sent the information to their iPhone with the details (with the iPhone representing smartphones). The plumber does the job and fills in an invoice via Freshbooks App on the iPhone. This is sent immediately to the customer who can then choose to pay via the payment application on the plumber’s iPhone.

The invoice details are logged to Xero, the accountancy software, which is then followed up with the payment details when payment is made. Any supplies consumed during the plumbing task are sent to the supply management software which logs the usage and re-orders as required. The details of the job and correspondence with the customer are all logged in 37Signals' Highrise.

By bringing in effective web technology (be it SaaS, mobile applications or whatever) into the workflow the administrative burden is reduced (no one had to copy numbers from one system to another) while improving customer service. It allows the plumber (small business) to focus on the plumbing and not on coordinating moving parts.

All of what I described in the example can be done now. It isn’t something for the future but something for now. The glue that binds them together are APIs. It is APIs that allow the requisite data to move between the various applications and create the seemless workflow.

The key to a digital economy is effective use of technology in small business; something that is sadly lacking in the various grand plans for digital economy from Governments and interest groups.

Turning Book Publishing Inside Out

Image via WikipediaContinuing the series of collaboration and coordination platforms (C&C platforms) I wanted to look at a more concrete example of how cc platforms can change an industry and how they work to turn existing businesses inside out. My example will look at book publishing.

Book publishing has a range of steps to produce an end result (a book in the shops that is purchased by a consumer). Let’s look at the steps:

1. Author a book
2. Find a publisher willing to publish your book
3. Edit the book
4. Make changes to your book
5. Create Cover art
6. Print and bind book
7. Market Book
8. Ship book to stores
9. Distribute revenue

Now I realise it isn’t necessarily as smooth as the list makes out and some of the items happen in parallel, but for purposes of the example it works. Steps 3 to 9 are currently the realm of the publisher. That arose as a publisher was the only one who could organise the resources needed to complete steps 3 to 9 at reasonable cost.

But with Collaboration and Coordination platforms this is no longer true. C&C platforms reduce the costs of coordinating the various activities that you don’t need a vertically integrated publisher to achieve a sold book.

For an author to publish their book, the C&C platform would provide them with access to a market of editors, designers to create cover art, print-on-demand services, book marketing services and payment features. The C&C platform doesn’t provide all the services rather it organises them and reduces the transaction costs of using services from multiple providers.

Say I am an author of a book. I upload my manuscript to the C&C platform. I then begin the workflow of publishing the book with the system creating an alert to editors that a new manuscript is ready for editing. The editor that I select then makes the edits and uploads the changes to the manuscript; I review and accept or push-back on until we arrive at something both are happy with. I then need to create the cover art and the system again creates an alert for designers. I select a designer and they get access to the manuscript so they can create relevant cover art. Once the cover art is agreed, the art is uploaded to the system and I progress to the next stage.

I now have a book ready to go, so I need to pass the manuscript and cover art along to a POD provider. This I do with a simple click on a button loading the book into the system of the POD provider I have selected. This also puts advertising and listing of the book into key retailers and I can begin the marketing of the book. It may be that I need support for the marketing which I can source through the platform as well.

The C&C platform guides the user through the steps necessary to complete a task (publishing a book), handles the necessary communications, makes sure everyone is paid and manages the media. It coordinates the various markets for each service so that together they can achieve a task. C&C platforms reduce the transaction costs to the point that a Firm in the Coaseian sense is not the most effective manner in achieving a task.

Related articles by Zemanta

• Publishers Beware: Amazon has you in their sights
• HarperCollins to Authors: Sorry, no money... but we will publish your book
• Thinking About Self-Publishing? Read This Book

Answering API requests or not (QOOP)

Image via WikipediaOk in this day and age, it shouldn't, nay, mustn't take 2 business days (and counting) to respond to a request for an API key. Unfortunately, that is what QOOP is doing.

I fail to understand why they do not have an automated system for generating API keys so developers can get started integrating QOOP into their sites. Even if the key is restricted in usage, the verfication of commercial intent can happend later. But it is vital that any company offering an API make it dead simple to get started. Doing otherwise is simply the creation of a needless barrier to entry and will drive potential partners to competitors that don't have that barrier.

Honestly, this is simple stuff. You can't rely on human vetting in an environment like the internet. People do not scale, automation does.

Technorati Tag: QOOP

In support of James’ Cloud

Image by Simon Cast via FlickrJames Governor recently did a re-run of his 15 Ways to Tell Its Not Cloud Computing with a post addressing some of the critical reaction he has got in a post 15 Ways I Am Wrong About Enterprise Cloud Computing. I broadly support James’ original thesis and I’ll explain why.

Definitions of Cloud Computing abound and some are really wordy and well confusing. When I think of Cloud Computing I keep it to the following definition:

“Cloud Computing abstracts the where and how of computing to allow users to focus on the what”

By this I mean that developers no longer need to worry about the details of how computing is delivered or where the computing is located (i.e. what server) instead they can focus on making sure their application achieves what they want and is reliable.

So a Cloud is more than simply a grid or utility computing as it also needs to support software stacks without the developer worrying about how it is done. A full on cloud negates many of the low level management requirements and simply provides computing and storage resource that is on-demand and easy to use, like booting an OS and running an application.

Now what we will have is internal and external clouds to an enterprise. Think Internet versus Intranet. The reason for deploying an internal cloud is to reduce the hardware capital inefficiency most enterprises face along with providing internal developers access to the benefits of Cloud Computing will still meeting the desire for security and control of data and applications.

Companies like Sun, IBM and HP will rollout “cloud-in-a-box” that allow Enterprises to replace existing hardware with Clouds. Will the enterprise own each server that makes up the “cloud-in-a-box”? Probably not. Instead they will own the “cloud” with a maintenance contract that sees the vendor swap out the hardware regularly to keep the computing capability of the cloud growing.

The reason for enterprises to deploy internal clouds is simple – it increases capital efficiency of IT while allowing developers and system administrators to focus on the application and less on keeping a mass of hardware and low-level software running and up-to-date. External clouds will work for many businesses that don’t need massive internal applications. It isn’t a really an either/or proposition.

James notes at the end of his reply they he is half-right and half wrong. I agree with the caveat that I think he is more than half-right and less than half-wrong. Until enterprises can tick off his 15 points they will not be taking full advantage of the potential of Cloud Computing.

Related articles by Zemanta

• 15 Ways I Am Wrong About Enterprise Cloud Computing
• Giving The Manifesto A Haircut: What is Cloud Computing?
• Web 2.0 and Cloud Computing

Optimisation of Workflow and Collaboration Platforms

Image by Simon Cast via FlickrOptimisation of workflow is the aim of the game. We want to reach the goal with as little expenditure of resources as possible. Unfortunately, much of the optimisation game has been played at the level of the individual action, which usually results in a destabilised system. This post in the series on collaboration and workflow will look at how workflow should be optimised and the role that collaboration platforms can play.

Reasonable question to start with is why optimise? Optimising or improving workflow increases the throughput. More simply, optimising workflow means more gets down with fewer resources. For business it means they can focus on producing the most value without wasting resources.

Optimisation of workflow is not about making a single action overly productive but instead about balancing the various actions in a workflow to produce the best overall throughput. It is about making the system robust rather than optimising for a particular scenario. “The Goal” by Goldratt provides a useful case study on optimising workflow.

Actions within a workflow can be re-arranged, removed, melded together and improved. The key is the modification of actions within the workflow all needs to focus on improving the overall workflow throughput. This may mean that while an individual action’s throughput can be increased from say 80 to 90%, there is no point in doing this if it does not increase the overall throughput of the workflow.
So where do collaboration platforms come in? Collaboration platforms have two functions (1) they serve as a framework within which to improve workflow and (2) they offer a way of improving individual actions.
The improvement of individual actions is a tried and tested use of collaboration frameworks. Think parallel editing of a client document and the management of tasks for a project. The improvement of individual actions is a well developed use of collaboration platforms but this only works so far as it does not optimise the action in context of the wider throughput of the workflow in question.

The framework aspect of collaboration platforms is very under developed and in terms of overall impact on business this is where changes will have the most dramatic impact on a workflow. The collaboration framework allows users to optimise and control actions within the context of the overall workflow.
The idea of a framework is to allow users to build a workflow from individual actions, examine how this workflow works and selectively change (add, remove, optimise) actions all with the aim to improve the overall workflow. It is like using plant control software to change various flow rates and values in order to change the amount of a chemical produced.

To illustrate what I mean let us look at the example of putting on an event. An event requires the coordination and completion of a series of actions such as booking and managing the venue, managing attendance and paying various entities. Each of the activities would be arranged as required into a workflow with the collaboration platform ensuring the smooth handoff between activities. None of the activities need to be powered by the platform rather they are coordinated and controlled using the collaboration platform in order to achieve the goal of the workflow. Using real world companies the venue would be booked and managed through BookingBug, RegOnline handles the registration and attendance management, Moo.com produces the tickets and ID, PayPal is used to manage payments to various entities and Huddle coordinates all these activities and manages the communication and information between the event organisers. The event organiser can focus on creating a compelling event that runs smoothly.

It is the coordination and control of actions that produces the dramatic improvement in workflow and consequently the value created by the business. As workflow now and increasingly extends across multiple organisations coordination is key to ensuring that workflow is as effective as possible. At the same time, as collaboration platforms improve the coordination of workflow then increasingly workflow will become made up of various groups working on the actions to which they add the greatest value (think of the example above). It is a positive feedback back cycle – improved collaboration increases the value of various groups working together which in turn drives improvement in collaboration and so on.

We are seeing the rise of workflow specific platforms such as Amiando and RegOnline in the case of event management. I suspect this trend will continue but there is a lack of flexibility to this approach. The real revolution will happen as the current crop of collaboration platforms along with new entrants evolve towards workflow coordination platforms that support the plug-in and specialist modules. Most groups require more than a single workflow to operate and I expect single workflow services such as Amiando and RegOnline will work within generalised coordination and collaboration platforms.

Understanding Workflow

In the first post, Collaboration Reformation, of this series on collaboration and workflow, I made the statement about the transformation of collaboration from resources to activities. It is worthwhile looking at activities, or more accurately workflow, specifically before moving on to further discussion on collaboration and workflow.

Workflow is essentially a series of discrete actions that when group together produce a desired outcome. The obvious example is a manufacturing assembly line. A series of actions such as screwing on a door and adding an engine are arranged together in a line in order to assemble a car. Manufacturing assembly lines are obvious but don’t become hooked on the assembly line example. Workflow is simply a series of discrete actions performed together to achieve a goal. The goal can easily be the development and roll out of new features for a web service as it is the assembly of a car.

Goals can range from a product or service (say a car or a massage) to something more intangible (say increasing support for a candidate). For most of the post I will focus on the product and service goals but it applies equally to intangibles as well.

Workflow can be broken down into three categories: operational, development and overhead. Operational is the workflow that delivers the goal. Development is workflow that is necessary to create, improve or fix a goal. Overhead is workflow necessary to keep the organisation and group going in order that it can achieve the operational and development workflow. There is overlap between the three categories of workflow and you can represent it as a Venn diagram.

The workflows necessary to complete a goal are unlikely to be fully contained within a single group. A car maker doesn’t make the bolts or wires that go into a car. Toyota recognised this which is why The Toyota Production System works to coordinate workflow in suppliers not only within a Toyota plant.

Each workflow is made up of lots of different actions. Optimising an action without consideration for the workflow de-stabilises the whole workflow and produces counter-productive results. It is no good having one action of a workflow produce more than can be processed by downstream actions. In “The Goal” Goldratt provides a series of good examples of what happens when optimising a single action versus optimisation across the workflow.

In the next of this series, we’ll look at how optimisation can be achieved and look in more detail at how collaboration platforms are a part of this optimisation.

Keywords from Questions

In a recent article Google Search Quality Tech Lead Daniel Russell talks about an example of a user using keywords to find ferry timetable. What struck me as interesting was how the user didn’t hit upon using the keyword “ferry” until further in through their search task.

I suspect this was caused by the user starting with a question with words to the effect of “When does the ferry leave San Francisco to Larkspur?” and then attempting to turn this into a series of keywords by knocking out works such as does, when etc. The word “ferry” got knocked out of the user’s first run of keywords as it was a generic reference to ferry. In this case ferry was thought of as a common noun rather a proper noun.

If my hypothesis is reasonable then quality of keywords is going to depend on how the user first structures the question in their mind. For example if the user had used the following structure for the question “When does the San Francisco to Larkspur Ferry leave?” the word “ferry” would have been used as a keyword.

The potential importance to the way a user structures their initial question mentally points to a severe limitation to keyword and ranking search paradigm. The speed and quality of the search experience is heavily dependent on the user structuring the initial question so as to readily identify effective keywords, something that the search engines can do little to effect.

On the other hand question and fact paradigm based search engines, such as True Knowledge, will not suffer this problem.

Micro-startups in a Collaboration and Coordinating world

Image by Simon Cast via FlickrIn Jason Calacanis’s recent email missive, he explores where the value for start-ups lie. Two points “The Age of the Micro-startup” and “The Try Everything Era” touches on the long on-going battle between features versus products. Put succinctly, many start-ups are little more than a feature (albeit useful) and in and of itself not a sustainable product.

Jason’s theme is that the high capital efficiency of today’s web will allow features to blossom and expand on existing services. I remain sceptical of feature companies’ (micro-startup in Jason’s terminology) as a standalone going concern but I see the value that these micro-startups can create in a collaboration and coordination world.

Collaboration and coordination platforms will enable micro-startup’s to create value by increasing the overall value of the platform by adding functionality. The advantage for the micro-startups is that the value of their feature is increased by being coordinated with various other features allowing users to achieve a goal. The value of the whole is more than the sum of its parts. Further they get access to a framework that simplifies coordination and business operations (e.g. getting revenue).

For collaboration and coordination platforms also benefit from the multiplicative effective as well as allowing features to be added to the platform cheaply and in response to demand. While they could build a lot of the features, the development resources needed that would limit what and how quickly new features can be rolled out. Micro-startups form a eco-system that is self organising about what to build and the resources to devote to it.

This differs from existing platforms, Facebook and OpenSocial, in that these platforms are about building micro-apps that have little to no coordination with other applications on the platform. The applications are standalone. These platforms essentially act as a hosting service with access to a social graph.
Coordination and Collaboration platforms are about coordinating actions (features) in order to achieve something. Achieving a goal is going to produce greater value in the long run and produce more value producing companies than simply tapping a social graph.

Resources versus Answers – Asking a Question of Search

Image via WikipediaSearch is very broad in meaning and it is easy to lose sight that search actually consists of two distinct sub-sets of queries. Both sub-sets aim to find something; one is looking for resource and another for an answer. At this time we use the same approach – keywords matched in a document that is ranked for relevancy via some method (human and/or algorithm) – across both of these sub-sets of queries. This works somewhat but we are rapidly approaching the limit of effectiveness for this approach. This limit is Marissa Mayers 80/20 problem of search.

The first sub-set is finding resources (e.g. documents). The current keyword and ranking method works well for this type of query. This is what has fuelled Google’s growth. Keyword and ranking when a user is looking for one or more resources on a topic such as blog posts talking about an election. Where it falls down is answering specific queries such as “How old is the Eiffel Tower?” The user in this case is looking for a fact. Users have gotten around this problem by using the returned resources from a search as the basis to find the answer they are looking for, a human adaptation to a systemic problem.

Finding answers is the second sub-set. While we currently rely on keywords and ranking to navigate to an answer it is cumbersome and not effective. Instead the paradigm of keywords and ranking needs to be tossed out. Finding answers works better with a question and fact. A question (as opposed to queries) allows the system to quantify what fact is being asked about. For example the question “How old is the Eiffel Tower?” focuses the particular answer to be found to the age instead of potentially the location, who built it, what it is made of etc.

Using the question and fact paradigm to find answers creates new approaches to using web services and usefulness of the web to everyday life. This isn’t to say that question and fact will replace keyword and ranking rather it is complimentary and produces better results for a sub-set of search.

Consider the example of finding flights for a holiday. Using keyword and ranking the user would type in something along the lines of “flights cheap [destination]”. The engine would then return a series of web sites that match those keywords. The user then navigates to those pages and then drills through the pages to find the answer to their question. If, however, question and fact is used the user would type in “What is the cheapest flight to [destination] leaving on the 21st of December?” The web then returns the fact that flight y priced at x leaving at 10 am on the 21st is the cheapest flight. How much quicker and easier is that to understand?

For many people the web and search are still too difficult to use. But they know how to ask a question and this opens up the utility of search and the web to a whole range of users that are intimidated by it. It is worth repeating that question and fact will not replace keyword and ranking. There are queries with which question and fact doesn’t work for just as there are queries for which keyword and ranking doesn’t work for. They are complimentary.

Question and fact does have the potential to boost the growth of paid search results. The boost arises from question and fact providing a better signal of the user’s intention and so improves the targeting of advertising that better answers the query. For example a user asks the question “what is the cheapest holiday for a 16 year old girl in Mexico?” it a very reasonable to assume the intent is to find a holiday for a 16 year old girl in Mexico. A keyword and ranking would produce results about holiday’s in Mexico without any knowledge of whom or why he is searching although an assumption could be made that the person is looking for themselves. Interestingly, through in demographics and/or behavioural data and the system will produce completely the wrong answer. Say for example the person is 52 year old male in which case the system is likely to return Mexico holidays for a 52 year old man when his intention was to find a holiday for his 16 year old daughter.

Question and fact will go a long way to addressing the 20% of search remaining. Many web services implement crude methods for asking a question, ones that are frankly laborious and time consuming to use. The key to unlocking the power of question and fact is to make it as easy as possible to ask the questions. The pitfall to implementing question and fact is knowing when to use it. Question and fact works when the question can be answered by a fact e.g. “How old is the Eiffel Tower?” It doesn’t work when the answer is not a fact e.g. “What is the best holiday in Mexico?”

The Collaborative Reformation

Collaboration has become the buzz word of the times. It holds out the promise of making all work productive all of the time, an attractive carrot in times of economic crisis. While collaboration tools will improve the effectiveness of existing processes and businesses its true impact and most dramatic effect lies in how collaboration tools can bring about a reformation of business.

Improving the efficiency of existing process works up to a point and indeed most, if not all, collaboration tools are predicated on somehow improving the efficiency of existing processes. The real promise of collaboration tools is how they can help users reform the fundamental processes of businesses. I’m not simply talking about getting rid of layers of approval but the complete overhaul of how new work is brought in, how it is created, how it is charged and how it is produced.

The impact comes from allowing business and users to focus on the work that adds value and streamline and eliminate the non-value add work. Elimination may involve out-sourcing to another business where the particular process or work is their value-add. Think designing cover art for a new book. It is not a value add proposition for a publisher but is a value-add proposition for a designer.

The reformation extends beyond simply managing documents and information to completing the core tasks of the business whether it is a plumber, development agency or a manufacturer. Collaboration services need to be given access to the physical world that plays such an important part in many businesses whether it is tasking plumbers and ordering plumbing supplies for delivery or controlling a CNC machine.

In effect collaboration services need to evolve into a framework within which business operates; a framework which supports agile business processes, modules for specialist features (think CAM control) and management of information within the business. This is the path for development of collaboration services such as Huddle.

The ultimate goal is supporting the ideal of the networked business. A “Business” is a network of smaller businesses using a common collaboration platform with specialist modules from various providers. Business becomes a network of networks in which the collaboration service coordinates activities. It is this that is the root of the dramatic and sustainable change that collaboration services can bring to business.

Can Hubdub Survive?

I’ve been playing with Hubdub recently and all I can say is it has a looong way to go. In fact I think that unless some major changes are made Hubdub won’t survive 2009. Unfortunately, I think Hubdub faces some major, major hurdles, that will make the struggle to build a decent revenue stream (remember now cash is king...angel funding will only get them so far).

Hubdub for those that have not come across it is as predication market based around news. The idea is to combine news aggregation of some sort with predictive markets.

I’ve put down bets and created questions. My consequent experience has been less than heart warming. But to illustrate my concerns let’s look at the questions I created.

My first question was “Will Digg buy Hubdub in 2009?” Speculative yes, but Hubdub is a predictive market – questions are by nature speculative. Background to the question: Kevin Rose discussed Digg’s international expansion plans in his talk at FOWA London. This talk was widely reported in the media. Some other facts:

• Digg has just closed a funding round of $29m in September
• Hubdub has only raised angel funding and has 4 employees
• In this economic climate cash is king. Getting revenue positive is the holy grail
• Hubdub has not articulated a source of revenue that is sustaining

From these data points (which are all easily available with a quick search on the web) one concludes that Hubdub is a good target for acquisition. It has a decent (although it requires some work) prediction platform but other than that it has nothing special. Kevin Rose wants to expand internationally and the prediction market technology would work well with the Digg platform. It would certainly give Digg greater number of potential revenue streams. I will be the first to admit that this is speculative but it is based on facts. But all it took was one person to raise a question and the question was voided.

Second question was “How far will UK house prices fall by December 2008?” This question was voided as it didn’t have an option for housing prices fall being below 15%. Let’s look at the logic of this. As of September 2008 both Halifax and Nationwide have reported house price falls of 12.4%. There are three more months to go before the end of December and these things don’t turn around on a dime. Being below 15% is not an option as credit is still tight and the UK has entered recession. There is no sound possibility that house prices will not fall less 15%. Now let’s look at this from angle of question creator. I may not want to provide an option so why should I? Why do questions have to be modified to provide gamblers with an option they want?

The issues with the questions are merely symptoms of what I see as major flaws of the Hubdub system. They are:

1. The rules for creating questions are vague and easily open to interpretation.
2. The very act of creating questions is daunting and annoying
3. The site is rapidly becoming dominated by power users

Hubdub is seeing the play out of Clay Shirky’s maxim “A Group Is Its Own Worst Enemy”. Power users and early adopters will band together to void questions and in other ways hassle newbies merely because they don’t like the question (rather than it’s predictive quality) or because newbies are falling afoul of capricious unwritten rules. There is no penalty against power users for this type of behaviour. This is over and above the effort needed to create questions in the first place. It is extremely disconcerting to put the effort into questions only to have them void on relatively spurious grounds.

The single most worrying aspect about the flaws in Hubdub is everyone has been talking about the development of communities and their interaction for the last 11 years. Let’s recap – Usenet went through the same problem, Slashdot went through the same problem, Digg went through the same problem. See a pattern here?

Clay Shirky has been shouting from the roof tops about it for years. Hugh McLeod and Tara Hunt have all discussed it. At what point do people pay attention? Angel investing or not, for a service that is based on community and users, to not have the necessary tools needed to manage the community’s interaction with the platform is simply, well, scary. It speaks to a company that has a fundamental lack of understand of community base services.

Is it important? Very. Hubdub needs a diverse, large and vibrant community not only of speculators but also question creators. The way Hubdub is going to make money is from selling premium access to data and audience to businesses. However, companies will only pay if the Hubdub community is diverse and large as then the data and audience has value to them. As it is the current community is doing very well at driving new members away. Hardly a good method of growth.

Hubdub can possibly turn this around. The first step is to build the tools and features necessary to manage the community interaction. This will piss off the power users and early adopters as it blunts their power. That is the price to pay for improving the experience and engagement for a broader and more diverse people. Actions must have consequence. When actions have no consequence poor behaviour soon dominates. Slashdot found this out the hard way as has Digg.

The other part of the engagement issue is question creation. Relying on people to read FAQs about question creation is very, well, RTFM. Most people don’t RTFM and nor should they. If there are rules about question creation they need to be clear and objective with no room for abuse to void questions that someone doesn’t like. Of course, if the rules are clear and objective the system should not allow questions to be created in the first place that don’t meet those rules. People should not have to RTFM.

In fact, I wonder why voiding is necessary at all – isn’t the very act of betting on a question a vote on the question's quality? Why not just use the activity on the questions as a way to surface or subsume questions? Activity is a much better method than voiding or voting. Using activity blunts the prejudices and power of any single person or group of people.

The interleaving idea through the emails from Hubdub and site of not being a speculative market still stumps me. It’s a predictive market they are, by definition, speculative. It’s like being a fish and trying not to drink the water. Hubdub is a speculative market – if the problem is with questions that don’t have any news or are a long time in the future have a special section for these types of questions. Don’t ban them.

I did hope that Hubdub would be good. But I am sorely disappointed and I now doubt the company’s survival. I certainly would not invest any money in the company without some major changes to the platform.

Widgets, Communities and the Edge

The web is making it easier and easier for groups and communities to form. Groups foster social cohesion by having members demonstrate affiliation and by the use of objects to create community identity. Think Star Trek fans wearing Star Trek uniforms at conventions or fans of Metallica wearing Metallica branded tee-shirts.

Unfortunately web based methods of indicating affiliation don’t really translate to the real world. This is important as groups are increasingly rooted in the real world, indeed traditional line between cyberspace and the real world is becoming increasingly blurry.

Personalisation services offer the ability to create physical objects that indicate affiliation and community identity. These services are centralised and therein lays the problem. By being centralised they impose a coordination cost on the groups.

Widgets offer services like MOO.com and Ninjazoo the opportunity to offer personalised and communitised products directly into the community without getting in the way. Widgets provide a means of removing the coordination cost on groups by meshing the service within the normal activities and sites of the group.

It is taking the mountain to Muhammad rather taking Muhammad to the mountain.

It is the distribution of core functionality where the true value of widgets lies. Not with the distribution of content but allowing web services to adjust to an Edge Economy.

Tags: MOO.com, Ninjazoo, Edge Economy, Web Services, Web 3.0

Privacy Filters and Facebook

In my previous post I used privacy in Facebook as an example of how data filters could work. One point I glossed over was how currently Facebook, indeed all social sites, fail with social distance. Unfortunately, social distance is a necessary for privacy filters to work satisfactorily.

Facebook has one major flaw, once a person is a friend in Facebook they are treated the same as all other contacts whether the connection comes from bumping into the person at a pub or someone you grew up with. It collapses the privacy or social distance between two people. The social distance can be considered how strong the connection between two people is. Social distance provides a measure of both strong and weak ties as articulated by Mark Granovetter.

Without some measure of social distance or strength of connections, any privacy filter is going to fail. The social graph fails to represent the real world connections between people properly.

Facebook attempts to use groupings of friends to approximate social distance but this is cumbersome to use. The manual nature of setting up and categorising everyone into groups is a major barrier to use. People are lazy.

What is needed is an automated method for calculating social distance. Social distance is calculated (and this is how Mark Granovetter categorised connections) by the frequency of communications. Measuring frequency of communications is difficult for Facebook. While Facebook can measure wall posts, internal emails, poking etc., so much more of our communication occurs outside of Facebook, outside of the wall; whether through email, IMs, phone calls, SMS, twitter parties attended etc.; that the frequency of communication within the wall is not a reasonable approximation for the wider frequency of communication.

The key measure of social distance – communication – is hard to quantify as it is dispersed through many different channels. Trying to capture the frequency of communication via porting the data in is one method of dealing with the issue. The other, probably more realistic, method is to start off with some rules and use what can be easily quantified to refine the measure of connection strength overtime.

The rules would look at what is known generically about social connections. Some of rules are:

1. Married is a strong connection
2. The same surname is a strong connection
3. If strong connections to friends with which you have strong connections then you probably have a strong connection

Some of these rules will dictate a very strong connection (first rule) while others will dictate varying strengths dependent on factors such as prior connections with other friends (third rule). All connections start as very weak and are refined first by application of the rules and then overtime by measures of frequency of communication.

Privacy filters all start with knowing the distance between two end points whether physical in case of centuries before or by social distance in the case of today. Until Facebook and any other social-based site has a measure of social distance privacy filters are going to be mediocre at best and more often prone to failure.

Tags: Privacy, Facebook, Filters

Failure of Filters

The title from this post is taken from the keynote that Clay Shirky delivered at the NY Web2.0 Expo in September 2008. The premise of the keynote is that the “information overload” we are facing is not a problem but a fact (one that has been around since Gutenberg and his movable type press) and what we are seeing now is the collapse of the traditional filters that mediated the information overload.

The existing filters for information were founded in the difficulty of moving information over distance. The various communications technologies of the 20th Century have steadily eroded the tyranny of distance. The web completed the destruction of distance filters by removing all concept of spatial distance for information.
Our sense of privacy is again bounded up in the hassle in moving information over distance. This physical distance is the basis for the whole concept of privacy. The closer we are to other people the less privacy we expect. We found that to be a reasonable rule of thumb as those closest to us (community, family, friends) are likely to spatially close to us. We only now need privacy safeguards because the rule of thumb no longer applies – spatial distance is meaningless for information now.

Information overload and privacy issues are a rooted in us expecting that filters based on spatial distance to continue working in a world where information has no spatial component. Any filters built with this expectation don’t work. Instead we have to create a new framework for filters that don’t rely on spatial distance.

By borrowing ideas from science we can create a framework that doesn’t rely on spatial distance. The framework is based on data half-life, data permeability and data potential. Data half-life is the measure of how long the bit of data takes to lose half of its relevancy/ importance. Data permeability is a measure of how hard it is for data to move over a period of time – think fluid moving through a filter. Data potential is the initial potential for the data to move – think potential energy in Newtonian dynamics.

The interaction of these three parameters determines how far and how quickly information can travel within an environment where spatial distance has no meaning. An analogy will help illustrate how the parameters behave together to filter information.

Let’s say we have some information – death of the chief of a village. The village has good roads and the news is to be sent by horse. This information will go far as it important (chief of a village), it is easy for the information to move on the road and the horse is quick. If, however, the death is not the chief then the news won’t travel as far it is not as important. It is the interaction between the data half-life (how important the person is), the data permeability (how easy it is to move the information) and data potential (how fast the information can move) which determines how far the information will travel.

Changing the parameters creates a varied set of filters that determines how far and how fast information will defuse. Each connection has a level of data permeability with information coming in assigned a data half-life and a data potential. The information only passes the filter when the data half-life and data potential are enough to overcome the data permeability.

To illustrate consider changing your relationship status in Facebook. If someone changes their status from relationship to single they don’t necessarily want the information to spread quickly through their “facebook friends” as their friends will include work colleagues and friends of friends only met once. Instead each of their connections should have different data permeability and depending on information (data half-life and data potential) it will show up in some of the connections news feeds right away, some in days, some in weeks and others never at all.

There is no single way to create and calculate data half-life, data potential and data permeability. Various developers will come up with their own methods. Some of which will work and others that won’t. Hopefully further down the track we will see a standardisation on calculating the parameters based on accepted criteria for each type of information – personal, communications, knowledge etc.

Tags: Filters, Information Overload, Privacy, Clay Shirky

Intersection of the web and tangible objects (manufacturing-as-a-service)

MIT's Technology Review have an interesting article examining the rise of web services that allow users to design and make objects via the web. These services have the potential to disrupt traditional manufacturing. While at an early stage its useful to consider what disruption these services could lead to.

This disruption potential comes from two different causes. The first is the rise of micro-businesses around manufactured products (I am excluding already existing craft companies which simply use the web to sell their products). One example company would allow users to find and order fittings such as taps and facets that are no longer being made by the original manufacturer. The company creates and ships the taps and facets on demand to the user. Think printing-on-demand but for products.

Manufacturing based web services open the long tail of manufactured products. Suddenly, manufactured products never really stop being made just like books now never really go out of print with print-on-demand services from Amazon et al.

The other cause of the disruption is the way these web services change the economics of manufacturing and how this ties into increasing conciousness of energy/carbon cost of manufactured products. Why buy simple products like plates, pots, pans etc. that are made in China and shipped half-way around the world when you can purchase the same products at roughly the same price that is shipped from someone down the road?

Carbon concerns will change the economics of manufacturing by making smaller factories closer to large markets more economical than large centeralised factories. Web services based around manufacturing will further erode the economics of large scale factories by removing the need for companies to purchase or rent expensive manufacturing machines. These companies are the first of manufacturing-as-a-service.

Manufacturing-as-a-service removes the equipment and capital barrier to entry for new product companies. These companies can focus on product design and building the community around the product and leave the operation of the manufacturing hardware to manufacturing-as-a-service guys.

There will be a range of responses to manufacturing-as-a-service. Some manufacturers and factories will ignore it only to be over taken by the tide. Others will embrace it and will aim to become the Amazon Web Services of manufacturing-as-a-service. What the response will be is largely going to be determined by the DNA of the company.

Manufacturing-as-a-Service, Web Applications, Trends

The iPhone 2.0 is NOT a mobile phone

One of the big concerns I have with the analysis of the mobile phone (for example Om's round up here) is the implicit assumption the iPhone 2.0 is a mobile phone.

It is not.

The iPhone 2.0 is a mobile computing platform. Why do you think the keynote spent so much time looking at the Apps?

Its the mobile computing platform is the game changer. As a straight phone (even smartphone) the iPhone has great usability but is only so-so in terms of extra features (as many a blogger will tell ad nauseum). But as a mobile computing platform nothing compares.

More importantly, Apple is repeating the iTunes/iPod strategy of building a seamless end-to-end system. In this case it is a seamless end-to-end mobile computing platform. One that includes development, hardware and distribution of the applications.

Apple is pursuing an edge strategy that re-defines the general idea of mobile towards the definition used by Tim O'Reilly in his recent Web 2.0 keynote in San Francisco, 2008. It drives innovation to the edge and upturns the existing industry.

iPhone 2.0 is NOT a mobile phone. It is a mobile computing platform.

Tags: iPhone, mobile phone, mobile computing

The Social Network has little Value in a World of Flow

Fred Wilson recently wrote a post about how the flow of data is important versus the data itself. Primarily in response to the ongoing bushfire in the blogsphere about Facebook and Google Friend Connect and the larger context of data portability.

The conceptual point of flow versus data is important one to highlight.

It is very easy to confuse the two. To an extent this is an artifact of our language which emphasises objects (nouns) over flow (verbs). But it is also influenced by trying to use an existing frame-of-reference to discuss a new frame-of-reference that is only just beginning to come into focus. This is always going to make everything more difficult.

The web is moving into uncharted territory. Up to now we have been dealing with the conversion of existing real-world into an online equivalent. Now the web has reached the point that it is moving beyond the confines of being a real-world analogy. This is creating vast new opportunities, few of which are known to us now. Data portability discussion exits within this new framework.

To make headway understanding this new framework, we need to converse using language that properly describes this new framework. The language of flow will help us frame problems and hold conversations that enable solutions and new opportunities.

In Wither Social Networks, Arise Communities I pointed out that social networks are glorified contact books. A better way to look at social networks is that the merely describe a connection between two people. They are the pipes, wave-guides, tubes along which guide the flow. What happens at the end points is not part of the social network.

In addition to the guides, we have process points. The process points are where points along a flow something happens to the flow. Whether its received (such as email), or processed (such as Wesabe). A process point is not necessarily where the flow stops, merely where it undergoes some sort of processing.

Facebook's aim is to become the primary process point. They know (or suspect) that merely having a description of a flow network is not enough. They have to be a processing point, but here is their dilemma: Facebook was never designed with being a processing point in mind, merely a description of a flow network. So their strategy is to try and control of the description of flow networks by restricting access while they shift to being a processing point, Facebook Connect being an example.

Bear in mind that the flow network description has little intrinsic value. It is the flows along the network where the value lies. In this Robert Scoble is wrong. It is not the flow network where the value lies but flow along the guides that is important.

So is Facebook right or is Google right? How about neither? Facebook's move is entirely about trying provide themselves with time to become a processing point and less of a pure flow network. Google's aim is to get access to the flow network in order to get access to the processing points. Google looses out if it doesn't know about the processing points. Neither are taking the positions they are merely out of moral indignation. I do find Google's behaviour less obnoxious than Facebook as Google's move is about access where as Facebook's is about control. Not really surprising given Facebook's past behaviour and in the words Umair, evilness.

Ultimately, it is a meaningless argument. The web is shifting so fast that both companies actions will soon be lost in the momentum in the move to flow. The flow based web even looks like it will overtake the Data Portability movement. Rendering the broader discussion irrelevant as well.

Tags: Social Networks, Facebook, Data Portability, Google, Web Next, Web 2.0, Fred Wilson, Robert Scoble, Umair Haque, Web Services

Wither Social Networks, Arise Communities

A community is an assemblage of people around a common interest. What Hugh MacLeod calls a social object. Social Networks, like Facebook, are glorified contact books. And as Facebook is finding out, people get stroppy when you get between them and their contact books.

A community, on the other hand, behaves differently. The members of a community are there because of the shared interest or bond, the social object. Consider the rise of communities around particular diseases as highlighted in this week’s NewScientist (vol 198, issue 2656). These communities are generating a wealth of data about these diseases that would otherwise be expensive or impossible to obtain. Communities tend to generate data around the particular shared interest beyond simply demographics that you get in a Social Network.

The data is hugely important. As Tim O’Reilly is fond of saying, “data inside” is the new “Intel Inside” (between time point 2.15 to 3:20 in the presentation). The value of web companies is entirely determined by the data they can aggregate and turn into new knowledge. Communities generate large amounts of data by harnessing the network effect. As each member adds more data around the shared interest this creates a positive feedback loop encouraging more people to add more data and so on in a virtuous cycle. A good example of this principle in action is the company Wesabe (also discussed by Tim O’Reilly in his keynote at 2008 Web2.0 Expo).

Social Networks don’t have this positive feedback loop that generates great swathes of data. While they do have network effects this is merely increasing the size social network by members rather than adding large amounts of data. We are even seeing indications now of limits to how far network effects work in maintaining growth of membership. The amount of data in social networks is relatively limited and most of this information is limited to who knows who and simple demographic data.

Social advertising is the “next big thing” in advertising. However, achieving this on a social network has not been easy. An outcome that is not surprising. The effectiveness of advertising comes down to two things: attention and intent. Attention being what is the person doing at the moment. Intent being why are they doing what they are doing at this time. The more closely you can determine the attention and intent of the user the better the advertising can be made to be of interest to the user.

Social Networks do not offer great data to determine attention and intent. Just because you are a 46 year-old climate researcher, does that determine why you are looking for a holiday? An advertiser could assume you are looking for a holiday for yourself but there is no evidence for this. Nor can a social network really say whether you are looking for holiday in the first place. There is little data to indicate attention and intent. Communities on the other hand do offer good data for determining attention and intent. Consider the climate researcher. He joins a community around travel and holidays and asks the question of the community “what is a good holiday as a birthday present for my 16 year old daughter?” Now we know attention (searching for a holiday) and intent (as a present for his 16 year old daughter). Having this data allows the advertiser to very accurately target with information about holidays suitable for a 16-year old girl.

People like being part of a community. We are tribal at heart. A social network is a mathematical abstraction that merely indicates a connection. The tribe will always win over the maths. Communities will win over social networks.

Tags: Tim O'Reilly, Social Networks, Hugh Macleod, Social Object, Community, Data Inside, Attention and Intent, Facebook, Wesabe

Mobilty, Facebook, Twitter and Social Cohesion

The Economist has an interesting article about mobility and its effect on society. In particular one anecdote of a plumber and a sociologist had a strong resonance.

"Richard Ling, a sociologist at Telenor, the largest Norwegian telephone company, and author of “New Tech, New Ties: How Mobile Communication Is Reshaping Social Cohesion”, was standing on his porch in Oslo one day, saying farewell to a few guests, when a plumber walked around the corner, talking on his mobile phone to what appeared to be his wife. Mr Ling, who had a leak in the kitchen, was expecting him. But the plumber took Mr Ling and his guests aback by walking right past them and into the house, where he took off his shoes and headed for the kitchen, chattering into his handset all the while."

The article goes on to talk about weak and strong social interactions. In this case the plumbers weak social interaction with the sociologist was overcome by the stronger interaction between the plumber and his wife. Or it could easily be the girl at the checkout counter chatting away to someone on their mobile phone while barely paying attention to the task of paying for their shopping. The anecdotes are there and very strong.

Now that strong social interactions are rarely limited by distance, they are easily overwhelming weak social interactions. But is this a problem? I think so. The weak social interactions are often with random people in every day life - bus drivers, commuters, shop assistants, doctors, police, people on the street, customers in a cafe - rather than with people we've selected as being a part of our "tribe". They are unlikely to be similar to us and this variety of social interaction helps us be less insular. In reading the article I was strongly reminded of one of my favourite quotes from Terry Pratchett.

"Individuals aren’t naturally paid-up members of the human race, except biologically. They need to be bounced around by the Brownian motion of society, which is a mechanism by which human beings constantly remind one another that they are... well....human beings."

The dominance of the stronger ties reduces the "Brownian" motion needed to make us human. This loss is what is disturbing about the "mobility" society. We are rapidly shutting out random acts of chance, of serendipity. Nor is it simply mobile phones. Social Networks such as Facebook also produce the same effect. Anything that promotes strong social interactions at the expense of weak ones are culpable.

Services such as Twitter and FriendFeed go towards promoting weak social interactions as a tweeter will not always "know" or have a previous strong social interaction with a follower. I do wonder, however, whether these virtual weak social interactions will again come to dominate over the face-to-face weak social interactions of every day lives.

Banning mobile phones, Facebook and Twitter is not an answer. These services and devices serve a strong purpose of facilitating communication and strengthening social ties. What we need to be aware of is the here and now. To recognise that at certain points the here and now of weak social interactions out weights strong social interactions.

Tags: Social Interactions, Mobile, The Economist, Facebook, Twitter, FriendFeed

Amazon to buy a CDN?

Amazon is beefing up its AWS offerings. In the end making the menu of services much more enticing. Amazon's has reached a point that a whole startup can be cheaply and easily hosted with robust and reliable services. But that isn't the point of the post.

With S3 video startup's can store and transmit their video. Which if you take off like YouTube quickly becomes costly (for both Amazon and the startup). So how to address this issue? One method would be to use a CDN like system to distribute content to the edge. Amazon has several options 1) build its own, 2) buy an existing CDN or 3) partner with CDNs.

The structure of Amazon's computation infrastructure (particularly with the Zoning) could already allow for CDN like behaviour. The question is whether this is enough. I expect that it would require more development to get a proper CDN like behaviour working.

The second option buy an existing CDN would allow Amazon to avoid having to complicate S3 storage to give it behaviour like a CDN. Rather Amazon would like S3 to the CDN's existing infrastructure. Here I think Akamai is probably the best target for purchase. Allthough Limelight Networks is a possibility as well.

The third option partnerting with CDNs would provide AWS customers with the most choice. They get to choose which CDN they use and AWS makes the link between the S3 and the CDN very straight forward.

Dealing with the video/rich content distribution to the masses is something that Amazon is going to have to work on. It will be interesting to see the finial solution Amazon chooses to pursue.

Tags: Amazon, Amazon Web Services, CDN, Akamai, Limelight Networks