Thursday, 31 January 2008

The Wisdom of Crowds

I was in two minds whether to read James Surowiecki's book, thinking that the main conclusions could be drawn from the title. And it was pretty hard to wade through, given that I could only read it in 22 minute bursts (the length of my train journey each day). But I'm glad I persevered!

The subtext is "Why the many are smarter than the few", but it could just as easily have been "How to get the best out of a team". It presents a remarkable stream of evidence to suggest that certain types of decisions made by groups of people are consistently better than decisions made by leaders or experts. The inevitable errors made by individuals in a group cancel themselves out time and time again. Numerous examples are given, ranging from poor decisions made by NASA prior to the Colombia disaster, to great decisions made by groups of people betting on the points spread at an American Football game, to the mechanics of the stock market. It's compelling stuff.

The repercussions for a team or a company are immense; top down leadership doesn't work unless the leaders can find a mechanism for harnessing the collective 'wisdom' of the people below them in the org chart. The deeper the hierarchy (or the bigger the ego of those in charge), the tougher the challenge.

Surowiecki identifies three characteristics that make a crowd 'wise':
  • Independence - the people in the crowd need to be unbiased
  • Diversity - that's cognitive diversity rather than e.g. race or gender, so they should come from as wide a range of backgrounds as possible, collectively bringing a wide range of information and perspectives with them
  • Private judgment - be allowed to make their decision without duress
Hmm. Independence, diversity and private judgment. Sounds rather like an open source project to me.

The book casts damning judgment on CEOs and their ilk, claiming that their success is more down to luck than skill. After analysing several CEOs who crashed and burned in a new job after success in their previous role, Surowiecki says:
It's natural for us to look at successful people and assume that their success is due to some innate quality they have rather than to think that it might be the result of circumstance or chance...."CEOs should come with the same disclaimer as mutual funds: Past success is no guarantee of future success".
Quite a humbling notion for those on the board who pay huge salaries to their CEO, or the trader who buys stock just because a new CEO has a good track record.

Of course there are some decisions which aren't suitable for collective decision making. Anyone who's tried to design a website by committee can tell you that. The challenge for anyone leading a team or a company is to try and identify which decisions are best made alone, which are best made by the crowd, and - if the latter - how this wisdom is harvested.

Tuesday, 29 January 2008

Designing User Interfaces

One of the biggest criticisms leveled at open source projects is that the user experience of open source products sucks. This post is about an attempt to introduce design standards into an open source project, particularly one without designers. If you already know how personas are used in a development product, you might want to skip this post. Otherwise, read on!

Anyone who's been within shouting distance of myself or Paul Downey over the past six months will know that we embrace the mantra laid out in "The Inmates Are Running the Asylum" by Alan Cooper (free preview available on Google Books). For those who haven't read this book, the main contentions are:
  • Products should be designed based on end users and their goals, rather than on the underlying system or code
  • Features shouldn't be included just because they're easy to add. Unnecessary features result in a bloated interface and a poor user experience. Features should help end users reach their defined goals. If they don't, maybe they should be removed.
  • Developers aren't well positioned to design great user interfaces because they have too intimate a relationship with the underlying code.
The TiddlyWiki community in particular might disagree with some of this. After all, where TiddlyWiki is concerned, the unique functionality is the experience - and some see it as more of a toolset than a web site. And how can you design the front end without a deep understanding of how javascript and css works?

I still think there's plenty for us (the open source community) to learn from Cooper's book. Even for those who think open source is all about scratching your own itch. If you want other people to use your product (open source or otherwise), it will greatly improve the chances of adoption if you try and understand the end user's goals, and design the user interface to help them meet those goals.

Cooper provides guidance on how to achieve this. We use personas. We make up pretend people, and design for them. We attribute them with believable characteristics, give them goals and motives, and then base the user experience around those goals.

It sounds simple, but creating good personas is an art, and this is my first attempt. But I have tried to learn from Cooper, as well as my former (rather talented) colleagues at LBi. Below you can see the two personas I've developed for the next phase of RippleRap development. I'm publishing these for the following reasons:
  • Developers who are new to the concept of personas might find this process interesting, and might want to develop their own personas (in which case, read Inmates first! You can copy my format, but would be much better off developing specific personas for your product. Here's a decent introduction)
  • People interested in RippleRap might be curious why our development priorities are changing
  • People with more experience than I might weigh in with suggestions (fingers crossed)
  • We want to be open about our process to see what we can learn from this transparency
So, have a look. Some of the text is too small to read here, if you want to read it, click on Menu and then the slide title.

Dan - "Fastidious Notetaker"



Sally - "Social Butterfly"



The next stage is paper prototyping. We're developing RippleRap for BlogTalk on 3/4 March 2008. Watch this space.

Tuesday, 22 January 2008

The brouhaha about the ownership of social data

(Image courtesy of Imamon)

Nearly a month after the Robert Scoble Facebook fiasco, the Blogosphere is still awash with people arguing over who owns the personal data exchanged over the web.

I've been thinking about this, and have reached the conclusion that it isn't a question of ownership. This is a red herring. It's like arguing over the ownership of knowledge acquired from a book. Or a piece of mail while in transit. We'll never reach consensus and will get no closer to solving the problem.

I think, instead, we should be focusing on permission and trust.

When I add someone as a friend on Facebook, I give Facebook permission to share my details with them. I also give that person permission to use that data however they wish. And I trust them not to use the data for nefarious ends. Same as in real life, except there is an initial broker.

It gets more interesting when permission is more implicit than explicit. But I still think we should be guided by the same principles, by aiming to establish what permission was granted and therefore how the recipient is trusted to behave.

When I buy something on Amazon, I would argue that, implicitly, I give them permission to use that data. Not just for their own accounting and stock management needs, but if they want to profile me and target me with offers they think I'd like, then fine. I know they're in it for the money. In fact I think they do a great job of this, given the discreet nature of their recommendations, and trust is established as a result. And of course, they'll lose at least one customer if they sell this data onto a third party without checking with me first.

But should I be allowed to use that data as well? This is where the question of ownership often pops up, and I think we're asking the wrong question. If Amazon wishes to maintain the trusting relationship, they should be willing to give this information up, for sake of maintaining the relationship. If they need to charge a modest sum for it, that seems fair (because it takes time and effort to produce it).

Coming back to the Robert Scoble incident, I gave permission to him to use my contact data however he wanted. Facebook acted as the broker, but after the initial connection was made, they have no place in preventing Scoble from using that data. Terms of Use be damned. If he abused the trust I place in him (he didn't), then that's a matter between me and him.

I suppose I've opened myself up to accusations of naivety. Yes, we are all guided by a legal construct that is based on ownership. But let's not forget that this construct originated in a need for humanity to behave in a civilised manner, and emerging social technologies ought to take heed rather than buck the trend.

Monday, 21 January 2008

RippleRap - The Next Phase

We've now published RippleRap for external consumption, and I'm now planning the next iteration.

Based on our own experiences, and on feedback from members of the community (thanks!), I've identified the following priority items. Let me know if you have additional items you'd like to see on the roadmap, or if you think I've defined the problems incorrectly - or if you think my approach is all wrong!

New features

Given that this is predominantly a conference tool, we can build additional tools into the interface for that environment (or similar environments). Suggestions so far have included:
  • Blogging tool (publish notes to your blog - useful for BlogTalk!)
  • Twitter tool (post to twitter, pull in your twitterstream for processing or analysis)
  • Conference-in-a-box (download conference content from multiple sources e.g. Flickr, Twitter, Blogosphere, for reading offline)
Technical issues

Improved back-end - The current version was developed in tight deadlines and, although it works, it isn't easy to replicate elsewhere.

User Interface

Some known issues include:
  • Irritating amount of scrolling required when clicking on an agenda item which has gone off the page
  • Calls to action can be clearer (e.g. 'done' appears near 'close')
Bugs

Some known bugs will be fixed. These include:
  • CSS issue with images. If you resize the browser the images disappear until 'content' is refreshed
  • On RippleRap.com, IE6 running 1024x768, all tiddlers are opening below the Main Menu, even though it looks like there should be enough space for them to open to the right of the MainMenu.
As part of this exercise we'll be undertaking some more cross-browser testing.

Promotional materials
  • Demo - We created a demo for Le Web which was displayed on plasma screens on our stall. I'll be created an updated generalised version.
  • Cartoon - If I have sufficient time, I might use the Comic Life application to create something like this (created by my former colleagues at LBi)...to illustrate how RippleRap works. But given that I have zero artistic skills, it all depends on how much skill I actually need!
Once this roadmap has taken shape, I'll be posting it on RippleRap.com. I'm also keen to identify people who might be willing to contribute (from inside and outside Osmosoft). Note that RippleRap is composed of components which can be used elsewhere; therefore if someone is keen to develop e.g. a generalised publish / subscribe component for their own use, and is willing to collaborate so that it can be used on RippleRap, that would be just peachy.

In any event, whatever work is undertaken by people in Osmosoft will be generalised and available to use in other TiddlyWikis in other places. Reuable components are the order of the day.

Thursday, 17 January 2008

Managing open source projects

One of the biggest challenges we face at Osmosoft is figuring out how to manage projects. Might seem like a simple problem to solve, but if we're trying to help BT to understand open source, then what's the point in using traditional (project) management techniques? We already know how to manage these kinds of projects.

Surprising fact: What we (Osmosoft) can learn about open source projects is much more important than the quantity and timing of software that we create. We're interested in the innovation mechanism. The notion of delivering projects 'on time, on budget' doesn't apply in the same way here.

In order to learn about innovation in open source, we ought to individually behave like open source developers. In this case, we would only work on the things that interest us on an individual basis (scratching our own itch). And we wouldn't work to any imposed or superficial deadlines. If teamwork is needed, it will happen without management interference.

This raises some tough questions, which I've had a stab at answering below. These answers are all quite crude, partly for reasons of brevity, but I'm keen to share the learning process in case others have a view they'd like to share.

Q. How do we (Osmosoft) reconcile this open source model with the needs and expectations of a large enterprise?

A. It depends on what the needs are.

If we're talking about something with a tight deadline, then open source mightn't be the right way to go. But is that deadline really necessary? People don't remember when software is late, they remember when it's poor.

If we're talking about a job that needs to meet a pre-defined specification, then again perhaps this open source model isn't the right approach. Telling open source developers what to do is bad enough; telling them how to do it is fatal! One can simply pay a programmer or team to do the job in this case. Releasing it to the open source community afterwards, to benefit from broader scrutiny and improvements made by other developers, remains an option.

So there are optimum projects which lend themselves toward open source, particularly where commodities are concerned.

Q. What if we (Osmosoft) have projects that depend on input from people who aren't interested in taking part?

A. Well, what happens in the open source space? The person who needs it to be done finds a way. It might involve them learning the required skills to do the job themselves. From my perspective, this is getting easier as software becomes more powerful.

But this puts my role as product owner for RippleRap in a precarious position. I can't code, so I'll be relying on the goodwill of the TiddlyWiki community to move things forward. This might involve my colleagues, and then again it might not. But if this was a commercial product, I'd be inclined to default to more traditional processes (paying someone to do the job). And in fact if I had to go down this path, I'd be questioning whether further RippleRap development is really the best use of my time here at Osmosoft. Such is the nature of open source projects.

Q. One analogy made regarding open source projects is that it's like living in a shared house, where everyone wants to choose the wallpaper (do the fun jobs) but no-one wants to clean the toilet. How do we make sure that the un-fun jobs still get done?

A. Again, what happens in the open source space? Usually, someone in the group is compelled to get this done. It takes a special kind of person to do this. But if this person didn't exist, then one answer is trying to make these jobs more fun. We're organising a hack day of sorts to try and achieve this, where some of our guys are joining together (voluntarily) to work through some of the bugs in the TiddlyWiki core. Good experience for all concerned.

Q. If we try and remove salary and management pressure from the picture, what other motivational forces are at play? Just scratching your own itch, or something more?

A. This is a big one! Many others have tried to answer this question. Peer pressure and pride are, I suppose, the obvious ones. Not to mention the architectural purity that developers often pursue (partly down to pride, of course). I want to get better at understanding this - watch this space.

And my own personal favorite.....

Q. What is the role of a project manager in all this?

A. I'm not a project manager! Not any more anyway. But I do come from this background, and I can't help but use it as my frame of reference. I think there's still a role for someone in an open source project to coordinate efforts, especially when the project is above a certain size, and possibly alongside a benevolent dictator that is the hallmark of many successful open source projects. I'm finding it fascinating to consider this role in contrast to the old role; I believe that many of the skills are transferable, especially diplomacy skills! A subject for a future post.

++

In reality, I think we'll still occasionally have to do things that we don't want to do. For example when people in BT need help with TiddlyWiki. But as far as practical we'll be pushing the above agenda and I'll keep you posted on the things we learn.

This was a challenging post to write. I'd welcome any feedback, no matter how critical.

++

UPDATE: Later that day....so I've been urged to reconsider whether we're doing open source a disservice by assuming all open source projects and communities are like TiddlyWiki. And also by assuming that all open source developers are 'scratch your own itch' kind of guys. So perhaps we should say for now that we should act like beekeepers (analogy stolen from psd), and aim to develop an environment in Osmosoft where people assume the roles that they feel comfortable in, to most closely match an open source environment.

Wednesday, 9 January 2008

Open source in BT - Part Two

Sorry that this has taken a long time coming, but I hope you'll find it was worth the wait....further to my blog post on the BT Open Source event on 31 October, I've managed to track down some of the video footage for publication. You can now see the Doc Searls presentation on "Why All Business Will Be Based On Open Source":



And also Jeremy Ruston's presentation (Disclaimer: he's my boss) on "How to Start an Open Source Project":

Tuesday, 8 January 2008

RippleRap - Next Steps

Now that things have settled back down after New York, Le Web and TiddlyYuletide, we're now focusing on packaging RippleRap up for general consumption. The edition used at Le Web is now a historical artifact (and for now it can still be downloaded from the official site, complete with notes from Le Web), but we'd like to try and make it as easy as possible for people to either modify RippleRap for their own conference situation, or use the components within RippleRap for other purposes.

So we've been thinking about the best way of doing this, and would like to share our plans to gather in feedback. Note that, at this stage, we're focusing on how best to expose the product, not on how it should be improved (that time will come!). We're trying to figure out how the open source community would like us to share our work in general; this will set a precedent for all future releases, not just RippleRap.

Our suggestion is that RippleRap.com will be the place where one goes for:
  • A written "how to" for developers
  • A written "how to" for conference organisers
  • Links to new and modified components on the subversion repository (each component will be documented in due course)
  • A link to the server side code (location to be confirmed, and implementation guidelines included in "how to" for developers)
  • A link to a new 'ripplerap' component on the TiddlyWiki.org bug tracker, which will serve as a bug report / feature request mechanism
  • A link to a complete copy of RippleRap, without any data (ready to populate with conference agenda, etc.)
Is there anything we're forgotten that you'd like, or something you'd like done differently?

One other question is whether future RippleRap discussions should take place on the TiddlyWiki Google group, or if we should create a new RippleRap Google group. My personal view is that we should stick with the TiddlyWiki group, because some of the current and future components may be of use to the wider TiddlyWiki community. We can always re-open the debate at a later stage if the noise becomes too loud. But please let me know if you have strong views to the contrary and we'll take them under advisement!

Saturday, 5 January 2008

The Ten Commandments of Twitter

Thanks to everyone for your feedback, both on the blog and on Twitter! I've made several tweaks which I hope will keep most people happy.

If you haven't seen the original post putting forward the suggested list and seeking feedback, you can see it here.

Before the flame war begins, I'd just like to say that I know these shouldn't really be called commandments. But "Etiquette Requests" or somesuch just doesn't have the same ring to it....

You can see these commandments on Twitter (if you're a believer, sign up with a 'Follow'). If you want to shame someone being antisocial on Twitter, just point them at @tencommandments. Job done.

The image is also on Flickr.

Finally, if anyone out there with mad photoshopping skillz wants to mock up an image of some better commandment tablets, I'll be happy to add these instead (although please keep the roman numerals!). Just post them to flickr with a creative commons licence, add a comment with a link here and I'll do the rest.

This was fun!

Wednesday, 2 January 2008

Tweetaholics

Happy New Year everyone!

Much has been made of the ambient intimacy afforded by Twitter, and seeing as JP has kindly mentioned my tweet on the subject, I thought I'd expand on my current feelings. Plus I didn't want the first thing a bunch of new people see on my blog to be the front cover of Oklahoma - The Musical....

Recent developments have made me realise just how sensitive the balance is on Twitter. It was fascinating to see how my old team adopted Twitter in the early days, and while some people have fallen away several still remain. It's a bit like marmite, really; when people try it, they either love it or hate it. And I'm chuffed that so many colleagues (past and present) and friends love it as it's the volume of fellow twitterers that gives the service it's value. I get a lot out of it.

And just as happened with blogs, people are still trying to figure out the best way to use it. Who would've thought it would take so long for something so simple? A distinct etiquette is forming, even as new people are still arriving in their droves. Who should you follow? What kind of conversations can you have? How often should one tweet? When do you unfollow people?

Of course, the answer to these questions is whoever / whatever / whenever you want. And I suppose there is an argument that we should just let nature takes its course. But as with blogging, people will judge you on the nature and quality of your tweets, and it therefore makes sense to exercise some restraint and good taste when tweeting. Just like with the broader web - if we're good citizens, it's better for everybody.

So I submit for your consideration a rough draft of the 10 commandments of Twitter. Please let me know if you have improvements or additions. I'll make a poster with the best ones.
  1. Thou shalt not tweet more than 20 times a day. This has a detrimental effect on everyone else's ambient intimacy with their group.
  2. Thou shalt assume that everyone following you is following 150 people or less (because they probably are).
  3. Thou shalt not tweet more than 10 times in an hour*. That's what blogging is for.
  4. Thou shalt not treat Twitter like a private chat room*. That's what IRC is for.
  5. Thou shalt not engage in lengthy one-on-one conversations in Twitter*. That's what Instant Messaging is for.
  6. Thou shalt not forget that the question being asked is "What are you doing?".
  7. Thou shalt not be too boastful or pompous. Arrogance is offensive in real life, and Twitter is no different.
  8. Thou shalt learn how to use @, L:, D: and #, -- and ++. If you're not sure, watch others.
  9. Thou shalt type people's usernames correctly (e.g. @Casablanca); otherwise the 'replies' function doesn't work.
  10. Thou shalt welcome new users publicly and with enthusiasm; this helps them make connections so they can share the excitement.
*Unless thou art being very funny or entertaining.

Thoughts?

++++++

UPDATE 5 January 2007, 22:57: The consultancy period has now closed and results can be seen here. Please head over there to continue with recommendations, flames, abuse, agreements, etc.