Daniel Steinbock

In a recent post, I introduced Kluster, a new web startup that is trying to build a community for democratized design, i.e. crowdsourcing.

As an experiment, I recently sponsored my own design challenge on Kluster (you’ll need a Kluster account to see it). I offered $50 of my own money and challenged the Kluster community to design a killer location-aware application for the iPhone. The challenge ran for about a week and a half, during which time 47 proposals were made and 68,523 watts (Kluster currency) were invested to determine the best among them.

The main purpose of my experiment was to try and re-create the human-centered design process within Kluster, which, according to my previous post, is difficult if not impossible. Not to be a mere critic, I gave my own best shot and making it happen.

I’d say the results were so-so. While some really good ideas were generated (some even innovative), on the whole, the design process I provided wasn’t strongly adhered to. I created three phases: 1) Needfinding: identifying unfilled user needs based on actual personal experiences, 2) Wild brainstorm: going for a quantity of unconstrained ideas, 3) Final solution: the final brainstorm.
Also, in every phase I stressed the importance of being succinct. I’ve noticed that most Kluster projects are plagued by long-winded proposals that could really be said in a few sentences. This makes for really tedious evaluation of the idea’s quality. I went so far as to require a two sentence limit for phases 1 and 2 — which people promptly ignored.

Lessons learned:

1. You can’t expect wild brainstorms when you title your design challenge with a specific solution in mind (i.e. “location-aware iPhone app”). People basically treated phase 2 and 3 the same.
2. People have a hard time discovering needs based on real personal experience. It’s so much easier to imagine “what if…” or “wouldn’t it be cool if…”, instead of developing insights about the current state of the art.
3. Kluster’s algorithmic selection process, based on an investment market metaphor, doesn’t seem to work very well for the small-scale collaborations happening on the site. (The final winning idea was mediocre, at best.) The only projects that draw a large enough crowd are the ones that offer hundreds of dollars or more as reward. Hardly a platform for democratized design… But potentially a good recipe for well-funded crowdsourcing.

After it was all over I received two interesting messages from people who had participated in my challenge. One asked if I would send him a check for $4.70 (his return on investment for the winning idea). Kluster has yet to tell me how I’m actually supposed to pay out my reward. Symptom of a brand-new startup I suppose.

The other person wanted to know how the final idea had been chosen since its stats seemed to indicate it was lower-rated and had less investment than other proposals. My inquiry to my Kluster contact revealed that the timeline of investment plays a big part in preventing people from gaming the system by investing big right before the challenge ends. In other words, earlier investments carry more weight than last-minute ones. While that makes sense, early investments are also much-less informed: there are fewer ideas to compare it to.

I suggested to Kluster that they put this information in a FAQ so people aren’t investing in the dark. I actually lost several thousand Watts investing last minute like that. That was after I’d won over $100 with some good investments in the earliest Kluster projects. I figured I’d use that money to run two experiments. Now you know about the results of the first. Stay tuned for the second.