A Framework for Innovation Through Experiments

I know all readers of this blog agree that change is the only constant in technology , marketing and media today and that anyone proclaiming to have all the answers is quite frankly a fraud. Constantly emerging technologies and best practices in search, social and mobile mean that any marketer, communications professional or media executive has only one option today and that is experimentation. The challenge is that although every organisation talks about innovation most corporate and many national cultures do not allow for the idea of experimentation and its associated failure as a norm. So providing a framework for experimentation in organisations is much needed. Fortunately the authors of the recent New York Times Innovation Report have provided a framework for experimentation which I believe is worth highlighting and using within many companies today. Here are their recommended 7 steps to successful and no doubt unsuccessful experimentation.


1. Launch Efforts Quickly then Iterate

Marketing and media organisations are full of perfectionists who hold back campaigns or stories until they are in their eyes perfect. This mindset needs to change and whilst quality will always be important it does not trump speed and real user feedback. Marketers need to take a lesson from the lean startup playbook and go for minimum viable products which are launched quickly, get instant user feedback and can be improved constantly. If you need a lesson in this regard look no further than the team at Vox Media Group that recently launched Vox.com in 9 weeks, much quicker than all their previous efforts and to much acclaim.

2. Set Goals and Track Progress

 As with anything what cannot be measured cannot be managed and we should not run away from measurement which is something that many creative marketers/agencies have been accused of in the past. Every project should have goal(s) and success metric(s) attached at the start and they should be measured in realtime. However getting the right KPIs in place are key – in the age of big or even little data there is no excuse for running away from the numbers.

3. Reward Experimentation

At many organisations the risk of failing far outweighs the reward of succeeding. This needs to change and pioneering projects which contribute to innovation of a company, department or project need to be celebrated and rewarded whether they succeed or fail. However a solid framework for learning and information sharing needs to be  created for those failures to be worth it.

4. Communicate & Share Best Practice

All organisations need to do a better job of communicating our digital and change goals and sharing what we know about the best practices we know about in how to achieve them. It is often legacy organizational structures, silos and their associated egos that stop this happening. The culture of constant learning needs to introduced to ensure the entire organization learns from every projects success and even more important failure.

5. Kill Off Mediocre Efforts

 Success will always be about the optimisation of resources as they will always be scarce. It is essential that resources are freed up to concentrate on projects that will deliver value. This is why using the discipline in your project management process to evaluate progress and be brave enough to “kill your darlings” will be key to every organisations success with experimentation.  Get the egos and vested interests out of the way and concentrate of the success of learning.

6. Plan for Version 2.0 and beyond

 Most of the resource plan for new experiments cease at launch, as they can be moved onto another project. This is short-sighted and launch is just the beginning. Whether it’s a marketing campaign or technology project, take a lesson from the lean startup playbook and ensure you constantly iterate and include user feedback.

7. Make it Easier to Launch and Experiment

 This is all about organisational culture. Innovation and the experimentation needed to be innovative will always be about balancing the risk/reward equation. In many organizations today the focus is very much on the risk rather than the reward side of that equation and that is no recipe success. Enabling your organization to experiment with small projects with limited approval levels will be key to unleashing the power of innovation in your organization. A one page lean or business model canvas should be all that is requited to test an idea. Forget your 5 year business plans, many organisations are still wedded to. In the world of technology, media and marketing we do not know where we will be in 5 months let alone 5 years and that’s why its such an exciting space to be in.

Strategy is all about winning. I will leave it to a man who has won and lost many big battles and was forced to innovate in his time to have a final say on experimentation. As Winston Churchill

“Success is the ability to go from one failure to another without the loss of enthusiasm”

Organisations and their leaders would do well to learn from the man who lost many battles but ended up winning the war.


  1. Good points made by you and NYT.

    Re #1 Vox​ took nine weeks to plan, design, build, test, and go live (six weeks from the time development began). By comparison, the initial launches of The Verge and Polygon occurred about eight months after the respective editorial leads joined the company.

    Success seems to be centred on flexibility of thinking – therefore creativity. This an area dear to my heart.

    • Thanks Scott. Yes Vox Media Group and their multiple brands through one technology platform are a true inspirtion for both legacy and new media companies. Their gradual improvement through The Verge, Polygon and now Vox just show the improvements that can be made through agile iteration.

  2. Will be interesting times as big corporates wrestle with becoming more agile and innovative, with cultures that encourage experiments. Who will be the top 10 corporations in 5-10yrs time?
    A key shift for me is the ability to break from short term thinking only (metrics/ dashboards etc) and more about learning, dynamic scanning of what’s next and being genuinely inquisitive.

    • Absolutely Greg…great comment. Couldn’t agree more. Its not about being the smartest animal in the bush but being the quickest to adapt to your environment. As external contexts are changing at rates never know before due to technology those that get the learning organisation in place and an agile or emergent strategic approach will be the winners.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s