There has been a lot of fuzz over the last decade on how to assess an organization’s innovation performance. I developed some frameworks myself back in 2008-2009, but during those days innovation management as a discipline was still very premature and most existing models were struggling with what to measure, where to measure, when to measure and how to measure it. The general confusion led to a plethora of models, which certainly gave options to organizations, but also difficulties in knowing which models were superior to others. And to complicate things further most were differently described, using terms as innovation assessment, innovation readiness, innovation maturity index, etc, though they all meant the same thing.
Standards enter the scene
The CEN-16555 deployment model (click to enlarge)In 2012-2013 I was part of the CEN Technical Committee 389 who had a work group for developing a European standard for innovation management assessments (CEN/TS 16555-7). Things started to clear up for many, but what standards prescribe is just how, when and where to measure, not what to measure, which is still the most critical part of an assessment. The reason why standards do not capture the details of what to measure is because they depend on such things as company size, industry, practice, etc, and they may also change over time, thus they are leaving it out for the market to handle. Since 2014, there is now a working group within the ISO Technical Committee 279 that are developing an international standard for innovation management assessments. The problem remains though. Even if the international standard is highly useful for organizations to set up the structure for their assessment work, as with CEN it still leaves out the areas to measure. As it should.
What assessment model should I choose then?
Good question. How can one become any wiser regarding what tool or method to use? To answer that, let me first give you some feedback on how data is commonly collected for innovation assessments.
- Self-assessments. Online check-box tools that let you fill in your organization’s data and you get a fairly quick response on you approximate maturity level. The strength with these questionnaires is that they are fast, they are cheap, and they are easily accessible on the Internet. The flip side is that they generally give pretty inaccurate results. And that is not due to poor performance of the assessment model, but the plausible discrepancy between the model’s intent and the assessor’s skills. They need to be extremely aligned for the results to be any type of valid.
- Workshops. Physical workshops where strategic managers from the target firm are present that are facilitated by assessment professionals. The strength is that you can get more accurate results than with an online form and in a fairly cheap and speedy manner. The weakness is first of all that it can be hard to get the right people in the same room at the same time, but also that you will just have enough time to scratch the surface. No in-depth analysis can be delivered in 3-4 hours.
- Interviews. Physical interviews of strategic managers and other relevant respondents, conducted by innovation management professionals. The strength is that they can be thorough and dig deep and thus provide excellent results, while also asking counter-questions to the respondents to further increase validity and quality. The drawback is that they take up much of senior managers’ time and are relatively expensive using experts for many hours.
- Combinations. To reach more respondents several of the above can be utilized together and, when relevant, in sequence. For instance, the process can start with somebody at the firm making a simple online assessment, followed by a workshop with executives, and if necessary lead to a set of strategic interviews complemented with a number of online forms sent to e.g. middle management. The structure can be established alongside the first workshop.
But are all models the same?
No, of course not. The trick is to find an assessment model that suits your purposes. Massively generalized you could say that assessment models belong to either of four categories; building on 1) a proprietary management model, 2) an established management model, 3) an innovation management model (such as a standard), or 4) on innovation management best practices. The former two require more interpretation, but can be very useful if managed right. The latter two are more aligned from the offset and may thus be easier to utilize. Just make sure you pick the right model and the right data collection scheme for what you intend to do. So if you don’t over-simplify the evaluation process, then you’ll be fine. 😉
In the second article I go through what the assessment is primarily used for and why it has such important implications.
