ITasITis

Mike Rasmussen has blogged a critique of Forrester’s recent Wave on Global Risk and Compliance, in a piece which deserves an audience outside the GRC community.

Mike is the doyen of GRC analysts and, in his time at Forrester, authored two previous versions of this Wave. He’s quite explicitly not getting at the authors of the new update; it’s the process that’s at issue. And the comments are well worth reading if you use any Wave, Magic Quadrant, or similar tool to help your purchasing decisions. Especially if your management board won’t approve a purchase unless these tools “endorse” the choice.

Mike has two criticisms. One of them isn’t a surprise, but it’s worth a reminder. The picture presented in a point-of-time report is, almost by definition, out of date before it’s published. Vendors don’t stand still while a report’s being researched. Indeed, some vendors opt out of supporting an evaluation because they’re close to a new release and don’t want to be judged on the old one.

InformationSpan has a similar issue: if you download our free report on insight service coverage for BI, you’ll find that its assessment of Forrester’s coverage is significantly out of date. It was written last November, and they’ve put out a lot of new coverage this year.

But secondly, Mike comments that the assessment criteria for the GRC Wave haven’t been updated while the discipline of GRC has moved on substantially. Analysis needs to recognise this.

These are process questions. The first one reflects the length of the evaluation process; things go out of date while being evaluated, and vendors sometimes decline to commit the resources. Mike asks whether this can be streamlined.

But the second question reflects the fact that the process was designed for systems delivered into a relatively mature marketplace, where the underlying concepts being modelled in software (for example) aren’t changing greatly.

GRC in the enterprise isn’t primarily about tools: it’s about management discipline and process. Tools simply support the process; tool capabilities develop rapidly as the discipline itself develops. And there are other areas like this.

For example, there’s an emergent unified approach to change management, configuration control and release management (CCRM) – I attended a workshop about this recently. Or there’s architecture, needing new approaches to integrate the benefits of cloud services securely into the enterprise while the first wave of architecture repositories and other tools are still evolving. No doubt you can identify other examples.

This isn’t about the pace of change. It’s whether a process implicitly predicated on a stable environment can cope with changes which are about much more than new technical ways of doing essentially the same thing.

So what’s the way forward? More than Mike suggests, I think. The key must be to separate the market snapshot from the analysis report. A vendor’s vision or ability to deliver don’t change over the timescale of report writing – nor with the actual issue of a new release, though the market’s reaction to it may be significant. So here’s my suggestion.

First: let the analysts get behind the versions to give a well researched, more stable view of the vendors and their contributions in a particular sector. Detach this from the assessment of current releases: they are data for the assessment, in this understanding, but not the core of it. Gartner’s MarketScope, an alternative to their better known Magic Quadrant, targets this: their aim is to provide “an overall market rating that indicates the strength and potential for the market in general. This is particularly important in emerging markets, when … it is difficult to assess the long-term viability or evolution of offerings. In mature markets, MarketScopes provide insight about the ongoing value of products and services“.

And second: make the process continuous, rather than point-in-time. Then it can respond continually not just to new products but to new assessment criteria as the underlying paradigms change. If  new tools were continually tested, and the results added to the database, a Wave or MQ could be dynamically delivered from the most current data. I don’t think anyone does this. Tell me if you know different!

It’s a step beyond what Mike’s asking for: not to streamline the process, but to change it.

So read Mike’s comments, and apply them to your own specialism. Understand the strengths and weaknesses of Waves and MQs as they currently exist. Comment back to me here – does anyone know of an assessment tool which is already dynamic in this way? And always remember, when you’re using any of these analyst tools, that they provide insight - not ready-made decisions!

Links:
• The Forrester GRC ?Ripple? … , Corporate Integrity (Mike Rasmussen), 2 July 2009
• Coverage report: Business Intelligence, InformationSpan, Nov 2008 (free download from this page)
• The Forrester Wave
• Magic Quadrants and MarketScopes: How Gartner Evaluates Vendors Within a Market, Gartner, Jan 2008 (this document appears to be openly available)