Carter Lusher of Sage Circle reports an interaction between Andreas Bitterer of Gartner and a smaller vendor who’s taken issue with coverage in a Magic Quadrant. Credit, as Carter says, to both parties for engaging in a public discussion about this.
If you’re a user IT organisation with a relevant subscription, you probably use the Magic Quadrant (or Forrester’s equivalent, the Wave) to assess the players in a particular marketplace. If so, it’s well worth reading this post to better understand these methodologies. It appeared over the holiday period, so you might have missed it.
Sage Circle looks at it from the vendor’s angle, for Analyst Relations professionals. Here’s a thought for the enterprise IT team.
If you are inclined to look at emerging or niche vendors, or at Open Source (the issue which triggered Carter’s posting), then the mainstream market assessments probably won’t see them, and the main insight providers probably won’t cover them. The same’s true if you are investigating a niche area. I was asked, once, to find a hosting service experienced with a specialised software product. In its own market area, this application was a leader; but it wasn’t exactly mainstream ERP! There was no analyst coverage at all, within our available subscriptions.
In this situation you can try three things.
First, find a niche insight provider which specifically covers the market, if there is one. Ask InformationSpan: we have the database, and the review is almost complete (I’ve got to “T”!), but one thing I do know now is that not all niches have this kind of specialist coverage.
Second, make your own assessment. Both Gartner and Forrester publish their criteria, though Forrester provide more complete details; if you’re in an area where there is no coverage then look at a related MQ or Wave for ideas what you ought to assess.
If you’re looking at niche vendors in a major area where there is coverage, remember that Forrester make available the full model including data and weightings. You can adjust the weightings to create your own Wave, and if some of those criteria aren’t of interest to you in relation to your niche interests, or you can’t either research or estimate them, then just zero-weight them. You can’t add data to a Wave, but you can work in parallel with it. And validate it with an analyst when you’ve done the work: they might well see something you’ve missed.
Third, you can ask the analyst to do custom research. Again, with a niche product there will be criteria they can’t assess – there won’t be a sizeable user community to ask questions of, for example. This may be useful for a niche product in a major area; in a specialist area it’s likely you’ll know more about it yourself and, because they’ll start from a lower base, it will be expensive. They may even say “no”. But they will work, so far as possible, to their standard methodology and this will make comparison easier; aside from not having to do all the work yourself!
Oh – and a Happy New Year! There’s plenty of talk about managing through the recession. I still like George Colony’s take: including, hire the smart MBAs who are not now going into finance!
• Vendor complains in a very public blog post about Gartner’s Data Integration Magic Quadrant Sage Circle, 29 Dec 2008
• Setting the Record Straight Andreas Bitterer, Gartner blog, 28 Dec 2008
• CIO best practices for thriving in a recession Counterintuitive, George Colony, Forrester Research, 24 Sep 2008