How Markets and Vendors Are Evaluated in Gartner Magic Quadrants

Overview

Analysis

Assessing the market and its technology and service providers is a business-critical task. Vendor differentiation based on variations in operations, customer satisfaction, level of complexity and strategy can make comparisons of vendor offerings challenging. In addition, the market’s overall direction is often evolving.

Gartner’s Magic Quadrantssolve these problems by offering snapshots of markets and their participants, and help you map vendor strengths and cautions against your current and future needs.

Magic Quadrants provide a graphical comparative positioning of technology and service providers where market growth is high and provider differentiation is distinct. Magic Quadrants depict markets that are established, but not yet consolidated or in decline, by using a two-dimensional matrix that evaluates vendors based on their completeness of vision and ability to execute. The Magic Quadrant has 15 weighted criteria that are used to plot vendors on the graph (shown in Figure 1) based on their relative strengths in the market.

complement Magic Quadrant analysis to offer deeper insight into the products and services offered by multiple vendors, organized by use case.

Alternatively, Gartner may provide insights about a specific market with a Gartner Market Guide rather than a Magic Quadrant.

To evaluate vendors for Magic Quadrants, Gartner uses a comprehensive process that defines the boundaries of the market, the research focus and the steps taken to form the analysis.

To be considered for a Magic Quadrant, a market must be established, distinct and viable. Gartner selects a market for analysis based on the impact of emerging trends and the users’ need to understand changing market dynamics. Gartner analysts focus on markets where their insights can assist clients with planning, investment decision making and ongoing vendor relationship support.

A Magic Quadrant is not an exhaustive analysis of every vendor in a market, but rather a focused analysis of the most relevant providers for Gartner’s end-user clients. The criteria for inclusion may consist of installed base, types of products or services, target market or other defining characteristics. These criteria help narrow the scope of the research to those vendors Gartner considers to be the most relevant — or best suited to the evolving needs of Gartner’s clients as buyers in the market.

Providers that do not meet the Magic Quadrant criteria for inclusion may be cited in the research but are not part of the comparative evaluation. Those vendors are recognized for their ability to meet specific business needs and are included in an Honorable Mention section of the report. Vendors with notable offerings in a specific submarket may also be highlighted in Magic Quadrant additional perspectives.

Magic Quadrants use standard criteria in two categories: Ability to Execute and Completeness of Vision (see Figure 1). We then adapt the rating criteria to a market by weighting them based on a high, medium or low scale of importance. In some cases, a criterion may have a “no rating” weight because it has very low relevance for the market or does not contribute to sufficient differentiation.

While the 15 standard criteria are common to all Magic Quadrants, analysts may refine the standard criteria by adding subcriteria to further clarify how the standard criteria apply specifically to the market being evaluated.

This category includes these standard criteria:

• Products/services — Core goods and services offered by the vendor that compete in and serve the market. This criterion includes product and service capabilities, quality, feature sets and skills (offered natively or through original equipment manufacturers), as defined in the market definition and possibly further detailed by other criteria.

• Overall viability — Includes an assessment of the vendor’s overall financial health, the financial and practical success of the relevant business unit and the likelihood that business unit will continue to invest in the product.

• Sales execution/pricing — The vendor’s capabilities in presales and sales activities and the structure that supports them in this market. This criterion also includes deal management, pricing and negotiation, presales support and the overall effectiveness of the sales channel.

• Market responsiveness and track record — The vendor’s ability to respond, change direction, be flexible and achieve competitive success as opportunities develop, competitors act, customer needs evolve and market dynamics change. This criterion also considers how responsive the vendor has been over time.

• Marketing execution — The clarity, quality, creativity and efficacy of marketing programs designed to deliver the vendor’s message to influence the market, promote its brand and business, increase awareness of its products and services, and establish a positive identification with the product, brand or vendor with buyers. These programs may include, among other elements, a combination of advertising, promotions, thought leadership, word of mouth and sales activities.

• Customer experience — Relationships, products and services, and programs that enable clients to succeed with the products being evaluated. This criterion includes the ways in which customers receive technical or account support for the product being evaluated. It can also include ancillary tools, customer support programs (and their quality), availability of user groups and service-level agreements.

• Operations — The vendor’s ability to meet its goals and commitments. This criterion includes the quality of the organizational structure, such as skills, experiences, programs, systems and other vehicles that enable the vendor to operate effectively and efficiently.

This category includes these standard criteria:

• Market understanding — The ability of a vendor to understand buyers’ needs and translate them into products and services. A vendor that shows the highest degree of vision listens to and understands what buyers want and need, and can use that information to shape or enhance the relationship.

• Marketing strategy — A clear, differentiated set of messages consistently communicated throughout the organization and publicized through online presence, advertising, customer programs, events and positioning statements.

• Sales strategy — An approach for selling products or services that uses the appropriate network of direct and indirect sales, marketing, service and communication affiliates to extend the scope and depth of a vendor’s market reach, skills, expertise, technologies, services and customer base.

• Offering (product) strategy — A vendor’s approach to product development and service delivery that emphasizes differentiation, functions, methodology and feature set relative to current and future requirements.

• Business model — The validity and logic of a vendor’s underlying business proposition in this market.

• Vertical/industry strategy — A vendor’s strategy to direct resources, skills and offerings to meet the needs of individual market segments, including vertical industries.

• Innovation — Marshaling of resources, expertise or capital for competitive advantage, investment, consolidation or defense against acquisition.

• Geographic strategy — A vendor’s strategy to direct resources, skills and offerings to meet the needs of regions beyond its “home” or native area — directly or through partners, channels and subsidiaries — as appropriate for that region and market.

Research activities include, but are not limited to:

• Attending vendor briefings and product demonstrations

• Conducting surveys

• Reviewing the vendor’s customer feedback available to Gartner

• Interacting with industry contacts

• Discussing pertinent topics with clients and nonclients

• Consulting public sources, such as U.S. Securities and Exchange Commission filings, articles, speeches and published papers

• Seeking input from other Gartner analysts

Teams of analysts collaborate to evaluate and rate each vendor against each of the weighted criteria. The resulting scores are used to generate a Magic Quadrant.

The Magic Quadrant is published as an interactive tool, as well as static HTML content, which may be downloaded as a PDF. The research explains the vendor positions and ratings, as well as any new developments in the market. Prior to publishing, the research undergoes rigorous internal peer review and validation, and a factual review by those vendors included in the research.

Where appropriate, Gartner will provide “additional perspectives” based on important market differentiators to consider when selecting technology and service providers. These perspectives may include context for either a specified geography, industry or company size, depending on the market in question.

As significant events, such as a merger, occur in the market, the Magic Quadrant will contain links to research that further explores the impacts of such events on the market and specific provider. The graphical positioning, however, remains unchanged until the next full update because it takes time to see the impacts of significant events on a provider’s vision and execution.

A companion Critical Capabilities document typically publishes at the same time or soon after. Together, they explain each vendor’s position and rating, and compare its products or services. As a result, these documents provide context for the best use of any comparison.

The Magic Quadrant graph shown in Figure 1 is based on two axes:

• Completeness of Vision reflects the vendor’s innovation, whether the vendor drives or follows the market and if the vendor’s view of how the market will develop matches Gartner’s perspective.

• Ability to Execute summarizes factors such as the vendor’s financial viability, market responsiveness, product development, sales channels and customer base.

Vendors are positioned in the four quadrants — Leaders, Challengers, Visionaries and Niche Players.

Leaders provide mature offerings that meet market demand and have demonstrated the vision necessary to sustain their market position as requirements evolve. The hallmark of Leaders is that they focus on and invest in their offerings to the point where they lead the market and can affect its overall direction. As a result, Leaders can become the vendors to watch as you try to understand how new market offerings might evolve.

Leaders typically possess a large, satisfied customer base (relative to the size of the market) and enjoy high visibility within the market. Their size and financial strength enable them to remain viable in a challenging economy.

Leaders typically respond to a wide market audience by supporting broad market requirements. However, they may fail to meet the specific needs of vertical markets or other more specialized segments.

Challengers have a strong Ability to Execute but may not have a plan that will maintain a strong value proposition for new customers. Larger vendors in mature markets may be positioned as Challengers because they choose to minimize risk or avoid disrupting their customers or their own activities.

Although Challengers typically have significant size and financial resources, they may lack strong vision, innovation or an overall understanding of market needs. Challengers may offer products nearing the end of their lives that dominate a large but shrinking segment.

Challengers can become Leaders if their vision develops. Over time, large companies may fluctuate between the Challengers and Leaders quadrants as their product cycles and market needs shift.

Visionaries align with Gartner’s view of how a market will evolve, but their ability to deliver against that vision is less proven. In growing markets, this status is typical. In more mature markets, it may reflect a competitive strategy for a smaller vendor — such as selling an innovation ahead of mainstream demand — or a larger vendor trying to break out of a rut or differentiate itself.

For vendors and customers, Visionaries fall into the higher-risk-higher-reward category. They often introduce new technology, services or business models, and they may need to build financial strength, service and support, and sales and distribution channels. Whether Visionaries become Challengers or Leaders may depend on if customers accept the new technology or the vendors can develop partnerships that complement their strengths. Visionaries are sometimes attractive acquisition targets for Leaders or Challengers.

Niche Players do well in a segment of a market, or they have a limited ability to innovate or outperform other vendors in the wider market. This may be because they focus on a particular functionality or geographic region, or because they are new entrants to the market. Alternatively, they may be struggling to remain relevant in a market that is moving away from them. Niche Players may have reasonably broad functionality, but with limited implementation and support capabilities and relatively limited customer bases. Compared to other quadrants, they do not demonstrate a strong vision for their offerings.

For end users, assessing Niche Players is more challenging than assessing vendors in other quadrants. Some could make progress, while others do not execute well and may not have the vision and means to keep pace with broader market demands.

A Niche Player may be a perfect fit for your requirements. However, if it goes against the direction of the market — even if you like what it offers — then it may be a risky choice because its long-term viability will be threatened.

Your needs and circumstances should determine how you use the Magic Quadrant, not the other way around. We do not recommend evaluating vendors in the Leaders quadrant only and ignoring those in other quadrants. There are many reasons for this; for example, a vendor in the Niche Players quadrant could offer functions that are ideally suited to your needs. Similarly, a Leader may not offer functions that meet your requirements. For example, its offerings may cost more than those of its competitors’, it may not support your region or industry, or its delivery style and customer management processes might not align wellwith your requirements.

A vendor’s position on a Magic Quadrant illustrates useful information about that vendor. For example, if you want to make a strategic investment in a technology, then a vendor’s viability will be critical. In this case, you should weigh a vendor’s Ability to Execute more heavily than its Completeness of Vision and evaluate Challengers before Visionaries. Conversely, if you can gain a competitive advantage by investing in an emerging technology, then evaluate Visionaries before Challengers.

The comparative positioning of all vendors in a Magic Quadrant offers a high-level view of the market. In an established, mature market, most vendors may cluster in the Leaders quadrant because most will tend to offer a complete set of products and services, with little differentiation. Conversely, in a growing market, most vendors will cluster in the Niche Players quadrant, and most offerings will be incomplete, yet distinctive.

Gartner’s interactive Magic Quadrant features provide additional insight into a market and its vendors. You may use these features to tailor your view of the Magic Quadrant and align it with your business goals, needs and priorities. The Customize tab in the interactive view allows you to adjust the weightings applied to each of the evaluation criteria — to generate a new Magic Quadrant graphic for that market tailored to your own requirements. You can then save and share your customized Magic Quadrant as part of your internal analyses and decision making. This client-customized view of the Magic Quadrant is intended to help you gain relevant insight into a market; it does not represent Gartner’s view of the market. Client-customized views of the Magic Quadrant are for internal use only and are subject to standard Gartner usage guidelines.

Gartner’s interactive Magic Quadrant also brings together our expert opinion and Gartner Peer Insights user-contributed reviews into one experience. The User Reviews tab enables you to read and evaluate what your peers have to say about the enterprise technology and services they have implemented and use in their organizations.

Some Magic Quadrants are:

• Accompanied by additional perspectives that emphasize what is distinct in select submarkets, such as industry, company size or region, where differentiation is relevant and provides unique considerations for technology and service selection for the given context. The Notable Vendors included in this research are a sample of vendors in the given market that clients should consider and that may or may not have been included in the full Magic Quadrant

• Complemented by a separate write-up that aims to meet Gartner clients’ evolving needs by reflecting merger, acquisition or divestiture by a vendor featured in a Magic Quadrant graphic. These notifications highlight only significant corporate transactions most relevant to technology portfolio decisions. The Magic Quadrant graphic and scoring remain unchanged.

Markets vary in many ways, but all follow a predictable life cycle with five phases:

• Emerging

• High growth

• Established

• Consolidating

• Declining

When new markets emerge and the offerings and user requirements fluctuate, solutions are often difficult to compare, making competitive positioning less useful. When a market emerges, it lacks definition and typically has many vendors offering point solutions, which makes comparative vendor assessments difficult, if not impossible. Low barriers to entry mean many vendors can enter the market without significant investment, and they can leave without significant implications for their broader business, too.

Similarly, when the market is highly mature, the products and services offered will typically be provided by only a few vendors and solution features or services can lack differentiation. When a market matures to the point where the offerings become relatively interchangeable, comparative positioning is less important than an analysis of and recommendations about the market itself.

In both these cases, a Gartner Market Guide provides a meaningful perspective of the underlying market, with a high-level overview of representative participants. The Gartner Magic Quadrant exclusively evaluates markets during the period of time where a market is established.

How does Gartner decide between a Magic Quadrant and a Market Guide?

Market maturity is the most important factor in determining the type of research Gartner will publish. By the very nature of these markets, determining the growth and maturity of a market is not an exact science. The key characteristics of the market outlined earlier in this document are all considered, and the decision is made using that information.

What is a Gartner Critical Capabilities document?

Critical Capabilities is an essential companion to the Magic Quadrant. A Critical Capabilities helps Gartner clients gain a view of the positioning of vendors’ product and service offerings, allowing comparison against a critical set of differentiators to support their strategic technology investment decisions. It shows which products or services are the best fit in various use cases and provides actionable advice to end-user clients on which products or services they should add to their vendor shortlists for further evaluation.

How does a Gartner Critical Capabilities differ from a Magic Quadrant?

Critical Capabilities is a companion note to the Magic Quadrant. Critical Capabilities analyzes products or services based on a set of use cases that match client deployment scenarios, while the Magic Quadrant positions vendors in a market.

Can a Magic Quadrant be used as a vendor selection tool?

A Magic Quadrant analyzes a market and can help you focus your search on important criteria. However, it will not provide the details needed (during the vendor selection process) to align your requirements with a particular vendor. Gartner provides an extensive range of complementary research — such as Critical Capabilities and Vendor Ratings — that can assist you further in your vendor selection process. Discussions with analysts can also help during the vendor selection phase.

Why aren’t all vendors from a market included in a Magic Quadrant?

In most cases, it would be impossible to cover every vendor in a market; there are too many, and/or some are irrelevant based on the inclusion criteria. Therefore, Gartner focuses its research in a way that will provide the greatest value to Gartner clients.

Does a vendor’s status as a Gartner client influence whether it appears in a Magic Quadrant?

No. Sales contracts, vendor requests and client engagements do not influence or drive inclusion in Magic Quadrant research.

What does it mean if a vendor isn’t included in a Magic Quadrant?

It means they did not meet the inclusion criteria defined for that Magic Quadrant. It does not imply the vendor is not viable or competitive. It might indicate the vendor’s strategy or functions differ from the inclusion criteria, or it addresses a different target market. We publish the inclusion criteria with the Magic Quadrant to help you understand why a vendor might not have been included.

Is a market always analyzed the same way?

No. As a market matures, it changes and so does the analysis. From year to year, the inclusion criteria, weightings and Gartner’s opinion about what it takes to maintain market strength may change. Each report includes a detailed description of market trends and vendor dynamics for that year and details the vendors added or dropped since the previous report.

When will a Magic Quadrant have additional perspectives?

For most markets, the globally broad perspective of a Magic Quadrant meets the needs of Gartner’s clients. Where appropriate, Gartner analysts will add a company size, industry or geographical perspective. The decision to include these perspectives will be based on whether sufficient submarket differentiation exists and is important to clients.

Will the Notable Vendors section of a Magic Quadrant contextualization include vendors not in the Magic Quadrant?

A vendor that did not meet the inclusion criteria for the Magic Quadrant could be included in the Notable Vendors section of a Magic Quadrant contextualization.

Is there a schedule for when Magic Quadrants will be published?

Yes. Gartner provides the Magic Quadrant publication schedule, which is updated monthly to reflect any changes and can be accessed on the Gartner Magic Quadrant and Critical Capabilities page.

How often is Magic Quadrant research updated?

Gartner typically updates each Magic Quadrant annually unless it is retired. However, some may be refreshed more or less frequently in response to dynamic changes in the market.

What is the meaning of the “as of” date, and why is it different from the publication date?

The “as of” date on a Magic Quadrant indicates when the research was completed and closed to all further input. Any events in the market after the “as of” date are not considered and do not form part of the assessment or the final report. Generally, the “as of” date is prior to the publication date of the report. However, analysts may choose to make any necessary adjustments up to the day of publication, in which case the “as of” date will match the publication date.

How can I compare a current Magic Quadrant to a previous version?

Historical Magic Quadrant research is available online as archived files. The interactive Magic Quadrant also shows the history of a specific market when past versions are available. When performing direct comparisons, you should understand the market definition and criteria may change from year to year and thus, will affect vendor positions and/or ratings. Also, due to mergers, acquisitions and rebranding, some vendors may appear under different names from one year to the next. Vendor and product names are accurate as of the date of publication and are not maintained or revised to reflect market activities that may occur after publication of that report.

Where can I get more information about interactive Magic Quadrants?

Interactive Magic Quadrants are available only to Gartner clients on gartner.com. Further information and FAQs about the interactive experience are available on the main Magic Quadrant page.

What is the relationship between Magic Quadrants, Critical Capabilities and Gartner Peer Insights?

Gartner Peer Insights provides ratings and reviews of software and services contributed by end-user professionals. Contributors share their firsthand experience of a vendor and product through the various stages of IT life cycle management, from evaluation through deployment. Gartner Peer Insights leverages the structure of the Market Description, Vendor List, Product List and Critical Capabilities (from the Magic Quadrant and Critical Capabilities) for its user review survey structure and the client experience. With this innovative design (analyst-led Magic Quadrants and Critical Capabilities, complemented by Gartner Peer Insights), organizations can count on Gartner as their must-have source for a full-spectrum perspective on markets, vendors, products and services.

What is the difference between Gartner Peer Insights Voice of the Customer, the Gartner Magic Quadrant and the Gartner Critical Capabilities?

The Voice of the Customer document represents the views of end users about the products they are using in a given market. It provides an aggregated view of those reviews published on Gartner Peer Insight as calculated under the Gartner Voice of the Customer methodology. A Gartner Magic Quadrant positions vendors in a market. It contains broader analysis of the vendors in a given market, as identified by the inclusion criteria. The Magic Quadrant is a snapshot in time that assesses the vendors’ vision and Ability to Execute. A Gartner Critical Capabilities complements Magic Quadrant analysis to offer deeper insight into the products and services offered by multiple vendors. Magic Quadrant, Critical Capabilities and Voice of the Customer are complementary perspectives to help end users that are making technology purchase decisions with their shortlisting process. They provide a view of expert-led research about vendors’ positioning in a market (Gartner Magic Quadrant), their products (Gartner Critical Capabilities), and the peer perspective on the use of individual products in their own organizations (Gartner Peer Insight).

What if there is a difference between how a vendor and product is rated in Gartner Peer Insights and Gartner research such as Magic Quadrants and Critical Capabilities?

Gartner Peer Insights reflects the individual opinions and experiences of end users who have submitted reviews and ratings through a rigorous verification, validation and moderation process. Analysts draw upon a large base of information from multiple sources to contribute to their research. While end-user feedback is important, so is feedback and input from vendors, public sources, investors, the press and social media. Lead authors of Magic Quadrants and Critical Capabilities will get access to Peer Insights review data that has been verified, validated and moderated. Analysts will be able to see data throughout the year to review crowd sentiment. A single Gartner Peer Insights review will not affect the placement of a vendor or product in a Magic Quadrant or Critical Capabilities. The Magic Quadrant and Critical Capabilities research process is based on a structured methodology to maintain independence and objectivity, and provide unbiased advice to clients.

Do Gartner’s research analysts use Gartner Peer Insights in their research?

Yes. Gartner Peer Insights reviews are one of the sources of customer input information that may be considered by Gartner experts as part of our rigorous research process. Gartner may consider other sources of customer input information in addition to Gartner Peer Insights, such as analysts’ inquiries with Gartner end-user clients, and primary research survey results. While end-user feedback is important, it is just one aspect in a vast range of criteria that are considered.

Gartner IT Glossary: Magic Quadrant

Gartner Magic Quadrant: Positioning Technology Players Within a Specific Market

Gartner Magic Quadrant: Frequently Asked Questions

Gartner Recommended Reading