APM Tools are a massive investment. This is our evaluation criteria we recommend to all of our clients.
1. Does the product help you solve problems?
This one may seem obvious, and it is. However, this simple requirement can sometimes get lost when looking at a lot of slick displays. When the money is spent and the vendor is gone, will the product improve your situation?
2. Can the monitoring solution harm the environment it is meant to protect?
If it can, it almost certainly, at some point will. Human beings make mistakes. The APM solution you’re buying, no matter how automated, is still run by human beings. It is up to you to consider the risk that brings to your application. Normally the benefits outweigh the risks, but sometimes a purely passive solution is the only way to eliminate the risks entirely.
3. Have you calculated the true Total Cost of Ownership?
The obvious costs are licensing fees and maintenance fees. Naturally you’ll need a person or people to run the tool and use it. However, many fail to account for the hardware costs, administration costs beyond the primary install point (agent management, etc) and the training costs associated with learning the tool. Ultimately, if you’re not seeing a net improvement on all of that, then you probably need to keep looking for other tools.
4. Does the product integrate well with your other solutions?
An APM tool looks at the end user experience and generates events based upon that. Can the tool integrate well into an event aggregator? Can the APM tool itself become the single pane of glass for your component level monitoring? Does the tool have seamless integration with your Incident Management System? These are things to consider before pulling the trigger on an APM tool purchase.
5. Does the licensing model make sense for your organization’s needs?
Some products charge by an agent, some charge based on usage, and other based on appliances. Each have their pro’s and con’s. Make sure the product you land on fits with the overall growth plan you have in mind for the use of the product. Often, agent-based solutions start out cheap and then quickly become unmanageably expensive, while appliance-based solutions have a higher up-front, but are easy to scale to large environments.
6. How readily available is support?
When considering an APM vendor, ensure that support will be reliable, quick, and efficient. If they don’t have a local support office, make sure you have clear escalation paths to make absolutely certain that your issues will be solved in minutes, not in days or months.
7. What is the learning curve to use the tool?
Is the tool too hard to use? Is there appropriate community and/or vendor support? Is the vendor going to use their bad user interface design to sell you more services? If so, you may want to look elsewhere.
8. Does the product scale well for your environment?
At the end of a proof of concept, be sure to ask about capacity issues. A key indicator is how much capacity of their product was used to complete the proof of concept? If the answer is more than 15-20%, then you may want to consider looking closely at what it would take to expand the capacity to what you’ll need before making a buying decision.
So that’s it! Just a bit of full disclosure though… obviously this list is going to be biased… I mean come on, we sell APM software! However, another service we offer is consulting with companies on how to implement APM strategies with or without our tool. This is a list of considerations we give to everyone. That said, our product suite was built around these principles as opposed to this list being built around our product. Good luck out there!
ABOUT THE AUTHOR
Matthew Bradford has been in the I.T. Performance Business for 15 years and has been critical to the success of many Fortune 500 Performance Management groups. He is currently the CTO of InsightETE, an I.T. Performance Management company specializing in passive monitoring and big data analytics with a focus on real business metrics.