Monitoring Order Processing with AppDynamics
Published: June 2019
Delaying a customer order can have a big impact on our business and the business of our customers. Working to process customer orders in a way that is timely, accurate, and efficient is one of the most critical IT services for Cisco's sales and operations. But sometimes order processing gets delayed, and finding out why and fixing it is extremely difficult.
Figure 1: Cisco order processing architecture
Every Cisco product order goes through multiple steps, databases, and applications in both cloud and on-premises systems. (Figure 1) IT and application teams need the ability to look inside that technology stack to monitor issues and performance in every layer and at every connection point. By using AppDynamics solutions, we have the right tools for monitoring the end-to-end processing of order-related transactions.
Challenges of managing complex order processing
Before a received order is released to production, it must go through several processing steps in our control systems, such as validation and booking. Additionally, the order data must be sent to several systems that handle tasks such as customer notification, sales compensation, and reporting before the order can be fulfilled and invoiced.
Behind these systems is a MongoDB database that was difficult for us to monitor because it required multiple custom scripts and tools, and none could provide an end-to-end view of performance and status. These limitations made it difficult and time-consuming to pinpoint the root cause of database problems and performance issues. In most cases, we had to find, combine, and analyze historical data from multiple logs in an attempt to identify the source of the problem.
We wanted to replace these multiple tools with a single, unified monitoring solution that would provide end-to-end visibility for all components in the ordering applications, including the MongoDB database, Java applications, elastic search, and data collection clusters.
Monitoring with AppDynamics solutions
Solutions from AppDynamics give us the capabilities we sought for database and application monitoring.
We use the AppDynamics Database Visibility solution as a single tool to monitor approximately 130 MongoDB replica sets. This solution gives our database administrators and application teams full visibility into the performance of the database; we can also see server performance through metrics such as CPU and memory utilization.
The feature to define alerts with predefined thresholds helps us monitor metrics such as database availability, member states, connections, and replication lag. The AppDynamics web hooks publish alerts to our custom dashboard and also to a unified monitoring system, where IT and application teams can easily integrate the alerts into their own dashboards and notification systems.
We also use the AppDynamics Application Performance Management (APM) solution to monitor all application components because it offers granular details of application transactions with a drill-down capability. APM provides alerts on metrics for application performance levels and compliance with service-level agreements (SLAs). We use the AppDynamics software-as-a-service (SaaS) deployment model and have installed database agent components in on-premises servers.
We leveraged AppDynamics APIs to automate the collector configuration and define health rules that enable monitoring as part of our automation for self-service infrastructure provisioning.
Benefits for IT and applications teams
By using AppDynamics solutions, we are able to proactively react to any issue in the application, database, or platform layer for our order processing applications--often before that issue is detected by our customers and business users. This proactive monitoring has reduced our IT support case volume and allows us to address issues proactively when an order is stalled or progressing slowly.
Additional benefits gained by IT and application teams include:
- A single monitoring solution for all database and application components with real-time, systemwide visibility of application and database performance metrics
- Database visibility monitoring continues even if the database host is disabled, eliminating the need for multiple monitoring layers
- Less time is required to resolve issues, which reduces work for the IT and application teams and the impact on business activity
- Increased customer and partner satisfaction because fewer orders experience problems
- Improved transaction efficiency with the capability to track order processing end-to-end, giving us more insights about where delays may occur
- Historical performance data helps us optimize cloud database resources with auto-scaling and gain input for future capacity planning
- Monitoring is easy to implement for new database instances using AppDynamics APIs