“Big Data” is the big technology buzz word of the moment.
Due to the promise of “Big Data”, most enterprises are now exploring the power and potential of Big Data and predictive analytics, as well as starting to learn how they can harness Big Data to help them accomplish business goals.
Some of the earliest adopters of Big Data have been financial services companies. However, as these earlier adopters have discovered, one of the biggest issues they face is with monitoring the performance and availability of their large, mission-critical Big Data data warehouses.
This of course naturally leads to an important question — as new technologies related to Big Data emerge, are your IT operations agile and flexible enough to accommodate them?
If not, then Big Data could become your next Big Problem.
New Technologies Like Big Data Require a Highly Extensible Monitoring Approach
In order to quickly incorporate new technologies into your enterprise IT monitoring processes, you need a flexible, highly extensible monitoring solution — one that is agile enough to be quickly adapted and extended as your enterprise IT infrastructure changes in response to new technologies.
By design, Zenoss Service Dynamics was built to easily extend and accommodate new and emerging technologies like those that enable Big Data. Our flexible, highly extensible ZenPack model allows both Zenoss itself, as well as Zenoss customers, to easily extend discovery and monitoring of any new technology that comes along. As long as it uses some type of protocol to communicate, we can incorporate it.
A great example of our extensible approach can be seen in our newly released ZenPacks for Apache Hadoop, HBase, and ZooKeeper.
Hadoop, HBase, and ZooKeeper are highly complex technologies, but it took our internal ZenPack development team less than two months to build and deliver them to our QA team for testing. Now that these ZenPacks are out and available, you can install them and start monitoring Hadoop, HBase, and ZooKeeper in a matter of hours.
New Hadoop, HBase, and ZooKeeper ZenPacks for Big Data
If you are currently using or plan to use Hadoop, HBase, or ZooKeeper in your environment, check out some of the key performance and availability monitoring capabilities for Big Data these new ZenPacks provide.
Hadoop ZenPack
The Hadoop ZenPack includes the following key capabilities:
- Discovery: Automatically discovers data node, name node, job tracker roles, and what systems they are running on.
- Performance Monitoring: Tracks memory utilization, file blocks written/read/removed, cluster health, number of threads running on nodes, jobs running/failed/completed.
- Event Management: Alerts administrators if nodes are down, job tracker is down, or there are errors with receiving data.
- Service Impact: Provides real-time view of how Hadoop cluster, node, or job tracker failures impact supported services.
HBase ZenPack
The HBase ZenPack includes the following key capabilities:
- Discovery: Automatically discovers region servers, regions, and tables.
- Performance Monitoring: Tracks memory utilization, disk space, read/write metrics, average region server load, and number of dead/live/total region servers.
- Event Management: Alerts administrators if percentage of dead region servers exceeds threshold, memory store is nearing or exceeding maximum or flush size, region servers are added or removed, or tables are dropped or disabled.
- Service Impact: Provides real-time view of how region server or table failures impact supported services.
ZooKeeper ZenPack
The ZooKeeper ZenPack includes the following key capabilities:
- Discovery: Automatically discovers ZooKeeper nodes and node count for ZooKeeper clusters.
- Performance Monitoring: Monitors the status of ZooKeeper nodes and metrics around number of requests, number of connections, and latencies — including average, minimum, and maximum latencies — so you can understand node health and whether any nodes are under duress. Also monitors for sent and received packets.
- Event Management: Provides built-in thresholds for number of requests, connections, and latencies that administrators can use to alert them when issues arise.
- Service Impact: Provides real-time notification if any device that ZooKeeper is running on fails. If ZooKeeper itself fails, notifies you that all attached Hadoop and HBase components are also impacted or down due to the ZooKeeper failure.
Next Steps
The effort and compute power going into Big Data projects is considerable. Monitoring is a key aspect for projects that include Big Data, and ensuring that you have the resources you need to support your goals is critical.
If you’re starting to plan Big Data projects with using Hadoop, HBase, or ZooKeeper technology, ensure you carve out at least a few minutes to explore our new Hadoop, HBase, and ZooKeeper ZenPacks so you can see how these ZenPacks can give you the monitoring support you need for your Big Data projects.
New to Zenoss Service Dynamics?
If you are new to Zenoss Service Dynamics, check out the following links, which provide more information about Zenoss and the Zenoss Service Dynamics platform:
- Watch a demo that gives you an overview of Zenoss and how Zenoss can help your more efficiently and cost-effectively monitor your environment.
- Read Unified Service Insight from Zenoss to get a quick overview of how Zenoss Service Dynamics provides Unified Service Insight into the end-to-end operation of IT infrastructures and supports service delivery, improves service quality, and reduces your IT operational costs.
- Read Zenoss Service Dynamics: Technical Feature Overview to learn more about the technical features available in Zenoss Service Dynamics.
- Request a free trial! See how you can use Zenoss more in your environment to more effectively and cost-effectively monitor and manage your environment using a single, unified monitoring view and unified monitoring processes.
