Contents: The evolution Leveraging the Java platform JMX specification JMX model Inside the JMX agent Growing support Conclusions Resources About the author Rate this article Related content: Management application programming Subscriptions: dW newsletters dW Subscription (CDs and downloads)

Sing Li (mailto:westmakaha@yahoo.com?cc=&subject=Management, JMX 1.1 style)
Author, Wrox Press
24 September 2002

In this article, the first in a three-part series on Java Management Extension (JMX), Sing Li examines the history of network management software and how it evolved from humble beginnings into today's complex and sophisticated enterprise management systems. He also explores the origin of many common problems that plague these systems and how JMX can be leveraged to solve them.

Java Management Extension (JMX) is a hot new addition to the Java platform that promises a scalable, low-implementation cost, and legacy-compatible solution to the age-old problems associated with enterprise network management. Modern software servers, including popular open source servers like Jakarta Tomcat and JBoss, are quick to adopt JMX as their management standard. We'll begin our exploration of JMX by looking at the history of network management software and how it has evolved.

The evolution of network management
Early network bridges, protocol converters, and routers were simple dedicated hardware units typically configured through a terminal directly connected to a serial port on the unit itself. Configuration commands were usually used to enable or disable ports or change characteristics of the protocols supported by the device. The number of configurable parameters on these "black boxes" was limited, and the serial terminal interface was often cryptic and intelligible only to highly trained network operators, as shown in Figure 1:

Figure 1. Ad hoc serial connection

The proprietary network era
As networks grew in size, and the number of these "black boxes" proliferated, it became evident that some way to address and control a large number of these networking devices was necessary. Some vendors elected to provide a separate "out-of-band" interconnection or concentrator network for administration of these devices, as shown in Figure 2:

Figure 2. Out-of-band multiplexed control

Others used "in band" technology in which the addressing and control information was sent on the same network that these "black boxes" operated on, as shown in Figure 3:

Figure 3. In-band device control and management

Client software (called management consoles, a name inherited from the terminal console analogy) evolved slowly from the cryptic terminal-style interface, to GUI-enhanced configuration tools that enabled each addressable device on the network to be configured individually. This gave birth to early network management systems (NMSs). For the most part, proprietary networking protocols dominated during this time, and early NMSs typically used proprietary protocols to address, control, and monitor a vendor's devices.

TCP/IP becomes the de facto networking standard

Don't miss the rest of this series Part 2, "Beans, JMX 1.1 style" (October 2002) Part 3, "Hands-on JMX integration" (December 2002)

Enter the era of the TCP/IP network. Through an early incarnation of the Internet, the TCP/IP protocol suite became the standard for globally connected networks. Large businesses discovered they needed to connect their separately evolved, proprietary islands of heterogeneous internal networks (many of which were from different vendors on a global scale), yet at the same time not lose control of all their devices across the entire enterprise.

In addition, infrastructure builders were maintaining more and more networking black boxes in their operations. Frequently, this spanned multiple network operation centers across separate geographic regions. There was a need to administer, control, and monitor all these devices from a centralized location. The urgent requirements from these two large groups of customers spawned the concept of early enterprise NMSs.

The management consoles (clients) during this time used the latest graphics workstation and processing technology, and often included interactive geographic maps of where managed devices were located, as well schematic displays of the network mesh. By clicking on a map or a node of a GUI network graph, the network operations center staff could monitor the status of any particular device and change its setting and configurations. SNMP (Simple Network Management Protocol -- a protocol designed from the ground up for network management on top of TCP/IP networks) became more popular as the de facto standard for management of heterogeneous networks. While initially resistant to SNMP, for both practical and economical reasons, vendors of proprietary networking hardware eventually adopted it as an alternative way to manage their networking devices.

The PC and LAN revolution: extending enterprise management
Around the same time as the TCP/IP revolution, the popularity of the PC- and LAN-equipped office exploded. This created a need for some enterprise NMSs to take on a new dimension: managing and controlling PC workstations, printers, peripherals, and other LAN devices, as shown in Figure 4:

Figure 4. Adding PC and LAN devices to the management mix

Many vendors used their newly acquired capability as a marketing advantage to push their enterprise management systems, or EMSs (notice the de-emphasis of the word "network"). Commercial EMSs from leading vendors grew in both size and complexity during this time, while many vendors' products still maintained a large proprietary code base and supported all legacy vendor-specific systems. Existing GUI-based management console (client) software was "ported" again and again to the many different versions of emerging desktop operating systems, pushing the envelope and increasing the complexity of the underlying code base. The SNMP protocol was stretched to its limit supporting new and varying devices that it was never designed for. The number of managed network endpoints proliferated beyond manageable levels, and added-value intelligent management assistance was standard in almost every commercial EMS.

Internet and e-commerce revolution: Adding manageable software services to EMS
Continuing with the explosion of EMS-managed endpoints, the Internet revolution sent most EMS systems and architects into shock. Suddenly, large networks that were once disparate were connected through the Internet, and customers were requesting that this "network of networks" be manageable in the same user-friendly management consoles they knew and loved (see Figure 5).

The emergence of increasingly intelligent network devices, PCs, and peripherals created a demand for more and more intelligence in the daily management and monitoring of these endpoints. Furthermore, the increasing demand for conducting business over the Internet created a new breed of endpoints for the EMS to support: that of intelligent software servers/services.

It is not uncommon for a modern EMS system to have management access to every single database server, Web server, application server, and disk farm (new, highly intelligent, wide-bandwidth, adaptive disk farms are now called storage area networks, or SANs) in an enterprise across the globe!

Figure 5. Modern EMS

The need for distributed intelligence in managed networks
In the present age of EMS, intelligent management assistance is no longer an option, but a necessity. The need to deal with dynamic networks that are constantly changing in composition and topology is also a must. In addition, the ability to adapt to and support any new and future devices is a definite requirement. Yet these new EMSs must continue to support and work in harmony with the mishmash of legacy devices, protocols, and software that customers have invested heavily in during the arduous evolution. In fact, there is a strong demand to support existing GUI-based EMS clients due to the training invested in these complex tools. At the same time, new users to EMS are demanding the most modern Web-based management interface possible to enable remote management and monitoring.

Using these new EMS systems, you can still enable and disable ports on a legacy "dumb" networking black box, but you can just as easily issue commands like:

Automatically switch to backup application server clusters and database servers as necessary to maintain a 95% uptime over the next 24 hours; alert network operator on pager 808-555-1212 upon catastrophic failure.

In other words, you can also rely on intelligent management assistance to manage network failures as they occur in a typical application service provider (ASP) environment.

What sort of changes in software architecture and networking protocols enable this sort of sophisticated network management in the Internet age? In fact, the EMS scenario we've painted is still pretty much a "work in progress." JMX is a key step toward realizing the universal management of dynamic networks with complete backward compatibility and support for existing and legacy EMS infrastructures.

Leveraging the Java platform for network management
There are various features of the Java platform that make it a natural candidate for the implementation of complex network management solutions, including platform and OS independence, networking, and dynamic adaptability.

Platform and OS independence
Because of the Java language's platform and OS independence, it is no longer necessary for NMS developers to maintain a version of the code base for every OS and hardware platform combination. Instead, energy and effort to enhance product features can now be focused behind one single code base.

Networking
Unlike most other programming languages, networking is a core part of the Java platform. It is not an afterthought or a third-party library, and as such it is guaranteed and certified to work in unison with the rest of the OS/hardware platform. This is important because older network management products often rely on third-party networking libraries to fulfill requirements not satisfied by the platform/OS combination or to achieve targeted stability.

Dynamic adaptability
The ability to dynamically and securely load classes across the network can be readily leveraged by network management software. For example, a Java-based EMS can support new devices or services simply by "loading" its support module across a network, and software modules that implement network management intelligence can be dynamically upgraded on demand.

JMX: A network management specification
JMX is an industry-wide collaborative effort to create a set of specifications that will describe an extensible architecture, API, and a set of distributed services for network management using the Java programming language. It leverages the strengths of the Java platform for network management as detailed earlier. At the time of this writing, the most current specification is Java Management Extension Instrumentation and Agent Specification, v1.1.

Deliverables of JMX include a set of specifications, documentation for the APIs, a reference implementation conforming to the JMX spec, and a compatibility test suite. See Resources for access to JMX 1.1 specs and more information.

The goal of JMX is to define only the interfaces that make up the systems within the JMX architecture, but not to dictate implentations and policies when they are unneccessary. This philosophy is essential to gain support from vendors with vested proprietary interest in existing network and service management technologies. It will also enable new JMX-based applications to enjoy the maximum freedom and flexibility in design -- avoiding situations where a prescribed implementation and policy may not be appropriate for future needs. In this way, the reference implementation provides only one example of how the set of interfaces specified can be satisfied, but does not recommend or suggest ways to accomplish this.

Architecture and operational model of JMX
The architecture and operational model of JMX aim to satisfy the following goals:

• Scalability: The ability to adapt from the management of a handful of devices or services to the tens of thousands of manageable endpoints that an Internet-age enterprise may have
  
  
• Legacy integration and compatibility: The ability to work in harmony with existing NMS or EMS solutions, as well as legacy-manageable endpoints that may not support JMX
  
  
• Low cost implementation: JMX compatibility can be engineered into existing software products and devices easily, without major design and coding effort

In the JMX architecture, taking a three-level, divide-and-conquer, architectural approach reduces the complexity of scalable network management. The three loosely coupled levels are illustrated in Figure 6. They are:

• Instrumentation Level: In this level, the manageable endpoints (device, software service, and so on) are made accessible through JMX-specified interfaces. This is done by creating Java objects that expose configurable attributes, accessible operations, and events. These objects are called ManagedBeans (MBeans for short). Endpoints that are manageable through these objects are called JMX-manageable resources in the specification. Legacy non-JMX devices and servers (such as an SNMP-compatible device or subnetwork) can be readily "adapted" into a JMX-manageable resource by providing a Java MBean wrapper. JMX-manageable resources at this level can be designed completely independently of objects at any other JMX architectural levels.
  
  
• Agent Level: In this level, the internal architecture of JMX agents is exposed. A JMX agent is a software component that exposes a set of standardized agent services to remote management components and directly controls the JMX-manageable resources through their MBean interfaces. In fact, MBeans are managed inside a JMX agent through an MBean server that can load and unload MBeans dynamically. The interface to access the MBean server is specified by JMX. Value-added in a large EMS system, such as localized intelligent monitoring and automated response, can be implemented at this level of the JMX architecture. Agents at this level can be designed independent of objects at the other levels. Agents communicate with management applications through connectors or protocol adapters. However, the specification for these components are still works in progress.
  
  
• Distributed Services Level: In this level, the aim is to specify the interfaces that provide for JMX Manager components. JMX Managers can access agents or groups of agents to manage the JMX-manageable resources that are exposed by the agents. Essentially, these are interfaces for EMS application developers to program against. Manager components can be EMS applications or an intermediate intelligence layer that manages multiple agents and associated resources.

Figure 6. Three levels of architecture for JMX

Figure 6 illustrates the JMX objects at the three levels, as well as the interfaces specified by JMX 1.1. Leveraging object-oriented design and the Java programming language, each level in the JMX architecture is highly componentized and separated by well-defined interfaces. Writing compliant code to these well-specified interfaces ensures our instrumentation and agent logic will work with any JMX 1.1-compatible implementation. While the instrumentation and agent levels are mapped out in the JMX 1.1 specification, the distributed services level is still the subject of future specifications (shown in Figure 6 as a horizontal gray dotted line and as the dotted boxes above). Standard JMX interfaces to existing network management standards, including SNMP and CIM/WBEM are being worked on concurrently by other groups.

Standard versus dynamic MBeans
Core to the instrumentation level are the MBean interfaces. The MBean interfaces specify how attributes are to be accessed, how operations (similar to methods in the Java programming language) can be invoked, and how events are sent from the MBean. There are two main types of MBeans, detailed in Table 1.

Table 1. Standard and dynamic MBeans

MBean Type	Description
Standard MBean	All the attributes, operations, and events associated with a standard MBean are specified statically in its interface. They are fixed and do not change with time. Standard MBeans must implement a management interface that is coded to fixed coding conventions (called lexical design patterns in the JMX 1.1 specs), detailed in the MBean specification. For example, to instrument a class called WebServer using a standard MBean, its management interface must be called WebServerMBean. A JMX agent will typically use introspection to discover the management interface of a standard MBean.
Dynamic MBean	All dynamic MBeans implement the javax.management.DynamicMBean interface. The set of attributes, operations, and events associated with the MBean is not determined until run time, using the DynamicMBean interface. This caters to manageable JMX resources that have attributes, operations, and events that may vary over time (for example, an instrumented application server may support different attributes with different release levels -- Tomcat 4.1.4 versus Tomcat 4.1.5). It is also a natural fit for instrumentation of federated network technologies such as Jini/Jiro.

Dynamic MBean specializations: Model and Open MBeans
There are two subtypes of Dynamic MBeans for special purposes, shown in Table 2:

Table 2. Model and Open MBeans

MBean Type	Description
Model MBean	All JMX implementations must supply an instance of a Model MBean -- implementing the javax.management.modelmbean.ModelMBean interface. It must be called javax.management.modelmbean.RequiredModelMBean. Model MBeans provide a "ready-to-go" MBean implementation that you can use immediately to quickly instrument any JMX-manageable resources. It is a pre-fabricated, generic, dynamic MBean class, offered as part of the reference implementation, that already has implementations for all the required default behaviors -- allowing you to add or override those that you need to customize at run time. This enables Java-based, non-instrumented resources to provide a guaranteed compatible MBean facade at run time, enabling them to be managed through the JMX architecture.
Open MBean	An Open MBean is a dynamic MBean where all the attributes of the MBean belong to a set of specified Java data types (String, Integer, Float, and so on), and the bean provides self-describing data through the javax.management.openmbean.* set of interfaces. Any agent and any manager/EMS can easily manage the JMX-manageable resources represented by these beans. Details of Open MBean are not fully specified in JMX 1.1, and the reference implementation does not include them.

Figure 7 illustrates how MBean is used to add instrumentation to a device and a software service. Note that the corresponding MBean is the JMX-internal representation of the device or service. Any MBean registered with the MBean server (discussed later) in the JMX agent can have its management interface (attributes, operations, and events) exposed to remote NMSs or other JMX applications. When we add MBeans to instrument a device or software service, however, we do not have to consider what sort of JMX agent or NMS will be used for management.

Figure 7. Operational model of JMX

Persistent MBeans
Some MBeans may require persistence support to ensure proper operation. These MBeans should always implement the javax.management.PersistentMBean interface. This interface only has a save() and load() method. The persistent MBean implementation is responsible for calling the load() method during bean construction to initialize the MBean's state based on persisted values.

Inside the JMX agent
Figure 7 reveals the anatomy of a typical JMX agent. Note that the four major components inside an agent are the MBean server, a set of agent services, connectors and protocol adapters, and custom agent logic.

MBean server
The MBean server is a core component inside an agent. All MBeans must be registered with the MBean server before they can be accessed by remote applications. Registered MBeans are addressed by unique object names when working with the MBean server. Remote manager applications (or distributed services) can discover and access an MBean only through its management interface (exposed attributes, operations, and events).

Agent services
Agents also provide a set of agent services that can be used by the custom agent logic to operate on the registered MBeans in the MBean server. These services are mandatory for compliance to JMX 1.1 -- all agents must provide them. Interestingly, these services can be implemented in the form of MBeans themselves. Implementing a service in the form of an MBean can provide several benefits:

• The service's operations can be remotely accessed through a Manager component or EMS.
• The service itself can be remotely managed by an EMS, accessed from remote manager applications.
• Update of the service logic can be performed dynamically at run time by downloading the MBean.

Table 3 shows the set of agent services defined in the JMX 1.1 specification.

Table 3. Agent services required by JMX 1.1

m-let or Management Applet service	Enables dynamic class loading across the network from the URL location (see javax.management.loading.MLetMBeanand associated classes/interfaces).
Monitor Service	Converts an expensive remote-poll operation to a local one; will monitor an MBean's attribute for specified changes and send an event when the change is observed.
Timer Service	Sends an event when a specified amount of time elapses or periodically at a specified interval (see javax.management.monitor.MonitorMBean and associated/interfaces).
Relation Service	Enables the definition of relationships between MBeans and enforces integrity of the relationship(s) (see javax.management.relation.RelationServiceMBean and associated classes/interfaces).

Value-added agent logic
Value-added agent logic is typically the code that we write. It is the custom agent logic that may provide localized intelligence to manage the JMX-manageable resources registered with this agent. For example, if we have two redundant clusters of application servers, we can create a custom agent that monitors load levels and redirects incoming requests to different clusters dynamically -- by operating on the MBeans of the registered servers. Often, the vendor of an NMS may also supply custom logic. Once the specification of the distributed services solidifies, it is foreseeable that certain custom agent logic may work in unison with custom remote JMX manager components to provide higher-level network-management functions.

Connectors and protocol adapters
The agent does not communicate with distributed services, NMSs, or other management applications directly. Instead, connectors and protocol adapters are used. This architecture is consistent with the J2EE Connector Architecture (see Resources). A protocol adapter is a software component that provides access to the agent-managed resources through a standardized protocol, such as HTTP and SNMP.

A connector is a specialty software component that provides a remote interface to the agent and/or managed resources at the agent (typically done using remote procedure call technologies such as CORBA or RMI -- often over SSL for security). When an agent has multiple connectors and protocols active, the resource being managed may be accessed by multiple heterogeneous applications or NMSs simultaneously. Protocol adapters can be used to provide backward compatibility to existing and established NMSs. For example, we can create a common information model/Web-based enterprise management (CIM/WEBM) adapter for NMSs that supports this standard, or we could create a telecommunications management network (TMN) protocol adapter to enable operation, administration, and maintenance (OAM) on telecommunications resources managed by JMX agents. The precise specifications for JMX connectors and protocol adapters are subjects of specifications being developed concurrently (see Resources).

Growing application support for JMX 1.1
While only the instrumentation and agent level of the JMX architecture is fully defined, it has already gained widespread support among Java-based enterprise technology developers.

J2EE 1.4 and JMX
The August 18, 2002, proposed final draft version of the J2EE v1.4 (see Resources) details how JMX-based instrumentation will become a standard part of the J2EE Web container (JSP/Servlet Engine), EJB container, and application client container. This is a natural evolution because the wide deployment of J2EE-based servers is making manageability a definite requirement. (As you can imagine, manual or ad hoc configuration, management, and monitoring in a large site with hundreds of deployed servers is nearly impossible.)

Thus far, commercial server products have supported manageability through proprietary means or special additions that make the product compatible with existing NMS products. The standardization of JMX as the management substrate in the J2EE v1.4 specs will go a long way to ensure that server products from different vendors will be compatible with (and can be managed by) NMSs universally.

Implementation case study: Apache Jakarta Tomcat 4.1.x
Consistent with the J2EE v1.4 specifications, the latest beta version of the Tomcat 4.1.x server has already integrated JMX instrumentation. See the Resources section for information on where to download it. Most of the Tomcat instrumentation is centered on the configuration components, which correspond to run-time objects within Tomcat's Catalina engine. Configuration components (such as <Engine>, <Host>, <Server>, <Service>, <Connector>, <Context>, and <Realm>), formerly modifiable only through the server.xml and web.xml file in the conf directory, are now all instrumented through JMX MBean implementations.

If you examine the org.apache.catalina.mbeans package (either in the source code distribution or the API Javadoc), you will see all the JMX instrumentation MBean classes that expose these Tomcat configuration components. Some examples include org.apache.catalina.mbeans.StandardContextMBean that instruments the <Context> configuration component, while the org.apache.catalina.mbeans.MemoryUserDatabaseMBean instruments the <MemoryUserdatabase> component. If you examine the actual source code, you will find that the Tomcat 4 instrumentation makes extensive use of the Model MBean template supplied by the JMX implementation.

Currently, Tomcat 4.1.x uses the MX4J JMX implementation, an open source licensed JMX implementation, instead of Sun's reference JMX implementation. However, both implementations are completely JMX 1.1 compatible. See the Resources section to find a link to the latest version of MX4J.

One immediate benefit of the extensive Tomcat instrumentation is that a Web-based GUI administration application can be readily created. Tomcat 4.1.x provides such an application, called admin. The application itself is created using the Struts application framework and JSP technology. It accesses all the configuration components (same as those defined in the server.xml file) via the MBean server of the local JMX agent. The JMX agent is created, in Tomcat's case, by the org.apache.catalina.mbeans.ServerLifeCycleListener class. This is a LifeCycleListener that will be invoked upon Tomcat startup and shutdown.

To try out this JMX-based admin utility, you must first edit the tomcat-users.xml file under the conf directory of the Tomcat distribution. Make sure you add the admin role to the tomcat user:

  <user username="tomcat" password="tomcat" roles="tomcat,admin"/>

This is the file that will be used by the <MemoryUserdatabase> component to authenticate access during runtime, and a user must have the admin role to run the utility. Next, start up Tomcat 4.1.x.

Open a browser and type the URL:

http://localhost:8080/admin/

On the login page, type:

Username: tomcat
Password: tomcat

Now, explore the admin utility and observe how JMX MBean has enabled the real-time display and modification of the Catalina configuration component values. For example, Figure 8 shows the configuration page for the attributes of a <Context> configuration component:

Figure 8. Tomcat 4 GUI admin utility based on JMX 1.1

Of course, in addition to the admin Web-based application, the instrumentation of Tomcat also enables it to be managed by other agents and EMS.

Conclusions
JMX achieves its design goal of scalability, low implementation cost, and compatibility with existing technology by leveraging the Java platform and careful design. In the next article, we will work with the reference JMX 1.1 code, implement our own Standard MBean and Dynamic MBean, leverage the required Model MBean for quick instrumentation, and experiment with a JMX agent.

Resources

• Download the latest JMX specification, reference implementation, and compatibility test suite.
  
  
• For a completely open source implementation of JMX 1.1, also the current JMX used by Jakarta Tomcat, check out the MX4J Project.
  
  
• For more information on SNMP, consult the Internet Engineering Task Force's Request For Comments 1157 document, or see the latest developments in SNMP Version 3.
  
  
• For detailed information and the latest developments on CIM/WEBM, take a look at the Distributed Management Task Force's Web site.
  
  
• The latest version of Tomcat 4.1.x server, with JMX 1.1 instrumentation, can be downloaded from The Jakarta Project. Source code distribution is also available.
  
  
• For on-going work on the specification of JMX distributed services, protocol adapters, and connectors see JSR 160 JMX Remoting 1.2, JSR 146 WEBM Services, and JSR 70 IIOP Protocol Adapter for JMX.
  
  
• JMX features an agent connector/adapter architecture similar to the J2EE Connector architecture.
  
  
• J2EE 1.4 will feature completely JMX-instrumented containers. Download the latest version of the emerging J2EE 1.4 specification.
  
  
• Tivoli software from IBM features secure enterprise management solutions that support a wide variety of existing and emerging network management standards, including TMX4J, a JMX-compatible implementation, available from alphaWorks.
  
  
• "Management application programming" (developerWorks, May 2001) is the first in a three-part series that looks at how Sun Microsystems's Jiro technology and the Distributed Management Task Force's Web-Based Enterprise Management Initiative (WBEM) can simplify the creation of management applications for heterogeneous environments.
  
  
• In "Developing JSP tag libraries with Apache Tomcat" (WebSphere Developer Domain, May 2001), get hands-on experience attaching VisualAge for Java to an installed Apache Tomcat 4.0 server.
  
  
• JSR109, Web services for J2EE, (developerWorks, August 2001) defines how Web services are supported in a J2EE 1.3 or J2EE 1.4 application server.
  
  
• Be sure to visit the newest developerWorks product domain Tivoli Developer Domain for technical information about Tivoli software and security products.
  
  
• Find hundreds of Java technology-related resources on the developerWorks Java technology zone.
  
  

About the author Sing Li is the author of Early Adopter JXTA and Professional Jini, as well as numerous other books with Wrox Press. He is a regular contributor to technical magazines, and is an active evangelist of the P2P evolution. Sing is a consultant and freelance writer, and can be reached at westmakaha@yahoo.com.

What do you think of this document? Killer! (5) Good stuff (4) So-so; not bad (3) Needs work (2) Lame! (1) Comments?

developerWorks > Java technology