Monitoring JBoss Wildfly with Jolokia and Graphite

Once you have a running Java application server (in production), you are need of some kind of monitoring of the JVM’s main parameter (Memory, Threads), the datasources, queues, calls and different metrics provided by the applications. In this post I’ll show how to set up a monitoring chain for JBoss Wildfly. The main ingredients are the JMX subsystem of the Java Virtual Machine, Jolokia, collectd and Graphite.

Graphite is a widely used monitoring system. There are many good tutorials which show how to set it up and running, so in this tutorial you have already a graphite server running at a server graphite.example.com. I also assume that the JBoss server is already installed, so I’ll skip this too. The JBoss server shall be running at jboss01.example.com

First you need to download the WAR provided by Jolokia and deploy it to JBoss Wildfly. Jolokia is a JMX agent which allows you to query all JMX Beans over HTTP.


/opt/wildfly/bin/jboss-cli.sh --connect "deploy jolokia.war"

Now you can check http://jboss01.example.com:8080/jolokia/ that Jolokia is deployed. The response will return some basic information about Jolokia as JSON.

Next we’ll need to install collectd, a general monitoring agent which can retrieve monitoring data from many services through its plugins. We will install collectd and the JSON-curl plugin which is needed to read the data from Jolokia on jboss01.example.com. In Fedora Linux this can be done with


yum install collectd collectd_json-curl
systemctl enable collectd

Before we start the collecd agent we need some configuration. In /etc/collectd.conf you need to enable the plugins for graphite and curl_json.


#
# Config file for collectd(1).
# Please read collectd.conf(5) for a list of options.
# http://collectd.org/
#
...
LoadPlugin curl_json
...
LoadPlugin write_graphite
...
<Plugin write_graphite>
<Node "jboss01">
Host "graphite.example.com"
Port "2003"
Protocol "tcp"
LogSendErrors true
# Prefix "collectd"
# Postfix "collectd"
# StoreRates true
# AlwaysAppendDS false
# EscapeCharacter "_"
</Node>
</Plugin>
...
Include "/etc/collectd.d"

and for specifying the JMX values which we want to monitor we set up a new file /etc/collectd.d/wildfly.conf with


<Plugin curl_json>
<URL "http://jboss01.example.com:8080/jolokia/read/java.lang:type=Memory">
Instance "Memory"
<Key "value/NonHeapMemoryUsage/*">
Type "memory"
</Key>
<Key "value/HeapMemoryUsage/*">
Type "memory"
</Key>
</URL>
<URL "http://jboss01.example.com:8080/jolokia/read/java.lang:type=Threading/ThreadCount">
Instance "ThreadCount"
&;ltKey "value">
Type "gauge"
</Key>
</URL>
<URL "http://jboss01.example.com:8080/jolokia/read/java.lang:type=OperatingSystem/OpenFileDescriptorCount">
Instance "OpenFiles"
<Key "value">
Type "gauge"
</Key>
</URL>
<URL "http://jboss01.example.com:8080/jolokia/read/jboss.as:subsystem=datasources,data-source=ExampleDS,statistics=pool">
Instance "ExampleDS"
<Key "value/ActiveCount">
Type "gauge"
</Key>
<Key "value/AvailableCount">
Type "gauge"
</Key>
</URL>
</Plugin>

The syntax is pretty straightforward. Each JMX bean you want to query is specified with the URL with which it will be retrieved from Jolokia. Each attribute from the JSON result set is defined by the Key. You can use wildcards within the keys. This example retrieves the JVM parameters for Memory, Threads and Open Files as well as the key values of the DataSource ExampleDS. If you want to monitor different values you need to provide the name of the JMX bean. jvisualvm is a great tool to inspect JMX beans. JBoss Wildfly provides a lot of JMX beans out of the box (e.g. for JMS queues) and if you enable the statistics for EJBs, Transactions, Servlets etc. all of those can be retrieved like this. You can also write your own JMX beans and deploy it with your application code.

Now you can start the collectd service with

systemctl start collectd

and should the the values appear at jboss01_example_com.curl_json-INSTANCE, with the name of the instance you have defined in the configuration file above.

Tagged : ,

7 Responses to “Monitoring JBoss Wildfly with Jolokia and Graphite”

  1. Seriously?

    I’m searching google for information about collectd, curl_json and jolokia, because of jboss queue graphs, and I get this page?

    Your Operations Manager IT

  2. Asaf Mesika Says:
    November 27, 2015 at 10:03 pm

    You can save your self the trouble of:
    * Installing collectd
    * configuring it
    * Writing down any MBean and any attribute you want to send to Graphite – very tedious.

    How? Check out jmx2graphite (https://github.com/logzio/jmx2graphite). It’s one liner command line that polls JMX (Jolokia based which is easier) and writes it to Graphite every interval seconds. It polls all metrics so you won’t have to specify any bean by hand.

  3. Jolokia Newbie Says:
    March 3, 2016 at 2:43 pm

    Hello,

    i have tried this tutorial but i’m stuck at the Point of checking, if Jolokia is deployed corretly. When i try to call http://localhost:8080/Jolokia/ it just returns 404 not found. I just downloaded the .war file and put it into the deploy directory of my Wilffly Server and rebooted. What went wrong ?

    • If you spell it “/Jolokia/” then your War-File needs to be “Jolokia.war” (with an upper J). You should have a look at the file you have downloaded and rename it to “jolokia.war”

  4. Abhishek Chordia Says:
    August 11, 2016 at 11:24 am

    I am getting below error while staring collectd :

    [root@app03 abhishek.chordia]# /etc/init.d/collectd start
    Starting collectd: Could not find plugin write_graphite.
    [ OK ]

    Can you please help me on this.

    • Some hints:
      – in the configuration of collectd you need to have a line “LoadPlugin write_graphite”
      – depending on your OS, you may need an additional package for the graphite support; I’m using Fedora, there are many plugins (dnf search collectd), but the support for graphite is part of the core package collectd.

Leave a Reply to admin