Monitoring Your PiCluster with Elasticsearch and Kibana

https://www.linux-toys.com/archives/1494

Share

PiCluster is a simple way to manage Docker containers on multiple hosts. It will work on x86 or ARM CPU architectures that can run Docker and Node.js. PiCluster 2.3 will be out soon and I wanted to share the exciting new Elasticsearch integration currently in the dev branch.  Traditionally, PiCluster had the ability to store its log output in Elasticsearch. Starting in PiCluster 2.3, node metrics such as disk, cpu, and memory utilization is stored also. With this ability, users can easily view their host metrics over time.

Requirements

• PiCluster
• Elasticsearch 6+
• Kibana 6+

Requirements

If you want to quickly spin up an Elasticsearch cluster with Kibana in Docker,  check this out.

PiCluster Configuration

PiCluster needs to be configured for Elasticsearch so it knows where to send data. All you need to do is add the following config option in config.json that points to your Elasticsearch endpoint and restart the PiCluster server process:
"elasticsearch": "http://my-es-endpoint:9200"

Kibana Configuration

Starting in PiCluster 2.3, there are pre-set graphs available for viewing host metrics (cpu, memory, and disk).

Creating the Index Pattern

1. Click on Management -> Index Patterns.
2. Click  Create Index.
3. Under Index Pattern, type  picluster-monitoring
4. Click  Next Step
5. Under  Time Filter field name, choose date.
6. Click  Create.
7. Repeat steps 1-6 for the  picluster-logging Index Pattern.

Importing the Graphs

To import the PiCluster Monitoring graphs in Kibana:

1. 1. Click on Management -> Saved Objects.
   2. Click import.
   3. Navigate to the PiCluster Git directory and choose kibana-graphs.json
   4. Click open.
   5. Repeat steps 1-4 for kibana-dashboard.json

Viewing the Dashboard

1. Click on Dashboard
2. Choose PiCluster Monitoring

Viewing the Graphs

1. Click on  Visualize.
2. Choose CPU, Disk, or Memory.

Viewing the PiCluster Logs

1. Click on  Discover.
2. Change the index to  picluster-logging


Feel like trying it out? Check out our dev branch to start playing with it now or wait for it to hit our main branch! To learn more about how PiCluster works, Check out our Wiki page.