Here's the task list:
- find the required data
- gather the data somehow
- scrub with prometheus
- display with grafana
- admire with satisfaction ;)
First we need to find some source of air pollution data, ideally it should provide an API we can use easily. One such case is World Air Quality Index. They gather data for more than 9k stations in 800 cities from 70 countries thanks to worlds Environmental Protection Agencies. API is easy, available and exports as JSON, all you really need is a token. A searchable cities list is also available: http://aqicn.org/city/all/.
Now let's gather the data by creating our own exporter for prometheus. We can use the code created by robustperception and documented by the prometheus team:
#!/usr/bin/python
# -*- coding: UTF-8 -*-
from prometheus_client import start_http_server, Metric, REGISTRY
import json
import requests
import sys
import time
class JsonCollector(object):
def __init__(self):
pass
def collect(self):
# Fetch the JSON
token = 'demo-token-won\'t-work' #apply for a token here: http://aqicn.org/data-platform/token/
url = 'http://api.waqi.info/feed/'
city = 'Wrocław' # Cities: http://aqicn.org/city/all/
response = json.loads(requests.get(url+city+"/?token="+token).content.decode('Utf-8'))
metric = Metric('aqi_airquality', 'Air Quality Index', 'gauge')
metric.add_sample('aqi_airquality', value=response['data']['aqi'], labels={})
yield metric
metric = Metric('aqi_co', 'CO gases', 'gauge')
metric.add_sample('aqi_co', value=response['data']['iaqi']['co']['v'], labels={})
yield metric
metric = Metric('aqi_humidity', 'Air Humidity', 'gauge')
metric.add_sample('aqi_humidity', value=response['data']['iaqi']['h']['v'], labels={})
yield metric
metric = Metric('aqi_no2', 'NO2 gases', 'gauge')
metric.add_sample('aqi_no2', value=response['data']['iaqi']['no2']['v'], labels={})
yield metric
metric = Metric('aqi_pressure', 'Air Pressure', 'gauge')
metric.add_sample('aqi_pressure', value=response['data']['iaqi']['p']['v'], labels={})
yield metric
metric = Metric('aqi_pm25', 'pm2.5 particles', 'gauge')
metric.add_sample('aqi_pm25', value=response['data']['iaqi']['pm25']['v'], labels={})
yield metric
metric = Metric('aqi_temp', 'Air temperature', 'gauge')
metric.add_sample('aqi_temp', value=response['data']['iaqi']['t']['v'], labels={})
yield metric
if __name__ == '__main__':
# Usage: airq_exporter.py port
start_http_server(int(sys.argv[1]))
REGISTRY.register(JsonCollector())
while True: time.sleep(1)
Ok so now to run our exporter on localhost and port 9999:
./airq_exporter.py 9999
Opening http://localhost:9999 should give a similar result:
telnet localhost 9999
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET /metrics HTTP/1.0
HTTP/1.0 200 OK
Server: BaseHTTP/0.3 Python/2.7.12
Date: Wed, 15 Feb 2017 12:06:22 GMT
Content-Type: text/plain; version=0.0.4; charset=utf-8
# HELP aqi_airquality Air Quality Index
# TYPE aqi_airquality gauge
aqi_airquality 219.0
# HELP aqi_co CO gases
# TYPE aqi_co gauge
aqi_co 12.7
# HELP aqi_humidity Air Humidity
# TYPE aqi_humidity gauge
aqi_humidity 80.0
# HELP aqi_no2 NO2 gases
# TYPE aqi_no2 gauge
aqi_no2 42.6
# HELP aqi_pressure Air Pressure
# TYPE aqi_pressure gauge
aqi_pressure 1037.0
# HELP aqi_pm25 pm25
# TYPE aqi_pm25 gauge
aqi_pm25 219.0
# HELP aqi_temp Air temperature
# TYPE aqi_temp gauge
aqi_temp 4.77
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 132415488.0
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 19132416.0
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1487149538.19
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 6.08
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 7.0
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 1024.0
Connection closed by foreign host.
So as you can see we gathered all the important data and there are also some typical prometheus client metrics.
Scrubbing the data is really simple, just edit your prometheus.conf and add:
- job_name: 'airq'
static_configs:
- targets: ['IP:9999']
Remember to reload prometheus;)
Next step -> grafana. This is fairly easy now, since we have the data and need to present it. Choose anything that suits your needs. End result could look like this:
 |
| Grafana Dashboard for AirQuality |
Looks like all the TODOs have been checked by now... so the last thing to do is 'admire with satisfaction;)'
All the code is available on github.
