/ OPERATIONS

Custom Metrics with Wavefront

Hi there! I’ve been playing with Wavefront on and off not long after the acquisition (May, 2017), trying to wrap my head around the space Wavefront operates in. Previously Wavefront’s audience wasn’t typlically in VMware’s domain as the primary use case is application metrics by instrumenting code, others include infrastructure metrics, cloud metrics and custom metrics. Now, because Wavefront is a time series metric platform, you can ingest any metric so long as the formatting is correct.

A good Youtube video by Coda Hale was recommended to me, it helped me understand why metrics were so important to developers. Long video, not the best quality but worth the watch.

In this blog, I will focus on custom metrics. Ingesting electricity production metrics from my solar array, consumption metrics from the mains, determining net kW, import kW and export kW all visualised through Wavefront!

Background

I had a solar array installed shortly after our first born because there was going to be consumption during the day (wife being home), rise of electricity costs and having a north facing roof was a bonus. All these factors were going to guarantee savings after a few years, however after some time I wasn’t convinced I was getting the most value out of them or something was wrong with the panels and/or inverters, the quarterly electricity bill was proof!

My goal was to see how I was consuming electricity and when, then try to move the consumption to when production was occurring. For example, by shifting when we use the clothes washing/drying and dishwashing.

Luckily, I choose a solar platform that included a device to monitor the arrays health and operational status. In fact, it would send data back home and there was even a online portal to view production and consumption statistics. The problem was the data being reported was aggregated to a 15-minute data point, was often slow, delayed and individual panel monitoring attracted a fee.

The good news was that I was able to read directly of the device via HTTPS and better yet, based on the URL the response body was in JSON. During my testing, every refresh showed a different measurement and epoch timestamp. I knew this was going to give me far better accuracy, included individual panels data and near real-time.

Prerequisites

  1. Install Wavefront Proxy1
  2. Scheduler to execute the script based on a 1 minute schedule2

If your Wavefront Proxy is external then you need to change the line sock.connect(('127.0.0.1', 2878)) or make it a variable, mine is local to where its executed.

I used a cron job executing based on a 1 minute schedule, translating to one data point per minute, this was more than sufficient. I installed the Wavefront Proxy on the same host as the Cron job was executing from.

Exporting Data to Wavefront Proxy

There are two sections to the Python script:

  1. Reading the data from the Enphase Envoy and then loading the response into a JSON object
  2. Sending the formatted metric to the Wavefront Proxy

I won’t go into details about reading the data as its probably different depending on your use case, what’s important is formatting the data so that it can be sent to Wavefront.

Metric Format

Wavefront accepts the following format metricName metricValue timestamp source=[source] pointTags, the fields must be space separated and each line must be terminated with the newline character (\n or ASCII hex 0A)

In my case, the production metric on October 2nd 2018 at 11:05:58AM is envoy.production.watts 3215 1538442358 source=tp-lid-env01.tphome.local

Python Script

Here is the script I created, for the most recent version grab it off GitHub

envoy_fqdn = 'tp-lid-env01.tphome.local'
envoy_username = 'envoy'
envoy_password = '000112'

import datetime

def read_envoy_prod_data( envoy_ip_addr ):
	# gets production json data from the envoy

	import urllib.request
	import json
	import socket
	import time

	socket.setdefaulttimeout(30)

	prodStr = '/production.json'

	# build the full url to get the production data
	url = 'http://' + envoy_fqdn + prodStr

	# https://docs.python.org/3.4/howto/urllib2.html#id5

	try:
		response = urllib.request.urlopen(url,  timeout=30)
	except urllib.error.URLError as error:
		print('Data was not retrieved because error: {}\nURL: {}'.format(error.reason, url) )
		quit()  # exit the script, some error happened
	except socket.timeout:
		print('Connection to {} timed out, '.format( url))
		quit()  # exit the script, cannot connect

	try:
		# convert bytes to string type and string type to dict
		string = response.read().decode('utf-8')
	except urllib.error.URLError as error:
		print('Reading of data stopped because error:{}\nURL: {}'.format(error.reason, url) )
		response.close()  # close the connection on error
		quit()  # exit the script, some error happened
	except socket.timeout:
		print('Reading data at {} had a socket timeout getting inventory, '.format( url))
		response.close()  # close the connection on error
		quit()  # exit the script, read data timeout

	json_prod_data = json.loads(string)

	# close the open response object
	#urllib.request.urlcleanup()
	response.close()

	# print pretty JSON
	#print(json.dumps(json_prod_data, indent=4))

	return json_prod_data
	# end of read_envoy_prod_data() function

def main():

	# calls functions to get envoy data and pushes to Wavefront Proxy

	import json
	import time
	import socket

	sock = socket.socket()
	# wavefront proxy on localhost
	sock.connect(('127.0.0.1', 2878))

	prod_data = read_envoy_prod_data(envoy_fqdn)
	# reading timestamp
	#timestamp = time.strftime("%a, %d %b %Y %H:%M:%S %Z", time.localtime(prod_data['production'][1]['readingTime']))
	epoch = str(prod_data['production'][1]['readingTime'])
	#print('Timestamp: ' + epoch)

	# calculating production
	cur_prod = round(float(prod_data['production'][1]['wNow']),2)
	# overriding envoy data when value is less than zero
	if cur_prod < 0:
		upd_cur_prod  = 0
	else:
		upd_cur_prod  = cur_prod
	# sending production metric to wavefront
	met_cur_prod = 'envoy.production.watts' + ' ' +  str(upd_cur_prod ) + ' ' + epoch + ' ' +  'source=' + envoy_fqdn + ' \n'
	sock.sendall(met_cur_prod.encode('utf-8'))
	#print(met_cur_prod)

	# calculating consumption
	cur_cons = round(float(prod_data['consumption'][0]['wNow']),2)
	# sending consumption metric to wavefront
	met_cur_cons = 'envoy.consumption.watts' + ' ' + str(cur_cons) + ' ' + epoch + ' ' +  'source=' + envoy_fqdn + ' \n'
	sock.sendall(met_cur_cons.encode('utf-8'))
	#print(met_cur_cons)

	# calculating net consumption
	upd_cur_net_cons = round(float(upd_cur_prod  - cur_cons),2)
	# sending net consumption metric to wavefront
	met_cur_net_cons = 'envoy.net.watts' + ' ' + str(upd_cur_net_cons) + ' ' + epoch + ' ' +  'source=' + envoy_fqdn + ' \n'
	sock.sendall(met_cur_net_cons.encode('utf-8'))
	#print(met_cur_net_cons)

	sock.close()

	# end of main() function

# call main() function to run program
main()

The Chart

Notice the production line in yellow and consumption in red? Looks like some heating using approximately 2.5kW turned on at 5:30AM and continued to run until 9:00AM. There wasn’t much else happening during the day, a fair amount of produced electricity being wasted on this day. In Australia, we export any unused production back to the grid for a small amount of credit, however its much more valuable to use it.

First post done… you’re turn Aaron Evans and Simon Lynch!