AppFirst StatsD

Table of Contents:

About AppFirst StatsD

We have taken the open source project StatsD and extended it! Instead of using UDP for data transfer (which StatsD does natively), we use HTTPS and automatically encrypt your data to ensure it’s transmitted securely. Once the data hits the AppFirst big data store, your business metrics get integrated into our AppFirst Dashboards.

Extensions

1. Hierarchical namespace gives you the ability to manage how buckets are organized. For example, using the table below, if you were using the application component Redis and you wanted to see how much network traffic Redis was using, you would use the namespace pattern: app.redis.byte_sent

Metric Group Namespace Pattern
Business biz.< >biz.gauges.< >

biz.timers.< >

Process sys.processes.< >
Application app.< >
Server sys.server.< >
Polled Data sys.server.< >.polled_data.<>
Log sys.log.< >

2. For standard StatsD as defined by Etsy uses gauges which provide you with a metric at a point in time. AppFirst StatsD extends this capability by allowing you to provide a timestamp defining when this measurement was taken.

We are delivering these extension in a set of client binding that are open source that are available on the AppFirst GitHub. The AppFirst Statsd is supported by the same client binding that Etsy supports (Java, C#, Perl, PHP, Python, Ruby, Go, Erlang).

AppFirst StatsD supports all the existing StatsD behavior and will work with or without the AppFirst collector.

Using Other StatsD Clients

You are able to use other StatsD clients with AppFirst. If you are using Linux you can use Etsy StatsD. However, if you do you won’t be able to get the benefits of the AppFirst StatsD extensions. To continue to use your Etsy StatsD client, follow the directions below:

1. Add the following line inside /etc/AppFirst: StatsDPassThrough TRUE
2. Restart collector (http://community.appfirst.com/questions/30/how-do-i-restart-a-collector.html).
3. Change StatsD configuration to send to local host (localhost:8125) and not the ip address of Graphite.

If you are using Windows, you’ll need to download a new library and use our AppFirst StatsD library. Our libraries can be found at our GitHub page. This requires no necessary changes to the collector or your code.

StatsD Holt Winters

Holt-Winters is an exponential smoothing method of data prediction. Basic exponential smoothing calculates a weighted average of historical points, where the weights are a geometric sequence. This smoothed average can be calculated using only the current point and previous prediction. Holt added a slope factor to account for series that are trending up or down over time, and Winters added a periodic or “seasonal” factor to account for repeated patterns in data which requires an additional value to calculate corresponding to the point one period prior. Our implementation also uses a standard method to calculate a smoothed deviation for calculating the upper & lower bounds (which are ±3 deviations.)

Holt-Winters predictive analysis is used in our product when aggregating StatsD buckets. The confidence band of ±3 deviations is displayed on StatsD dashboard widgets as a grey band, and bars corresponding to values outside the band are colored yellow instead of green.

Adding StatsD Holt-Winters

Holt-Winters can be added when adding a StatsD widget within the dashboard. There is a tab labeled “Holt-Winters” that will give the user the option to display Holt-Winters.