Skip to content

last9/last9-cdk

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Last9 CDK for HTTP

Last9 Golang CDK is a way to emit commonly aggregated metrics straight from your Golang application that serves HTTP traffic.

Why?

Traditional metrics emitters emit, per-path or handler-based metrics. Both these approaches have their drawbacks.

  • Per-path-based metrics can explode the cardinality because /user/foo /user/bar and so on can result in millions of endpoints.
  • handler-based metrics can absorb details of the route details.

What metrics does CDK emit?

RED Metrics

The very first metrics to be emitted are Rate, Error, and Duration.

httpRequestsDuration = prometheus.NewHistogramVec(
		prometheus.HistogramOpts{
			Name: "http_requests_duration_milliseconds",
			Help: "HTTP requests duration per path",
		},
		[]string{
			"per", "hostname", "domain", "method", "program", "status",
			"tenant", "cluster",
		},
	)

Labels Explained

Name Description
hostname current Hostname where the metric is emitted from
program Binary / Process name where the metric is emitted from
per This is the "main" label, which contains the pathname or an identifier that you emit per request.
By default, it is the path pattern if the Mux is one of the supported List and the entire URL Path where Mux does not have a path parameter.
Optionally, this can ALSO be a custom string (Read below for details)
domain domain at which the request was received
method HTTP method
status HTTP Status returned
tenant Optional Field for a multi-tenant application
cluster Optional Field for a multi-cluster deployment

Say more about Tenancy

Most modern SaaS applications end up being multi-tenant or multi-clustered where it's crucial to identify the behavior across each, separately.

CDK honors this need as a first-class property and has reserved two label fields for this purpose. These two are:

  • tenant
  • cluster

Features

  • Simple configuration for multi-tenancy or multi-cluster using an additional label.
  • Cross-tenant aggregation or segregation later via PromQL.
  • Allow data with no tenant or cluster information to be written or queried.

How are metrics exposed?

CDK emits metrics in Openmetrics Prometheus Exposition format. The metric port and endpoint expose the freshest metrics to be pulled by a Prometheus.

You can read more about the Prometheus exposition format on the link

Golang Support

There may be parts to the CDK which are activated ONLY when supported frameworks are detected or declared.

import (
	"github.com/last9/last9-cdk/go/httpmetrics"
)

// given you have a http.Handler or a Router already
// Only line that needs to be changed
httpmetrics.REDHandler(handler)
Name Supported Mux Supported
http.ServeMux Yes Yes
Gorilla Yes Yes
Pat Yes Yes
Chi Yes Yes

An Example Golang Application

go get the httpmetrics

go get -v github.com/last9/last9-cdk/go/httpmetrics

Assuming a basic main.go

package main

import (
	"fmt"
	"log"
	"net/http"

	"github.com/last9/last9-cdk/go/httpmetrics"
)

func handler(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path[1:])
}

func main() {
	http.Handle("/golang", httpmetrics.REDHandler(http.HandlerFunc(handler)))
	go httpmetrics.ServeMetrics(9091)
	log.Fatal(http.ListenAndServe(":9090", nil))
}

Once set up, you can send a couple of test requests on your application

➜  example curl -XPOST http://localhost:9090/golang -d '{}'
Hi there, I love golang!%                                                                                                                                                        ➜  example curl -XGET http://localhost:9090/golang -d '{}' 
Hi there, I love mount!

Visit localhost:9091/metrics to see your RED metrics


Custom Labels

If you want to change the default path being emitted, it is extremely easy to do so.

Say, my application handler is something like this

m := mux.NewRouter()
m.Handle("/api/category/{category}/item/{id}", itemHandler())
m.Use(REDHandler)

This will emit metrics where the per label will look like /api/category/{category}/item/{id}

But you want the category to NOT be abstracted. For situations like these, you can use the

REDHandlerWithLabelMaker function to assist the label-making process.

// REDHandlerWithLabelMaker accepts a function that in-turn accepts
// both the request and the mux.
m.Use(REDHandlerWithLabelMaker(
	func(r *http.Request, m http.Handler) map[string]string {

		// Gorilla exposes the variables using a request local mux
		vars := mux.Vars(r)
		category := vars["category"]

		return map[string]string{
			"per":     strings.Replace(r.URL.Path, "{category}", category),
			"tenant":  "possible_override", // You may also override other labels
		}

	},
))

Voila! That's it

The above example is for Gorilla Mux, but it's extremely straightforward to draw inspiration for other mux like Pat, etc.


About Last9

This project is sponsored and maintained by Last9. Last9 builds reliability tools for SRE and DevOps.