Plugin info

name: restful-metrics

Generates metrics or events by issuing HTTP request on any type of endpoint - rest APIs or otherwise. The metrics can be generated from data in response bodies, HTTP headers and other response attributes. The following types of response bodies are supported:

• json, scalar objects or arrays of objects. Scalars may generate a single metric, while arrays may generated multiple metrics
• ndjson, new line delimited json objects which can generated one metric per object

Furthermore, multiple metrics from the same request can be aggregated into by their respective dimensions.

Prerequisites

None

Events

None

Metrics

User defined

Configuration

This section describes the configuration settings for this plugin

IMPORTANT

Sensitive data should be stored inside the agent secrets store and referenced in this file using ${secret key} notation.

Name	Type	Required	Default	Description
default_timeout	duration string	No	5s	default request timeout
default_proxy	string	No		default proxy for requests (supports secrets)
default_max_redirects	int	No	10	default max redirects to follow
outputs	[]Output	Yes		defines the endpoints to collect metrics or events from

Output Configuration

Name	Type	Required	Default	Secrets	Description
id	string	Yes			A unique Human friendly name for this endpoint. Max length is 128
url	string	Yes		Yes	The endpoint URL. Supports path parameters via {param name} format
username	string	No		Yes	If using HTTP basic-auth set this to the username
password	string	No		Yes	If using HTTP basic-auth set this to the password
auth_token	string	No		Yes	Authorization: Bearer header token
method	string	Yes			HTTP method to use. One of GET, HEAD, POST, PUT, DELETE, PATCH
path_params	map[string]string	No			Name/value pairs to substitute in url
headers	map[string]string	No			Name/value pairs to pass as HTTP header to request
query_params	map[string]string	No			Name/value pairs to pass as query parameters to request
query_string	string	No			Query string to append to url
form_data	map[string]string	No			Name/value pairs to submit as a form to the request
body	string	No			Raw request body
data_path	string	No			The json path in the response body to read metric values from. Default is to read data from the body root
source	string	No	hostname		Value to assign as metric/event source.
response_type	string	No	json		Configures how to interpret the response body. One of json (JSON object or JSON array), ndjson (sequence of JSON objects delimited by new lines), status (do not interpret the response body and work only with response HTTP headers). For types json and ndjson, the response HTTP status code must be in the range of 200-299 otherwise an error is triggered.
metric	Metric	No			Defines metrics generated from the endpoint response.
max_redirects	int	No	default_max_redirects		Max number of HTTP redirect to follow
proxy	string	No	default_proxy		Proxy URL
no_proxy	bool	No	false		Do not use default_proxy
timeout	duration string	No	default_timeout		default request timeout
tls_ca_path	string	No			The path to the server TLS CA root certificate path if using self signed certs
tls_cert_path	string	No			The path to the client certificate path if using mTLS
tls_key_path	string	No			The path to the client key path if using mTLS
tls_key_passphrase	string	No		Yes	The passphrase for TLS key in tls_key_path
tls_insecure_skip_verify	bool	No			Set to true if server TLS cert should always be trusted

Metric Configuration

Name	Type	Required	Default	Secrets	Description
name	string	Yes			The name for the metric. For example security/2fa_disabled/login
aggregation	string	No			Aggregate a sequence of metrics by their dimensions. One of: count, sum, avg, max, min
value	map[string]string	Yes			How to extract the value for the metric. One of: path, duration, status, param, header, size
source	map[string]string	No	Output.source		How to extract the metric source. One of: path, duration, status, param, header
dimensions	map[string]string	No			Zero or more dimensions by name to add to metric. Each named dimension is configured with a mapping which extracts its value. One of: path, duration, status, param, header

Examples

Github admins with 2fa disabled

outputs:
  - id: "2fa_disabled"
    url: "https://api.github.com/orgs/{org}/members"
    method: GET
    auth_token: ${github_oauth_token}
    path_params:
      "org": "arisant"
    headers:
      "Accept": "application/vnd.github.v3+json"
    query_params:
      "filter": "2fa_disabled"
      "role": "admin"
    timeout: 10s
    response_type: json
    metric:
      name: "test/github/security/2fa_disabled/login"
      source:
        header: Server  
      value:
        path: login
      dimensions:
        org:
          param: org

Count of Github admins with 2fa disabled

outputs:
  - id: "2fa_disabled_counts"
    url: "https://api.github.com/orgs/{org}/members"
    method: GET
    auth_token: ${github_oauth_token}
    path_params:
      "org": "arisant"
    headers:
      "Accept": "application/vnd.github.v3+json"
    query_params:
      "filter": "2fa_disabled"
      "role": "admin"
    timeout: 10s
    response_type: json
    metric:
      name: "test/github/security/2fa_disabled/count"
      aggregation: count
      source:
        header: Server  
      value:
        path: login
      dimensions:
        org:
          param: org

Response Time

outputs:
  - id: "response_time"
    url: "https://api.github.com/orgs/{org}/members"
    method: GET
    auth_token: ${github_oauth_token}
    path_params:
      "org": "arisant"
    headers:
      "Accept": "application/vnd.github.v3+json"
    query_params:
      "filter": "2fa_disabled"
      "role": "admin"
    timeout: 10s
    response_type: status
    source: github
    metric:
      name: "test/github/response/millis"
      value:
        duration:
      dimensions:
        org:
          param: org  

Service Status

outputs:
  - id: "response_time"
    url: "https://api.github.com/orgs/{org}/members"
    method: GET
    auth_token: ${github_oauth_token}
    path_params:
      "org": "arisant"
    headers:
      "Accept": "application/vnd.github.v3+json"
    query_params:
      "filter": "2fa_disabled"
      "role": "admin"
    timeout: 10s
    response_type: status
    source: github
    metric:
      name: "test/github/status"
      value:
        status:
      dimensions:
        org:
          param: org  

Validate Configuration

restful-metrics --run-conf /path/to/config/file.yaml --validate

Testing

Run the plugin from the command line and get any emitted metrics on stdout

• simulate, do not try to register metrics

restful-metrics --run-conf /path/to/config/file.yaml --simulate

List the metrics emitted by a plugin configuration

restful-metrics --run-conf /path/to/config/file.yaml --metrics