Data Blending
- Last Updated 3/9/2021, 11:49:41 AM UTC
- About 4 min read
Data blending allows you to combine data from your collected metrics with auxiliary data from other sources. The blended data is available to metric analytics and alert notification templates. Currently, the system supports data blending in the form of metric dimension aliasing.
# Metric Dimension Aliasing
Blends metric dimension aliases from yaml
files in your git
repository. Create directory data-blending/sources/file/aliasing
under your assets
folder in your git
repo, then drop alias files under any path in this directory. The system will watch this directory for changes and automatically sync their contents with the timeseries database and the alerting meta data store.
assets
└── data-blending
└── sources
└── file
└── aliasing
├── agents.yaml
└── url.yaml
# Dimension Aliasing Files
Each file contains aliases for one dimension, for one metric set. A metric set is the namespace
of a metric minus the trailing segment, for example the metric set for metric myrmex/http/status/url/status_code
is myrmex/http/status/url
. This means that you define the same dimension aliases for all metrics with a common parent namespace. You may have multiple files for the same metric set and dimension, the system will merge their contents. If you have duplicate aliases in the same file or different files, the system will blend only one of them but not in a deterministic order.
Inside an aliasing file, define the following fields:
metric_set
, the metric set that the aliases are defined fordim
, the aliased dimension name segment inmetric_set
aliases
, an array of dimension aliases. Each array element is a 2-dimensional array where the first element is the source value and the second is the alias.
The below example provides short names for the url
dimension in myrmex/http/status/url
:
# URL short names
metric_set: "myrmex/http/status/url"
dim: "url"
aliases:
- ["www.test.com/very_long_url/1", "API 1"]
- ["www.test.com/very_long_url/6", "API 2"]
- ["www.test.com/very_long_url/6", "API 3"]
- ["www.test.com/very_long_url/8", "API 4"]
- ["www.test.com/very_long_url/9", "API 5"]
- ["www.test.com/very_long_url/10", "API 6"]
# Use metric dimension aliases in analytics
Tick the Display Dimension Aliases
box in an analysis definition. The system will replace dimension values for any metric with their corresponding aliases if they exist.
# Use metric dimension aliases in alert notification templates
The system provides functions DimAlias
and DimAliasWithValue
which you can use in your templates to get hold of dimension aliases if they exist.
# DimAlias(alertData, dimName, dimValue)
Returns the alias for dimension dimName of alertData.Metric with value dimValue or an empty string (""
) if no alias exists
Print
.Name: .Value (alias)
{{$data := .}} {{range .MetricDimensions}} // - given dim name = "url", dim value = "www.some.com/very/long/url?with_params=param" and dim alias = "Super API", it will print: // url: www.some.com/very/long/url?with_params=param (Super API) // // - given dim name = "url", dim value = "www.some.com/very/long/url?with_params=other" and no dim alias defined, it will print: // url: www.some.com/very/long/url?with_params=other () - {{.Name}}: {{.Value}} ({{DimAlias $data .Name .Value}}) {{end}}
Print
.Name: alias
if alias exists, otherwise.Name: .Value
{{$data := .}} {{range .MetricDimensions}} // - given dim name = "url", dim value = "www.some.com/very/long/url?with_params=param" and dim alias = "Super API", it will print: // url: Super API // // - given dim name = "url", dim value = "www.some.com/very/long/url?with_params=other" and no dim alias defined, it will print: // url: www.some.com/very/long/url?with_params=other - {{$a := DimAlias $data .Name .Value}}{{.Name}}: {{or $a .Value}} {{end}}
Print
.Name: alias (.Value)
if dim alias exists, otherwise.Name: .Value
{{$data := .}} {{range .MetricDimensions}} // - given dim name = "url", dim value = "www.some.com/very/long/url?with_params=param" and dim alias = "Super API", it will print: // url: Super API (www.some.com/very/long/url?with_params=param) // // - given dim name = "url", dim value = "www.some.com/very/long/url?with_params=other" and no dim alias defined, it will print: // url: www.some.com/very/long/url?with_params=other - {{$a := DimAlias $data .Name .Value}}{{or (and $a (printf "%s (%s)" $a .Value)) .Value}} {{end}}
# DimAliasWithValue(alertData, dimName, dimValue, format)
Formats dim alias and dim value with a format parameter if the alias exists. Otherwise returns the dimension value. format is called with 2 arguments, the first is the alias and the second is the value. The format string specification must use golang fmt package verbs (opens new window)
{{$data := .}}
{{range .MetricDimensions}}
// - given dim name = "url", dim value = "www.some.com/very/long/url?with_params=param" and dim alias = "Super API", it will print:
// url: Super API (www.some.com/very/long/url?with_params=param)
//
// - given dim name = "url", dim value = "www.some.com/very/long/url?with_params=other" and no dim alias defined, it will print:
// url: www.some.com/very/long/url?with_params=other
- {{.Name}}: {{DimAliasWithValue $data .Name .Value "%s (%s)"}}
{{end}}