- Generated on Thu Nov 29 2018 12:03:46 for AnyCollect by
1.8.14
|
AnyCollect
1.1.2
|
Please be careful about metric instantiation rules. They are detailed below in Metrics and substitution.
AnyCollect can gather metrics from two different kinds of sources:
Each of these sources have an array of expressions which are used to filter and match the contents. Each expression is a regex and an array of metrics templates, which use regex matches to form metrics.
Metrics are defined in a JSON file:
Metrics from reading file contents are specified in the top-level Files array. Each require two fields:
Paths, an array of strings representing the paths of files to readExpressions, an array of expressions (see below) used to match file contentsPaths support standard POSIX globbing (with wildcards, etc.). You can specify multiple paths to be matched against the same expressions.
Metrics from command output are specified in the top-level Command array. It requires three fields:
Program, the main program to executeArguments, an array of additional arguments to give to the programExpressions, an array of expressions (see below) used to match command outputPrior to execution, program and arguments are concatenated with spaces in between.
An expression is defined by two fields:
Regex, a regular expression string (backslashes \ and quotes " should be escaped)Metrics, an array of metric templatesA metric template is in turn defined in JSON by six fields:
Name, an array of strings representing the name of the metricValue, a string representing the value of the metricUnit, a string representing the value's unitTags, a map associating strings to strings adding metadata to a metricComputeRate, a boolean indicating whether the variation of the value, rather than the value, should be collectedConvertToUnitsPerSecond, a boolean indicating whether the value should be converted to units per secondThe regex will be applied for each line of content. If a match is found, metric templates are attempted to be filled with variable substitution.
One expression may have more than one metric template to facilitate parsing: if a line contains multiple metrics, the whole line can be matched by the regex and each metric template will extract one metric from the regex match.
String fields of a metric template (Name, Value, Unit and Tags) are subject to variable substitution:
$1, $2, $3, etc. will be replaced by the regex submatch with the same index. $0 matches the whole match$path_0, $path_1, $path_2, etc. will be replaced by the path component at the same indexImportant notes after substitution:
Name and Tags fields: if two metrics have the same Name and Tags, they are considered to represent the same thing. The Unit field is not taken into account. This equivalence is used to compute rates from an iteration to the next, and in case two metrics are found to be equivalent during the same iteration then their values are added.Value field may be a simple mathematic expressions. Integer or floating point numbers are supported, as well as operands +, -, *, /, % (modulo), ^ (power), ( and ); some function are also supported (sqrt, exp, log, cos, sin, tan, ...). If the expression is not valid, or can't be converted to a number, the metric is dropped.Name, Value, or Tags field is empty, the metric is considered deficient and is dropped. The Unit field may be empty.Name field should only contain lower-case alphanumeric characters and underscores _. Upper-case letters will be converted to lower-case, and symbols will be replaced by underscores.Please refer to the examples config files in the example directory of this repo.
Memory stats in /proc/meminfo
Regex: ^(\w+) (\d+) (\w)B$
Matches:
| Index | Submatch |
|---|---|
| 0 | MemTotal: 949448 kB |
| 1 | MemTotal |
| 2 | 949448 |
| 3 | k |
Metrics:
| Template | Metric |
|---|---|
|
|
CPU stats in /proc/stat
Regex: ^cpu (\d+) \d+ (\d+) (\d+)
Matches:
| Index | Submatch |
|---|---|
| 0 | cpu 14574109 23322 24156706 688820875 |
| 1 | cpu |
| 2 | 14574109 |
| 3 | 24156706 |
| 4 | 688820875 |
Metrics:
| Template | Metric |
|---|---|
|
|
|
|
|
|
Get network latency with ping
Regex: bytes from ([\d.]+): .* time=([\d.]+) (\w+)$
Matches:
| Index | Submatch |
|---|---|
| 0 | bytes from 8.8.8.8: icmp_seq=1 ttl=61 time=2.74 ms |
| 1 | 8.8.8.8 |
| 2 | 2.74 |
| 3 | ms |
Metrics:
| Template | Metric |
|---|---|
|
|
This program collects all available metrics and prints them on the standard output. For each metric, it shows its name, its value and its unit.
AnyCollectValues can take up to three arguments:
For example, ./AnyCollectValues 60 ./config.json 10 will read the config file ./config.json, and then collect metrics and print them every 60 seconds; it will do so 10 times. The program will thus run for 10 minutes.
In order for the AnyCollect plugin to be aware of its configuration before Snap launches a task (otherwise metrics won't be registered), the configuration file must be specified in snapteld global config:
However, if you choose to send all metrics described by a configuration file, this global configuration is not needed. You can tell the plugin to send all metrics in the task file, either by setting SendAllMetrics to true or by requesting the metric /cfm/anycollect/*.
First set up the Snap framework.
The default configuration for a AnyCollect Snap task is the following:
The parameters are (default values are given above):
ConfigFile (type string): path to AnyCollect's JSON configuration fileSamplingInterval (type int): delay in seconds between two readings of the kernel valuesSendAllMetrics (type boolean): whether to send all metrics to Snap, ignoring requested metrics in the task. This is a workaround: if the config file is modified and the Snap daemon not restarted, Snap doesn't update the metric list and new metrics won't be sentMaxMetricsBuffer (type int): maximum number of metrics to send to Snap at onceMaxCollectDuration (type int): maximum waiting time (in seconds) before sending metrics to SnapCollected metrics are described above.
Each metric template described in the configuration file will, after regex substitution, be transformed into a Snap metric. The metric name is converted into a metric namespace which you can filter in the Snap task file.
You can use wildcards to specify groups of metrics easily:
1.8.14