Skip to content

Commit

Permalink
feat: updating telegraf plugins for latest release (#20861)
Browse files Browse the repository at this point in the history
  • Loading branch information
russorat authored Mar 22, 2021
1 parent 73e4e70 commit 080341b
Show file tree
Hide file tree
Showing 27 changed files with 921 additions and 30 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ or `/query` HTTP endpoints.
1. [20971](https://github.com/influxdata/influxdb/pull/20971): Add `--http-read-header-timeout`, `--http-read-timeout`, `--http-write-timeout`, and `--http-idle-timeout` options to `influxd`.
1. [20971](https://github.com/influxdata/influxdb/pull/20971): Set a default `--http-read-header-timeout` of 10s in `influxd`.
1. [20971](https://github.com/influxdata/influxdb/pull/20971): Set a default `--http-idle-timeout` of 3m in `influxd`.
1. [20861](https://github.com/influxdata/influxdb/pull/20861): Update Telegraf plugins in UI to include additions and changes in 1.18 release.

### Bug Fixes

Expand Down
142 changes: 142 additions & 0 deletions ui/src/writeData/components/telegrafPlugins/aliyuncms.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
# Alibaba (aka Aliyun) CloudMonitor Service Statistics Input
Here and after we use `Aliyun` instead `Alibaba` as it is default naming across web console and docs.

This plugin will pull Metric Statistics from Aliyun CMS.

### Aliyun Authentication

This plugin uses an [AccessKey](https://www.alibabacloud.com/help/doc-detail/53045.htm?spm=a2c63.p38356.b99.127.5cba21fdt5MJKr&parentId=28572) credential for Authentication with the Aliyun OpenAPI endpoint.
In the following order the plugin will attempt to authenticate.
1. Ram RoleARN credential if `access_key_id`, `access_key_secret`, `role_arn`, `role_session_name` is specified
2. AccessKey STS token credential if `access_key_id`, `access_key_secret`, `access_key_sts_token` is specified
3. AccessKey credential if `access_key_id`, `access_key_secret` is specified
4. Ecs Ram Role Credential if `role_name` is specified
5. RSA keypair credential if `private_key`, `public_key_id` is specified
6. Environment variables credential
7. Instance metadata credential

### Configuration:

```toml
## Aliyun Credentials
## Credentials are loaded in the following order
## 1) Ram RoleArn credential
## 2) AccessKey STS token credential
## 3) AccessKey credential
## 4) Ecs Ram Role credential
## 5) RSA keypair credential
## 6) Environment variables credential
## 7) Instance metadata credential

# access_key_id = ""
# access_key_secret = ""
# access_key_sts_token = ""
# role_arn = ""
# role_session_name = ""
# private_key = ""
# public_key_id = ""
# role_name = ""

# The minimum period for AliyunCMS metrics is 1 minute (60s). However not all
# metrics are made available to the 1 minute period. Some are collected at
# 3 minute, 5 minute, or larger intervals.
# See: https://help.aliyun.com/document_detail/51936.html?spm=a2c4g.11186623.2.18.2bc1750eeOw1Pv
# Note that if a period is configured that is smaller than the minimum for a
# particular metric, that metric will not be returned by the Aliyun OpenAPI
# and will not be collected by Telegraf.
#
## Requested AliyunCMS aggregation Period (required - must be a multiple of 60s)
period = "5m"

## Collection Delay (required - must account for metrics availability via AliyunCMS API)
delay = "1m"

## Recommended: use metric 'interval' that is a multiple of 'period' to avoid
## gaps or overlap in pulled data
interval = "5m"

## Metric Statistic Project (required)
project = "acs_slb_dashboard"

## Maximum requests per second, default value is 200
ratelimit = 200

## Discovery regions set the scope for object discovery, the discovered info can be used to enrich
## the metrics with objects attributes/tags. Discovery is supported not for all projects (if not supported, then
## it will be reported on the start - foo example for 'acs_cdn' project:
## 'E! [inputs.aliyuncms] Discovery tool is not activated: no discovery support for project "acs_cdn"' )
## Currently, discovery supported for the following projects:
## - acs_ecs_dashboard
## - acs_rds_dashboard
## - acs_slb_dashboard
## - acs_vpc_eip
##
## If not set, all regions would be covered, it can provide a significant load on API, so the recommendation here
## is to limit the list as much as possible. Allowed values: https://www.alibabacloud.com/help/zh/doc-detail/40654.htm
discovery_regions = ["cn-hongkong"]

## how often the discovery API call executed (default 1m)
#discovery_interval = "1m"

## Metrics to Pull (Required)
[[inputs.aliyuncms.metrics]]
## Metrics names to be requested,
## described here (per project): https://help.aliyun.com/document_detail/28619.html?spm=a2c4g.11186623.6.690.1938ad41wg8QSq
names = ["InstanceActiveConnection", "InstanceNewConnection"]

## Dimension filters for Metric (these are optional).
## This allows to get additional metric dimension. If dimension is not specified it can be returned or
## the data can be aggregated - it depends on particular metric, you can find details here: https://help.aliyun.com/document_detail/28619.html?spm=a2c4g.11186623.6.690.1938ad41wg8QSq
##
## Note, that by default dimension filter includes the list of discovered objects in scope (if discovery is enabled)
## Values specified here would be added into the list of discovered objects.
## You can specify either single dimension:
#dimensions = '{"instanceId": "p-example"}'

## Or you can specify several dimensions at once:
#dimensions = '[{"instanceId": "p-example"},{"instanceId": "q-example"}]'

## Enrichment tags, can be added from discovery (if supported)
## Notation is <measurement_tag_name>:<JMES query path (https://jmespath.org/tutorial.html)>
## To figure out which fields are available, consult the Describe<ObjectType> API per project.
## For example, for SLB: https://api.aliyun.com/#/?product=Slb&version=2014-05-15&api=DescribeLoadBalancers&params={}&tab=MOCK&lang=GO
#tag_query_path = [
# "address:Address",
# "name:LoadBalancerName",
# "cluster_owner:Tags.Tag[?TagKey=='cs.cluster.name'].TagValue | [0]"
# ]
## The following tags added by default: regionId (if discovery enabled), userId, instanceId.

## Allow metrics without discovery data, if discovery is enabled. If set to true, then metric without discovery
## data would be emitted, otherwise dropped. This cane be of help, in case debugging dimension filters, or partial coverage
## of discovery scope vs monitoring scope
#allow_dps_without_discovery = false
```

#### Requirements and Terminology

Plugin Configuration utilizes [preset metric items references](https://www.alibabacloud.com/help/doc-detail/28619.htm?spm=a2c63.p38356.a3.2.389f233d0kPJn0)

- `discovery_region` must be a valid Aliyun [Region](https://www.alibabacloud.com/help/doc-detail/40654.htm) value
- `period` must be a valid duration value
- `project` must be a preset project value
- `names` must be preset metric names
- `dimensions` must be preset dimension values

### Measurements & Fields:

Each Aliyun CMS Project monitored records a measurement with fields for each available Metric Statistic
Project and Metrics are represented in [snake case](https://en.wikipedia.org/wiki/Snake_case)

- aliyuncms_{project}
- {metric}_average (metric Average value)
- {metric}_minimum (metric Minimum value)
- {metric}_maximum (metric Maximum value)
- {metric}_value (metric Value value)

### Example Output:

```
$ ./telegraf --config telegraf.conf --input-filter aliyuncms --test
> aliyuncms_acs_slb_dashboard,instanceId=p-example,regionId=cn-hangzhou,userId=1234567890 latency_average=0.004810798017284538,latency_maximum=0.1100282669067383,latency_minimum=0.0006084442138671875
```
1 change: 0 additions & 1 deletion ui/src/writeData/components/telegrafPlugins/beat.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Beat Input Plugin
The Beat plugin will collect metrics from the given Beat instances. It is
known to work with Filebeat and Kafkabeat.

### Configuration:
```toml
## An URL from which to read beat-formatted JSON
Expand Down
64 changes: 62 additions & 2 deletions ui/src/writeData/components/telegrafPlugins/cisco_telemetry_mdt.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Cisco Model-Driven Telemetry (MDT) Input Plugin
# Cisco model-driven telemetry (MDT) Input Plugin

Cisco model-driven telemetry (MDT) is an input plugin that consumes
telemetry data from Cisco IOS XR, IOS XE and NX-OS platforms. It supports TCP & GRPC dialout transports.
GRPC-based transport can utilize TLS for authentication and encryption.
RPC-based transport can utilize TLS for authentication and encryption.
Telemetry data is expected to be GPB-KV (self-describing-gpb) encoded.

The GRPC dialout transport is supported on various IOS XR (64-bit) 6.1.x and later, IOS XE 16.10 and later, as well as NX-OS 7.x and later platforms.
Expand All @@ -21,6 +21,9 @@ The TCP dialout transport is supported on IOS XR (32-bit and 64-bit) 6.1.x and l
## Address and port to host telemetry listener
service_address = ":57000"

## Grpc Maximum Message Size, default is 4MB, increase the size.
max_msg_size = 20000000

## Enable TLS; grpc transport only.
# tls_cert = "/etc/telegraf/cert.pem"
# tls_key = "/etc/telegraf/key.pem"
Expand All @@ -35,10 +38,67 @@ The TCP dialout transport is supported on IOS XR (32-bit and 64-bit) 6.1.x and l
## Define aliases to map telemetry encoding paths to simple measurement names
[inputs.cisco_telemetry_mdt.aliases]
ifstats = "ietf-interfaces:interfaces-state/interface/statistics"
[inputs.cisco_telemetry_mdt.dmes]
# Global Property Xformation.
# prop1 = "uint64 to int"
# prop2 = "uint64 to string"
# prop3 = "string to uint64"
# prop4 = "string to int64"
# prop5 = "string to float64"
# auto-prop-xfrom = "auto-float-xfrom" #Xform any property which is string, and has float number to type float64
# Per Path property xformation, Name is telemetry configuration under sensor-group, path configuration "WORD Distinguished Name"
# Per Path configuration is better as it avoid property collision issue of types.
# dnpath = '{"Name": "show ip route summary","prop": [{"Key": "routes","Value": "string"}, {"Key": "best-paths","Value": "string"}]}'
# dnpath2 = '{"Name": "show processes cpu","prop": [{"Key": "kernel_percent","Value": "float"}, {"Key": "idle_percent","Value": "float"}, {"Key": "process","Value": "string"}, {"Key": "user_percent","Value": "float"}, {"Key": "onesec","Value": "float"}]}'
# dnpath3 = '{"Name": "show processes memory physical","prop": [{"Key": "processname","Value": "string"}]}'
```

### Example Output:
```
ifstats,path=ietf-interfaces:interfaces-state/interface/statistics,host=linux,name=GigabitEthernet2,source=csr1kv,subscription=101 in-unicast-pkts=27i,in-multicast-pkts=0i,discontinuity-time="2019-05-23T07:40:23.000362+00:00",in-octets=5233i,in-errors=0i,out-multicast-pkts=0i,out-discards=0i,in-broadcast-pkts=0i,in-discards=0i,in-unknown-protos=0i,out-unicast-pkts=0i,out-broadcast-pkts=0i,out-octets=0i,out-errors=0i 1559150462624000000
ifstats,path=ietf-interfaces:interfaces-state/interface/statistics,host=linux,name=GigabitEthernet1,source=csr1kv,subscription=101 in-octets=3394770806i,in-broadcast-pkts=0i,in-multicast-pkts=0i,out-broadcast-pkts=0i,in-unknown-protos=0i,out-octets=350212i,in-unicast-pkts=9477273i,in-discards=0i,out-unicast-pkts=2726i,out-discards=0i,discontinuity-time="2019-05-23T07:40:23.000363+00:00",in-errors=30i,out-multicast-pkts=0i,out-errors=0i 1559150462624000000
```

### NX-OS Configuration Example:
```
Requirement DATA-SOURCE Configuration
-----------------------------------------
Environment DME path sys/ch query-condition query-target=subtree&target-subtree-class=eqptPsuSlot,eqptFtSlot,eqptSupCSlot,eqptPsu,eqptFt,eqptSensor,eqptLCSlot
DME path sys/ch depth 5 (Another configuration option)
Environment NXAPI show environment power
NXAPI show environment fan
NXAPI show environment temperature
Interface Stats DME path sys/intf query-condition query-target=subtree&target-subtree-class=rmonIfIn,rmonIfOut,rmonIfHCIn,rmonIfHCOut,rmonEtherStats
Interface State DME path sys/intf depth unbounded query-condition query-target=subtree&target-subtree-class=l1PhysIf,pcAggrIf,l3EncRtdIf,l3LbRtdIf,ethpmPhysIf
VPC DME path sys/vpc query-condition query-target=subtree&target-subtree-class=vpcDom,vpcIf
Resources cpu DME path sys/procsys query-condition query-target=subtree&target-subtree-class=procSystem,procSysCore,procSysCpuSummary,procSysCpu,procIdle,procIrq,procKernel,procNice,procSoftirq,procTotal,procUser,procWait,procSysCpuHistory,procSysLoad
Resources Mem DME path sys/procsys/sysmem/sysmemused
path sys/procsys/sysmem/sysmemusage
path sys/procsys/sysmem/sysmemfree
Per Process cpu DME path sys/proc depth unbounded query-condition rsp-foreign-subtree=ephemeral
vxlan(svi stats) DME path sys/bd query-condition query-target=subtree&target-subtree-class=l2VlanStats
BGP DME path sys/bgp query-condition query-target=subtree&target-subtree-class=bgpDom,bgpPeer,bgpPeerAf,bgpDomAf,bgpPeerAfEntry,bgpOperRtctrlL3,bgpOperRttP,bgpOperRttEntry,bgpOperAfCtrl
mac dynamic DME path sys/mac query-condition query-target=subtree&target-subtree-class=l2MacAddressTable
bfd DME path sys/bfd/inst depth unbounded
lldp DME path sys/lldp depth unbounded
urib DME path sys/urib depth unbounded query-condition rsp-foreign-subtree=ephemeral
u6rib DME path sys/u6rib depth unbounded query-condition rsp-foreign-subtree=ephemeral
multicast flow DME path sys/mca/show/flows depth unbounded
multicast stats DME path sys/mca/show/stats depth unbounded
multicast igmp NXAPI show ip igmp groups vrf all
multicast igmp NXAPI show ip igmp interface vrf all
multicast igmp NXAPI show ip igmp snooping
multicast igmp NXAPI show ip igmp snooping groups
multicast igmp NXAPI show ip igmp snooping groups detail
multicast igmp NXAPI show ip igmp snooping groups summary
multicast igmp NXAPI show ip igmp snooping mrouter
multicast igmp NXAPI show ip igmp snooping statistics
multicast pim NXAPI show ip pim interface vrf all
multicast pim NXAPI show ip pim neighbor vrf all
multicast pim NXAPI show ip pim route vrf all
multicast pim NXAPI show ip pim rp vrf all
multicast pim NXAPI show ip pim statistics vrf all
multicast pim NXAPI show ip pim vrf all
```
3 changes: 3 additions & 0 deletions ui/src/writeData/components/telegrafPlugins/cloudwatch.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ API endpoint. In the following order the plugin will attempt to authenticate.
## ex: endpoint_url = "http://localhost:8000"
# endpoint_url = ""

## Set http_proxy (telegraf uses the system wide proxy settings if it's is not set)
# http_proxy_url = "http://localhost:8888"

# The minimum period for Cloudwatch metrics is 1 minute (60s). However not all
# metrics are made available to the 1 minute period. Some are collected at
# 3 minute, 5 minute, or larger intervals. See https://aws.amazon.com/cloudwatch/faqs/#monitoring.
Expand Down
37 changes: 37 additions & 0 deletions ui/src/writeData/components/telegrafPlugins/csgo.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Counter-Strike: Global Offensive (CSGO) Input Plugin

The `csgo` plugin gather metrics from Counter-Strike: Global Offensive servers.

#### Configuration
```toml
[[inputs.csgo]]
## Specify servers using the following format:
## servers = [
## ["ip1:port1", "rcon_password1"],
## ["ip2:port2", "rcon_password2"],
## ]
#
## If no servers are specified, no data will be collected
servers = []
```

### Metrics

The plugin retrieves the output of the `stats` command that is executed via rcon.

If no servers are specified, no data will be collected

- csgo
- tags:
- host
- fields:
- cpu (float)
- net_in (float)
- net_out (float)
- uptime_minutes (float)
- maps (float)
- fps (float)
- players (float)
- sv_ms (float)
- variance_ms (float)
- tick_ms (float)
48 changes: 48 additions & 0 deletions ui/src/writeData/components/telegrafPlugins/directory_monitor.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Directory Monitor Input Plugin

This plugin monitors a single directory (without looking at sub-directories), and takes in each file placed in the directory.
The plugin will gather all files in the directory at a configurable interval (`monitor_interval`), and parse the ones that haven't been picked up yet.

This plugin is intended to read files that are moved or copied to the monitored directory, and thus files should also not be used by another process or else they may fail to be gathered. Please be advised that this plugin pulls files directly after they've been in the directory for the length of the configurable `directory_duration_threshold`, and thus files should not be written 'live' to the monitored directory. If you absolutely must write files directly, they must be guaranteed to finish writing before the `directory_duration_threshold`.

### Configuration:

```toml
[[inputs.directory_monitor]]
## The directory to monitor and read files from.
directory = ""
#
## The directory to move finished files to.
finished_directory = ""
#
## The directory to move files to upon file error.
## If not provided, erroring files will stay in the monitored directory.
# error_directory = ""
#
## The amount of time a file is allowed to sit in the directory before it is picked up.
## This time can generally be low but if you choose to have a very large file written to the directory and it's potentially slow,
## set this higher so that the plugin will wait until the file is fully copied to the directory.
# directory_duration_threshold = "50ms"
#
## A list of the only file names to monitor, if necessary. Supports regex. If left blank, all files are ingested.
# files_to_monitor = ["^.*\.csv"]
#
## A list of files to ignore, if necessary. Supports regex.
# files_to_ignore = [".DS_Store"]
#
## Maximum lines of the file to process that have not yet be written by the
## output. For best throughput set to the size of the output's metric_buffer_limit.
## Warning: setting this number higher than the output's metric_buffer_limit can cause dropped metrics.
# max_buffered_metrics = 10000
#
## The maximum amount of file paths to queue up for processing at once, before waiting until files are processed to find more files.
## Lowering this value will result in *slightly* less memory use, with a potential sacrifice in speed efficiency, if absolutely necessary.
# file_queue_size = 100000
#
## The dataformat to be read from the files.
## Each data format has its own unique set of configuration options, read
## more about them here:
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
## NOTE: We currently only support parsing newline-delimited JSON. See the format here: https://github.com/ndjson/ndjson-spec
data_format = "influx"
```
Loading

0 comments on commit 080341b

Please sign in to comment.