Checks reference
Checks work with Sensu agents to produce observability events automatically. You can use checks to monitor server resources, services, and application health as well as collect and analyze metrics. Read Monitor server resources to get started. Use Bonsai, the Sensu asset hub, to discover, download, and share Sensu check dynamic runtime assets.
Check example (minimum recommended attributes)
This example shows a check resource definition that includes the minimum recommended attributes.
NOTE: The attribute interval
is not required if a valid cron
schedule is defined.
Read scheduling for more information.
---
type: CheckConfig
api_version: core/v2
metadata:
name: check_minimum
spec:
command: collect.sh
handlers:
- slack
interval: 10
publish: true
subscriptions:
- system
{
"type": "CheckConfig",
"api_version": "core/v2",
"metadata": {
"name": "check_minimum"
},
"spec": {
"command": "collect.sh",
"subscriptions": [
"system"
],
"handlers": [
"slack"
],
"interval": 10,
"publish": true
}
}
Check commands
Each Sensu check definition specifies a command and the schedule at which it should be executed. Check commands are executable commands that the Sensu agent executes.
A command may include command line arguments for controlling the behavior of the command executable. Many common checks are available as dynamic runtime assets from Bonsai and support command line arguments so different check definitions can use the same executable.
NOTE: Sensu advises against requiring root privileges to execute check commands or scripts. The Sensu user is not permitted to kill timed-out processes invoked by the root user, which could result in zombie processes.
Check command execution
All check commands are executed by Sensu agents as the sensu
user.
Commands must be executable files that are discoverable on the Sensu agent system (for example, installed in a system $PATH
directory).
Check result specification
Although Sensu agents attempt to execute any command defined for a check, successful check result processing requires adherence to a simple specification.
- Result data is output to stdout or stderr.
- For service checks, this output is typically a human-readable message.
- For metric checks, this output contains the measurements gathered by the check.
- Exit status code indicates state.
0
indicates OK.1
indicates WARNING.2
indicates CRITICAL.- Exit status codes other than
0
,1
, and2
indicate an UNKNOWN or custom status
PRO TIP: If you’re familiar with the Nagios monitoring system, you may recognize this specification — it is the same one that Nagios plugins use. As a result, you can use Nagios plugins with Sensu without any modification.
At every execution of a check command, regardless of success or failure, the Sensu agent publishes the check’s result for eventual handling by the event processor (the Sensu backend).
Check scheduling
The Sensu backend schedules checks and publishes check execution requests to entities via a publish/subscribe model.
Checks have a defined set of subscriptions: transport topics to which the Sensu backend publishes check requests.
Sensu entities become subscribers to these topics (called subscriptions) via their individual subscriptions
attribute.
You can schedule checks using the interval
, cron
, and publish
attributes.
Sensu requires that checks include either an interval
attribute (interval scheduling) or a cron
attribute (cron scheduling).
Round robin checks
By default, Sensu schedules checks once per interval for each agent with a matching subscription: one check execution per agent per interval.
Sensu also supports deduplicated check execution when configured with the round_robin
check attribute.
For checks with round_robin
set to true
, Sensu executes the check once per interval, cycling through the available agents alphabetically according to agent name.
For example, for three agents configured with the system
subscription (agents A, B, and C), a check configured with the system
subscription and round_robin
set to true
results in one observability event per interval, with the agent creating the event following the pattern A -> B -> C -> A -> B -> C for the first six intervals.
In the diagram above, the standard check is executed by agents A, B, and C every 60 seconds. The round robin check cycles through the available agents, resulting in each agent executing the check every 180 seconds.
To use check ttl
and round_robin
together, your check configuration must also specify a proxy_entity_name
.
If you do not specify a proxy_entity_name
when using check ttl
and round_robin
together, your check will stop executing.
PRO TIP: Use round robin to distribute check execution workload across multiple agents when using proxy checks.
Event storage for round robin scheduling
If you use round robin scheduling for check execution, we recommend using PostgreSQL rather than etcd for event storage. Etcd leases are unreliable as the scheduling mechanism for round robin check execution, and etcd will not produce precise round robin behavior.
When you enable round robin scheduling on PostgreSQL, any existing round robin scheduling will stop and migrate to PostgreSQL as entities check in with keepalives. Sensu will gradually delete the existing etcd scheduler state as keepalives on the etcd scheduler keys expire over time.
Interval scheduling
You can schedule a check to be executed at regular intervals using the interval
and publish
check attributes.
For example, to schedule a check to execute every 60 seconds, set the interval
attribute to 60
and the publish
attribute to true
.
NOTE: When creating an interval check, Sensu calculates an initial offset to splay the check’s first scheduled request. This helps balance the load of both the backend and the agent and may result in a delay before initial check execution.
Example interval check
---
type: CheckConfig
api_version: core/v2
metadata:
name: interval_check
spec:
command: check-cpu.sh -w 75 -c 90
handlers:
- slack
interval: 60
publish: true
subscriptions:
- system
{
"type": "CheckConfig",
"api_version": "core/v2",
"metadata": {
"name": "interval_check"
},
"spec": {
"command": "check-cpu.sh -w 75 -c 90",
"subscriptions": ["system"],
"handlers": ["slack"],
"interval": 60,
"publish": true
}
}
Cron scheduling
You can also schedule checks using cron syntax.
Examples of valid cron values include:
cron: CRON_TZ=Asia/Tokyo * * * * *
cron: TZ=Asia/Tokyo * * * * *
cron: '* * * * *'
NOTE: If you’re using YAML to create a check that uses cron scheduling and the first character of the cron schedule is an asterisk (*
), place the entire cron schedule inside single or double quotes (for example, cron: '* * * * *'
).
Example cron checks
To schedule a check to execute once a minute at the start of the minute, set the cron
attribute to * * * * *
and the publish
attribute to true
:
---
type: CheckConfig
api_version: core/v2
metadata:
name: cron_check
spec:
command: check-cpu.sh -w 75 -c 90
cron: '* * * * *'
handlers:
- slack
publish: true
subscriptions:
- system
{
"type": "CheckConfig",
"api_version": "core/v2",
"metadata": {
"name": "cron_check"
},
"spec": {
"command": "check-cpu.sh -w 75 -c 90",
"subscriptions": ["system"],
"handlers": ["slack"],
"cron": "* * * * *",
"publish": true
}
}
Use a prefix of TZ=
or CRON_TZ=
to set a timezone for the cron
attribute:
---
type: CheckConfig
api_version: core/v2
metadata:
name: cron_check
spec:
check_hooks: null
command: hi
cron: CRON_TZ=Asia/Tokyo * * * * *
env_vars: null
handlers: []
high_flap_threshold: 0
interval: 0
low_flap_threshold: 0
output_metric_format: ""
output_metric_handlers: null
output_metric_tags: null
proxy_entity_name: ""
publish: true
round_robin: false
runtime_assets: null
stdin: false
subdue: null
subscriptions:
- sys
timeout: 0
ttl: 0
{
"type": "CheckConfig",
"api_version": "core/v2",
"metadata": {
"name": "cron_check"
},
"spec": {
"check_hooks": null,
"command": "hi",
"cron": "CRON_TZ=Asia/Tokyo * * * * *",
"env_vars": null,
"handlers": [],
"high_flap_threshold": 0,
"interval": 0,
"low_flap_threshold": 0,
"output_metric_format": "",
"output_metric_handlers": null,
"output_metric_tags": null,
"proxy_entity_name": "",
"publish": true,
"round_robin": false,
"runtime_assets": null,
"stdin": false,
"subdue": null,
"subscriptions": [
"sys"
],
"timeout": 0,
"ttl": 0
}
}
Ad hoc scheduling
In addition to automatic execution, you can create checks to be scheduled manually using core/v2/checks API endpoints.
To create a check with ad-hoc scheduling, set the publish
attribute to false
in addition to an interval
or cron
schedule.
Example ad hoc check
---
type: CheckConfig
api_version: core/v2
metadata:
name: ad_hoc_check
spec:
command: check-cpu.sh -w 75 -c 90
handlers:
- slack
interval: 60
publish: false
subscriptions:
- system
{
"type": "CheckConfig",
"api_version": "core/v2",
"metadata": {
"name": "ad_hoc_check"
},
"spec": {
"command": "check-cpu.sh -w 75 -c 90",
"subscriptions": ["system"],
"handlers": ["slack"],
"interval": 60,
"publish": false
}
}
Proxy checks
Sensu supports running proxy checks that associate events with a entity that isn’t actually executing the check, regardless of whether that entity is an agent entity or a proxy entity.
Proxy entities allow Sensu to monitor external resources on systems and devices where a Sensu agent cannot be installed, like a network switch or a website.
You can create a proxy check using proxy_entity_name
or proxy_requests
.
When you create a proxy check, make sure the check definition includes a subscription that matches the subscription of at least one agent entity to define which agents will run the check. Proxy entities do not use subscriptions.
To avoid duplicate events, use the round_robin
check attribute with proxy checks.
Read Round robin checks and Proxy entities and round robin scheduling to learn more.
Read Monitor external resources with proxy entities to learn how to create proxy checks to generate events for one or more proxy entities.
Use a proxy check to monitor a proxy entity
When executing checks that include a proxy_entity_name
, Sensu agents report the resulting events under the specified proxy entity instead of the agent entity.
If the proxy entity doesn’t exist, Sensu creates the proxy entity when the backend receives the event.
Example proxy check using proxy_entity_name
The following proxy check runs every 60 seconds, cycling through the agents with the run_proxies
subscription alphabetically according to the agent name, for the proxy entity sensu-site
.
---
type: CheckConfig
api_version: core/v2
metadata:
name: proxy_check
spec:
command: http_check.sh https://sensu.io
handlers:
- slack
interval: 60
proxy_entity_name: sensu-site
publish: true
round_robin: true
subscriptions:
- run_proxies
{
"type": "CheckConfig",
"api_version": "core/v2",
"metadata": {
"name": "proxy_check"
},
"spec": {
"command": "http_check.sh https://sensu.io",
"subscriptions": ["run_proxies"],
"handlers": ["slack"],
"interval": 60,
"publish": true,
"round_robin": true,
"proxy_entity_name": "sensu-site"
}
}
Use a proxy check to monitor multiple proxy entities
The proxy_requests
check attributes allow Sensu to run a check for each entity that matches the expressions specified in the entity_attributes
, resulting in observability events that represent each matching proxy entity.
The entity attributes must match exactly as stated.
No variables or directives have any special meaning, but you can use Sensu query expressions to perform more complicated filtering on the available value, such as finding entities with a particular class or label.
Combine proxy_requests
attributes with with token substitution as shown in the example proxy check below to monitor multiple entities using a single check definition.
Example proxy check using proxy_requests
The following proxy check runs every 60 seconds, cycling through the agents with the run_proxies
subscription alphabetically according to the agent name, for all existing proxy entities with the custom label proxy_type
set to website
.
This check uses token substitution to import the value of the custom entity label url
to complete the check command.
Read the entities reference for information about adding custom labels to entities.
---
type: CheckConfig
api_version: core/v2
metadata:
name: proxy_check_proxy_requests
spec:
command: http_check.sh {{ .labels.url }}
handlers:
- slack
interval: 60
proxy_requests:
entity_attributes:
- entity.labels.proxy_type == 'website'
publish: true
round_robin: true
subscriptions:
- run_proxies
{
"type": "CheckConfig",
"api_version": "core/v2",
"metadata": {
"name": "proxy_check_proxy_requests"
},
"spec": {
"command": "http_check.sh {{ .labels.url }}",
"subscriptions": ["run_proxies"],
"handlers": ["slack"],
"interval": 60,
"publish": true,
"proxy_requests": {
"entity_attributes": [
"entity.labels.proxy_type == 'website'"
]
},
"round_robin": true
}
}
Fine-tune proxy check scheduling with splay
Use the splay
and splay_coverage
attributes to distribute proxy check executions across the check interval.
To continue the example proxy_check_proxy_requests check, if the check matches three proxy entities, you will get a single burst of three check executions (with the resulting events) every 60 seconds.
Use the splay
and splay_coverage
attributes to distribute the three check executions over the specified check interval instead of all at the same time.
The following example adds splay
set to true
and splay_coverage
set to 90
within the proxy_requests
object.
With this addition, instead of three check executions in a single burst every 60 seconds, Sensu will distribute the three check executions evenly across a 54-second period (90% of the 60-second interval):
---
type: CheckConfig
api_version: core/v2
metadata:
name: proxy_check_proxy_requests
spec:
command: http_check.sh {{ .labels.url }}
handlers:
- slack
interval: 60
proxy_requests:
entity_attributes:
- entity.labels.proxy_type == 'website'
splay: true
splay_coverage: 90
publish: true
round_robin: true
subscriptions:
- run_proxies
{
"type": "CheckConfig",
"api_version": "core/v2",
"metadata": {
"name": "proxy_check_proxy_requests"
},
"spec": {
"command": "http_check.sh {{ .labels.url }}",
"handlers": [
"slack"
],
"interval": 60,
"proxy_requests": {
"entity_attributes": [
"entity.labels.proxy_type == 'website'"
],
"splay": true,
"splay_coverage": 90
},
"publish": true,
"round_robin": true,
"subscriptions": [
"run_proxies"
]
}
}
Check token substitution
Sensu check definitions may include attributes that you wish to override on an entity-by-entity basis. For example, check commands, which may include command line arguments for controlling the behavior of the check command, may benefit from entity-specific thresholds. Sensu check tokens are check definition placeholders that the Sensu agent will replace with the corresponding entity definition attribute values (including custom attributes).
Learn how to use check tokens with the Sensu tokens reference documentation.
NOTE: Check tokens are processed before check execution, so token substitutions will not apply to check data delivered via the local agent socket input.
Subdues
Use the subdues
attribute in check definitions to set specific periods of time when Sensu should not execute the check.
Subdues allow you to schedule alert-free periods of time, such as during sleeping hours, weekends, or special maintenance periods.
You can set more than one subdue at a time. Each subdue includes a begin and end time as well as how often to repeat the subdue, if desired.
For example, this check will be subdued (in other words, will not be executed) from 5:00 p.m. until 8:00 a.m. PDT on every weekday, and for the entire day on weekends, as well as every Friday from 10:00 until 11:00 a.m. PDT:
---
type: CheckConfig
api_version: core/v2
metadata:
name: check_cpu
spec:
command: check-cpu-usage -w 75 -c 90
interval: 60
handlers:
- slack
publish: true
round_robin: false
runtime_assets:
- check-cpu-usage
subdues:
- begin: "2022-04-18T17:00:00-07:00"
end: "2022-04-19T08:00:00-07:00"
repeat:
- weekdays
- begin: "2022-04-23T00:00:00-07:00"
end: "2022-04-23T23:59:59-07:00"
repeat:
- weekends
- begin: "2022-04-22T10:00:00-07:00"
end: "2022-04-22T11:00:00-07:00"
repeat:
- fridays
subscriptions:
- system
{
"type": "CheckConfig",
"api_version": "core/v2",
"metadata": {
"name": "check_cpu"
},
"spec": {
"command": "check-cpu-usage -w 75 -c 90",
"interval": 60,
"handlers": [
"slack"
],
"publish": true,
"round_robin": false,
"runtime_assets": [
"check-cpu-usage"
],
"subdues": [
{
"begin": "2022-04-18T17:00:00-07:00",
"end": "2022-04-19T08:00:00-07:00",
"repeat": [
"weekdays"
]
},
{
"begin": "2022-04-23T00:00:00-07:00",
"end": "2022-04-23T23:59:59-07:00",
"repeat": [
"weekends"
]
},
{
"begin": "2022-04-22T10:00:00-07:00",
"end": "2022-04-22T11:00:00-07:00",
"repeat": [
"fridays"
]
}
],
"subscriptions": [
"system"
]
}
}
Subdues and repeat
If you include a repeat
array in a subdues
object, Sensu will start the subdue period on the date you specify.
After the first subdue, Sensu uses the begin
and end
values only to determine the time of day to start and stop the subdue.
NOTE: Check subdue repeats are based on the specified begin
and end
times and not duration or the difference between the begin
and end
times.
Read Repeat and multi-day subdues for more information.
In the above example, on April 18, 2022, Sensu will apply the weekdays
subdue at 5:00 p.m. PDT and end it on April 19 at 8:00 a.m. PDT.
On April 19, Sensu will apply the weekdays
subdue again at 5:00 p.m. PDT and end it on April 20 at 8:00 a.m. PDT, and so on.
Valid values for repeat arrays
This table lists and describes valid values for the repeat
array:
Value | Description |
---|---|
mondays tuesdays wednesdays thursdays fridays saturdays sundays |
Subdue on the specified day, at the same time of day |
weekdays |
Subdue on all Mondays, Tuesdays, Wednesdays, Thursdays, and Fridays, at the same time of day |
weekends |
Subdue on all Saturdays and Sundays, at the same time of day |
daily |
Subdue once every day, at the same time of day |
weekly |
Subdue once per week on the same day, at the same time of day |
monthly |
Subdue once per month on the same day, at the same time of day |
annually |
Subdue once per year on the same day, at the same time of day |
Repeat and multi-day subdues
Because repeat schedules for subdues are based only on the specified time of day, you may need to configure more than one repeat for multi-day subdues.
For example, suppose that you want to subdue a check on the weekends. You might set a repeat that starts on a Friday at 5:00 p.m. PDT and ends on Monday at 8:00 a.m. PDT:
subdues:
- begin: "2022-05-06T17:00:00-07:00"
end: "2022-05-09T08:00:00-07:00"
repeat:
- fridays
{
"subdues": [
{
"begin": "2022-05-06T17:00:00-07:00",
"end": "2022-05-09T08:00:00-07:00",
"repeat": [
"fridays"
]
}
]
}
The first weekend, your repeat will work as expected to subdue the check from 5:00 p.m. PDT on Friday until 8:00 a.m. PDT on Monday.
After the first weekend, the subdue will start as expected at 5:00 p.m. PDT on Friday, but it will expire at 8:00 a.m. PDT on Saturday instead of Monday.
This is because after the first instance, repeats are based only on the specified begin
and end
times.
Sensu uses the dates to schedule only the first subdue.
Instead, use the following three-part configuration to achieve the desired repeat schedule (every Friday at 5:00 p.m. PDT until Monday at 8:00 a.m. PDT):
subdues:
- begin: "2022-05-06T17:00:00-07:00"
end: "2022-05-06T23:59:59-07:00"
repeat:
- fridays
- begin: "2022-05-07T00:00:00-07:00"
end: "2022-05-07T23:59:59-07:00"
repeat:
- weekends
- begin: "2022-05-09T00:00:00-07:00"
end: "2022-05-09T08:00:00-07:00"
repeat:
- mondays
{
"subdues": [
{
"begin": "2022-05-06T17:00:00-07:00",
"end": "2022-05-06T23:59:59-07:00",
"repeat": [
"fridays"
]
},
{
"begin": "2022-05-07T00:00:00-07:00",
"end": "2022-05-07T23:59:59-07:00",
"repeat": [
"weekends"
]
},
{
"begin": "2022-05-09T00:00:00-07:00",
"end": "2022-05-09T08:00:00-07:00",
"repeat": [
"mondays"
]
}
]
}
With this configuration, the repeat schedule will subdue the check every Friday from 5:00 p.m. PDT until midnight, the entire 24 hours on every Saturday and Sunday, and every Monday from midnight until 8:00 a.m. PDT.
Check hooks
Check hooks are commands run by the Sensu agent in response to the result of check command execution.
The Sensu agent will execute the appropriate configured hook command, depending on the check execution status (for example, 0
, 1
, or 2
).
Learn how to use check hooks with the Sensu hooks reference documentation.
Check specification
Top-level attributes
api_version | |
---|---|
description | Top-level attribute that specifies the Sensu API group and version. For checks in this version of Sensu, this attribute should always be core/v2 . |
required | Required for check definitions in wrapped-json or yaml format for use with sensuctl create . |
type | String |
example |
|
metadata | |
---|---|
description | Top-level collection of metadata about the check, including name , namespace , and created_by as well as custom labels and annotations . The metadata map is always at the top level of the check definition. This means that in wrapped-json and yaml formats, the metadata scope occurs outside the spec scope. Read metadata attributes for details. |
required | Required for check definitions in wrapped-json or yaml format for use with sensuctl create . |
type | Map of key-value pairs |
example |
|
spec | |
---|---|
description | Top-level map that includes the check spec attributes. |
required | Required for check definitions in wrapped-json or yaml format for use with sensuctl create . |
type | Map of key-value pairs |
example |
|
type | |
---|---|
description | Top-level attribute that specifies the sensuctl create resource type. Checks should always be type CheckConfig . |
required | Required for check definitions in wrapped-json or yaml format for use with sensuctl create . |
type | String |
example |
|
Metadata attributes
annotations | |
---|---|
description | Non-identifying metadata to include with observation event data that you can access with event filters. You can use annotations to add data that’s meaningful to people or external tools that interact with Sensu. In contrast to labels, you cannot use annotations in API response filtering, sensuctl response filtering, or web UI views. |
required | false |
type | Map of key-value pairs. Keys and values can be any valid UTF-8 string. |
default | null |
example |
|
created_by | |
---|---|
description | Username of the Sensu user who created the check or last updated the check. Sensu automatically populates the created_by field when the check is created or updated. |
required | false |
type | String |
example |
|
labels | |
---|---|
description | Custom attributes to include with observation event data that you can use for response and web UI view filtering. If you include labels in your event data, you can filter API responses, sensuctl responses, and web UI views based on them. In other words, labels allow you to create meaningful groupings for your data. Limit labels to metadata you need to use for response filtering. For complex, non-identifying metadata that you will not need to use in response filtering, use annotations rather than labels. |
required | false |
type | Map of key-value pairs. Keys can contain only letters, numbers, and underscores and must start with a letter. Values can be any valid UTF-8 string. |
default | null |
example |
|
name | |
---|---|
description | Unique string used to identify the check. Check names cannot contain special characters or spaces (validated with Go regex \A[\w\.\-]+\z ). Each check must have a unique name within its namespace. |
required | true |
type | String |
example |
|
namespace | |
---|---|
description | Sensu RBAC namespace that the check belongs to. |
required | false |
type | String |
default | default |
example |
|
Spec attributes
NOTE: Spec attributes are not required when sending an HTTP POST
request to the agent events API or the backend core/v2/events API.
When doing so, the spec attributes are listed as individual top-level attributes in the check definition instead.
check_hooks | |
---|---|
description | Array of check response types with respective arrays of Sensu hook names. Sensu hooks are commands run by the Sensu agent in response to the result of the check command execution. Hooks are executed in order of precedence based on their severity type: 1 to 255 , ok , warning , critical , unknown , and finally non-zero . |
required | false |
type | Array |
example |
|
command | |
---|---|
description | Check command to be executed. |
required | true |
type | String |
example |
|
cron | |
---|---|
description | When the check should be executed, using cron syntax or a predefined schedule. Use a prefix of TZ= or CRON_TZ= to set a timezone for the cron attribute.
NOTE: If you’re using YAML to create a check that uses cron scheduling and the first character of the cron schedule is an asterisk ( |
required | true (unless interval is configured) |
type | String |
example |
|
env_vars | |
---|---|
description | Array of environment variables to use with command execution.
NOTE: To add |
required | false |
type | Array |
example |
|
handlers | |
---|---|
description | Array of Sensu event handlers (names) to use for events created by the check. Each array item must be a string.
NOTE: The names of Sumo Logic metrics handlers and TCP stream handlers are not valid values for the handlers array.
Only traditional handlers are valid for the handlers array. |
required | false |
type | Array |
example |
|
high_flap_threshold | |
---|---|
description | Flap detection high threshold (% state change) for the check. Sensu uses the same flap detection algorithm as Nagios. Read the event reference to learn more about how Sensu uses the high_flap_threshold value. |
required | true (if low_flap_threshold is configured) |
type | Integer |
example |
|
interval | |
---|---|
description | How often the check is executed. In seconds. |
required | true (unless cron is configured) |
type | Integer |
example |
|
low_flap_threshold | |
---|---|
description | Flap detection low threshold (% state change) for the check. Sensu uses the same flap detection algorithm as Nagios. Read the event reference to learn more about how Sensu uses the low_flap_threshold value. |
required | false |
type | Integer |
example |
|
output_metric_format | |
---|---|
description | Metric format generated by the check command. Sensu supports the following metric formats:nagios_perfdata (Nagios Performance Data)graphite_plaintext (Graphite Plaintext Protocol)influxdb_line (InfluxDB Line Protocol)opentsdb_line (OpenTSDB Data Specification)prometheus_text (Prometheus Exposition Text)When a check includes an output_metric_format , Sensu will extract the metrics from the check output and add them to the event data in Sensu metric format. Read Collect metrics with Sensu checks. |
required | false |
type | String |
example |
|
output_metric_handlers | |
---|---|
description | Array of Sensu handlers to use for events created by the check. Each array item must be a string. Use output_metric_handlers in place of the handlers attribute if output_metric_format is configured. Metric handlers must be able to process Sensu metric format. The Sensu InfluxDB handler provides an example. |
required | false |
type | Array |
example |
|
output_metric_tags | |
---|---|
description | Custom tags to enrich metric points produced by check output metric extraction. One name/value pair make up a single tag. The output_metric_tags array can contain multiple tags.You can use check token substitution for the value attribute in output metric tags. |
required | false |
type | Array |
example |
|
output_metric_thresholds | |
---|---|
description | Array of metric names and threshold values to compare to check output metrics for metric threshold evaluation.
NOTE: To apply metric threshold evaluation, check definitions must include the |
required | false |
type | Array |
example |
|
pipelines | |
---|---|
description | Name, type, and API version for the pipelines to use for event processing. All the observability events that the check produces will be processed according to the pipelines listed in the pipeline array. Read pipelines attributes for more information. |
required | false |
type | Array |
example |
|
proxy_entity_name | |
---|---|
description | Entity name. Used to create a proxy entity for an external resource (for example, a network switch). |
required | false |
type | String |
validated | \A[\w\.\-]+\z |
example |
|
proxy_requests | |
---|---|
description | Assigns a check to run for multiple entities according to their entity_attributes . In the example below, the check executes for all entities with entity class proxy and the custom proxy type label website . The proxy_requests attributes allow you to reuse check definitions for a group of entities. For more information, read Proxy requests attributes and Monitor external resources with proxy entities. |
required | false |
type | Hash |
example |
|
publish | |
---|---|
description | true if check requests are published for the check. Otherwise, false . |
required | false |
type | Boolean |
default | false |
example |
|
round_robin | |
---|---|
description | When set to true , Sensu executes the check once per interval, cycling through each subscribing agent in turn. Read round robin checks for more information.Use the round_robin attribute with proxy checks to avoid duplicate events and distribute proxy check executions evenly across multiple agents. Read about proxy checks for more information.To use check ttl and round_robin together, your check configuration must also specify a proxy_entity_name . If you do not specify a proxy_entity_name when using check ttl and round_robin together, your check will stop executing. |
required | false |
type | Boolean |
default | false |
example |
|
runtime_assets | |
---|---|
description | Array of Sensu dynamic runtime assets (names). Required at runtime for the execution of the command . |
required | false |
type | Array |
example |
|
scheduler | |
---|---|
description | Type of scheduler that schedules the check. Sensu automatically sets the scheduler value and overrides any user-entered values. Value may be:
|
required | false |
type | String |
example |
|
secrets | |
---|---|
description | Array of the name/secret pairs to use with command execution. |
required | false |
type | Array |
example |
|
silenced | |
---|---|
description | Silences that apply to the check. |
type | Array |
example |
|
stdin | |
---|---|
description | true if the Sensu agent writes JSON serialized Sensu entity and check data to the command process’ stdin. The command must expect the JSON data via stdin, read it, and close stdin. Otherwise, false . This attribute cannot be used with existing Sensu check plugins or Nagios plugins because the Sensu agent will wait indefinitely for the check process to read and close stdin. |
required | false |
type | Boolean |
default | false |
example |
|
subdue (placeholder) | |
---|---|
description | Use the subdues attribute to stop check execution during specific periods. This subdue attribute appears in check definitions by default, but it is a placeholder and should not be modified. |
example |
|
subdues | |
---|---|
description | Specific periods of time when Sensu should not send alerts based on the events the check produces. Use to schedule alert-free periods of time, such as during sleeping hours, weekends, or special maintenance periods. Read subdues attributes for more information. |
required | false |
type | Array |
example |
|
subscriptions | |
---|---|
description | Array of Sensu entity subscriptions that check requests will be sent to. The array cannot be empty and its items must each be a string. |
required | true |
type | Array |
example |
|
timeout | |
---|---|
description | Check execution duration timeout (hard stop). In seconds. |
required | false |
type | Integer |
example |
|
ttl | |
---|---|
description | The time-to-live (TTL) until check results are considered stale. In seconds. If an agent stops publishing results for the check and the TTL expires, an event will be created for the agent’s entity. The check ttl must be greater than the check interval and should allow enough time for the check execution and result processing to complete. For example, for a check that has an interval of 60 (seconds) and a timeout of 30 (seconds), the appropriate ttl is at least 90 (seconds).To use check ttl and round_robin together, your check configuration must also specify a proxy_entity_name . If you do not specify a proxy_entity_name when using check ttl and round_robin together, your check will stop executing.
NOTE: Adding TTLs to checks adds overhead, so use the |
required | false |
type | Integer |
example |
|
Check output truncation attributes
max_output_size | |
---|---|
description | Maximum size of stored check outputs. In bytes. When set to a non-zero value, the Sensu backend truncates check outputs larger than this value before storing to etcd. max_output_size does not affect data sent to Sensu filters, mutators, and handlers.When check output is truncated due to the max_output_size configuration, the events the check produces will include the label sensu.io/output_truncated_bytes . |
required | false |
type | Integer |
example |
|
discard_output | |
---|---|
description | If true , discard check output after extracting metrics. No check output will be sent to the Sensu backend. Otherwise, false . |
required | false |
type | Boolean |
example |
|
output_metric_tags
attributes
name | |
---|---|
description | Name for the output metric tag. |
required | true |
type | String |
example |
|
value | |
---|---|
description | Value for the output metric tag. Use check token substitution syntax for the value attribute, with dot-notation access to any event attribute. |
required | true |
type | String |
example |
|
output_metric_thresholds
attributes
name | |
---|---|
description | Name of the metric to use for metric threshold evaluation. Must match the event.metrics.points[].name value for a metric point in the check results.
NOTE: To produce values for the output metrics you specify, the check definition must include a valid |
required | true |
type | String |
example |
|
null_status | |
---|---|
description | Event check status to use if a metric specified for metric threshold evaluation is missing from the event data.
NOTE: Sensu only overrides the event check status if it is less than the specified |
required | false |
type | Integer |
default | 0 |
example |
|
tags | |
---|---|
description | Tags of the metric to use for metric threshold evaluation. If provided, must match the event.metrics.points[].tags name and value for a metric point in the check results. Read tags attributes for more information. |
required | false |
type | Array |
example |
|
thresholds | |
---|---|
description | Rules to apply for metric threshold evaluation. Read thresholds attributes for more information. |
required | true |
type | Array |
example |
|
Pipelines attributes
api_version | |
---|---|
description | The Sensu API group and version for the pipeline. For pipelines in this version of Sensu, the api_version should always be core/v2 . |
required | true |
type | String |
default | null |
example |
|
name | |
---|---|
description | Name of the Sensu pipeline for the check to use. |
required | true |
type | String |
default | null |
example |
|
type | |
---|---|
description | The sensuctl create resource type for the pipeline. Pipelines should always be type Pipeline . |
required | true |
type | String |
default | null |
example |
|
Proxy requests attributes
entity_attributes | |
---|---|
description | Sensu entity attributes to match entities in the registry using Sensu query expressions. |
required | false |
type | Array |
example |
|
splay | |
---|---|
description | true if proxy check requests should be splayed, published evenly over a window of time, determined by the check interval and a configurable splay_coverage percentage. Otherwise, false . |
required | false |
type | Boolean |
default | false |
example |
|
splay_coverage | |
---|---|
description | Percentage of the check interval over which Sensu can execute the check for all applicable entities, as defined in the entity attributes. Sensu uses the splay_coverage attribute to determine the period of time to publish check requests over, before the next check interval begins. For example, if a check’s interval is 60 seconds and splay_coverage is 90, Sensu will distribute its proxy check requests evenly over a time window of 54 seconds (60 seconds * 90%). This leaves 6 seconds after the last proxy check execution before the the next round of proxy check requests for the same check. |
required | true if splay attribute is set to true (otherwise, false ) |
type | Integer |
example |
|
secrets
attributes
name | |
---|---|
description | Name of the secret defined in the executable command. Becomes the environment variable presented to the check. Read Use secrets management in Sensu for more information. |
required | true |
type | String |
example |
|
secret | |
---|---|
description | Name of the Sensu secret resource that defines how to retrieve the secret. |
required | true |
type | String |
example |
|
Tags attributes
name | |
---|---|
description | Tag name for the metric to use for metric threshold evaluation. If provided, must match the event.metrics.points[].tags.name value for a metric point in the check results.
NOTE: If provided, you must also provide the value for the same metric point tag. |
required | false |
type | String |
example |
|
value | |
---|---|
description | Tag value of the metric to use for metric threshold evaluation. If provided, must match the event.metrics.points[].tags.value value for a metric point in the check results.
NOTE: If provided, you must also provide the name for the same metric point tag. |
required | false |
type | String |
example |
|
Thresholds attributes
max | |
---|---|
description | Maximum threshold for the metric for metric threshold evaluation. You must provide a thresholds max value if you do not provide a min value. |
required | false (if a thresholds min value is provided) |
type | String |
example |
|
min | |
---|---|
description | Minimum threshold for the metric for metric threshold evaluation. You must provide a thresholds min value if you do not provide a max value. |
required | false (if a thresholds max value is provided) |
type | String |
example |
|
status | |
---|---|
description | Event check status to use if the check’s output metric value is equal to or greater than the specified max threshold or equal to or less than the specified min threshold in metric threshold evaluation.
NOTE: Sensu only overrides the event check status if it is less than the specified threshold unknown if the status does not equal 0 , 1 , or 2 . |
required | true |
type | Integer |
example |
|
subdues
attributes
begin | |
---|---|
description | Date and time at which the subdue should begin. In RFC 3339 format with numeric zone offset (2022-01-01T07:30:00-07:00 or 2022-01-01T14:30:00Z ). |
required | true |
type | String |
example |
|
end | |
---|---|
description | Date and time at which the subdue should end. In RFC 3339 format with numeric zone offset (2022-01-01T07:30:00-07:00 or 2022-01-01T14:30:00Z ). |
required | true |
type | String |
example |
|
repeat | |
---|---|
description | Interval at which the subdue should repeat. weekdays includes Mondays, Tuesdays, Wednesdays, Thursdays, and Fridays. weekends includes Saturdays and Sundays. Read Subdues and repeat for more information.
NOTE: Check subdue repeats are based on the specified |
required | false |
type | Array |
allowed values | mondays , tuesdays , wednesdays , thursdays , fridays , saturdays , sundays , weekdays , weekends , daily , weekly , monthly , annually |
example |
|
Metric check example
The following example shows the resource definition for a check that collects metrics in Nagios Performance Data format:
---
type: CheckConfig
api_version: core/v2
metadata:
annotations:
slack-channel: '#monitoring'
labels:
region: us-west-1
name: collect-metrics
spec:
check_hooks: null
command: collect.sh
discard_output: true
env_vars: null
handlers: []
high_flap_threshold: 0
interval: 10
low_flap_threshold: 0
output_metric_format: prometheus_text
output_metric_tags:
- name: instance
value: '{{ .name }}'
- name: namespace
value: '{{ .namespace }}'
- name: service
value: '{{ .labels.service }}'
output_metric_thresholds:
- name: system_mem_used
tags: null
null_status: 1
thresholds:
- max: "75.0"
min: ""
status: 1
- max: "90.0"
min: ""
status: 2
- name: system_host_processes
tags:
- name: namespace
value: production
null_status: 1
thresholds:
- max: "50"
min: "5"
status: 1
- max: "75"
min: "2"
status: 2
pipelines:
- type: Pipeline
api_version: core/v2
name: prometheus_gateway_workflows
proxy_entity_name: ""
publish: true
round_robin: false
runtime_assets: null
stdin: false
subscriptions:
- system
timeout: 0
ttl: 0
{
"type": "CheckConfig",
"api_version": "core/v2",
"metadata": {
"annotations": {
"slack-channel": "#monitoring"
},
"labels": {
"region": "us-west-1"
},
"name": "collect-metrics"
},
"spec": {
"check_hooks": null,
"command": "collect.sh",
"discard_output": true,
"env_vars": null,
"handlers": [],
"high_flap_threshold": 0,
"interval": 10,
"low_flap_threshold": 0,
"output_metric_format": "prometheus_text",
"output_metric_tags": [
{
"name": "instance",
"value": "{{ .name }}"
},
{
"name": "namespace",
"value": "{{ .namespace }}"
},
{
"name": "service",
"value": "{{ .labels.service }}"
}
],
"output_metric_thresholds": [
{
"name": "system_mem_used",
"tags": null,
"null_status": 1,
"thresholds": [
{
"max": "75.0",
"min": "",
"status": 1
},
{
"max": "90.0",
"min": "",
"status": 2
}
]
},
{
"name": "system_host_processes",
"tags": [
{
"name": "namespace",
"value": "production"
}
],
"null_status": 1,
"thresholds": [
{
"max": "50",
"min": "5",
"status": 1
},
{
"max": "75",
"min": "2",
"status": 2
}
]
}
],
"pipelines": [
{
"type": "Pipeline",
"api_version": "core/v2",
"name": "prometheus_gateway_workflows"
}
],
"proxy_entity_name": "",
"publish": true,
"round_robin": false,
"runtime_assets": null,
"stdin": false,
"subscriptions": [
"system"
],
"timeout": 0,
"ttl": 0
}
}
Check example that uses secrets management
The check in the following example uses secrets management to keep a GitHub token private. Learn more about secrets management for your Sensu configuration in the secrets and secrets providers references.
---
type: CheckConfig
api_version: core/v2
metadata:
name: ping-github-api
spec:
check_hooks: null
command: ping-github-api.sh $GITHUB_TOKEN
secrets:
- name: GITHUB_TOKEN
secret: github-token-vault
{
"type": "CheckConfig",
"api_version": "core/v2",
"metadata": {
"name": "ping-github-api"
},
"spec": {
"check_hooks": null,
"command": "ping-github-api.sh $GITHUB_TOKEN",
"secrets": [
{
"name": "GITHUB_TOKEN",
"secret": "github-token-vault"
}
]
}
}
Check example with a PowerShell script command
If you use a PowerShell script in your check command, make sure to include the -f
flag in the command.
The -f
flag ensures that the proper exit code is passed into Sensu.
For example:
---
type: CheckConfig
api_version: core/v2
metadata:
name: interval_test
spec:
command: powershell.exe -f c:\\users\\tester\\test.ps1
subscriptions:
- system
interval: 60
pipelines:
- type: Pipeline
api_version: core/v2
name: interval_pipeline
publish: true
{
"type": "CheckConfig",
"api_version": "core/v2",
"metadata": {
"name": "interval_test"
},
"spec": {
"command": "powershell.exe -f c:\\\\users\\ ester\\ est.ps1",
"subscriptions": [
"system"
],
"interval": 60,
"pipelines": [
{
"type": "Pipeline",
"api_version": "core/v2",
"name": "interval_pipeline"
}
],
"publish": true
}
}
The dynamic runtime asset reference includes an example check definition that uses the asset path to correctly capture exit status codes from PowerShell plugins distributed as dynamic runtime assets.