Agent reference

Example Sensu agent configuration file (download)

The Sensu agent is a lightweight client that runs on the infrastructure components you want to monitor. Agents register with the Sensu backend as entities with type: "agent". Agent entities are responsible for creating check and metrics events to send to the backend event pipeline.

The Sensu agent is available for Linux, macOS, and Windows. For Windows operating systems, the Sensu agent uses cmd.exe for the execution environment. For all other operating systems, the Sensu agent uses the Bourne shell (sh).

Read the installation guide to install the agent.

Agent authentication

The Sensu agent authenticates to the Sensu backend via WebSocket transport by either built-in basic authentication (username and password) or mutual transport layer security (mTLS) authentication.

Username and password authentication

The default mechanism for agent authentication is built-in basic authentication with username and password. The Sensu agent uses username and password authentication unless mTLS authentication has been explicitly configured.

For username and password authentication, sensu-agent joins the username and password with a colon and encodes them as a Base64 value. Sensu provides the encoded string as the value of the Authorization HTTP header — for example, Authorization: Basic YWdlbnQ6UEBzc3cwcmQh — to authenticate to the Sensu backend.

When using username and password authentication, sensu-agent also sends the following HTTP headers in requests to the backend:

• Sensu-User: the username in plaintext
• Sensu-AgentName: the agent’s configured name in plaintext
• Sensu-Subscriptions: the agent’s subscriptions in a comma-separated plaintext list
• Sensu-Namespace: the agent’s configured namespace in plaintext

mTLS authentication

When mTLS is configured for both the Sensu agent and backend, the agent uses mTLS authentication instead of the default username and password authentication.

Sensu backends that are configured for mTLS authentication will no longer accept agent authentication via username and password. Agents that are configured to use mTLS authentication cannot authenticate with the backend unless the backend is configured for mTLS.

To configure the agent and backend for mTLS authentication:

• In the backend configuration, specify valid certificate and key files as values for the agent-auth-cert-file and agent-auth-key-file parameters (e.g. backend-1.pem and backend-1-key.pem, respectively).
• In the agent configuration, specify valid certificate and key files as values for the cert-file and key-file parameters (e.g. agent.pem and agent-key.pem, respectively).

The agent and backend will compare the provided certificates with the trusted CA certificate either in the system trust store or specified explicitly as the agent-auth-trusted-ca-file in the backend configuration and trusted-ca-file in the agent configuration.

When using mTLS authentication, sensu-agent sends the following HTTP headers in requests to the backend:

• Sensu-AgentName: the agent’s configured name in plaintext
• Sensu-Subscriptions: the agent’s subscriptions in a comma-separated, plaintext list
• Sensu-Namespace: the agent’s configured namespace in plaintext

If the Sensu agent is configured for mTLS authentication, it will not send the Authorization HTTP header.

Certificate bundles or chains

The Sensu agent supports all types of certificate bundles (or chains) as long as the agent (or leaf) certificate is the first certificate in the bundle. This is because the Go standard library assumes that the first certificate listed in the PEM file is the leaf certificate — the certificate that the program will use to show its own identity.

If you send the leaf certificate alone instead of sending the whole bundle with the leaf certificate first, you will receive a certificate not signed by trusted authority error. You must present the whole chain to the remote so it can determine whether it trusts the presented certificate through the chain.

Certificate revocation check

The Sensu backend checks certificate revocation list (CRL) and Online Certificate Status Protocol (OCSP) endpoints for agent mTLS, etcd client, and etcd peer connections whose remote sides present X.509 certificates that provide CRL and OCSP revocation information.

Communication between the agent and backend

The Sensu agent uses WebSocket (ws) protocol to send and receive JSON messages with the Sensu backend. For optimal network throughput, agents will attempt to negotiate the use of Protobuf serialization when communicating with a Sensu backend that supports it. This communication is via clear text by default.

Follow Secure Sensu to configure the backend and agent for WebSocket Secure (wss) encrypted communication.

Connection failure

Although connection failure may be due to socket errors like unexpectedly closed connections and TLS handshake failures, the Sensu agent generally keeps retrying connections to each URL in the backend-url list until it is successfully connected to a backend URL or you stop the process.

When you start up a Sensu agent configured with multiple backend-url values, the agent shuffles the backend-url list and attempts to connect to the first URL in the shuffled list. If the agent cannot establish a WebSocket connection with the first URL within the number of seconds specified for the backend-handshake-timeout, the agent abandons the connection attempt and tries the next URL in the shuffled list.

When the agent establishes a WebSocket connection with a backend URL within the backend-handshake-timeout period, the agent sends a heartbeat message to the backend at the specified backend-heartbeat-interval. For every heartbeat the agent sends, the agent expects the connected backend to send a heartbeat response within the number of seconds specified for the backend-heartbeat-timeout. If the connected backend does not respond within the backend-heartbeat-timeout period, the agent closes the connection and attempts to connect to the next backend URL in the shuffled list.

The agent iterates through the shuffled backend-url list until it successfully establishes a WebSocket connection with a backend, returning to the first URL if it fails to connect with the last URL in the list.

Synchronize time between agents and the backend

System clocks between agents and the backend should be synchronized to a central NTP server. If system time is out of sync, it may cause issues with keepalive, metric, and check alerts.

Agent connection to a cluster

Agents can connect to a Sensu cluster by specifying any Sensu backend URL in the cluster in the backend-url configuration option.

For more information about clustering, read Backend datastore configuration and Run a Sensu cluster.

Keepalive monitoring

Sensu keepalives are the heartbeat mechanism used to ensure that all registered agents are operational and able to reach the Sensu backend. Sensu agents publish keepalive events containing entity configuration data to the Sensu backend according to the interval specified by the keepalive-interval configuration option. All Sensu agent data provided in keepalive events is stored in the agent registry and used to add context to Sensu events and detect Sensu agents in an unhealthy state.

If a Sensu agent fails to send keepalive events over the period specified by the keepalive-critical-timeout configuration option, the Sensu backend creates a keepalive critical alert in the Sensu web UI. The keepalive-critical-timeout is set to 0 (disabled) by default to help ensure that it will not interfere with your keepalive-warning-timeout setting.

If a Sensu agent fails to send keepalive events over the period specified by the keepalive-warning-timeout configuration option, the Sensu backend creates a keepalive warning alert in the Sensu web UI. The value you specify for keepalive-warning-timeout must be lower than the value you specify for keepalive-critical-timeout.

You can use keepalives to identify unhealthy systems and network partitions, send notifications, trigger auto-remediation, and automatically register and deregister agent entities, among other useful actions. The agent maps keepalive-critical-timeout and keepalive-warning-timeout values to certain event check attributes, so you can also create time-based event filters to reduce alert fatigue for agent keepalive events.

Process keepalive events

Process keepalive events with a pipeline or handler.

Keepalive pipelines

Use the keepalive-pipelines configuration option to send keepalive events to any pipeline you have configured.

To specify pipelines for the keepalive-pipelines option, use the fully qualified name for pipelines (core/v2.Pipeline) plus the pipeline name (e.g. slack or store-keepalives). For example:

sensu-agent start --keepalive-pipelines core/v2.Pipeline.slack,core/v2.Pipeline.store-keepalives

keepalive-pipelines:
- core/v2.Pipeline.slack
- core/v2.Pipeline.store-keepalives

If you do not specify a pipeline with the keepalive-pipelines option, the Sensu backend will use the default keepalive handler and create an event in sensuctl and the Sensu web UI for keepalives.

Keepalive handlers

You can use a keepalive handler to connect keepalive events to your monitoring workflows. Sensu looks for an event handler named keepalive and automatically uses it to process keepalive events.

Suppose you want to receive Slack notifications for keepalive alerts, and you already have a Slack handler set up to process events. To process keepalive events using the Slack handler, create a handler set named keepalive and add the slack handler to the handlers array. The resulting keepalive handler set configuration looks like this:

---
type: Handler
api_version: core/v2
metadata:
  name: keepalive
spec:
  handlers:
  - slack
  type: set

{
  "type": "Handler",
  "api_version": "core/v2",
  "metadata" : {
    "name": "keepalive"
  },
  "spec": {
    "type": "set",
    "handlers": [
      "slack"
    ]
  }
}

You can also use the keepalive-handlers configuration option to send keepalive events to any handler you have configured. If you do not specify a keepalive handler with the keepalive-handlers option, the Sensu backend will use the default keepalive handler and create an event in sensuctl and the Sensu web UI.

Create observability events using service checks

The Sensu backend coordinates check execution for you by comparing the subscriptions you specify in your checks and entities to determine which entities should receive execution requests for a given check.

Sensu uses the publish/subscribe pattern of communication, which allows automated registration and deregistration of ephemeral systems. At the core of this model are Sensu subscriptions, which you specify in checks and entities to determine which entities should receive execution requests for a given check. Subscriptions often correspond with the roles and responsibilities assigned to the entity, such as webserver or database.

Subscriptions determine which checks the agent will execute. For an agent to execute a check, at least one entity must include a subscription that matches a subscription in the check definition. Read the subscriptions reference for more information.

After receiving a check request from the Sensu backend, the agent:

1. Applies any tokens that match attribute values in the check definition.

2. Fetches dynamic runtime assets and stores them in its local cache.

   By default, agents cache dynamic runtime asset data at /var/cache/sensu/sensu-agent (Linux) or C:\ProgramData\sensu\cache\sensu-agent (Windows). To specify a different cache location, use the cache-dir configuration attribute.

3. Executes the check command.

4. Executes any hooks specified by the check based on the exit status.

5. Creates an event that contains information about the applicable entity, check, and metric.

The Sensu backend then processes the event by applying event filters, mutators, and handlers.

Proxy entities

Proxy entities allow Sensu to monitor external resources on systems or devices where a Sensu agent cannot be installed, like a network switch.

The Sensu backend stores proxy entity definitions (unlike agent entities, which the agent stores). When the backend requests a check that includes a proxy_entity_name, the agent includes the provided entity information in the observation data in events in place of the agent entity data.

Read the entities reference and Monitor external resources for more information about monitoring proxy entities.

Create observability events using the agent API

The Sensu agent API allows external sources to send monitoring data to Sensu without requiring the external sources to know anything about Sensu’s internal implementation. The agent API listens on the address and port specified with the agent API configuration options.

The agent API supports only unsecured HTTP requests (no HTTPS). Requests for unknown endpoints will result in an HTTP 404 Not Found response.

/events (POST)

The agent API provides HTTP POST access to publish observability events to the Sensu backend via the /events endpoint.

The agent places events created via the agent API /events endpoint into a queue stored on disk. In case of a loss of connection with the backend or agent shutdown, the agent preserves queued event data. When the connection is reestablished, the agent sends the queued events to the backend.

The agent API /events endpoint uses a configurable burst limit and rate limit for relaying events to the backend. Read API configuration to configure the events-burst-limit and events-rate-limit options.

Example POST request to events endpoint

The following example submits an HTTP POST request to the agent API /events endpoint. The request creates an event for a check named check-mysql-status with the output could not connect to mysql and a status of 1 (warning). The agent responds with an HTTP 202 Accepted response to indicate that the event has been added to the queue to be sent to the backend.

In this example, the event will be processed according to an incident_alerts pipeline.

curl -X POST \
-H 'Content-Type: application/json' \
-d '{
  "check": {
    "metadata": {
      "name": "check-mysql-status"
    },
    "status": 1,
    "output": "could not connect to mysql"
  },
  "pipelines": [
    {
      "api_version": "core/v2",
      "type": "Pipeline",
      "name": "incident_alerts"
    }
  ]
}' \
http://127.0.0.1:3031/events

Detect silent failures

You can use the Sensu agent API in combination with the check time-to-live (TTL) attribute to detect silent failures. This creates what’s commonly referred to as a “dead man’s switch”.

With check TTLs, Sensu can set an expectation that a Sensu agent will publish additional events for a check within the period of time specified by the TTL attribute. If a Sensu agent fails to publish an event before the check TTL expires, the Sensu backend creates an event with a status of 1 (warning) to indicate the expected event was not received. For more information about check TTLs, read the checks reference.

If you use the check TTL attribute along with the Sensu agent API to enable tasks that run outside of Sensu’s check scheduling to emit events, these events create a dead man’s switch: if the task fails for any reason, the lack of an “all clear” event from the task will notify operators of the silent failure, which might otherwise be missed. If an external source sends an event with a check TTL to the Sensu agent API, Sensu expects another event from the same external source before the TTL expires.

Here’s an example of external event input via the Sensu agent API that uses a check TTL to create a dead man’s switch for MySQL backups. Assume that a MySQL backup script runs periodically, and you expect the job to take a little less than 7 hours to complete.

• If the job completes successfully, you want a record of it, but you don’t need to receive an alert.
• If the job fails or continues running longer than the expected 7 hours, you do need to receive an alert.

The script can send an event that tells the Sensu backend to expect an additional event with the same name within 7 hours of the first event:

curl -X POST \
-H 'Content-Type: application/json' \
-d '{
  "check": {
    "metadata": {
      "name": "mysql-backup-job"
    },
    "status": 0,
    "output": "mysql backup initiated",
    "ttl": 25200
  }
}' \
http://127.0.0.1:3031/events

When the script submitted this initial event to the agent API, you recorded in the Sensu backend that your script started. You also configured the dead man’s switch by including the ttl attribute, so you’ll receive an alert if the job fails or runs for too long. Although it is possible for your script to handle errors gracefully and emit additional observability events, this approach allows you to worry less about handling every possible error case. A lack of additional events before the 7-hour period elapses results in an alert.

If your backup script runs successfully, it can send an additional event without the TTL attribute, which removes the dead man’s switch:

curl -X POST \
-H 'Content-Type: application/json' \
-d '{
  "check": {
    "metadata": {
      "name": "mysql-backup-job"
    },
    "status": 0,
    "output": "mysql backup ran successfully!"
  }
}' \
http://127.0.0.1:3031/events

Omitting the TTL attribute from this event also removes the dead man’s switch being monitored by the Sensu backend. This effectively sounds the “all clear” for this iteration of the task.

API specification

{
  "check": {
    "metadata": {
      "name": "check-mysql-status"
    },
    "status": 1,
    "output": "could not connect to mysql"
  }
}

/healthz (GET)

The agent API /healthz endpoint provides HTTP GET access to the status of the Sensu agent via the agent API.

Example

In the following example, an HTTP GET request is submitted to the agent API /healthz endpoint:

curl http://127.0.0.1:3031/healthz

The request results in a healthy response:

ok

API specification

Create observability events using the StatsD listener

Sensu agents include a listener to send StatsD metrics to the event pipeline. By default, Sensu agents listen on UDP socket 8125 for messages that follow the StatsD line protocol and send metric events for handling by the Sensu backend.

For example, you can use the Netcat utility to send metrics to the StatsD listener:

echo 'abc.def.g:10|c' | nc -w1 -u localhost 8125

Sensu does not store metrics received through the StatsD listener, so it’s important to configure event handlers.

StatsD line protocol

The Sensu StatsD listener accepts messages that are formatted according to the StatsD line protocol:

<metricname>:<value>|<type>

For more information about StatsD, read the StatsD documentation.

Configure the StatsD listener

To configure the StatsD listener, specify the statsd-event-handlers configuration option in the agent configuration and start the agent. For example, to start an agent that sends StatsD metrics to InfluxDB, run:

sensu-agent --statsd-event-handlers influx-db

Use the StatsD configuration options to change the default settings for the StatsD listener address, port, and flush interval. For example, to start an agent with a customized address and flush interval, run:

sensu-agent --statsd-event-handlers influx-db --statsd-flush-interval 1 --statsd-metrics-host 123.4.5.11 --statsd-metrics-port 8125

Create observability events using the agent TCP and UDP sockets

Sensu agents listen for external monitoring data using TCP and UDP sockets. The agent sockets accept JSON event data and pass events to the Sensu backend event pipeline for processing. The TCP and UDP sockets listen on the address and port specified by the socket configuration options.

Use the TCP socket

This example demonstrates external monitoring data input via the Sensu agent TCP socket. The example uses Bash’s built-in /dev/tcp file to communicate with the Sensu agent socket:

echo '{"name": "check-mysql-status", "status": 1, "output": "error!"}' > /dev/tcp/localhost/3030

You can also use the Netcat utility to send monitoring data to the agent socket:

echo '{"name": "check-mysql-status", "status": 1, "output": "error!"}' | nc localhost 3030

Use the UDP socket

This example demonstrates external monitoring data input via the Sensu agent UDP socket. The example uses Bash’s built-in /dev/udp file to communicate with the Sensu agent socket:

echo '{"name": "check-mysql-status", "status": 1, "output": "error!"}' > /dev/udp/127.0.0.1/3030

You can also use the Netcat utility to send monitoring data to the agent socket:

echo '{"name": "check-mysql-status", "status": 1, "output": "error!"}' | nc -u -v 127.0.0.1 3030

Socket event format

The agent TCP and UDP sockets use a special event data format designed for backward compatibility with Sensu Core 1.x check results. Attributes specified in socket events appear in the resulting event data passed to the Sensu backend.

Example socket input: Minimum required attributes

{
  "name": "check-mysql-status",
  "status": 1,
  "output": "error!"
}

Example socket input: All attributes

{
  "name": "check-http",
  "status": 1,
  "output": "404",
  "source": "sensu-docs-site",
  "executed": 1550013435,
  "duration": 1.903135228,
  "handlers": ["slack", "influxdb"]
}

Socket event specification

{
  "name": "check-mysql-status"
}

{
  "status": 0
}

{
  "output": "CheckHttp OK: 200, 78572 bytes"
}

{
  "source": "sensu-docs-site"
}

{
  "client": "sensu-docs-site"
}

{
  "executed": 1458934742
}

{
  "duration": 1.903135228
}

{
  "command": "http-check --url https://sensu.io"
}

{
  "interval": 60
}

{
  "handlers": ["slack", "influxdb"]
}

Registration, endpoint management, and service discovery

Sensu agents automatically discover and register infrastructure components and the services running on them. When an agent process stops, the Sensu backend can automatically create and process a deregistration event for the agent entities.

Read Automatically register and deregister entities for more information.

Agent configuration options

Agent configuration is customizable. This section describes each configuration option in more detail, including examples for each configuration method.

You can customize agent configuration with the agent configuration file (Linux and Windows), command line flag arguments (Linux), or environment variables (Linux and Windows).

To view available configuration options for the sensu-agent start command, run:

sensu-agent start --help

The response will list configuration options as command line flags for sensu-agent start:

start the sensu agent

Usage:
  sensu-agent start [flags]

Flags:
      --agent-managed-entity                manage this entity via the agent
      --allow-list string                   path to agent execution allow list configuration file
      --annotations stringToString          entity annotations map (default [])
      --api-host string                     address to bind the Sensu client HTTP API to (default "127.0.0.1")
      --api-port int                        port the Sensu client HTTP API listens on (default 3031)
      --assets-burst-limit int              asset fetch burst limit (default 100)
      --assets-rate-limit float             maximum number of assets fetched per second
      --backend-handshake-timeout int       number of seconds the agent should wait when negotiating a new WebSocket connection (default 15)
      --backend-heartbeat-interval int      interval at which the agent should send heartbeats to the backend (default 30)
      --backend-heartbeat-timeout int       number of seconds the agent should wait for a response to a hearbeat (default 45)
      --backend-url strings                 comma-delimited list of ws/wss URLs of Sensu backend servers. This flag can also be invoked multiple times (default [ws://127.0.0.1:8081])
      --cache-dir string                    path to store cached data (default "/var/cache/sensu/sensu-agent")
      --cert-file string                    certificate for TLS authentication
  -c, --config-file string                  path to sensu-agent config file (default "/etc/sensu/agent.yml")
      --deregister                          ephemeral agent
      --deregistration-handler string       deregistration handler that should process the entity deregistration event
      --detect-cloud-provider               enable cloud provider detection
      --disable-api                         disable the Agent HTTP API
      --disable-assets                      disable check assets on this agent
      --disable-sockets                     disable the Agent TCP and UDP event sockets
      --discover-processes                  indicates whether process discovery should be enabled
      --events-burst-limit int              /events api burst limit (default 10)
      --events-rate-limit float             maximum number of events transmitted to the backend through the /events api
  -h, --help                                help for start
      --insecure-skip-tls-verify            skip TLS verification (not recommended!)
      --keepalive-critical-timeout uint32   number of seconds until agent is considered dead by backend to create a critical event
      --keepalive-handlers strings          comma-delimited list of keepalive handlers for this entity. This flag can also be invoked multiple times
      --keepalive-interval int              number of seconds to send between keepalive events (default 20)
      --keepalive-pipelines strings         comma-delimited list of pipeline references for keepalive event
      --keepalive-warning-timeout uint32    number of seconds until agent is considered dead by backend to create a warning event (default 120)
      --key-file string                     key for TLS authentication
      --labels stringToString               entity labels map (default [])
      --log-level string                    logging level [panic, fatal, error, warn, info, debug] (default "info")
      --max-session-length                  maximum amount of time after which the agent will reconnect to one of the configured backends (no maximum by default)
      --name string                         agent name (defaults to hostname) (default "my_hostname")
      --namespace string                    agent namespace (default "default")
      --password string                     agent password (default "P@ssw0rd!")
      --redact strings                      comma-delimited list of fields to redact, overwrites the default fields. This flag can also be invoked multiple times (default [password,passwd,pass,api_key,api_token,access_key,secret_key,private_key,secret])
      --require-fips                        indicates whether fips support should be required in openssl
      --retry-max                           maximum amount of time to wait before retrying an agent connection to the backend
      --retry-min                           minimum amount of time to wait before retrying an agent connection to the backend
      --retry-multiplier                    value multiplied with the current retry delay to produce a longer retry delay (bounded by --retry-max)
      --socket-host string                  address to bind the Sensu client socket to (default "127.0.0.1")
      --socket-port int                     port the Sensu client socket listens on (default 3030)
      --statsd-disable                      disables the statsd listener and metrics server
      --statsd-event-handlers strings       comma-delimited list of event handlers for statsd metrics. This flag can also be invoked multiple times
      --statsd-flush-interval int           number of seconds between statsd flush (default 10)
      --statsd-metrics-host string          address used for the statsd metrics server (default "127.0.0.1")
      --statsd-metrics-port int             port used for the statsd metrics server (default 8125)
      --strip-networks                      do not include Network info in agent entity state
      --subscriptions strings               comma-delimited list of agent subscriptions. This flag can also be invoked multiple times
      --trusted-ca-file string              TLS CA certificate bundle in PEM format
      --user string                         agent user (default "agent")

General configuration

sensu-agent start --agent-managed-entity

agent-managed-entity: true

sensu-agent start --allow-list /etc/sensu/check-allow-list.yaml

allow-list: /etc/sensu/check-allow-list.yaml

sensu-agent start --annotations sensu.io/plugins/slack/config/webhook-url=https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX
sensu-agent start --annotations example-key="example value" --annotations example-key2="example value"

annotations:
  sensu.io/plugins/slack/config/webhook-url: "https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX"

sensu-agent start --assets-burst-limit 100

assets-burst-limit: 100

sensu-agent start --assets-rate-limit 1.39

assets-rate-limit: 1.39

sensu-agent start --backend-handshake-timeout 20

backend-handshake-timeout: 20

sensu-agent start --backend-heartbeat-interval 45

backend-heartbeat-interval: 45

sensu-agent start --backend-heartbeat-timeout 60

backend-heartbeat-timeout: 60

sensu-agent start --backend-url ws://127.0.0.1:8081
sensu-agent start --backend-url ws://127.0.0.1:8081 --backend-url ws://127.0.0.1:8082

sensu-agent start --backend-url wss://127.0.0.1:8081
sensu-agent start --backend-url wss://127.0.0.1:8081 --backend-url wss://127.0.0.1:8082

backend-url:
  - "ws://127.0.0.1:8081"
  - "ws://127.0.0.1:8082"

backend-url:
  - "wss://127.0.0.1:8081"
  - "wss://127.0.0.1:8082"

sensu-agent start --cache-dir /cache/sensu-agent

cache-dir: "/cache/sensu-agent"

sensu-agent start --config-file /sensu/agent.yml
sensu-agent start -c /sensu/agent.yml

sensu-agent start --disable-assets

disable-assets: true

sensu-agent start --discover-processes

discover-processes: true

sensu-agent start --labels proxy_type=website
sensu-agent start --labels example_key1="example value" example_key2="example value"

labels:
  proxy_type: website

sensu-agent start --log-level debug

log-level: debug

sensu-agent start --max-session-length 15m

max-session-length: 15m

sensu-agent start --name agent-01

name: "agent-01"

sensu-agent start --retry-max 120s

retry-max: 120s

sensu-agent start --retry-min 1s

retry-min: 1s

sensu-agent start --retry-multiplier 2.0

retry-multiplier: 2.0

sensu-agent start --strip-networks

strip-networks: true

sensu-agent start --subscriptions disk-checks,process-checks
sensu-agent start --subscriptions disk-checks --subscriptions process-checks

subscriptions:
  - disk-checks
  - process-checks

API configuration

sensu-agent start --api-host 127.0.0.1

api-host: "127.0.0.1"

sensu-agent start --api-port 3031

api-port: 3031

sensu-agent start --disable-api

disable-api: true

sensu-agent start --events-burst-limit 20

events-burst-limit: 20

sensu-agent start --events-rate-limit 20.0

events-rate-limit: 20.0

Ephemeral agent configuration

sensu-agent start --deregister

deregister: true

sensu-agent start --deregistration-handler deregister

deregistration-handler: deregister

sensu-agent start --detect-cloud-provider false

detect-cloud-provider: false

Keepalive configuration

sensu-agent start --keepalive-critical-timeout 300

keepalive-critical-timeout: 300

sensu-agent start --keepalive-handlers slack,email

keepalive-handlers:
- slack
- email

sensu-agent start --keepalive-interval 30

keepalive-interval: 30

sensu-agent start --keepalive-pipelines core/v2.Pipeline.slack,core/v2.Pipeline.store-keepalives

keepalive-pipelines:
- core/v2.Pipeline.slack
- core/v2.Pipeline.store-keepalives

sensu-agent start --keepalive-warning-timeout 300

keepalive-warning-timeout: 300

Security configuration

sensu-agent start --cert-file /path/to/tls/agent.pem

cert-file: "/path/to/tls/agent.pem"

sensu-agent start --insecure-skip-tls-verify

insecure-skip-tls-verify: true

sensu-agent start --key-file /path/to/tls/agent-key.pem

key-file: "/path/to/tls/agent-key.pem"

sensu-agent start --namespace ops

namespace: ops

sensu-agent start --password secure-password

password: secure-password

sensu-agent start --redact secret,ec2_access_key

redact:
  - secret
  - ec2_access_key

sensu-agent start --require-fips

require-fips: true

sensu-agent start --trusted-ca-file /path/to/tls/ca.pem

trusted-ca-file: "/path/to/tls/ca.pem"

sensu-agent start --user agent-01

user: "agent-01"

Socket configuration

sensu-agent start --disable-sockets

disable-sockets: true

sensu-agent start --socket-host 127.0.0.1

socket-host: "127.0.0.1"

sensu-agent start --socket-port 3030

socket-port: 3030

StatsD configuration

sensu-agent start --statsd-disable

statsd-disable: true

sensu-agent start --statsd-event-handlers influxdb,opentsdb
sensu-agent start --statsd-event-handlers influxdb --statsd-event-handlers opentsdb

statsd-event-handlers:
  - influxdb
  - opentsdb

sensu-agent start --statsd-flush-interval 30

statsd-flush-interval: 30

sensu-agent start --statsd-metrics-host 127.0.0.1

statsd-metrics-host: "127.0.0.1"

sensu-agent start --statsd-metrics-port 8125

statsd-metrics-port: 8125

Allow list configuration

The allow list includes check and hook commands the agent can execute. Use the allow-list configuration option to specify the path to the yaml or json file that contains your allow list.

Use these commands to build your allow list configuration file.

args:
- foo

{
  "args": ["foo"]
}

enable_env: true

{
  "enable_env": true
}

exec: "/usr/local/bin/check_memory.sh"

{
  "exec": "/usr/local/bin/check_memory.sh"
}

sha512: 4f926bf4328...

{
  "sha512": "4f926bf4328..."
}

Example allow list configuration

- exec: /usr/local/bin/check_memory.sh
  args:
  - ""
  sha512: 736ac120323772543fd3a08ee54afdd54d214e58c280707b63ce652424313ef9084ca5b247d226aa09be8f831034ff4991bfb95553291c8b3dc32cad034b4706
  enable_env: true
  foo: bar
- exec: /usr/local/bin/show_process_table.sh
  args:
  - ""
  sha512: 28d61f303136b16d20742268a896bde194cc99342e02cdffc1c2186f81c5adc53f8550635156bebeed7d87a0c19a7d4b7a690f1a337cc4737e240b62b827f78a
- exec: echo-asset.sh
  args:
  - "foo"
  sha512: cce3d16e5881ba829f271df778f9014f7c3659917f7acfd7a60a91bfcabb472eea72f9781194d310388ba046c21790364ad0308a5a897cde50022195ba90924b

[
  {
    "exec": "/usr/local/bin/check_memory.sh",
    "args": [
      ""
    ],
    "sha512": "736ac120323772543fd3a08ee54afdd54d214e58c280707b63ce652424313ef9084ca5b247d226aa09be8f831034ff4991bfb95553291c8b3dc32cad034b4706",
    "enable_env": true,
    "foo": "bar"
  },
  {
    "exec": "/usr/local/bin/show_process_table.sh",
    "args": [
      ""
    ],
    "sha512": "28d61f303136b16d20742268a896bde194cc99342e02cdffc1c2186f81c5adc53f8550635156bebeed7d87a0c19a7d4b7a690f1a337cc4737e240b62b827f78a"
  },
  {
    "exec": "echo-asset.sh",
    "args": [
      "foo"
    ],
    "sha512": "cce3d16e5881ba829f271df778f9014f7c3659917f7acfd7a60a91bfcabb472eea72f9781194d310388ba046c21790364ad0308a5a897cde50022195ba90924b"
  }
]

Agent configuration methods

Agent configuration file

For Linux and Windows agents, you can customize the agent configuration in a .yml configuration file.

The default agent configuration file path for Linux is /etc/sensu/agent.yml. The default agent configuration file path for Windows is C:\ProgramData\sensu\config\agent.yml.example.

To use the agent.yml file to configure the agent, list the desired configuration attributes and values. Review the example Sensu agent configuration file for a complete example.

Configuration via command line flags or environment variables overrides any configuration specified in the agent configuration file. Read Create overrides to learn more.

Command line flags

For Linux agents, you can customize the agent configuration with sensu-agent start command line flags.

To use command line flags, specify the desired configuration options and values along with the sensu-agent start command. For example:

sensu-agent start --name webserver_05 --keepalive-warning-timeout 60 --keepalive-critical-timeout 120

Configuration via command line flags overrides attributes specified in a configuration file or with environment variables. Read Create overrides to learn more.

Environment variables

Instead of using the agent configuration file or command line flags, you can use environment variables to configure your Sensu agent. Each agent configuration option has an associated environment variable. You can also create your own environment variables, as long as you name them correctly and save them in the correct place. Here’s how.

1. Create the files from which the sensu-agent service configured by our supported packages will read environment variables:

   sudo touch /etc/default/sensu-agent

   sudo touch /etc/sysconfig/sensu-agent

2. Make sure the environment variable is named correctly. All environment variables that control Sensu agent configuration begin with SENSU_.

   To rename a configuration option you wish to specify as an environment variable, prepend SENSU_, convert dashes to underscores, and capitalize all letters. For example, the environment variable for the flag api-host is SENSU_API_HOST.

   For a custom environment variable, you do not have to prepend SENSU. For example, TEST_VAR_1 is a valid custom environment variable name.

3. Add the environment variable to the environment file.

   In this example, the api-host flag is configured as an environment variable and set to "0.0.0.0":

   echo 'SENSU_API_HOST="0.0.0.0"' | sudo tee -a /etc/default/sensu-agent

   echo 'SENSU_API_HOST="0.0.0.0"' | sudo tee -a /etc/sysconfig/sensu-agent

4. Restart the sensu-agent service so these settings can take effect:

   sudo systemctl restart sensu-agent

   sudo systemctl restart sensu-agent

Environment variables on the Windows agent are located in the Windows Registry. Here’s how to add environment variables to the agent.

1. Open up the Run dialog:

windows + r

2. Open the registry editor by typing the following in the Run dialog:

regedit

3. Navigate to the following registry path: Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment

4. Click Edit –> New –> String Value

5. Edit the new string entry and change the “Value name” to be SENSU_API_HOST and the “Value data” to be 0.0.0.0

6. Log out of any sessions on your Windows device, or optionally reboot to have the environment variable loaded.

Format for label and annotation environment variables

To use labels and annotations as environment variables in your check and plugin configurations, you must use a specific format when you create the environment variables.

For example, to create the labels "region": "us-east-1" and "type": "website" as an environment variable:

echo 'SENSU_LABELS='{"region": "us-east-1", "type": "website"}'' | sudo tee -a /etc/default/sensu-agent

echo 'SENSU_LABELS='{"region": "us-east-1", "type": "website"}'' | sudo tee -a /etc/sysconfig/sensu-agent

To create the annotations "maintainer": "Team A" and "webhook-url": "https://hooks.slack.com/services/T0000/B00000/XXXXX" as an environment variable:

echo 'SENSU_ANNOTATIONS='{"maintainer": "Team A", "webhook-url": "https://hooks.slack.com/services/T0000/B00000/XXXXX"}'' | sudo tee -a /etc/default/sensu-agent

echo 'SENSU_ANNOTATIONS='{"maintainer": "Team A", "webhook-url": "https://hooks.slack.com/services/T0000/B00000/XXXXX"}'' | sudo tee -a /etc/sysconfig/sensu-agent

Use environment variables with the Sensu agent

Any environment variables you create in /etc/default/sensu-agent (Debian family) or /etc/sysconfig/sensu-agent (RHEL family) will be available to check and hook commands executed by the Sensu agent. This includes your checks and plugins.

For example, if you create a custom environment variable TEST_VARIABLE in your sensu-agent file, it will be available to use in your check and hook configurations as $TEST_VARIABLE.

The following check example demonstrates how to use a TEST_GITHUB_TOKEN environment variable (set to the token value in the sensu-agent file) in the check command to run a script that pings the GitHub API:

---
type: CheckConfig
api_version: core/v2
metadata:
  name: ping-github-api
spec:
  command: ping-github-api.sh $TEST_GITHUB_TOKEN
  handlers:
  - slack
  interval: 10
  publish: true
  subscriptions:
  - system

{
  "type": "CheckConfig",
  "api_version": "core/v2",
  "metadata": {
    "name": "ping-github-api"
  },
  "spec": {
    "command": "ping-github-api.sh $TEST_GITHUB_TOKEN",
    "handlers": [
      "slack"
    ],
    "interval": 10,
    "publish": true,
    "subscriptions": [
      "system"
    ]
  }
}

Use environment variables to specify an HTTP proxy for agent use

If an HTTP proxy is required to access the internet in your compute environment, you may need to configure the Sensu agent to successfully download dynamic runtime assets or execute commands that depend on internet access.

For Sensu agents that require a proxy server, define HTTP_PROXY and HTTPS_PROXY environment variables in your sensu-agent file.

HTTP_PROXY="http://YOUR_PROXY_SERVER:PORT"
HTTPS_PROXY="http://YOUR_PROXY_SERVER:PORT"

You can use the same proxy server URL for HTTP_PROXY and HTTPS_PROXY. The proxy server URL you specify for HTTPS_PROXY does not need to use https://.

After you add the HTTP_PROXY and HTTPS_PROXY environment variables and restart sensu-agent, they will be available to check and hook commands executed by the Sensu agent. You can then use HTTP_PROXY and HTTPS_PROXY to add dynamic runtime assets, run checks, and complete other tasks that typically require an internet connection for your unconnected entities.

Create configuration overrides

Sensu has default settings and limits for certain configuration attributes, like the default log level. Depending on your environment and preferences, you may want to create overrides for these Sensu-specific defaults and limits.

You can create configuration overrides in several ways:

• Command line configuration flag arguments for sensu-agent start.
• Environment variables in /etc/default/sensu-agent (Debian family) or /etc/sysconfig/sensu-agent (RHEL family).
• Configuration settings in the agent.yml config file.

Sensu applies the following precedence to override settings:

1. Arguments passed to the Sensu agent via command line configuration flags.
2. Environment variables in /etc/default/sensu-agent (Debian family) or /etc/sysconfig/sensu-agent (RHEL family).
3. Configuration in the agent.yml config file.

For example, if you create overrides using all three methods, the command line configuration flag values will take precedence over the values you specify in /etc/default/sensu-agent or /etc/sysconfig/sensu-agent or the agent.yml config file.

Example override: Log level

The default log level for the Sensu agent is warn.

To override the default and automatically apply a different log level for the agent, add the --log-level command line configuration flag when you start the Sensu agent. For example, to specify debug as the log level:

sensu-agent start --log-level debug

To configure an environment variable for the desired agent log level:

echo 'SENSU_LOG_LEVEL=debug' | sudo tee -a /etc/default/sensu-agent

echo 'SENSU_LOG_LEVEL=debug' | sudo tee -a /etc/sysconfig/sensu-agent

To configure the desired log level in the config file, add this line to agent.yml:

log-level: debug

Service management

Start the service

Use the sensu-agent tool to start the agent and apply configuration flags.

sensu-agent start --subscriptions disk-checks --log-level debug

sensu-agent start --help

sudo systemctl start sensu-agent

sensu-agent service install

sensu-agent service install --config-file 'C:\\ProgramData\\sensu\\config\\agent.yml' --log-file 'C:\\ProgramData\\sensu\\log\\sensu-agent.log'

Stop the service

Stop the agent service using a service manager:

sudo systemctl stop sensu-agent

sc.exe stop SensuAgent

Restart the service

You must restart the agent to implement any configuration updates.

sudo systemctl restart sensu-agent

sc.exe start SensuAgent

Enable on boot

Enable the agent to start on system boot:

sudo systemctl enable sensu-agent

The service is configured to start automatically on boot by default.

Disable on boot

Disable the agent from starting on system boot:

sudo systemctl disable sensu-agent

The service is configured to start automatically on boot by default.

Get service status

View the status of the agent service using a service manager:

sudo systemctl status sensu-agent

sc.exe query SensuAgent

Get service version

Get the version of the current sensu-agent tool:

sensu-agent version

Get the version of the running sensu-agent service:

curl http://127.0.0.1:3031/version

Uninstall the service

Uninstall the sensu-agent service:

sudo systemctl stop sensu-agent

sensu-agent service uninstall

Get help

The sensu-agent tool provides general and command-specific help flags.

View sensu-agent commands:

sensu-agent help

List options for a specific command (in this case, sensu-agent start):

sensu-agent start --help