Silencing reference

Sensu’s silencing capability allows you to suppress event handler execution on an ad hoc basis so you can plan maintenance and reduce alert fatigue. Silences are created on an ad hoc basis using sensuctl, the web UI, and the core/v2/silenced API endpoints.

Successfully created silencing entries are assigned a name in the format $SUBSCRIPTION:$CHECK, where $SUBSCRIPTION is the name of a Sensu entity subscription and $CHECK is the name of a Sensu check. You can use silences to silence checks on specific entities by taking advantage of per-entity subscriptions (for example, entity:$ENTITY_NAME).

When creating a silencing entry, you can specify a combination of checks and subscriptions, but only one or the other is strictly required. For example, if you create a silencing entry specifying only a check, its name will contain an asterisk (or wildcard) in the $SUBSCRIPTION position. This indicates that any event with a matching check name will be marked as silenced, regardless of the originating entities’ subscriptions.

Conversely, a silencing entry that specifies only a subscription will have a name with an asterisk in the $CHECK position. This indicates that any event where the originating entities’ subscriptions match the subscription specified in the entry will be marked as silenced, regardless of the check name.

These silences are persisted in the Sensu datastore. When the Sensu server processes subsequent check results, it retrieves matching silences from the store. If there are one or more matching entries, the event is updated with a list of silenced entry names. When the check name or subscription described in a silencing entry matches an event, the event will include the silenced attribute, which lists the silencing entries that match the event.

Silenced checks still create events, and events from silenced checks are still passed to handlers. To prevent handler execution for events from silenced checks, make sure the handler definition includes the built-in not_silenced event filter. The not_silenced event filter prevents handlers from processing events that include the silenced attribute.

Silencing examples

This example shows a silencing resource definition that uses a per-entity subscription to silence any alerts on a single Sensu entity, i-424242:

---
type: Silenced
api_version: core/v2
metadata:
  name: entity:i-424242:*
spec:
  begin: 1542671205
  check: null
  creator: admin
  expire: -1
  expire_at: 0
  expire_on_resolve: false
  reason: null
  subscription: entity:i-424242
{
  "type": "Silenced",
  "api_version": "core/v2",
  "metadata": {
    "name": "entity:i-424242:*"
  },
  "spec": {
    "expire": -1,
    "expire_at": 0,
    "expire_on_resolve": false,
    "creator": "admin",
    "reason": null,
    "check": null,
    "subscription": "entity:i-424242",
    "begin": 1542671205
  }
}

Silence a specific check on a specific entity

The following example shows how to silence a check named check_ntp on entity i-424242, ensuring the silencing entry is deleted after the underlying issue is resolved:

---
type: Silenced
api_version: core/v2
metadata:
  name: entity:i-424242:check_ntp
spec:
  subscription: entity:i-424242
  check: check_ntp
  expire_on_resolve: true
{
  "type": "Silenced",
  "api_version": "core/v2",
  "metadata": {
    "name": "entity:i-424242:check_ntp"
  },
  "spec": {
    "subscription": "entity:i-424242",
    "check": "check_ntp",
    "expire_on_resolve": true
  }
}

The optional expire_on_resolve attribute used in this example indicates that when the server processes a matching check from the specified entity with status OK, the silencing entry will be removed automatically.

When used in combination with other attributes (like creator and reason), this gives Sensu operators a way to acknowledge that they received an alert, suppress additional notifications, and automatically clear the silencing entry when the check status returns to normal.

Silencing specification

Silenced entry names

Silences must contain either a subscription or check name and are identified by the combination of $SUBSCRIPTION:$CHECK. If a check or subscription is not provided, it will be substituted with a wildcard (asterisk): $SUBSCRIPTION:* or *:$CHECK.

Top-level attributes

type
description Top-level attribute that specifies the sensuctl create resource type. Silences should always be type Silenced.
required Required for silencing entry definitions in wrapped-json or yaml format for use with sensuctl create.
type String
example 
type: Silenced
{
  "type": "Silenced"
}
api_version
description Top-level attribute that specifies the Sensu API group and version. For silences in this version of Sensu, the api_version should always be core/v2.
required Required for silencing entry definitions in wrapped-json or yaml format for use with sensuctl create.
type String
example 
api_version: core/v2
{
  "api_version": "core/v2"
}
metadata
description Top-level collection of metadata about the silencing entry that includes name, namespace, and created_by as well as custom labels and annotations. The metadata map is always at the top level of the silencing entry definition. This means that in wrapped-json and yaml formats, the metadata scope occurs outside the spec scope. Read metadata attributes for details.
required Required for silencing entry definitions in wrapped-json or yaml format for use with sensuctl create.
type Map of key-value pairs
example 
metadata:
  name: appserver:mysql_status
  namespace: default
  created_by: admin
  labels:
    region: us-west-1
{
  "metadata": {
    "name": "appserver:mysql_status",
    "namespace": "default",
    "created_by": "admin",
    "labels": {
      "region": "us-west-1"
    }
  }
}
spec
description Top-level map that includes the silencing entry spec attributes.
required Required for silences in wrapped-json or yaml format for use with sensuctl create.
type Map of key-value pairs
example 
spec:
  expire: -1
  expire_at: 0
  expire_on_resolve: false
  creator: admin
  reason:
  check:
  subscription: entity:i-424242
  begin: 1542671205
{
  "spec": {
    "expire": -1,
    "expire_at": 0,
    "expire_on_resolve": false,
    "creator": "admin",
    "reason": null,
    "check": null,
    "subscription": "entity:i-424242",
    "begin": 1542671205
  }
}

Metadata attributes

name
description Silencing identifier generated from the combination of a subscription name and check name.
required false - This value cannot be modified.
type String
example 
name: appserver:mysql_status
{
  "name": "appserver:mysql_status"
}
namespace
description Sensu RBAC namespace that the silencing entry belongs to.
required false
type String
default default
example 
namespace: production
{
  "namespace": "production"
}
created_by
description Username of the Sensu user who created the silence or last updated the silence. Sensu automatically populates the created_by field when the silence is created or updated.
required false
type String
example 
created_by: admin
{
  "created_by": "admin"
}
labels
description Custom attributes to include with observation event data that you can use for response and web UI view filtering.

If you include labels in your event data, you can filter API responses, sensuctl responses, and web UI views based on them. In other words, labels allow you to create meaningful groupings for your data.

Limit labels to metadata you need to use for response filtering. For complex, non-identifying metadata that you will not need to use in response filtering, use annotations rather than labels.
required false
type Map of key-value pairs. Keys can contain only letters, numbers, and underscores and must start with a letter. Values can be any valid UTF-8 string.
default null
example 
labels:
  environment: development
  region: us-west-2
{
  "labels": {
    "environment": "development",
    "region": "us-west-2"
  }
}
annotations
description Non-identifying metadata to include with observation event data that you can access with event filters. You can use annotations to add data that’s meaningful to people or external tools that interact with Sensu.

In contrast to labels, you cannot use annotations in API response filtering, sensuctl response filtering, or web UI views.
required false
type Map of key-value pairs. Keys and values can be any valid UTF-8 string.
default null
example 
annotations:
  managed-by: ops
  playbook: www.example.url
{
  "annotations": {
    "managed-by": "ops",
    "playbook": "www.example.url"
  }
}

Spec attributes

check
description Name of the check the entry should match.
required true, unless subscription is provided
type String
example 
check: haproxy_status
{
  "check": "haproxy_status"
}
subscription
description Name of the subscription the entry should match.
required true, unless check is provided
type String
example 
subscription: entity:i-424242
{
  "subscription": "entity:i-424242"
}
begin
description Time at which silence entry goes into effect. In epoch.
required false
type Integer
example 
begin: 1512512023
{
  "begin": 1512512023
}

expire
description Number of seconds until the entry should be deleted.

If the silence is set to expire when a check resolves, the expire value will be -1.

If the silence is set to expire at a specific time, the expire value will be 0.
required false
type Integer
default -1
example 
expire: 3600
{
  "expire": 3600
}

expire_at
description Time at which the entry should be deleted. In seconds since the Unix epoch.

Use expire_at in conjunction with expire_on_resolve to create silences that expire either when a check resolves or at a specific time, whichever comes first.
required false
type Integer
default 0
example 
expire_at: 1664550303
{
  "expire_at": 1664550303
}

expire_on_resolve
description true if the entry should be deleted when the specified check begins to return OK status (resolves). Otherwise, false.

Use expire_on_resolve in conjunction with expire_at to create silences that expire either when a check resolves or at a specific time, whichever comes first.
required false
type Boolean
default false
example 
expire_on_resolve: true
{
  "expire_on_resolve": true
}
creator
description Person, application, or entity responsible for creating the entry.
required false
type String
default null
example 
creator: Application Deploy Tool 5.0
{
  "creator": "Application Deploy Tool 5.0"
}
reason
description Explanation of the reason for creating the entry.
required false
type String
default null
example 
reason: rebooting the world
{
  "reason": "rebooting the world"
}

Silence all checks with a specific subscription

Use this example to create a silencing entry for all checks with the appserver subscription:

---
type: Silenced
api_version: core/v2
metadata:
  name: appserver
spec:
  subscription: appserver
{
  "type": "Silenced",
  "api_version": "core/v2",
  "metadata": {
    "name": "appserver"
  },
  "spec": {
    "subscription": "appserver"
  }
}

NOTE: This example will not silence entities with the appserver subscription. Checks that do not include the appserver subscription will still run on entities that include the appserver subscription.

To silence all checks for entities with a particular subscription, use the Sensu web UI.

Silence all checks for entities with a specific subscription

To silence all checks for entities with a particular subscription:

  1. Open the Entities page in the Sensu web UI.
  2. Use the search field to search the entities by subscription. For example, to search for entities with the system subscription, enter "system" in entity.subscriptions.
  3. Click the box to select all.
  4. Click SILENCE.
  5. In the New Silencing Entry dialog window, add any desired silence configuration options.
  6. Click CREATE.

The silencing entries will be listed on the Silences page in the Sensu web UI.

Silence entities by subscription in the Sensu web UI

Silence a specific check on entities with a specific subscription

To silence a check mysql_status that is running on Sensu entities with the subscription appserver:

---
type: Silenced
api_version: core/v2
metadata:
  name: appserver:mysql_status 
spec:
  subscription: appserver
  check: mysql_status
{
  "type": "Silenced",
  "api_version": "core/v2",
  "metadata": {
    "name": "appserver:mysql_status"
  },
  "spec": {
    "subscription": "appserver",
    "check": "mysql_status"
  }
}

Silence a specific check on every entity

To silence the check mysql_status on every entity in your infrastructure, regardless of subscriptions, you only need to provide the check name:

---
type: Silenced
api_version: core/v2
metadata:
  name: mysql_status
spec:
  check: mysql_status
{
  "type": "Silenced",
  "api_version": "core/v2",
  "metadata": {
    "name": "mysql_status"
  },
  "spec": {
    "check": "mysql_status"
  }
}

Delete a silence

To delete a silencing entry, you must provide its name.

Subscription-only silencing entry names will contain an asterisk (or wildcard) in the $SUBSCRIPTION position, similar to this example:

name: appserver:*
{
  "name": "appserver:*"
}

Check-only silencing entry names will contain an asterisk (or wildcard) in the $CHECK position, similar to this example:

name: '*:mysql_status'
{
  "name": "*:mysql_status"
}