Mutators reference

Sensu executes mutators during the transform stage of the observability pipeline.

Pipelines can specify a mutator to execute and transform observability event data before any handlers are applied. When the Sensu backend processes an event, it checks the pipeline for the presence of a mutator and executes that mutator before executing the handler.

Mutators accept input/data via stdin and can parse JSON event data. They output JSON data (modified event data) to stdout or stderr.

There are two types of mutators: pipe and JavaScript.

Pipe mutator examples

This example shows a pipe mutator resource definition with the minimum required attributes:

---
type: Mutator
api_version: core/v2
metadata:
  name: mutator_minimum
spec:
  command: example_mutator.go
  type: pipe

{
  "type": "Mutator",
  "api_version": "core/v2",
  "metadata": {
    "name": "mutator_minimum"
  },
  "spec": {
    "command": "example_mutator.go",
    "type": "pipe"
  }
}

The following mutator definition uses an imaginary Sensu plugin, example_mutator.go, to modify event data prior to handling the event:

---
type: Mutator
api_version: core/v2
metadata:
  name: example-mutator
spec:
  command: example_mutator.go
  eval: ""
  env_vars: []
  runtime_assets:
  - example-mutator-asset
  secrets: null
  timeout: 0
  type: pipe

{
  "type": "Mutator",
  "api_version": "core/v2",
  "metadata": {
    "name": "example-mutator"
  },  
  "spec": {
    "command": "example_mutator.go",
    "timeout": 0,
    "env_vars": [],
    "runtime_assets": [
      "example-mutator-asset"
    ],
    "secrets": null,
    "type": "pipe",
    "eval": ""
  }
}

JavaScript mutator example

JavaScript mutators use the eval attribute instead of the command attribute. The eval value must be an ECMAScript 5 (JavaScript) expression.

This example uses a JavaScript mutator to remove event attributes that are not required — in this case, the check name and entity app_id label:

---
type: Mutator
api_version: core/v2
metadata:
  name: remove_checkname_entitylabel
spec:
  eval: >-
    data = JSON.parse(JSON.stringify(event)); delete data.check.metadata.name;
    delete data.entity.metadata.labels.app_id; return JSON.stringify(data)    
  type: javascript

{
  "type": "Mutator",
  "api_version": "core/v2",
  "metadata": {
    "name": "remove_checkname_entitylabel"
  },
  "spec": {
    "eval": "data = JSON.parse(JSON.stringify(event)); delete data.check.metadata.name; delete data.entity.metadata.labels.app_id; return JSON.stringify(data)",
    "type": "javascript"
  }
}

You can also use JavaScript mutators to do things like add new attributes and combine existing attributes into a single new attribute.

Pipe mutators

Pipe mutators produce an exit status code to indicate state. A code of 0 indicates OK status. If the mutator executes successfully (returns an exit status code of 0), the modified event data return to the pipeline and the handler is executed.

Exit codes other than 0 indicate failure. If the mutator fails to execute (returns a non-zero exit status code or fails to complete within its configured timeout), an error is logged and the handler will not execute.

Pipe mutator commands

Each Sensu mutator definition defines a command to be executed. Mutator commands are executable commands that will be executed on a Sensu backend, run as the sensu user. Most mutator commands are provided by Sensu plugins.

Sensu mutator command attributes may include command line arguments for controlling the behavior of the command executable. Many Sensu mutator plugins provide support for command line arguments for reusability.

All mutator commands are executed by a Sensu backend as the sensu user. Commands must be executable files that are discoverable on the Sensu backend system (installed in a system $PATH directory).

JavaScript mutators

Mutators that use JavaScript are an efficient alternative to pipe mutators, which fork a process on each invocation. JavaScript mutators are evaluated by the Otto JavaScript VM as JavaScript programs, which enables greater mutator throughput at scale.

JavaScript mutators do not require you to return any value — you can mutate the events that are passed to the mutator instead. However, if you do return a value with a JavaScript mutator, it must be a string. If a JavaScript mutator returns a non-string value (an array, object, integer, or Boolean), an error is recorded in the Sensu backend log.

JavaScript mutators can use dynamic runtime assets as long as they are valid JavaScript assets.

Secrets are not available to JavaScript mutators. JavaScript mutators cannot look up events from the event store.

JavaScript mutator eval attribute

Each Sensu JavaScript mutator definition includes the eval attribute, whose value must be an ECMAScript 5 (JavaScript) expression. JavaScript mutators do not use the command attribute.

All mutator eval expressions are executed by a Sensu backend as the sensu user.

JavaScript mutator eval expressions can use the environment variables listed in the env_vars attribute. For JavaScript mutators, you can define environment variables and list the names of any environment variables that are available in your environment in the env_vars attribute.

Built-in mutators

Sensu includes built-in mutators to help you customize event pipelines for metrics and alerts.

Built-in mutator: only_check_output

To process an event, some handlers require only the check output, not the entire event definition. For example, when sending metrics to Graphite using a TCP handler, Graphite expects data that follows the Graphite plaintext protocol. By using the built-in only_check_output mutator, Sensu reduces the event to only the check output so Graphite can accept it.

To use only check output, include the only_check_output mutator in the pipeline mutator array:

---
type: Pipeline
api_version: core/v2
metadata:
  name: graphite_pipeline
spec:
  workflows:
  - name: graphite_check_output
    filters:
    - name: has_metrics
      type: EventFilter
      api_version: core/v2
    mutator:
      name: only_check_output
      type: Mutator
      api_version: core/v2
    handler:
      name: graphite
      type: Handler
      api_version: core/v2

{
  "type": "Pipeline",
  "api_version": "core/v2",
  "metadata": {
    "name": "graphite_pipeline"
  },
  "spec": {
    "workflows": [
      {
        "name": "graphite_check_output",
        "filters": [
          {
            "name": "has_metrics",
            "type": "EventFilter",
            "api_version": "core/v2"
          }
        ],
        "mutator": {
          "name": "only_check_output",
          "type": "Mutator",
          "api_version": "core/v2"
        },
        "handler": {
          "name": "graphite",
          "type": "Handler",
          "api_version": "core/v2"
        }
      }
    ]
  }
}

Mutator specification

Top-level attributes

api_version: core/v2

{
  "api_version": "core/v2"
}

metadata:
  name: example-mutator
  namespace: default
  created_by: admin
  labels:
    region: us-west-1
  annotations:
    slack-channel: "#monitoring"

{
  "metadata": {
    "name": "example-mutator",
    "namespace": "default",
    "created_by": "admin",
    "labels": {
      "region": "us-west-1"
    },
    "annotations": {
      "slack-channel": "#monitoring"
    }
  }
}

spec:
  command: example_mutator.go
  timeout: 0
  env_vars: []
  runtime_assets: []
  secrets: null
  type: pipe

{
  "spec": {
    "command": "example_mutator.go",
    "timeout": 0,
    "env_vars": [],
    "runtime_assets": [],
    "secrets": null,
    "type": "pipe"
  }
}

type: Mutator

{
  "type": "Mutator"
}

Metadata attributes

annotations:
  managed-by: ops
  playbook: www.example.url

{
  "annotations": {
    "managed-by": "ops",
    "playbook": "www.example.url"
  }
}

created_by: admin

{
  "created_by": "admin"
}

labels:
  environment: development
  region: us-west-2

{
  "labels": {
    "environment": "development",
    "region": "us-west-2"
  }
}

name: example-mutator

{
  "name": "example-mutator"
}

namespace: production

{
  "namespace": "production"
}

Spec attributes

command: /etc/sensu/plugins/mutated.go

{
  "command": "/etc/sensu/plugins/mutated.go"
}

env_vars:
- APP_VERSION=2.5.0

{
  "env_vars": [
    "APP_VERSION=2.5.0"
  ]
}

env_vars:
- APP_VERSION=2.5.0
- SHELL

{
  "env_vars": [
    "APP_VERSION=2.5.0",
    "SHELL"
  ]
}

eval: 'return JSON.stringify({"some stuff": "is here"});'

{
  "eval": "return JSON.stringify({\"some info\": \"is here\"});"
}

runtime_assets:
- metric-mutator

{
  "runtime_assets": [
    "metric-mutator"
  ]
}

secrets:
- name: ANSIBLE_HOST
  secret: sensu-ansible-host
- name: ANSIBLE_TOKEN
  secret: sensu-ansible-token

{
  "secrets": [
    {
      "name": "ANSIBLE_HOST",
      "secret": "sensu-ansible-host"
    },
    {
      "name": "ANSIBLE_TOKEN",
      "secret": "sensu-ansible-token"
    }
  ]
}

timeout: 30

{
  "timeout": 30
}

type: pipe

{
  "type": "pipe"
}

secrets attributes

name: ANSIBLE_HOST

{
  "name": "ANSIBLE_HOST"
}

secret: sensu-ansible-host

{
  "secret": "sensu-ansible-host"
}

Use secrets management in a mutator

Learn more about secrets management for your Sensu configuration in the secrets and secrets providers references.

---
type: Mutator 
api_version: core/v2 
metadata:
  name: ansible-tower
  namespace: ops
spec: 
  command: sensu-ansible-mutator -h $ANSIBLE_HOST -t $ANSIBLE_TOKEN
  secrets:
  - name: ANSIBLE_HOST
    secret: sensu-ansible-host
  - name: ANSIBLE_TOKEN
    secret: sensu-ansible-token

{
  "type": "Mutator",
  "api_version": "core/v2",
  "metadata": {
    "name": "ansible-tower",
    "namespace": "ops"
  },
  "spec": {
    "command": "sensu-ansible-mutator -h $ANSIBLE_HOST -t $ANSIBLE_TOKEN",
    "secrets": [
      {
        "name": "ANSIBLE_HOST",
        "secret": "sensu-ansible-host"
      },
      {
        "name": "ANSIBLE_TOKEN",
        "secret": "sensu-ansible-token"
      }
    ]
  }
}

Add new event attributes with JavaScript mutators

Use a JavaScript mutator to rewrite events with a new attribute added.

This example adds a new “organization” attribute to events at the top level, with a value of sec_ops:

---
type: Mutator
api_version: core/v2
metadata:
  name: add_org_sec_ops
spec:
  eval: >-
    data = JSON.parse(JSON.stringify(event)); data['organization'] = 'sec_ops';
    return JSON.stringify(data)    
  type: javascript

{
  "type": "Mutator",
  "api_version": "core/v2",
  "metadata": {
    "created_by": "admin",
    "name": "add_org_sec_ops"
  },
  "spec": {
    "eval": "data = JSON.parse(JSON.stringify(event)); data['organization'] = 'sec_ops'; return JSON.stringify(data)",
    "type": "javascript"
  }
}

Combine existing attributes with JavaScript mutators

Use a JavaScript mutator to create a new attribute from a combination of multiple existing attributes and add the new attribute to events.

This example combines the event namespace and the name of the check that generated the event into a single new attribute, origination:

---
type: Mutator
api_version: core/v2
metadata:
  name: add_origination_attribute
spec:
  eval: >-
    data = JSON.parse(JSON.stringify(event)); data.origination =
    data.metadata.namespace + data.check.metadata.name; return
    JSON.stringify(data)    
  type: javascript

{
  "type": "Mutator",
  "api_version": "core/v2",
  "metadata": {
    "name": "add_origination_attribute"
  },
  "spec": {
    "eval": "data = JSON.parse(JSON.stringify(event)); data.origination = data.metadata.namespace + data.check.metadata.name; return JSON.stringify(data)",
    "type": "javascript"
  }
}