TCP stream handlers reference
Sensu executes TCP stream handlers during the process stage of the observability pipeline.
Like traditional TCP handlers, TCP stream handlers send observability event data to TCP sockets for external services to consume.
However, TCP stream handlers can help prevent the data bottlenecks you may experience with traditional TCP handlers.
Traditional TCP handlers start a new UNIX process for every Sensu event they receive and require a new connection to send every event.
As you scale up and process more events per second, the rate at which the TCP handler can transmit observability event data decreases.
TCP stream handlers allow you to configure a connection pool with a maximum number of connections for the handler to use.
For example, suppose you configure a TCP stream handler with a pool of 10 connections, and 1000 events are queued for transmission.
As each connection finishes transmitting an event, it becomes available again and returns to the pool so the handler can use it to send the next event in the queue.
TCP stream handlers will reuse the available connections as long as they can rather than requiring a new connection for every event, which increases event throughput.
In addition to providing a persistent TCP connection to transmit Sensu observation events to a remote data storage service, TCP stream handlers allow you to use transport layer security (TLS) for secure data transmission.
TCP stream handlers are commercial resources available for use in pipeline definitions.
TCP stream handler example
This example shows a TCP stream handler resource definition configured to use TLS:
---
type: TCPStreamHandler
api_version: pipeline/v1
metadata:
name: logstash
spec:
address: 127.0.0.1:4242
tls_ca_cert_file: "/path/to/tls/ca.pem"
tls_cert_file: "/path/to/tls/cert.pem"
tls_key_file: "/path/to/tls/key.pem"
max_connections: 10
min_reconnect_delay: 10ms
max_reconnect_delay: 10s
{
"type": "TCPStreamHandler",
"api_version": "pipeline/v1",
"metadata": {
"name": "logstash"
},
"spec": {
"address": "127.0.0.1:4242",
"tls_ca_cert_file": "/path/to/tls/ca.pem",
"tls_cert_file": "/path/to/tls/cert.pem",
"tls_key_file": "/path/to/tls/key.pem",
"max_connections": 10,
"min_reconnect_delay": "10ms",
"max_reconnect_delay": "10s"
}
}
Use TCP stream handlers
TCP stream handlers are commercial resources and are available for use only in pipelines.
NOTE: TCP stream handlers are not used by listing the handler name in the check handlers attribute.
To use a TCP stream handler, list it as the handler in a pipeline definition.
For example, this pipeline definition uses the logstash example along with the built-in is_incident event filter:
---
type: Pipeline
api_version: core/v2
metadata:
name: tcp_logging_workflows
spec:
workflows:
- name: log_all_incidents
filters:
- name: is_incident
type: EventFilter
api_version: core/v2
handler:
name: logstash
type: TCPStreamHandler
api_version: pipeline/v1
{
"type": "Pipeline",
"api_version": "core/v2",
"metadata": {
"name": "tcp_logging_workflows"
},
"spec": {
"workflows": [
{
"name": "log_all_incidents",
"filters": [
{
"name": "is_incident",
"type": "EventFilter",
"api_version": "core/v2"
}
],
"handler": {
"name": "logstash",
"type": "TCPStreamHandler",
"api_version": "pipeline/v1"
}
}
]
}
}
TCP stream handler specification
Top-level attributes
type |
|
description |
Top-level attribute that specifies the sensuctl create resource type. TCP stream handlers should always be type TCPStreamHandler . |
required |
Required for TCP stream handler definitions in wrapped-json or yaml format for use with sensuctl create . |
type |
String |
example |
{
"type": "TCPStreamHandler"
}
|
api_version |
|
description |
Top-level attribute that specifies the Sensu API group and version. For TCP stream handlers in this version of Sensu, the api_version should always be pipeline/v1 . |
required |
Required for TCP stream handler definitions in wrapped-json or yaml format for use with sensuctl create . |
type |
String |
example |
{
"api_version": "pipeline/v1"
}
|
metadata |
|
description |
Top-level collection of metadata about the TCP stream handler that includes name , namespace , and created_by as well as custom labels and annotations . The metadata map is always at the top level of the handler definition. This means that in wrapped-json and yaml formats, the metadata scope occurs outside the spec scope. Read metadata attributes for details. |
required |
Required for TCP stream handler definitions in wrapped-json or yaml format for use with sensuctl create . |
type |
Map of key-value pairs |
example |
metadata:
name: logstash
namespace: default
created_by: admin
labels:
environment: development
region: us-west-2
annotations:
managed-by: ops
{
"metadata": {
"name": "logstash",
"namespace": "default",
"created_by": "admin",
"labels": {
"environment": "development",
"region": "us-west-2"
},
"annotations": {
"managed-by": "ops"
}
}
}
|
spec |
|
description |
Top-level map that includes the TCP stream handler spec attributes. |
required |
Required for TCP stream handler definitions in wrapped-json or yaml format for use with sensuctl create . |
type |
Map of key-value pairs |
example |
spec:
address: 127.0.0.1:4242
tls_ca_cert_file: "/path/to/tls/ca.pem"
tls_cert_file: "/path/to/tls/cert.pem"
tls_key_file: "/path/to/tls/key.pem"
max_connections: 10
min_reconnect_delay: 10ms
max_reconnect_delay: 10s
{
"spec": {
"address": "127.0.0.1:4242",
"tls_ca_cert_file": "/path/to/tls/ca.pem",
"tls_cert_file": "/path/to/tls/cert.pem",
"tls_key_file": "/path/to/tls/key.pem",
"max_connections": 10,
"min_reconnect_delay": "10ms",
"max_reconnect_delay": "10s"
}
}
|
name |
|
description |
Unique string used to identify the TCP stream handler. TCP stream handler names cannot contain special characters or spaces (validated with Go regex \A[\w\.\-]+\z ). Each TCP stream handler must have a unique name within its namespace. |
required |
true |
type |
String |
example |
|
namespace |
|
description |
Sensu RBAC namespace that the TCP stream handler belongs to. |
required |
false |
type |
String |
default |
default |
example |
{
"namespace": "default"
}
|
created_by |
|
description |
Username of the Sensu user who created the TCP stream handler or last updated the TCP stream handler. Sensu automatically populates the created_by field when the TCP stream handler is created or updated. |
required |
false |
type |
String |
example |
{
"created_by": "admin"
}
|
labels |
|
description |
Custom attributes to include with observation event data that you can use for response and web UI view filtering.
If you include labels in your event data, you can filter API responses, sensuctl responses, and web UI views based on them. In other words, labels allow you to create meaningful groupings for your data.
Limit labels to metadata you need to use for response filtering. For complex, non-identifying metadata that you will not need to use in response filtering, use annotations rather than labels. |
required |
false |
type |
Map of key-value pairs. Keys can contain only letters, numbers, and underscores and must start with a letter. Values can be any valid UTF-8 string. |
default |
null |
example |
labels:
environment: development
region: us-west-2
{
"labels": {
"environment": "development",
"region": "us-west-2"
}
}
|
annotations |
|
description |
Non-identifying metadata to include with observation event data that you can access with event filters. You can use annotations to add data that’s meaningful to people or external tools that interact with Sensu.
In contrast to labels, you cannot use annotations in API response filtering, sensuctl response filtering, or web UI views. |
required |
false |
type |
Map of key-value pairs. Keys and values can be any valid UTF-8 string. |
default |
null |
example |
annotations:
managed-by: ops
{
"annotations": {
"managed-by": "ops"
}
}
|
Spec attributes
address |
|
description |
The hostname:port combination the TCP stream handler should connect to. |
required |
true |
type |
String |
example |
{
"address": "127.0.0.1:4242"
}
|
tls_ca_cert_file |
|
description |
Path to the PEM-format CA certificate to use for TLS client authentication. |
required |
false |
type |
String |
example |
tls_ca_cert_file: "/path/to/tls/ca.pem"
{
"tls_ca_cert_file": "/path/to/tls/ca.pem"
}
|
tls_cert_file |
|
description |
Path to the PEM-format certificate to use for TLS client authentication. This certificate and its corresponding key are required for secure client communication. |
required |
false |
type |
String |
example |
tls_cert_file: "/path/to/tls/cert.pem"
{
"tls_cert_file": "/path/to/tls/cert.pem"
}
|
tls_key_file |
|
description |
Path to the PEM-format key file associated with the tls_cert_file to use for TLS client authentication. This key and its corresponding certificate are required for secure client communication. |
required |
false |
type |
String |
example |
tls_key_file: "/path/to/tls/key.pem"
{
"tls_key_file": "/path/to/tls/key.pem"
}
|
max_connections |
|
description |
Maximum number of connections to keep alive in the connection pool. If set to 0 , connection pooling is disabled. |
required |
true |
type |
Integer |
example |
{
"max_connections": 10
}
|
max_reconnect_delay |
|
description |
Maximum time to wait while retrying a broken connection. In seconds (s ) or milliseconds (ms ). |
required |
true |
type |
String |
example |
{
"max_reconnect_delay": "10s"
}
|
min_reconnect_delay |
|
description |
Minimum time to wait while retrying a broken connection. In seconds (s ) or milliseconds (ms ). |
required |
true |
type |
String |
example |
min_reconnect_delay: 10ms
{
"min_reconnect_delay": "10ms"
}
|