2.6 KiB
Export metrics to AWS Kinesis Data Streams
Prerequisites
To use AWS Kinesis for metric collecting and processing, you should first
install AWS SDK for C++.
libcrypto, libssl, and libcurl are also required to compile Netdata with Kinesis support enabled. Next, Netdata
should be re-installed from the source. The installer will detect that the required libraries are now available.
If the AWS SDK for C++ is being installed from source, it is useful to set -DBUILD_ONLY=kinesis. Otherwise, the
build process could take a very long time. Note, that the default installation path for the libraries is
/usr/local/lib64. Many Linux distributions don't include this path as the default one for a library search, so it is
advisable to use the following options to cmake while building the AWS SDK:
sudo cmake -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_ONLY=kinesis <aws-sdk-cpp sources>
The -DCMAKE_INSTALL_PREFIX=/usr option also ensures that
third party dependencies are installed in your system
during the SDK build process.
Configuration
To enable data sending to the Kinesis service, run ./edit-config exporting.conf in the Netdata configuration directory
and set the following options:
[kinesis:my_instance]
enabled = yes
destination = us-east-1
Set the destination option to an AWS region.
Set AWS credentials and stream name:
# AWS credentials
aws_access_key_id = your_access_key_id
aws_secret_access_key = your_secret_access_key
# destination stream
stream name = your_stream_name
Alternatively, you can set AWS credentials for the netdata user using AWS SDK for C++ standard methods.
Netdata automatically computes a partition key for every record with the purpose to distribute records across available shards evenly.