Providers Provisioning

Provider provisioning in Keep allows you to set up and manage data providers dynamically. This feature enables you to configure various data sources that Keep can interact with, such as monitoring systems, databases, or other services.

​

Configuring Providers

To provision providers and deduplication rules for them, we can configure via the environment variable. This can be done in two ways:

1. Using KEEP_PROVIDERS environment variable which either contains a JSON string or a path to a JSON file that contains the providers configurations.
2. Using KEEP_PROVIDERS_DIRECTORY environment variable which contains a path to a directory that contains the providers configurations (configured via YAML files). This is the recommended approach.

​

Providers provisioning using KEEP_PROVIDERS

Providers provisioning JSON example:

{
  "keepVictoriaMetrics": {
    "type": "victoriametrics",
    "authentication": {
      "VMAlertHost": "http://localhost",
      "VMAlertPort": 1234
    },
    "deduplication_rules": {
      "deduplication rule name example 1": {
        "description": "deduplication rule name example 1",
        "fingerprint_fields": ["fingerprint", "source", "service"],
        "full_deduplication": true,
        "ignore_fields": ["name", "lastReceived"]
      },
      "deduplication rule name example 2": {
        "description": "deduplication rule name example 2",
        "fingerprint_fields": ["fingerprint", "source", "service"],
        "full_deduplication": false,
      }
    }
  },
  "keepClickhouse1": {
    "type": "clickhouse",
    "authentication": {
      "host": "http://localhost",
      "port": 1234,
      "username": "keep",
      "password": "keep",
      "database": "keep-db"
    }
  }
}

Spin up Keep with this KEEP_PROVIDERS value:

# ENV
KEEP_PROVIDERS={"keepVictoriaMetrics":{"type":"victoriametrics","authentication":{"VMAlertHost":"http://localhost","VMAlertPort": 1234}},"keepClickhouse1":{"type":"clickhouse","authentication":{"host":"http://localhost","port":"4321","username":"keep","password":"1234","database":"keepdb"}}}

​

Providers provisioning using KEEP_PROVIDERS_DIRECTORY

Specify the path to the directory containing the providers configurations:

# ENV
KEEP_PROVIDERS_DIRECTORY=/path/to/providers

The directory should contain YAML files with the providers configurations.

Example of a provider configuration YAML file:

name: keepVictoriaMetrics
type: victoriametrics
authentication:
  VMAlertHost: http://localhost
  VMAlertPort: 1234
deduplication_rules:
  deduplication_rule_name_example_1:
    description: deduplication rule name example 1
    fingerprint_fields:
      - fingerprint
      - source
      - service
    full_deduplication: true
    ignore_fields:
      - name
      - lastReceived

​

Supported Providers

Keep supports a wide range of provider types. Each provider type has its own specific configuration requirements. To see the full list of supported providers and their detailed configuration options, please refer to our comprehensive provider documentation.

​

Update Provisioned Providers

​

Using KEEP_PROVIDERS

Provider configurations can be updated dynamically by changing the KEEP_PROVIDERS environment variable.

On every restart, Keep reads this environment variable and determines which providers need to be added or removed.

This process allows for flexible management of data sources without requiring manual intervention. By simply updating the KEEP_PROVIDERS variable and restarting the application, you can efficiently add new providers, remove existing ones, or modify their configurations.

The high-level provisioning mechanism:

1. Keep reads the KEEP_PROVIDERS value.
2. Keep checks if there are any provisioned providers that are no longer in the KEEP_PROVIDERS value, and deletes them.
3. Keep installs all providers from the KEEP_PROVIDERS value.

​

Using KEEP_PROVIDERS_DIRECTORY

Provider configurations can be updated dynamically by changing the YAML files in the KEEP_PROVIDERS_DIRECTORY directory.

On every restart, Keep reads the YAML files in the KEEP_PROVIDERS_DIRECTORY directory and determines which providers need to be added or removed.

The high-level provisioning mechanism:

1. Keep reads the YAML files in the KEEP_PROVIDERS_DIRECTORY directory.
2. Keep checks if there are any provisioned providers that are no longer in the YAML files, and deletes them.
3. Keep installs all providers from the YAML files.