# VictoriaMetrics Multi Alert Example Source: https://docs.keephq.dev/alertevaluation/examples/victoriametricsmulti This example demonstrates a simple CPU usage multi-alert based on a metric: ```yaml theme={null} workflow: # Unique identifier for this workflow id: query-victoriametrics-multi # Display name shown in the UI name: victoriametrics-multi-alert-example # Brief description of what this workflow does description: victoriametrics triggers: # This workflow can be triggered manually from the UI - type: manual steps: # Query VictoriaMetrics for CPU metrics - name: victoriametrics-step provider: # Use the VictoriaMetrics provider configuration config: "{{ providers.vm }}" type: victoriametrics with: # Query that returns the sum of CPU usage for each job # Example response: # [ # {'metric': {'job': 'victoriametrics'}, 'value': [1737808021, '0.022633333333333307']}, # {'metric': {'job': 'vmagent'}, 'value': [1737808021, '0.009299999999999998']} # ] query: sum(rate(process_cpu_seconds_total)) by (job) queryType: query actions: # Create an alert in Keep based on the query results - name: create-alert provider: type: keep with: # Only create alert if CPU usage is above threshold if: "{{ value.1 }} > 0.01 " # Alert must persist for 1 minute for: 1m # Use job label to create unique fingerprint for each alert fingerprint_fields: - labels.job alert: # Alert name includes the specific job name: "High CPU Usage on {{ metric.job }}" description: "CPU usage is high on the VM (created from VM metric)" # Set severity based on CPU usage thresholds: # > 0.9 = critical # > 0.7 = warning # else = info severity: '{{ value.1 }} > 0.9 ? "critical" : {{ value.1 }} > 0.7 ? "warning" : "info"' labels: # Job label is required for alert fingerprinting job: "{{ metric.job }}" # Additional context labels environment: production app: myapp service: api team: devops owner: alice ``` # VictoriaMetrics Single Alert Example Source: https://docs.keephq.dev/alertevaluation/examples/victoriametricssingle This example demonstrates a simple CPU usage alert based on a metric: ```yaml theme={null} # This workflow queries VictoriaMetrics metrics and creates alerts based on CPU usage workflow: # Unique identifier for this workflow id: query-victoriametrics # Display name shown in the UI name: victoriametrics-alert-example # Brief description of what this workflow does description: Monitors CPU usage metrics from VictoriaMetrics and creates alerts when thresholds are exceeded # Define how the workflow is triggered triggers: - type: manual # Can be triggered manually from the UI # Steps to execute in order steps: - name: victoriametrics-step provider: # Use VictoriaMetrics provider config defined in providers.vm config: "{{ providers.vm }}" type: victoriametrics with: # Query average CPU usage rate query: avg(rate(process_cpu_seconds_total)) queryType: query # Actions to take based on the query results actions: - name: create-alert provider: type: keep with: # Create alert if CPU usage exceeds threshold if: "{{ value.1 }} > 0.0040" alert: name: "High CPU Usage" description: "[Single] CPU usage is high on the VM (created from VM metric)" # Set severity based on CPU usage thresholds severity: '{{ value.1 }} > 0.9 ? "critical" : {{ value.1 }} > 0.7 ? "warning" : "info"' # Alert labels for filtering and routing labels: environment: production app: myapp service: api team: devops owner: alice ``` # Overview Source: https://docs.keephq.dev/alertevaluation/overview The Keep Alert Evaluation Engine is a flexible system that enables you to create alerts based on any data source and define evaluation rules. Unlike traditional monitoring solutions that are tied to specific metrics, Keep's engine allows you to combine data from multiple sources and apply complex logic to determine when and how alerts should be triggered. ## Core Features ### Generic Data Source Support * Query any data source (databases, APIs, metrics systems) * Combine multiple data sources in a single alert rule * Apply custom transformations to the data ### Flexible Alert Evaluation * Define custom conditions using templated expressions * Support for complex boolean logic and mathematical operations * State management for alert transitions (pending->firing->resolved) * Deduplication and alert instance tracking ### Customizable Alert Definition * Full control over alert metadata (name, description, severity) * Dynamic labels based on evaluation context * Template support for all alert fields * Custom fingerprinting for alert grouping ## Core Components ### Alert States * **Pending**: Initial state when alert condition is met (relevant only if `for` supplied) * **Firing**: Active alert that has met its duration condition * **Resolved**: Alert that is no longer active ### Alert Rule Components 1. **Data Collection**: Query steps to gather data from any source 2. **Condition (`if`)**: Expression that determines when to create/update an alert 3. **Duration (`for`)**: Optional time period the condition must be true before firing 4. **Alert Definition**: Complete control over how the alert looks and behaves: * Name and description * Severity levels * Labels for routing * Custom fields and annotations ### State Management * **Fingerprinting**: Unique identifier for alert deduplication and state tracking * **Keep-Firing**: Control how long alerts remain active * **State Transitions**: Rules for how alerts move between states ## Examples The following examples demonstrate different ways to use the alert evaluation engine: * [Single Metric Alert](/alertevaluation/examples/victoriametricssingle) - Basic example showing metrics-based alerting * [Multiple Metrics Alert](/alertevaluation/examples/victoriametricsmulti) - Advanced example with multiple alert instances # Action Menu Source: https://docs.keephq.dev/alerts/actionmenu The Action Menu in Keep provides quick access to common actions that can be performed on alerts. This menu enables teams to efficiently manage and interact with alerts directly from the table. ### (1) Run Workflow Trigger predefined workflows directly from the Action Menu. This allows automation of actions such as escalating alerts or notifying specific teams. ### (2) Create a New Workflow Quickly create a new workflow tailored to the selected alert. This is useful for handling unique cases that require a custom response. ### (3) View Alert History Access the full history of the alert, including changes to its status, comments, and any actions performed. This provides a clear timeline of the alert's lifecycle. ### (4) Manually Enrich Alert Add custom metadata or details to an alert manually. This can include additional context or information that assists with resolution. ### (5) Self Assign Assign the selected alert to yourself. This is ideal for team members who are taking ownership of specific alerts. ### (6) View Alert Open the alert details in the sidebar or dedicated alert view for a deeper dive into its metadata and context. ### (7) Source-Specific Actions Perform actions that are specific to the source of the alert. For example, linking directly to the monitoring tool or executing source-specific workflows. ### (8) Dismiss Alert Mark the alert as dismissed to indicate that no further action is required. This helps in managing and decluttering the alert table. ### (9) Change Status Update the status of the alert (e.g., from "firing" to "acknowledged"). This keeps the team informed about the current state of the alert. *** # Overview Source: https://docs.keephq.dev/alerts/overview **Alert Management** empowers teams to effectively manage, monitor, and act on critical alerts. With a robust and user-friendly interface, Keep allows users to gain deep insights into their alerts, filter through large volumes of data, and take swift actions to maintain system health. Everything related with Alert Management can be customized: 1. **Alert table** - view and manage the alerts. 2. **Search Bar** - use CEL to filter alerts which can be saved as "Customized Presets". 3. **Facets** - slice and dice alerts. 4. **Columns and Time** - customize columns and theme for your preset. # Customized Presets Source: https://docs.keephq.dev/alerts/presets You can think of a preset like a "Slack Channel" for your alerts - a logical container to follow only alerts that matter for you. With Keep's introduction of CEL (Common Expression Language) for alert filtering, users gain the flexibility to define more complex and precise alert filtering logic. This feature allows the creation of customizable filters using CEL expressions to refine alert visibility based on specific criteria. ## How It Works 1. **CEL Expression Creation**: Users craft CEL expressions that define the filtering criteria for alerts. 2. **Preset Definition**: These expressions can be saved as presets for easy application to different alert streams. 3. **Alert Filtering**: When applied, the CEL expressions evaluate each alert against the defined criteria, filtering the alert stream in real-time. ## Creating a CEL Expression There are two ways of creating a CEL expression in Keep ### Manually creating CEL query Use the [CEL Language Definition](https://github.com/google/cel-spec/blob/master/doc/langdef.md) documentation to better understand the capabilities of the Common Expression Language This is an example of how to query all the alerts that came from `Sentry` If the CEL syntax you typed in is invalid, an error message will show up (in this case, we used invalid `''` instead of `""`): ### Importing from an SQL query 1. Click on the "Import from SQL" button 2. Write/Paste your SQL query and hit the "Convert to CEL" button Which in turn will generate and apply a valid CEL query: ## Save Presets You can save your CEL queries into a `Preset` using the "Save current filter as a view" button You can name your `Preset` and configure whether it is "Private" (only the creating user will see this Preset) or account-wide available. The `Preset` will then be created and available for you to quickly navigate and used ## Practical Example For instance, a user could create a CEL expression to filter alerts by severity and source, such as `severity == 'critical' && service.contains('database')`, ensuring only critical alerts from database services are displayed. ## Best Practices * **Specificity in Expressions**: Craft expressions that precisely target the desired alerts to avoid filtering out relevant alerts. * **Presets Management**: Regularly review and update your presets to align with evolving alerting needs. * **Testing Expressions**: Before applying, test CEL expressions to ensure they correctly filter the desired alerts. ## Useful Links * [Common Expression Language](https://github.com/google/cel-spec?tab=readme-ov-file) * [CEL Language Definition](https://github.com/google/cel-spec/blob/master/doc/langdef.md) # Alert Sidebar Source: https://docs.keephq.dev/alerts/sidebar The Alert Sidebar in Keep provides a detailed view of a selected alert, offering in-depth context and information to aid in alert management and resolution. This feature is designed to give users a comprehensive understanding of the alert without leaving the main interface. ### (1) Alert Name Displays the name of the alert, which typically summarizes the issue or event being reported. This is the primary identifier for the alert. ### (2) Alert Related Service Shows the service associated with the alert. This helps teams quickly understand which part of the infrastructure or application is affected. ### (3) Alert Source Indicates the source of the alert, such as the monitoring tool or system that generated it (e.g., Prometheus, Datadog). This provides context on where the alert originated. ### (4) Alert Description A detailed description of the alert, including specifics about the issue. This section helps provide a deeper understanding of what triggered the alert. ### (5) Alert Fingerprint A unique identifier for the alert. The fingerprint is used to correlate alerts and track their lifecycle across systems. ### (6) Alert Timeline Displays a chronological history of the alert, including when it was created, acknowledged, updated, or resolved. The timeline provides insights into how the alert has been managed. ### (7) Alert Topology View Offers a visual representation of the alert's impact on the system's topology. This view helps identify affected components and their relationships to other parts of the infrastructure. *** # Sound Notifications Source: https://docs.keephq.dev/alerts/sound Sound notifications ensure you never miss important updates or alerts. ## How It Works 1. **Preset Notifications**: Mark a preset as "noisy," and any alert linked to it will play a sound. Alternatively, set individual alerts as `isNoisy=true` to trigger sounds through linked presets. 2. **Real-Time Alerts**: With WebSocket enabled, alerts arrive instantly. The server notifies the browser, which retrieves and processes new alerts immediately. ## Who Hears Notifications? Users with Keep open in their browser and the noisy preset visible in their navigation bar. Presets can be filtered to control notifications. ### Customizing 1. **Change the Default Sound**: Replace the `alert.mp3` file with a custom audio file of your choice. *** # Alert Table Source: https://docs.keephq.dev/alerts/table The Alert Table is the central interface for viewing and managing alerts in Keep. It provides a comprehensive view of all alerts with powerful filtering, sorting, and interaction capabilities. ### (1) Columns Columns in the alert table can be customized to display the most relevant data. Users can select which columns to display and reorder them using drag-and-drop functionality. ### (2) Alert Bulk Action Easily select one or more alerts for bulk actions. Actions include options like "assign to incident," "dismiss," or other available workflows. ### (3) Alert Actions Menu The actions menu provides quick access to various operations for each alert, such as linking to incidents, creating tickets, or escalating. ### (4) Alert Link Each alert includes a badge that links directly to the original alert in the monitoring tool. Clicking this badge opens the alert in its source system for further investigation. ### (5) Alert Ticket You can asign ticket to alert. If an alert is associated with a ticket, a ticket badge will be displayed. Clicking on this badge navigates directly to the assigned ticket in the ticketing tool. ### (6) Alert Comment Users can add comments to any alert to provide additional context or share insights with team members. This improves collaboration and ensures all relevant information is available. ### (7) Alert Related Workflows View and trigger related workflows for an alert directly from the table. This allows seamless integration with predefined processes like escalation, suppression, or custom automation. ### (8) Sorting The table supports sorting by any column using the "sort" icon. This makes it easy to prioritize or organize alerts based on specific criteria. *** # Alert enrich Source: https://docs.keephq.dev/cli/commands/alert-enrich Enrich an alert. ## Usage ``` Usage: keep alert enrich [OPTIONS] [PARAMS]... ``` ## Options ## CLI Help ``` Usage: keep alert enrich [OPTIONS] [PARAMS]... Enrich an alert. Options: --fingerprint TEXT The fingerprint of the alert to enrich. [required] --help Show this message and exit. ``` # Alert get Source: https://docs.keephq.dev/cli/commands/alert-get Get an alert. ## Usage ``` Usage: keep alert get [OPTIONS] FINGERPRINT ``` ## Options ## CLI Help ``` Usage: keep alert get [OPTIONS] FINGERPRINT Options: --help Show this message and exit. ``` # Alert list Source: https://docs.keephq.dev/cli/commands/alert-list List alerts. ## Usage ``` Usage: keep alert list [OPTIONS] ``` ## Options * `filter`: * Type: STRING * Default: `none` * Usage: `--filter -f` Filter alerts based on specific attributes. E.g., --filter source=datadog * `export`: * Type: Path * Default: `none` * Usage: `--export` Export alerts to a specified JSON file. * `help`: * Type: BOOL * Default: `false` * Usage: `--help` Show this message and exit. ## CLI Help ``` Usage: keep alert list [OPTIONS] List alerts. Options: -f, --filter TEXT Filter alerts based on specific attributes. E.g., --filter source=datadog --export PATH Export alerts to a specified JSON file. --help Show this message and exit. ``` # Cli Source: https://docs.keephq.dev/cli/commands/cli # cli Run Keep CLI. ## Usage ``` Usage: cli [OPTIONS] COMMAND [ARGS]... ``` ## Options * `verbose`: * Type: IntRange(0, None) * Default: `0` * Usage: `--verbose -v` Enable verbose output. * `json`: * Type: BOOL * Default: `false` * Usage: `--json -j` Enable json output. * `keep_config`: * Type: STRING * Default: `keep.yaml` * Usage: `--keep-config -c` The path to the keep config file (default keep.yaml) * `help`: * Type: BOOL * Default: `false` * Usage: `--help` Show this message and exit. ## CLI Help ``` Usage: cli [OPTIONS] COMMAND [ARGS]... Run Keep CLI. Options: -v, --verbose Enable verbose output. -j, --json Enable json output. -c, --keep-config TEXT The path to the keep config file (default keep.yaml) --help Show this message and exit. Commands: alert Manage alerts. api Start the API. config Get the config. provider Manage providers. run Run a workflow. version Get the library version. whoami Verify the api key auth. workflow Manage workflows. ``` # Cli alert Source: https://docs.keephq.dev/cli/commands/cli-alert # cli alert Manage alerts. ## Usage ``` Usage: cli alert [OPTIONS] COMMAND [ARGS]... ``` ## Options * `help`: * Type: BOOL * Default: `false` * Usage: `--help` Show this message and exit. ## CLI Help ``` Usage: cli alert [OPTIONS] COMMAND [ARGS]... Manage alerts. Options: --help Show this message and exit. Commands: enrich Enrich an alert. get list List alerts. ``` # api Source: https://docs.keephq.dev/cli/commands/cli-api Start the API. ## Usage ``` Usage: keep api [OPTIONS] ``` ## Options * `multi_tenant`: * Type: BOOL * Default: `false` * Usage: `--multi-tenant` Enable multi-tenant mode * `help`: * Type: BOOL * Default: `false` * Usage: `--help` Show this message and exit. ## CLI Help ``` Usage: keep api [OPTIONS] Start the API. Options: --multi-tenant Enable multi-tenant mode --help Show this message and exit. ``` # config Source: https://docs.keephq.dev/cli/commands/cli-config Set keep configuration. ## Usage ``` Usage: keep config [OPTIONS] COMMAND [ARGS]... ``` ## Options * `help`: * Type: BOOL * Default: `false` * Usage: `--help` Show this message and exit. ## CLI Help ``` Usage: keep config [OPTIONS] COMMAND [ARGS]... Manage the config. Options: --help Show this message and exit. Commands: new create new config. show show the current config. ``` # Cli config new Source: https://docs.keephq.dev/cli/commands/cli-config-new Create new config. ## Usage ``` Usage: keep config new [OPTIONS]... ``` ## Options * `interactive`: * Type: BOOL * Default: `True` * Usage: `--interactive` Create config interactively. * `url`: * Type: STRING * Default: `http://localhost:8080` * Usage: `--url` The URL of the Keep backend server. * `api-key`: * Type: STRING * Default: \`\` * Usage: `--api-key` The api key for authenticating over keep. * `help`: * Type: BOOL * Default: `false` * Usage: `--help` Show this message and exit. ## CLI Help ``` Usage: keep config new [OPTIONS] create new config. Options: -u, --url TEXT The url of the keep api -a, --api-key TEXT The api key for keep -i, --interactive Interactive mode creating keep config (default True) --help Show this message and exit. ``` # Cli config show Source: https://docs.keephq.dev/cli/commands/cli-config-show Show keep configuration. ## Usage ``` Usage: keep config show [OPTIONS]... ``` ## Options * `help`: * Type: BOOL * Default: `false` * Usage: `--help` Show this message and exit. ## CLI Help ``` Usage: keep config show [OPTIONS] show the current config. Options: --help Show this message and exit. ``` # Cli provider Source: https://docs.keephq.dev/cli/commands/cli-provider # cli provider Manage providers. ## Usage ``` Usage: cli provider [OPTIONS] COMMAND [ARGS]... ``` ## Options * `help`: * Type: BOOL * Default: `false` * Usage: `--help` Show this message and exit. ## CLI Help ``` Usage: cli provider [OPTIONS] COMMAND [ARGS]... Manage providers. Options: --help Show this message and exit. Commands: connect delete list List providers. ``` # run Source: https://docs.keephq.dev/cli/commands/cli-run Run the alert. ## Usage ``` Usage: keep run [OPTIONS] ``` ## Options * `alerts_directory`: * Type: STRING * Default: `none` * Usage: `--alerts-directory --alerts-file -af` The path to the alert yaml/alerts directory * `alert_url`: * Type: STRING * Default: `none` * Usage: `--alert-url -au` A url that can be used to download an alert yaml NOTE: This argument is mutually exclusive with alerts\_directory * `interval`: * Type: INT * Default: `0` * Usage: `--interval -i` When interval is set, Keep will run the alert every INTERVAL seconds * `providers_file`: * Type: STRING * Default: `providers.yaml` * Usage: `--providers-file -p` The path to the providers yaml * `tenant_id`: * Type: STRING * Default: `singletenant` * Usage: `--tenant-id -t` The tenant id * `api_key`: * Type: STRING * Default: `none` * Usage: `--api-key` The API key for keep's API * `api_url`: * Type: STRING * Default: `https://s.keephq.dev` * Usage: `--api-url` The URL for keep's API * `help`: * Type: BOOL * Default: `false` * Usage: `--help` Show this message and exit. ## CLI Help ``` Usage: keep run [OPTIONS] Run the alert. Options: -af, --alerts-directory, --alerts-file PATH The path to the alert yaml/alerts directory -au, --alert-url TEXT A url that can be used to download an alert yaml NOTE: This argument is mutually exclusive with alerts_directory -i, --interval INTEGER When interval is set, Keep will run the alert every INTERVAL seconds -p, --providers-file PATH The path to the providers yaml -t, --tenant-id TEXT The tenant id --api-key TEXT The API key for keep's API --api-url TEXT The URL for keep's API --help Show this message and exit. ``` # version Source: https://docs.keephq.dev/cli/commands/cli-version Get the library version. ## Usage ``` Usage: keep version [OPTIONS] ``` ## Options * `help`: * Type: BOOL * Default: `false` * Usage: `--help` Show this message and exit. ## CLI Help ``` Usage: keep version [OPTIONS] Get the library version. Options: --help Show this message and exit. ``` # whoami Source: https://docs.keephq.dev/cli/commands/cli-whoami Verify the api key auth. ## Usage ``` Usage: keep whoami [OPTIONS] ``` ## Options * `help`: * Type: BOOL * Default: `false` * Usage: `--help` Show this message and exit. ## CLI Help ``` Usage: keep whoami [OPTIONS] Verify the api key auth. Options: --help Show this message and exit. ``` # Cli workflow Source: https://docs.keephq.dev/cli/commands/cli-workflow # cli workflow Manage workflows. ## Usage ``` Usage: cli workflow [OPTIONS] COMMAND [ARGS]... ``` ## Options * `help`: * Type: BOOL * Default: `false` * Usage: `--help` Show this message and exit. ## CLI Help ``` Usage: cli workflow [OPTIONS] COMMAND [ARGS]... Manage workflows. Options: --help Show this message and exit. Commands: apply Apply a workflow. list List workflows. run Run a workflow with a specified ID and fingerprint. runs Manage workflows executions. ``` # Extraction create Source: https://docs.keephq.dev/cli/commands/extraction-create Create a extraction rule. ## Usage ``` Usage: keep extraction create [OPTIONS] ``` ## Options * `name` * Type: STRING * Default: \`\` * Usage: `--name ` The name of the extraction. * `description` * Type: STRING * Default: \`\` * Usage: `--description ` The description of the extraction. * `priority` * Type: INTEGER RANGE * Default: `0` * Usage: `--priority ` The priority of the extraction, higher priority means this rule will execute first. `0<=x<=100`. * `pre` * Type: BOOL * Default: `false` * Usage: `--pre 
`

  Whether this rule should be applied before or after the alert is standardized

* `attribute`

  * Type: STRING
  * Default: \`\`
  * Usage: `--attribute `

  Event attribute name to extract from.

* `regex`

  * Type: STRING
  * Default: \`\`
  * Usage: `--attribute `

  The regex rule to extract by. Regex format should be like python regex pattern for group matching.

* `condition`

  * Type: STRING
  * Default: \`\`
  * Usage: `--condition `

  CEL based condition.

* `help`:

  * Type: BOOL
  * Default: `false`
  * Usage: `--help`

  Show this message and exit.

## CLI Help

```
Usage: cli.py extraction create [OPTIONS]

  Create a extraction rule.

Options:
  -n, --name TEXT               The name of the extraction.  [required]
  -d, --description TEXT        The description of the extraction.
  -p, --priority INTEGER RANGE  The priority of the extraction, higher
                                priority means this rule will execute first.
                                [0<=x<=100]
  --pre BOOLEAN                 Whether this rule should be applied before or
                                after the alert is standardized.
  -a, --attribute TEXT          Event attribute name to extract from.
                                [required]
  -r, --regex TEXT              The regex rule to extract by. Regex format
                                should be like python regex pattern for group
                                matching.  [required]
  -c, --condition TEXT          CEL based condition.  [required]
  --help                        Show this message and exit.
```


# Extraction delete
Source: https://docs.keephq.dev/cli/commands/extraction-delete



Delete an extraction with a specified ID.

## Usage

```
Usage: keep extraction delete [OPTIONS]
```

## Options

* `extraction-id`

  * Type: STRING
  * Default: \`\`
  * Usage: `--extraction-id `

  The ID of the extraction to delete.

* `help`:

  * Type: BOOL
  * Default: `false`
  * Usage: `--help`

  Show this message and exit.

## CLI Help

```
Usage: cli.py extraction delete [OPTIONS]

  Delete a extraction with a specified ID.

Options:
  --extraction-id INTEGER  The ID of the extraction to delete.  [required]
  --help                   Show this message and exit.
```


# Extractions list
Source: https://docs.keephq.dev/cli/commands/extractions-list



List extractions.

## Usage

```
Usage: keep extraction list [OPTIONS]
```

List mappings.

## Options

* `help`:

  * Type: BOOL
  * Default: `false`
  * Usage: `--help`

  Show this message and exit.

## CLI Help

```
Usage: cli.py extraction list [OPTIONS]

  List extractions.

Options:
  --help  Show this message and exit.
```


# Mappings create
Source: https://docs.keephq.dev/cli/commands/mappings-create



Create a mapping rule.

## Usage

```
Usage: keep mappings create [OPTIONS]
```

## Options

* `name`

  * Type: STRING
  * Default: \`\`
  * Usage: `--name `

  The name of the mapping.

* `description`

  * Type: STRING
  * Default: \`\`
  * Usage: `--description `

  The description of the mapping.

* `file`

  * Type: STRING
  * Default: \`\`
  * Usage: `--file `

  The mapping file. Must be a CSV file.

* `matchers`

  * Type: STRING
  * Default: \`\`
  * Usage: `--matchers `

  The matchers of the mapping, as a comma-separated list of strings.

* `priority`

  * Type: INTEGER RANGE
  * Default: `0`
  * Usage: `--priority `

  The priority of the mapping, higher priority means this rule will execute first. `0<=x<=100`.

* `help`:

  * Type: BOOL
  * Default: `false`
  * Usage: `--help`

  Show this message and exit.

## CLI Help

```
Usage: keep mappings create [OPTIONS]

  Create a mapping rule.

Options:
  -n, --name TEXT               The name of the mapping.  [required]
  -d, --description TEXT        The description of the mapping.
  -f, --file PATH               The mapping file. Must be a CSV file.
                                [required]
  -m, --matchers TEXT           The matchers of the mapping, as a comma-
                                separated list of strings.  [required]
  -p, --priority INTEGER RANGE  The priority of the mapping, higher priority
                                means this rule will execute first.
                                [0<=x<=100]
  --help                        Show this message and exit.
```


# Mappings delete
Source: https://docs.keephq.dev/cli/commands/mappings-delete



Delete a mapping with a specified ID.

## Usage

```
Usage: keep mappings delete [OPTIONS]
```

## Options

* `mapping-id`

  * Type: STRING
  * Default: \`\`
  * Usage: `--mapping-id `

  The ID of the mapping to delete.

* `help`:

  * Type: BOOL
  * Default: `false`
  * Usage: `--help`

  Show this message and exit.

## CLI Help

```
Usage: keep mappings delete [OPTIONS]

  Delete a mapping with a specified ID

Options:
  --mapping-id INTEGER  The ID of the mapping to delete.  [required]
  --help                Show this message and exit.
```


# Mappings list
Source: https://docs.keephq.dev/cli/commands/mappings-list



List mappings.

## Usage

```
Usage: keep mappings [OPTIONS]
```

List mappings.

## Options

* `help`:

  * Type: BOOL
  * Default: `false`
  * Usage: `--help`

  Show this message and exit.

## CLI Help

```
Usage: keep mappings list [OPTIONS]

  List mappings.

Options:
  --help  Show this message and exit.
```


# Provider connect
Source: https://docs.keephq.dev/cli/commands/provider-connect



Connect a provider.

## Usage

```
Usage: keep provider connect [OPTIONS] PROVIDER_TYPE [PARAMS]...
```

## Options

## CLI Help

```
Usage: keep provider connect [OPTIONS] PROVIDER_TYPE [PARAMS]...

Options:
  -h, --help                Help on how to install this provider.
  -n, --provider-name TEXT  Every provider shuold have a name.
```


# Provider delete
Source: https://docs.keephq.dev/cli/commands/provider-delete



Delete a provider.

## Usage

```
Usage: keep provider delete [OPTIONS] [PROVIDER_ID]
```

## Options

## CLI Help

```
Usage: keep provider delete [OPTIONS] [PROVIDER_ID]

Options:
  --help  Show this message and exit.
```


# Provider list
Source: https://docs.keephq.dev/cli/commands/provider-list



List providers.

## Usage

```
Usage: keep provider list [OPTIONS]
```

## Options

* `available`:

  * Type: BOOL
  * Default: `false`
  * Usage: `--available
    -a`

  List provider that you can install.

* `help`:

  * Type: BOOL
  * Default: `false`
  * Usage: `--help`

  Show this message and exit.

## CLI Help

```
Usage: keep provider list [OPTIONS]

  List providers.

Options:
  -a, --available  List provider that you can install.
  --help           Show this message and exit.
```


# Runs list
Source: https://docs.keephq.dev/cli/commands/runs-list



List workflow executions.

## Usage

```
Usage: keep workflow runs list [OPTIONS]
```

## Options

* `help`:

  * Type: BOOL
  * Default: `false`
  * Usage: `--help`

  Show this message and exit.

## CLI Help

```
Usage: keep workflow runs list [OPTIONS]

  List workflow executions.

Options:
  --help  Show this message and exit.
```


# Runs logs
Source: https://docs.keephq.dev/cli/commands/runs-logs



Get workflow execution logs.

## Usage

```
Usage: keep workflow runs logs [OPTIONS] WORKFLOW_EXECUTION_ID
```

## Options

## CLI Help

```
Usage: keep workflow runs logs [OPTIONS] WORKFLOW_EXECUTION_ID

  Get workflow execution logs.

Options:
  --help  Show this message and exit.
```


# Workflow apply
Source: https://docs.keephq.dev/cli/commands/workflow-apply



Apply a workflow.

## Usage

```
Usage: keep workflow apply [OPTIONS]
```

## Options

* `file` (REQUIRED):

  * Type: Path
  * Default: `none`
  * Usage: `--file
    -f`

  The workflow file

* `help`:

  * Type: BOOL
  * Default: `false`
  * Usage: `--help`

  Show this message and exit.

## CLI Help

```
Usage: keep workflow apply [OPTIONS]

  Apply a workflow.

Options:
  -f, --file PATH  The workflow file  [required]
  --help           Show this message and exit.
```


# Workflow list
Source: https://docs.keephq.dev/cli/commands/workflow-list



List workflows.

## Usage

```
Usage: keep workflow list [OPTIONS]
```

## Options

* `help`:

  * Type: BOOL
  * Default: `false`
  * Usage: `--help`

  Show this message and exit.

## CLI Help

```
Usage: keep workflow list [OPTIONS]

  List workflows.

Options:
  --help  Show this message and exit.
```


# Workflow run
Source: https://docs.keephq.dev/cli/commands/workflow-run



Run a workflow with a specified ID and fingerprint.

## Usage

```
Usage: keep workflow run [OPTIONS]
```

## Options

* `workflow_id` (REQUIRED):

  * Type: STRING
  * Default: `none`
  * Usage: `--workflow-id`

  The ID (UUID or name) of the workflow to run

* `fingerprint` (REQUIRED):

  * Type: STRING
  * Default: `none`
  * Usage: `--fingerprint`

  The fingerprint to query the payload

* `help`:

  * Type: BOOL
  * Default: `false`
  * Usage: `--help`

  Show this message and exit.

## CLI Help

```
Usage: keep workflow run [OPTIONS]

  Run a workflow with a specified ID and fingerprint.

Options:
  --workflow-id TEXT  The ID (UUID or name) of the workflow to run  [required]
  --fingerprint TEXT  The fingerprint to query the payload  [required]
  --help              Show this message and exit.
```


# Workflow runs
Source: https://docs.keephq.dev/cli/commands/workflow-runs



Manage workflows executions.

## Usage

```
Usage: cli workflow runs [OPTIONS] COMMAND [ARGS]...
```

## Options

* `help`:

  * Type: BOOL
  * Default: `false`
  * Usage: `--help`

  Show this message and exit.

## CLI Help

```
Usage: cli workflow runs [OPTIONS] COMMAND [ARGS]...

  Manage workflows executions.

Options:
  --help  Show this message and exit.

Commands:
  list  List workflow executions.
  logs  Get workflow execution logs.
```


# Sync Keep Workflows With Github Action
Source: https://docs.keephq.dev/cli/github-actions



This documentation provides a detailed guide on how to use the Keep CLI within a GitHub Actions workflow to synchronize and manage Keep workflows from a directory. This setup automates the process of uploading workflows to Keep, making it easier to maintain and update them.

### Configuration

To set up this workflow in your repository:

* Add the workflow YAML file to your repository under `.github/workflows/`.
* Set your Keep API Key and URL as secrets in your repository settings if you haven't already.
* Make changes to your workflows in the specified directory or trigger the workflow manually through the GitHub UI.
* Change 'example/workflows/\*\*' to the directory you store your Keep Workflows.

### GitHub Action Workflow

This GitHub Actions workflow automatically synchronizes workflows from a specified directory to Keep whenever there are changes. It also allows for manual triggering with optional parameters.

```yaml theme={null}
# A workflow that sync Keep workflows from a directory
name: "Sync Keep Workflows"

on:
    push:
        paths:
          - 'examples/workflows/**'
    workflow_dispatch:
        inputs:
            keep_api_key:
              description: 'Keep API Key'
              required: false
            keep_api_url:
              description: 'Keep API URL'
              required: false
              default: 'https://api.keephq.dev'

jobs:
    sync-workflows:
        name: Sync workflows to Keep
        runs-on: ubuntu-latest
        container:
            image: us-central1-docker.pkg.dev/keephq/keep/keep-cli:latest
        env:
            KEEP_API_KEY: ${{ secrets.KEEP_API_KEY || github.event.inputs.keep_api_key }}
            KEEP_API_URL: ${{ secrets.KEEP_API_URL || github.event.inputs.keep_api_url }}

        steps:
        - name: Check out the repo
          uses: actions/checkout@v2

        - name: Run Keep CLI
          run: |
            keep workflow apply -f examples/workflows

```


# Installation
Source: https://docs.keephq.dev/cli/installation



Missing an installation? submit a new installation  request and we will add it as soon as we can.


  We recommend to install Keep CLI with Python version 3.11 for optimal compatibility and performance.
  This choice ensures seamless integration with all dependencies, including pyarrow, which currently does not support Python 3.12


Need Keep CLI on other versions? Feel free to contact us! 

## Clone and install (Option 1)

### Install

First, clone Keep repository:

```shell theme={null}
git clone https://github.com/keephq/keep.git && cd keep
```

Install Keep CLI with `pip`:

```shell theme={null}
# MacOS if python or pip not present:
# brew install python@3.11
# brew install postgresql
pip3.11 install .
```

or with `poetry`:

```shell theme={null}
poetry install
```

From now on, Keep should be installed locally and accessible from your CLI, test it by executing:

```
keep version
```

### Configuration

To get API key, check Keep UI -> your username (bottom left) -> Settings -> API Keys

```
keep config new --url http://backend.my_keep.my_awesome_org.com:backend_port --api-key your_personal_api_key
```

### Test

Now,

```
keep workflow apply -f examples/workflows/query_clickhouse.yml
```

Congrats 🥳 Check your UI for the new workflow uploaded from the YAML file.

## Docker image (Option 2)

### Install

```
docker run -v ${PWD}:/app -v ~/.keep.yaml:/root/.keep.yaml -it us-central1-docker.pkg.dev/keephq/keep/keep-cli keep config new --url http://backend.my_keep.my_awesome_org.com:backend_port --api-key your_personal_api_key
```

### Test

```
docker run -v ${PWD}:/app -v ~/.keep.yaml:/root/.keep.yaml -it us-central1-docker.pkg.dev/keephq/keep/keep-cli workflow apply -f examples/workflows/query_clickhouse.yml
```

## Enable Auto Completion

Keep's CLI supports shell auto-completion, which can make your life a whole lot easier 😌
If you're using zsh

```shell title=~/.zshrc theme={null}
eval "$(_KEEP_COMPLETE=zsh_source keep)"
```

If you're using bash

```bash title=~/.bashrc theme={null}
eval "$(_KEEP_COMPLETE=bash_source keep)"
```

Using eval means that the command is invoked and evaluated every time a shell is started, which can delay shell responsiveness. To speed it up, write the generated script to a file, then source that.


# Overview
Source: https://docs.keephq.dev/cli/overview



Keep CLI allow you to manage Keep from CLI.

Start by [installing](/cli/installation) Keep CLI and [running a workflow](/cli/commands/cli-run).

### Env variables

|           Env var          |                   Purpose                   | Required | Default Value |   Valid options   |
| :------------------------: | :-----------------------------------------: | :------: | :-----------: | :---------------: |
| **KEEP\_CLI\_IGNORE\_SSL** | Ignore SSL while connecting to the KEEP API |    No    |     false     | "true" or "false" |


# Auth0 Authentication
Source: https://docs.keephq.dev/deployment/authentication/auth0-auth




  Keep Cloud: ✅ 

  Keep Enterprise On-Premises: ✅ 

  Keep Open Source: ⛔️


Keep supports multi-tenant environments through Auth0, enabling separate tenants to operate independently within the same Keep platform.


  


### When to Use

* **Already using Auth0:** If you are already using Auth0 in your organization, you can leverage it as Keep authentication provider.
* **SSO/SAML:** Auth0 supports various Single Sign-On (SSO) and SAML protocols, allowing you to integrate Keep with your existing identity management systems.

### Setup Instructions

To start Keep with Auth0 authentication, set the following environment variables:

#### Frontend Environment Variables

| Environment Variable  | Description                             | Required | Default Value |
| --------------------- | --------------------------------------- | :------: | :-----------: |
| AUTH\_TYPE            | Set to 'AUTH0' for Auth0 authentication |    Yes   |       -       |
| AUTH0\_DOMAIN         | Your Auth0 domain                       |    Yes   |       -       |
| AUTH0\_CLIENT\_ID     | Your Auth0 client ID                    |    Yes   |       -       |
| AUTH0\_CLIENT\_SECRET | Your Auth0 client secret                |    Yes   |       -       |
| AUTH0\_ISSUER         | Your Auth0 API issuer                   |    Yes   |       -       |

#### Backend Environment Variables

| Environment Variable      | Description                             | Required | Default Value |
| ------------------------- | --------------------------------------- | :------: | :-----------: |
| AUTH\_TYPE                | Set to 'AUTH0' for Auth0 authentication |    Yes   |       -       |
| AUTH0\_MANAGEMENT\_DOMAIN | Your Auth0 management domain            |    Yes   |       -       |
| AUTH0\_CLIENT\_ID         | Your Auth0 client ID                    |    Yes   |       -       |
| AUTH0\_CLIENT\_SECRET     | Your Auth0 client secret                |    Yes   |       -       |
| AUTH0\_AUDIENCE           | Your Auth0 API audience                 |    Yes   |       -       |

### Example configuration

Use the `docker-compose-with-auth0.yml` for an easy setup, which includes necessary environment variables for enabling Auth0 authentication.


# Azure AD Authentication
Source: https://docs.keephq.dev/deployment/authentication/azuread-auth




  Keep Cloud: ✅ 

  Keep Enterprise On-Premises: ✅ 

  Keep Open Source: ⛔️


Keep supports enterprise authentication through Azure Entre ID (formerly known as Azure AD), enabling organizations to use their existing Microsoft identity platform for secure access management.

## When to Use

* **Microsoft Environment:** If your organization uses Microsoft 365 or Azure services, Azure AD integration provides seamless authentication.
* **Enterprise SSO:** Leverage Azure AD's Single Sign-On capabilities for unified access management.

## Setup Instructions (on Azure AD)

### Creating an Azure AD Application

1. Sign in to the [Azure Portal](https://portal.azure.com)
2. Navigate to **Microsoft Entra ID** > **App registrations** > **New registration**


  Azure AD App Registration


3. Configure the application:
   * Name: "Keep"

Note that we are using "Register an application to integrate with Microsoft Entra ID (App you're developing)" since you're self-hosting Keep and need direct control over the authentication flow and permissions for your specific instance - unlike the cloud/managed version where Keep's team has already configured a centralized application registration.


  Azure AD App Registration


4. Configure the application (continue)

* Supported account types: "Single tenant"


  We recommend using "Single tenant" for enhanced security as it restricts access to users within your organization only. While multi-tenant configuration is possible, it would allow users from any Azure AD directory to access your Keep instance, which could pose security risks unless you have specific cross-organization requirements.


* Redirect URI: "Web" + your redirect URI


  We use "Web" platform instead of "Single Page Application (SPA)" because Keep's backend handles the authentication flow using client credentials/secrets, which is more secure than the implicit flow used in SPAs. This prevents exposure of tokens in the browser and provides stronger security through server-side token validation and refresh token handling.



  For localhost, the redirect would be [http://localhost:3000/api/auth/callback/microsoft-entra-id](http://localhost:3000/api/auth/callback/microsoft-entra-id)

  For production, it should be something like http\://your\_keep\_frontend\_domain/api/auth/callback/microsoft-entra-id



  Azure AD App Registration


5. Finally, click "register"

### Configure Authentication

After we created the application, let's configure the authentication.

1. Go to "App Registrations" -> "All applications"


  Azure AD Authentication Configuration


2. Click on your application -> "Add a certificate or secret"


  Azure AD Authentication Configuration


3. Click on "New client secret" and give it a name


  Azure AD Authentication Configuration


4. Keep the "Value", we will use it soon as `KEEP_AZUREAD_CLIENT_SECRET`


  Azure AD Authentication Configuration


### Configure Groups

Keep maps Azure AD groups to roles with two default groups:

1. Admin Group (read + write)
2. NOC Group (read only)

To create those groups, go to Groups -> All groups and create two groups:


  Azure AD Authentication Configuration


Keep the Object id of these groups and use it as `KEEP_AZUREAD_ADMIN_GROUP_ID` and `KEEP_AZUREAD_NOC_GROUP_ID`.

### Configure Group Claims

1. Navigate to **Token configuration**


  Azure AD Authentication Configuration


2. Add groups claim:
   * Select "Security groups" and "Groups assigned to the application"
   * Choose "Group ID" as the claim value


  Azure AD Authentication Configuration



  Azure AD Authentication Configuration


### Configure Application Scopes

1. Go to "Expose an API" and click on "Add a scope"


  Azure AD Authentication Configuration


2. Keep the default Application ID and click "Save and continue"


  Azure AD Authentication Configuration


3. Add "default" as scope name, also give a display name and description


  Azure AD Authentication Configuration


3. Finally, click "Add scope"


  Azure AD Authentication Configuration


## Setup Instructions (on Keep)

After you configured Azure AD you should have the following:

1. Azure AD Tenant ID
2. Azure AD Client ID

How to get:


  Azure AD Authentication Configuration


3. Azure AD Client Secret [See Configure Authentication](#configure-authentication).
4. Azure AD Group ID's for Admins and NOC (read only) [See Configure Groups](#configure-groups).

### Configuration

#### Frontend

| Environment Variable          | Description                                  | Required | Default Value |
| ----------------------------- | -------------------------------------------- | :------: | :-----------: |
| AUTH\_TYPE                    | Set to 'AZUREAD' for Azure AD authentication |    Yes   |       -       |
| KEEP\_AZUREAD\_CLIENT\_ID     | Your Azure AD application (client) ID        |    Yes   |       -       |
| KEEP\_AZUREAD\_CLIENT\_SECRET | Your client secret                           |    Yes   |       -       |
| KEEP\_AZUREAD\_TENANT\_ID     | Your Azure AD tenant ID                      |    Yes   |       -       |
| NEXTAUTH\_URL                 | Your Keep application URL                    |    Yes   |       -       |
| NEXTAUTH\_SECRET              | Random string for NextAuth.js                |    Yes   |       -       |

#### Backend

| Environment Variable            | Description                                  | Required | Default Value |
| ------------------------------- | -------------------------------------------- | :------: | :-----------: |
| AUTH\_TYPE                      | Set to 'AZUREAD' for Azure AD authentication |    Yes   |       -       |
| KEEP\_AZUREAD\_TENANT\_ID       | Your Azure AD tenant ID                      |    Yes   |       -       |
| KEEP\_AZUREAD\_CLIENT\_ID       | Your Azure AD application (client) ID        |    Yes   |       -       |
| KEEP\_AZUREAD\_ADMIN\_GROUP\_ID | The group ID of Keep Admins (read write)     |    Yes   |       -       |
| KEEP\_AZUREAD\_NOC\_GROUP\_ID   | The group ID of Keep NOC (read only)         |    Yes   |       -       |

## Features and Limitations

#### Supported Features

* Single Sign-On (SSO)
* Role-based access control through Azure AD groups
* Multi-factor authentication (when configured in Azure AD)

#### Limitations

See [Overview](/deployment/authentication/overview)


# DB Authentication
Source: https://docs.keephq.dev/deployment/authentication/db-auth



For applications requiring user management and authentication, Keep supports basic authentication with username and password.


  


### When to Use

* **Self-Hosted Deployments:** When you're deploying Keep for individual use or within an organization.
* **Enhanced Security:** Provides a simple yet effective layer of security for your Keep instance.

### Setup Instructions

To start Keep with DB authentication, set the following environment variables:

| Environment Variable                  |               Description               | Required | Frontend/Backend | Default Value |
| ------------------------------------- | :-------------------------------------: | :------: | :--------------: | :-----------: |
| AUTH\_TYPE                            | Set to 'DB' for database authentication |    Yes   |       Both       |       -       |
| KEEP\_JWT\_SECRET                     |     Secret for JWT token generation     |    Yes   |      Backend     |       -       |
| KEEP\_DEFAULT\_USERNAME               |          Default admin username         |    No    |      Backend     |      keep     |
| KEEP\_DEFAULT\_PASSWORD               |          Default admin password         |    No    |      Backend     |      keep     |
| KEEP\_FORCE\_RESET\_DEFAULT\_PASSWORD |   Override the current admin password   |    No    |      Backend     |     false     |

### Example configuration

Use the `docker-compose-with-auth.yml` for an easy setup, which includes necessary environment variables for enabling basic authentication.


# Keycloak Authentication
Source: https://docs.keephq.dev/deployment/authentication/keycloak-auth




  Keep Cloud: ✅ 

  Keep Enterprise On-Premises: ✅ 

  Keep Open Source: ⛔️


Keep supports Keycloak in a "managed" way where Keep auto-provisions all resources (realm, client, etc.). Keep can also work with externally managed Keycloak. To learn how, please contact the team on [Slack](https://slack.keephq.dev).

Keep integrates with Keycloak to provide a powerful and flexible authentication system for multi-tenant applications, supporting Single Sign-On (SSO) and SAML.


  


### When to Use

* **On Prem:** When deploying Keep on-premises and requiring a robust authentication system.
* **OSS:** If you prefer using open-source software for your authentication needs.
* **Enterprise Protocols:** When you need support for enterprise-level protocols like SAML and OpenID Connect.
* **Fully Customized:** When you need a highly customizable authentication solution.
* **RBAC:** When you require Role-Based Access Control for managing user permissions.
* **User and Group Management:** When you need advanced user and group management capabilities.

### Setup Instructions

To start Keep with Keycloak authentication, set the following environment variables:

#### Frontend Environment Variables

| Environment Variable | Description                                                                                                                | Required |     Default Value    |
| -------------------- | -------------------------------------------------------------------------------------------------------------------------- | :------: | :------------------: |
| AUTH\_TYPE           | Set to 'KEYCLOAK' for Keycloak authentication                                                                              |    Yes   |           -          |
| KEYCLOAK\_ID         | Your Keycloak client ID (e.g. keep)                                                                                        |    Yes   |           -          |
| KEYCLOAK\_ISSUER     | Full URL to Your Keycloak issuer URL e.g. [http://localhost:8181/auth/realms/keep](http://localhost:8181/auth/realms/keep) |    Yes   |           -          |
| KEYCLOAK\_SECRET     | Your Keycloak client secret                                                                                                |    Yes   | keep-keycloak-secret |

#### Backend Environment Variables

| Environment Variable      | Description                                   | Required |                        Default Value                       |
| ------------------------- | --------------------------------------------- | :------: | :--------------------------------------------------------: |
| AUTH\_TYPE                | Set to 'KEYCLOAK' for Keycloak authentication |    Yes   |                              -                             |
| KEYCLOAK\_URL             | Full URL to your Keycloak server              |    Yes   | [http://localhost:8181/auth/](http://localhost:8181/auth/) |
| KEYCLOAK\_REALM           | Your Keycloak realm                           |    Yes   |                            keep                            |
| KEYCLOAK\_CLIENT\_ID      | Your Keycloak client ID                       |    Yes   |                            keep                            |
| KEYCLOAK\_CLIENT\_SECRET  | Your Keycloak client secret                   |    Yes   |                    keep-keycloak-secret                    |
| KEYCLOAK\_ADMIN\_USER     | Admin username for Keycloak                   |    Yes   |                         keep\_admin                        |
| KEYCLOAK\_ADMIN\_PASSWORD | Admin password for Keycloak                   |    Yes   |                         keep\_admin                        |
| KEYCLOAK\_AUDIENCE        | Audience for Keycloak                         |    Yes   |                      realm-management                      |

### Example configuration

To get a better understanding on how to use Keep together with Keycloak, you can:

* See [Keycloak](https://github.com/keephq/keep/tree/main/keycloak) directory for configuration, realm.json, etc
* See Keep + Keycloak [docker-compose example](https://github.com/keephq/keep/blob/main/keycloak/docker-compose.yaml)


# No Authentication
Source: https://docs.keephq.dev/deployment/authentication/no-auth



Using this configuration in production is not secure and strongly discouraged.

Deploying Keep without authentication is the quickest way to get up and running, ideal for local development or internal tools where security is not a concern.

## Setup Instructions

Either if you use docker-compose, kubernetes, openshift or any other deployment method, add the following environment variable:

```
# Frontend
AUTH_TYPE=NOAUTH

# Backend
AUTH_TYPE=NOAUTH
```

## Implications

With `AUTH_TYPE=NOAUTH`:

* Keep won't show any login page and will let you consume APIs without authentication.
* Keep will use a JWT with "keep" as the tenant id, but will not validate it.
* Any API key provided in the `x-api-key` header will be accepted without validation.

This configuration essentially bypasses all authentication checks, making it unsuitable for production environments where security is a concern.


# Example: OAuth2‑Proxy + Keep + GitLab SSO
Source: https://docs.keephq.dev/deployment/authentication/oauth2-proxy-gitlab



A **step‑by‑step cookbook** for adding single‑sign‑on to [Keep](https://github.com/keephq) with your **self‑hosted GitLab** using [oauth2‑proxy](https://oauth2‑proxy.github.io/) and the NGINX Ingress Controller.

> **Conventions used below**
>
> * ``             – public FQDN where users access Keep (e.g. `keep.example.com`)
> * ``           – URL of your GitLab instance (e.g. `gitlab.example.com`)
> * ``         – container registry that stores images (omit if you use the public images)
> * Kubernetes namespace **`keep`** – feel free to change it everywhere if you prefer another namespace.

***

## 1. Prerequisites

| What                                        | Why                                                   |
| ------------------------------------------- | ----------------------------------------------------- |
| Kubernetes cluster & `keep` namespace       | Where Keep, oauth2‑proxy and Services live            |
| **ingress‑nginx** (or compatible)           | Provides the `auth_request` feature oauth2‑proxy uses |
| GitLab 15 + at `https://`      | OpenID‑Connect issuer                                 |
| Helm 3.x & offline charts/images (optional) | If your cluster has no Internet egress                |

***

## 2. Create the GitLab OAuth application

1. **GitLab ▸ Admin → Applications → New**
2. Name → `keep‑sso`
3. Redirect URI → `https:///oauth2/callback`
4. Scopes → `openid profile email` (+ `read_api` if you plan to gate access by group/project)
5. Save – copy the generated **Application ID** and **Secret**.

***

## 3. Kubernetes secrets & config

```bash theme={null}
# 3.1 Generate a 32‑byte cookie secret
echo "$(openssl rand -base64 32 | head -c 32 | base64)" > cookie.b64

# 3.2 Store GitLab credentials and cookie secret
kubectl -n keep create secret generic oauth2-proxy \
  --from-literal=client-id= \
  --from-literal=client-secret= \
  --from-file=cookie-secret=cookie.b64

# 3.3 Add gitlab credentials and cookie secret using OAUTH2_PROXY ENV variables
OAUTH2_PROXY_CLIENT_ID=
OAUTH2_PROXY_CLIENT_SECRET=
OAUTH2_PROXY_COOKIE_SECRET=cookie.b64

# (optional) store GitLab’s custom CA certificate
kubectl -n keep create secret generic gitlab-ca \
  --from-file=gitlab-ca.pem
```

```yaml theme={null}
# 3.4 oauth2_proxy.cfg (ConfigMap)
apiVersion: v1
kind: ConfigMap
metadata:
  name: oauth2-proxy
  namespace: keep
data:
  oauth2_proxy.cfg: |
    email_domains = ["*"]
    upstreams     = ["file:///dev/null"]   # we only use auth‑request mode
    provider      = "gitlab"
    cookie_name   = "keep-dev" #if empty, will use default cookie name: _oauth2_proxy
    cookie_secure = true
```

***

## 4. Deploy **oauth2‑proxy** (Helm)

```yaml theme={null}
# values.oauth2-proxy.yaml – minimal baseline
image:                     # replace with public image if desired
  repository: /oauth2-proxy/oauth2-proxy
  tag: v7.9.0

config:
  configFile: |-
    # content comes from the ConfigMap above

extraArgs:
  oidc-issuer-url: https://
  set-xauthrequest: "true"            # add X-Auth-Request-*/X-Forwarded-* headers
  pass-authorization-header: "true"   # add Authorization: Bearer 
  # provider-ca-file: /ca/gitlab-ca.pem   # enable if you mounted a corporate CA or use ssl-insecure-skip-verify: "true" to disable SSL check.
extraVolumes:
  - name: gitlab-ca
    secret:
      secretName: gitlab-ca
extraVolumeMounts:
  - name: gitlab-ca
    mountPath: /ca/gitlab-ca.pem
    subPath: gitlab-ca.pem
    readOnly: true

service:
  type: ClusterIP

ingress:
  enabled: false   # we only need an internal Service
```

```bash theme={null}
helm repo add oauth2-proxy https://oauth2-proxy.github.io/manifests
helm upgrade --install oauth2-proxy oauth2-proxy/oauth2-proxy \
     -n keep -f values.oauth2-proxy.yaml
```

*Lab‑only shortcut*: instead of mounting the CA you can temporarily add
`ssl-insecure-skip-verify: "true"` under `extraArgs`.

***

## 5. Patch (or create) Keep’s Ingress resource

Add **three** annotations so ingress‑nginx delegates auth to the Service:

```yaml theme={null}
global:
  ingress:
    annotations:
      nginx.ingress.kubernetes.io/auth-url: "http://oauth2-proxy.keep.svc.cluster.local/oauth2/auth"
      nginx.ingress.kubernetes.io/auth-signin: "https:///oauth2/start?rd=$request_uri"
      nginx.ingress.kubernetes.io/auth-response-headers: "authorization,x-auth-request-user,x-auth-request-email,x-forwarded-user,x-forwarded-email,x-forwarded-groups"
```

Redeploy Keep (or patch the Ingress manually).

***

## 6. Environment variables for Keep

```yaml theme={null}
backend:
  env:
    - name: AUTH_TYPE
      value: OAUTH2PROXY
    - name: KEEP_OAUTH2_PROXY_USER_HEADER
      value: x-auth-request-email
    - name: KEEP_OAUTH2_PROXY_ROLE_HEADER
      value: x-auth-request-groups
    - name: KEEP_OAUTH2_PROXY_AUTO_CREATE_USER
      value: true
    - name: KEEP_OAUTH2_PROXY_ADMIN_ROLES
      value: 
    - name: KEEP_OAUTH2_PROXY_NOC_ROLES
      value: 

frontend:
  env:
    # Public URL the **browser** should use
    - name: NEXTAUTH_URL
      value: "https://"

    # URL the **server‑side** Next.js code can always reach
    - name: NEXTAUTH_URL_INTERNAL
      value: "http://keep-frontend.keep.svc.cluster.local:3000"

    # API URLs
    - name: API_URL_CLIENT   # browser → ingress
      value: "/v2"
    - name: API_URL          # server → backend Service (no auth‑proxy)
      value: "http://keep-backend.keep.svc.cluster.local:8080"

    #Oauth2-Proxy
    - name: AUTH_TYPE
      value: OAUTH2PROXY
    - name: KEEP_OAUTH2_PROXY_USER_HEADER
      value: x-auth-request-email
    - name: KEEP_OAUTH2_PROXY_ROLE_HEADER
      value: x-auth-request-groups
```

Roll out the frontend:

```bash theme={null}
kubectl -n keep rollout restart deploy/keep-frontend
```

***

## 7. Quick validation

```bash theme={null}
# 7.1 Call auth endpoint without cookie – expect 401
curl -I http://oauth2-proxy.keep.svc.cluster.local/oauth2/auth

# 7.2 Copy the keep-dev cookie from your browser session
curl -I --cookie "keep-dev=" \
     http://oauth2-proxy.keep.svc.cluster.local/oauth2/auth   # expect 200
```

Browser smoke‑test:

* `https://` → redirect to GitLab → sign in → return to Keep.
* DevTools ▸ Network → `/api/auth/session` returns **200**.

***

## 8. Troubleshooting

| Symptom                                                       | Common cause & remedy                                                                                                                       |
| ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| **TLS error** `x509: certificate signed by unknown authority` | Mount your GitLab CA (`provider-ca-file`) or set `ssl-insecure-skip-verify=true` (dev only).                                                |
| Ingress logs `auth request unexpected status: 502`            | `auth-url` is pointing at the external host – use the internal Service DNS (`http://oauth2-proxy.keep.svc.cluster.local`).                  |
| Browser loops at `/signin?callbackUrl=…`                      | ① `set-xauthrequest` not enabled, or ② `auth-response-headers` not set, or ③ backend receives calls through oauth2‑proxy (`API_URL` wrong). |
| Redirect to `0.0.0.0:3000` or pod name                        | `NEXTAUTH_URL` missing at **build time**; rebuild UI or override env.                                                                       |
| 401 from `/oauth2/auth` even with cookie                      | Cookie expired / clocks out of sync. Clear cookie and re‑login.                                                                             |

***

## 9. Clean‑up

```bash theme={null}
helm -n keep uninstall oauth2-proxy
helm -n keep uninstall keep          # if you want to remove Keep
kubectl -n keep delete secret oauth2-proxy gitlab-ca
```

***

## Appendix A – Generate a 32‑byte cookie secret

```bash theme={null}
openssl rand -hex 16 | xxd -r -p | base64
```

## Appendix B – Sync images to an offline registry (example)

```bash theme={null}
skopeo copy docker://quay.io/oauth2-proxy/oauth2-proxy:v7.9.0 \
             docker:///oauth2-proxy/oauth2-proxy:v7.9.0
```


# OAuth2Proxy Authentication
Source: https://docs.keephq.dev/deployment/authentication/oauth2proxy-auth




  Keep Cloud: ✅ 

  Keep Enterprise On-Premises: ✅ 

  Keep Open Source: (experimental)


Delegate authentication to Oauth2Proxy.

### When to Use

* **oauth2-proxy user:** Use this authentication method if you want to delegate authentication to an external Oauth2Proxy service.

### Setup Instructions

To start Keep with Oauth2Proxy authentication, set the following environment variables:

#### Frontend Environment Variables

| Environment Variable              | Description                                         | Required |    Default Value   |
| --------------------------------- | --------------------------------------------------- | :------: | :----------------: |
| AUTH\_TYPE                        | Set to 'OAUTH2PROXY' for OAUTH2PROXY authentication |    Yes   |          -         |
| KEEP\_OAUTH2\_PROXY\_USER\_HEADER | Header for the authenticated user's email           |    Yes   |  x-forwarded-email |
| KEEP\_OAUTH2\_PROXY\_ROLE\_HEADER | Header for the authenticated user's role            |    Yes   | x-forwarded-groups |

#### Backend Environment Variables

| Environment Variable                    | Description                                          | Required |    Default Value   |
| --------------------------------------- | ---------------------------------------------------- | :------: | :----------------: |
| AUTH\_TYPE                              | Set to 'OAUTH2PROXY' for OAUTH2PROXY authentication  |    Yes   |          -         |
| KEEP\_OAUTH2\_PROXY\_USER\_HEADER       | Header for the authenticated user's email            |    Yes   |  x-forwarded-email |
| KEEP\_OAUTH2\_PROXY\_ROLE\_HEADER       | Header for the authenticated user's role             |    Yes   | x-forwarded-groups |
| KEEP\_OAUTH2\_PROXY\_AUTO\_CREATE\_USER | Automatically create user if not exists              |    No    |        true        |
| KEEP\_OAUTH2\_PROXY\_ADMIN\_ROLES       | Role names for admin users                           |    No    |        admin       |
| KEEP\_OAUTH2\_PROXY\_NOC\_ROLES         | Role names for NOC (Network Operations Center) users |    No    |         noc        |
| KEEP\_OAUTH2\_PROXY\_WEBHOOK\_ROLES     | Role names for webhook users                         |    No    |       webhook      |


# Okta Authentication
Source: https://docs.keephq.dev/deployment/authentication/okta-auth



This document provides comprehensive information about the Okta integration in Keep.

## Overview

Keep supports Okta as an authentication provider, enabling:

* Single Sign-On (SSO) via Okta
* OAuth2/OIDC authentication flow
* JWT token verification with JWKS
* Role-based access control through token claims

## Environment Variables

### Backend Environment Variables

| Variable             | Description                                                                                         | Required |
| -------------------- | --------------------------------------------------------------------------------------------------- | -------- |
| `AUTH_TYPE`          | Set to `"OKTA"` to enable Okta authentication                                                       | Yes      |
| `OKTA_DOMAIN`        | Your Okta domain (e.g., `https://company.okta.com`)                                                 | Yes      |
| `OKTA_ISSUER`        | The issuer URL for your Okta authorization server (e.g., `https://company.okta.com/oauth2/default`) | Yes      |
| `OKTA_CLIENT_ID`     | Client ID of your Okta application                                                                  | Yes      |
| `OKTA_CLIENT_SECRET` | Client Secret of your Okta application                                                              | Yes      |
| `OKTA_AUDIENCE`      | Expected audience claim in the token. Falls back to `OKTA_CLIENT_ID` if not set                     | No       |
| `OKTA_JWKS_URL`      | Explicit JWKS URL. If not set, derived from `OKTA_ISSUER`                                           | No       |
| `OKTA_API_TOKEN`     | Okta API token for management operations                                                            | No       |

### Frontend Environment Variables

| Variable             | Description                                       | Example                                   |
| -------------------- | ------------------------------------------------- | ----------------------------------------- |
| `AUTH_TYPE`          | Set to `"OKTA"` to enable Okta authentication     | `OKTA`                                    |
| `OKTA_ISSUER`        | The issuer URL for your Okta authorization server | `https://company.okta.com/oauth2/default` |
| `OKTA_CLIENT_ID`     | Client ID of your Okta application                | `0oa1bcdef2ghijklm3n4`                    |
| `OKTA_CLIENT_SECRET` | Client Secret of your Okta application            | `abcd1234efgh5678`                        |

## Okta Configuration

### Creating an Okta Application

1. Sign in to your Okta Admin Console
2. Navigate to **Applications** > **Applications**
3. Click **Create App Integration**
4. Select **OIDC - OpenID Connect** as the sign-in method
5. Select **Web Application** as the application type
6. Click **Next**

### Application Settings

1. **App integration name**: Enter a name for your application (e.g., "Keep")
2. **Sign-in redirect URIs**: Add your callback URL: `https://your-keep-domain.com/api/auth/callback/okta`
3. **Sign-out redirect URIs**: Add your sign-out URL: `https://your-keep-domain.com`
4. **Assignments**: Assign the application to the appropriate users or groups
5. Click **Save**
6. Copy the **Client ID** and **Client Secret** from the application settings

### Role Mapping

Keep extracts the user role from the JWT token. The role is determined in the following order:

1. `keep_role` claim in the token
2. `role` claim in the token
3. First entry in the `groups` claim
4. Falls back to `user` role

To configure role mapping, add a custom claim to your Okta authorization server:

1. Navigate to **Security** > **API** > **Authorization Servers**
2. Select your authorization server (e.g., `default`)
3. Go to the **Claims** tab
4. Add a claim named `keep_role` or `groups` that maps to the user's Keep role


# OneLogin Authentication
Source: https://docs.keephq.dev/deployment/authentication/onelogin-auth



This document provides comprehensive information about the OneLogin integration in Keep

## Overview

Keep supports OneLogin as an authentication provider, enabling:

* Single Sign-On (SSO) via OneLogin
* OAuth2/OIDC authentication flow
* Token refresh capabilities
* Role-based access control through custom claims
* Session management through NextAuth.js

## Environment Variables

### Backend Environment Variables

| Variable                    | Description                                           | Example                               |
| --------------------------- | ----------------------------------------------------- | ------------------------------------- |
| `AUTH_TYPE`                 | Set to `"ONELOGIN"` to enable OneLogin authentication | `ONELOGIN`                            |
| `ONELOGIN_ISSUER`           | The issuer URL for your OneLogin application          | `https://company.onelogin.com/oidc/2` |
| `ONELOGIN_CLIENT_ID`        | Client ID of your OneLogin application                | `abc123def456ghi789`                  |
| `ONELOGIN_CLIENT_SECRET`    | Client Secret of your OneLogin application            | `abcd1234efgh5678ijkl9012`            |
| `ONELOGIN_ADMIN_ROLE`       | Role to be mapped to a keep admin role                | `KeepAdmin`                           |
| `ONELOGIN_NOC_ROLE`         | Role to be mapped to a keep noc role                  | `KeepNoc`                             |
| `ONELOGIN_WEBHOOK_ROLE`     | Role to be mapped to a keep webhook role              | `KeepWebhook`                         |
| `ONELOGIN_AUTO_CREATE_USER` | Whether to try and create autocreate users in keep    | `True`                                |

### Frontend Environment Variables

| Variable                 | Description                                           | Example                               |
| ------------------------ | ----------------------------------------------------- | ------------------------------------- |
| `AUTH_TYPE`              | Set to `"ONELOGIN"` to enable OneLogin authentication | `ONELOGIN`                            |
| `ONELOGIN_ISSUER`        | The issuer URL for your OneLogin application          | `https://company.onelogin.com/oidc/2` |
| `ONELOGIN_CLIENT_ID`     | Client ID of your OneLogin application                | `abc123def456ghi789`                  |
| `ONELOGIN_CLIENT_SECRET` | Client Secret of your OneLogin application            | `abcd1234efgh5678ijkl9012`            |

## OneLogin Configuration

### Creating a OneLogin Application

1. Sign in to your OneLogin Admin Console
2. Navigate to **Applications**
3. Click **Add App**
4. Search for **OpenId Connect (OIDC)** and select it
5. Click **Save**

### Application Settings

1. **Display Name**: Enter a name for your application (e.g., "Keep")
2. **Redirect URIs**: Enter your app's callback URL, e.g., `https://your-keep-domain.com/api/auth/callback/onelogin`
3. **Login URL**: Enter your app's login URL, e.g., `https://your-keep-domain.com/signin`
4. **Role Mapping**:
   * Go to the Parameters tab
   * Map the groups to user roles or groups with the default value being semicolon delimited input values
5. Go to the **SSO** tab and configure:
   * **Application Type**: Web
   * **Token Endpoint**: Client Secret Post
6. **Access**:
   * Assign to appropriate roles or users
7. Click **Save**
8. Copy the client id, client secret and issuer URL from the SSO tab


# Overview
Source: https://docs.keephq.dev/deployment/authentication/overview



For every authentication-related question or issue, please join our [Slack](https://slack.keephq.dev).

Keep supports various authentication providers and architectures to accommodate different deployment strategies and security needs, from development environments to production setups.

### Authentication Providers

* [**No Authentication**](/deployment/authentication/no-auth) - Quick setup for testing or internal use cases.
* [**DB**](/deployment/authentication/db-auth) - Simple username/password authentication. Works well for small teams or for dev/stage environments. Users and hashed password are stored on DB.
* [**Auth0**](/deployment/authentication/auth0-auth) - Utilize Auth0 for scalable, auth0-based authentication.
* [**Keycloak**](/deployment/authentication/keycloak-auth) - Utilize Keycloak for enterprise authentication methods such as SSO/SAML/OIDC, advanced RBAC with custom roles, resource-level permissions, and integration with user directories (LDAP).
* [**AzureAD**](/deployment/authentication/azuread-auth) - Utilize Azure AD for SSO/SAML/OIDC nterprise authentication.
* [**Okta**](/deployment/authentication/okta-auth) - Utilize Okta for SSO/OIDC authentication.
* [**OneLogin**](/deployment/authentication/onelogin-auth) - Utilize OneLogin for SSO/OIDC authentication.

Choosing the right authentication strategy depends on your specific use case, security requirements, and deployment environment. You can read more about each authentication provider.

### Authentication Features Comparison

| Identity Provider |            RBAC            | SAML/OIDC/SSO | LDAP | Resource-based permission | User Management | Group Management | On Prem | License |
| :---------------: | :------------------------: | :-----------: | :--: | :-----------------------: | :-------------: | :--------------: | :-----: | :-----: |
|    **No Auth**    |              ❌             |       ❌       |   ❌  |             ❌             |        ❌        |         ❌        |    ✅    | **OSS** |
|       **DB**      | ✅ 
(Predefiend roles) |       ❌       |   ❌  |             ✅             |        ✅        |         ❌        |    ✅    | **OSS** |
|     **Auth0**     | ✅ 
(Predefiend roles) |       ✅       |  🚧  |             🚧            |        ✅        |        🚧        |    ❌    |  **EE** |
|    **Keycloak**   |   ✅ 
(Custom roles)   |       ✅       |   ✅  |             ✅             |        ✅        |         ✅        |    ✅    |  **EE** |
|  **Oauth2Proxy**  | ✅ 
(Predefiend roles) |       ✅       |   ❌  |             ❌             |       N/A       |        N/A       |    ✅    | **OSS** |
|    **Azure AD**   | ✅ 
(Predefiend roles) |       ✅       |   ❌  |             ❌             |   By Azure AD   |    By Azure AD   |    ✅    |  **EE** |
|      **Okta**     | ✅ 
(Predefiend roles) |       ✅       |   ❌  |             ✅             |        ❌        |         ❌        |    ✅    | **OSS** |
|    **OneLogin**   | ✅ 
(Predefiend roles) |       ✅       |   ❌  |             ✅             |        ❌        |         ❌        |    ✅    | **OSS** |

### How To Configure


  Some authentication providers require additional environment variables. These will be covered in detail on the specific authentication provider pages.


The authentication scheme on Keep is controlled with environment variables both on the backend (Keep API) and the frontend (Keep UI).

| Identity Provider | Environment Variable    | Additional Variables Required                                                           |
| ----------------- | ----------------------- | --------------------------------------------------------------------------------------- |
| **No Auth**       | `AUTH_TYPE=NOAUTH`      | None                                                                                    |
| **DB**            | `AUTH_TYPE=DB`          | `KEEP_JWT_SECRET`                                                                       |
| **Auth0**         | `AUTH_TYPE=AUTH0`       | `AUTH0_DOMAIN`, `AUTH0_CLIENT_ID`, `AUTH0_CLIENT_SECRET`                                |
| **Keycloak**      | `AUTH_TYPE=KEYCLOAK`    | `KEYCLOAK_URL`, `KEYCLOAK_REALM`, `KEYCLOAK_CLIENT_ID`, `KEYCLOAK_CLIENT_SECRET`        |
| **Oauth2Proxy**   | `AUTH_TYPE=OAUTH2PROXY` | `OAUTH2_PROXY_USER_HEADER`, `OAUTH2_PROXY_ROLE_HEADER`, `OAUTH2_PROXY_AUTO_CREATE_USER` |
| **AzureAD**       | `AUTH_TYPE=AZUREAD`     | See [AzureAD Configuration](/deployment/authentication/azuread-auth)                    |
| **Okta**          | `AUTH_TYPE=OKTA`        | `OKTA_DOMAIN`, `OKTA_CLIENT_ID`, `OKTA_CLIENT_SECRET`                                   |
| **OneLogin**      | `AUTH_TYPE=ONELOGIN`    | See [OneLogin Configuration](/deployment/authentication/onelogin-auth)                  |

For more details on each authentication strategy, including setup instructions and implications, refer to the respective sections.


# Configuration
Source: https://docs.keephq.dev/deployment/configuration



## Background

Keep is highly configurable through environment variables. This allows you to customize various aspects of both the backend and frontend components without modifying the code. Environment variables can be set in your deployment environment, such as in your Kubernetes configuration, Docker Compose file, or directly on your host system.

## Backend Environment Variables

### General


  General configuration variables control the core behavior of the Keep server.
  These settings determine fundamental aspects such as the server's host, port,
  and whether certain components like the scheduler and consumer are enabled.


|                 Env var                 |                        Purpose                        | Required |          Default Value         |         Valid options        |
| :-------------------------------------: | :---------------------------------------------------: | :------: | :----------------------------: | :--------------------------: |
|              **KEEP\_HOST**             |         Specifies the host for the Keep server        |    No    |            "0.0.0.0"           | Valid hostname or IP address |
|                 **PORT**                |  Specifies the port on which the backend server runs  |    No    |              8080              |     Any valid port number    |
|              **SCHEDULER**              |       Enables or disables the workflow scheduler      |    No    |             "true"             |       "true" or "false"      |
|               **CONSUMER**              |            Enables or disables the consumer           |    No    |             "true"             |       "true" or "false"      |
|            **KEEP\_VERSION**            |               Specifies the Keep version              |    No    |            "unknown"           |     Valid version string     |
|            **KEEP\_API\_URL**           |               Specifies the Keep API URL              |    No    | Constructed from HOST and PORT |           Valid URL          |
|       **KEEP\_STORE\_RAW\_ALERTS**      |             Enables storing of raw alerts             |    No    |             "false"            |       "true" or "false"      |
| **TENANT\_CONFIGURATION\_RELOAD\_TIME** |    Time in minutes to reload tenant configurations    |    No    |                5               |       Positive integer       |
|        **KEEP\_LIVE\_DEMO\_MODE**       | Keep will simulate incoming alerts and other activity |    No    |             "false"            |       "true" or "false"      |

### Logging and Environment


  Logging and environment configuration determines how Keep generates and
  formats log output. These settings are crucial for debugging, monitoring, and
  understanding the behavior of your Keep instance in different environments.


|         Env var        |                         Purpose                         | Required |   Default Value   |                  Valid options                  |
| :--------------------: | :-----------------------------------------------------: | :------: | :---------------: | :---------------------------------------------: |
|     **LOG\_LEVEL**     |        Sets the logging level for the application       |    No    |       "INFO"      | "DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL" |
|     **ENVIRONMENT**    | Specifies the environment the application is running in |    No    |    "production"   |      "development", "staging", "production"     |
|     **LOG\_FORMAT**    |                 Specifies the log format                |    No    | "open\_telemetry" |        "open\_telemetry", "dev\_terminal"       |
| **LOG\_AUTH\_PAYLOAD** |        Enables logging of authentication payload        |    No    |      "false"      |                "true" or "false"                |

### Database


  Database configuration is crucial for Keep's data persistence. Keep supports
  various database backends through SQLAlchemy, allowing flexibility in choosing
  and configuring your preferred database system.


|              Env var             |                      Purpose                      | Required |           Default Value           |            Valid options           |
| :------------------------------: | :-----------------------------------------------: | :------: | :-------------------------------: | :--------------------------------: |
| **DATABASE\_CONNECTION\_STRING** |       Specifies the database connection URL       |    Yes   |                None               | Valid SQLAlchemy connection string |
|     **DATABASE\_POOL\_SIZE**     |       Sets the database connection pool size      |    No    |                 5                 |          Positive integer          |
|    **DATABASE\_MAX\_OVERFLOW**   | Sets the maximum overflow for the connection pool |    No    |                 10                |          Positive integer          |
|        **DATABASE\_ECHO**        |     Enables SQLAlchemy echo mode for debugging    |    No    |               False               |        Boolean (True/False)        |
|     **DB\_CONNECTION\_NAME**     |      Specifies the Cloud SQL connection name      |    No    | "keephq-sandbox:us-central1:keep" |  Valid Cloud SQL connection string |
|           **DB\_NAME**           |       Specifies the Cloud SQL database name       |    No    |              "keepdb"             |    Valid Cloud SQL database name   |
|     **DB\_SERVICE\_ACCOUNT**     |     Service account for database impersonation    |    No    |                None               |     Valid service account email    |
|         **DB\_IP\_TYPE**         |          Specifies the Cloud SQL IP type          |    No    |              "public"             |    "public", "private" or "psc"    |
|      **SKIP\_DB\_CREATION**      |       Skips database creation and migrations      |    No    |              "false"              |          "true" or "false"         |

### Resource Provisioning


  Resource provisioning settings control how Keep sets up initial resources.
  This configuration is particularly important for automating the setup process
  and ensuring that necessary resources are available when Keep starts.



  To elaborate on resource provisioning and its configuration, please see
  [provisioning docs](/deployment/provision/overview).


|          Env var         |                  Purpose                  | Required | Default Value |   Valid options   |
| :----------------------: | :---------------------------------------: | :------: | :-----------: | :---------------: |
| **PROVISION\_RESOURCES** | Enables or disables resource provisioning |    No    |     "true"    | "true" or "false" |

### Authentication


  Authentication configuration determines how Keep verifies user identities and
  manages access control. These settings are essential for securing your Keep
  instance and integrating with various authentication providers.



  For specific authentication type configuration, please see [authentication
  docs](/deployment/authentication/overview).


|                  Env var                  |                              Purpose                              | Required | Default Value |                              Valid options                             |
| :---------------------------------------: | :---------------------------------------------------------------: | :------: | :-----------: | :--------------------------------------------------------------------: |
|               **AUTH\_TYPE**              |                 Specifies the authentication type                 |    No    |    "NOAUTH"   | "AUTH0", "KEYCLOAK", "DB", "NOAUTH", "OAUTH2PROXY", "OKTA", "ONELOGIN" |
|           **KEEP\_JWT\_SECRET**           | Secret key for JWT token generation and validation (DB auth only) |    Yes   |      None     |                        Any strong secret string                        |
|        **KEEP\_DEFAULT\_USERNAME**        |         Default username for the admin user (DB auth only)        |    No    |     "keep"    |                        Any valid username string                       |
|        **KEEP\_DEFAULT\_PASSWORD**        |         Default password for the admin user (DB auth only)        |    No    |     "keep"    |                       Any strong password string                       |
| **KEEP\_FORCE\_RESET\_DEFAULT\_PASSWORD** |               Forces reset of default user password               |    No    |    "false"    |                            "true" or "false"                           |
|        **KEEP\_DEFAULT\_API\_KEYS**       |       Comma-separated list of default API keys to provision       |    No    |       ""      |               Format: "name:role:secret,name:role:secret"              |

### Service Mesh (Internal Alert Ingestion)


  These settings allow trusted services within the same Kubernetes cluster to
  POST alerts to Keep without requiring a Keep API key. This is intended for
  service-to-service communication where network-level authentication (e.g.
  Istio mTLS with AuthorizationPolicy) ensures only authorized callers can
  reach Keep's alert ingestion endpoints.


|                 Env var                 |                               Purpose                              | Required | Default Value |   Valid options   |
| :-------------------------------------: | :----------------------------------------------------------------: | :------: | :-----------: | :---------------: |
| **KEEP\_ALLOW\_MESH\_ALERT\_INGESTION** | Allows unauthenticated POST requests to `/alerts/event*` endpoints |    No    |    "false"    | "true" or "false" |

When `KEEP_ALLOW_MESH_ALERT_INGESTION` is set to `"true"`, requests to `/alerts/event*` that do not carry an API key or bearer token are accepted and authenticated as an internal service with the `webhook` role.

Calling services can optionally set the `X-Service-Name` HTTP header to identify themselves in Keep's logs and audit trail:

```bash theme={null}
curl -X POST http://keep-backend:8080/alerts/event \
  -H "Content-Type: application/json" \
  -H "X-Service-Name: my-service" \
  -d '[{"id":"alert-1","name":"Example Alert","severity":"info","status":"firing","source":["my-service"]}]'
```

The authenticated entity will have:

* **email**: `service:` (defaults to `service:unknown` if the header is not set)
* **role**: `webhook` (grants `write:alert` and `write:incident` scopes)


  This feature bypasses API key authentication for the alert ingestion
  endpoints. You **must** pair it with network-level access control (such as
  Istio AuthorizationPolicy) to restrict which services can reach these
  endpoints. Without network-level enforcement, any client that can reach
  Keep's backend can POST alerts.


### Secrets Management


  Secrets Management configuration specifies how Keep handles sensitive
  information. This is crucial for securely storing and accessing confidential
  data such as API keys and integrations credentials.


|             Env var            |                                Purpose                                | Required | Default Value |            Valid options            |
| :----------------------------: | :-------------------------------------------------------------------: | :------: | :-----------: | :---------------------------------: |
|    **SECRET\_MANAGER\_TYPE**   |               Defines the type of secret manager to use               |    Yes   |     "FILE"    | "FILE", "GCP", "K8S", "VAULT", "DB" |
| **SECRET\_MANAGER\_DIRECTORY** | Directory for storing secrets when using file-based secret management |    No    |    "/state"   |       Any valid directory path      |

### OpenTelemetry


  OpenTelemetry configuration enables comprehensive observability for Keep.
  These settings allow you to integrate Keep with various monitoring and tracing
  systems, enhancing your ability to debug and optimize performance.


|                   Env var                   |                     Purpose                    | Required | Default Value |       Valid options       |
| :-----------------------------------------: | :--------------------------------------------: | :------: | :-----------: | :-----------------------: |
|           **OTEL\_SERVICE\_NAME**           |           OpenTelemetry service name           |    No    |   "keep-api"  | Valid service name string |
|              **SERVICE\_NAME**              |       Alternative for OTEL\_SERVICE\_NAME      |    No    |   "keep-api"  | Valid service name string |
|      **OTEL\_EXPORTER\_OTLP\_ENDPOINT**     |        OpenTelemetry collector endpoint        |    No    |      None     |         Valid URL         |
|              **OTLP\_ENDPOINT**             | Alternative for OTEL\_EXPORTER\_OTLP\_ENDPOINT |    No    |      None     |         Valid URL         |
|  **OTEL\_EXPORTER\_OTLP\_TRACES\_ENDPOINT** |          OpenTelemetry traces endpoint         |    No    |      None     |         Valid URL         |
|   **OTEL\_EXPORTER\_OTLP\_LOGS\_ENDPOINT**  |           OpenTelemetry logs endpoint          |    No    |      None     |         Valid URL         |
| **OTEL\_EXPORTER\_OTLP\_METRICS\_ENDPOINT** |         OpenTelemetry metrics endpoint         |    No    |      None     |         Valid URL         |
|          **CLOUD\_TRACE\_ENABLED**          |       Enables Google Cloud Trace exporter      |    No    |    "false"    |     "true" or "false"     |
|          **METRIC\_OTEL\_ENABLED**          |          Enables OpenTelemetry metrics         |    No    |       ""      |     "true" or "false"     |

### WebSocket Server (Pusher/Soketi)


  WebSocket server configuration controls real-time communication capabilities
  in Keep. These settings are important for enabling features that require
  instant updates and notifications.


|         Env var         |              Purpose              |        Required       | Default Value |         Valid options        |
| :---------------------: | :-------------------------------: | :-------------------: | :-----------: | :--------------------------: |
|   **PUSHER\_DISABLED**  |    Disables Pusher integration    |           No          |    "false"    |       "true" or "false"      |
|     **PUSHER\_HOST**    |   Hostname of the Pusher server   |           No          |      None     | Valid hostname or IP address |
|     **PUSHER\_PORT**    |     Port of the Pusher server     |           No          |      None     |     Any valid port number    |
|   **PUSHER\_APP\_ID**   |       Pusher application ID       | Yes (if using Pusher) |      None     |      Valid Pusher App ID     |
|   **PUSHER\_APP\_KEY**  |       Pusher application key      | Yes (if using Pusher) |      None     |     Valid Pusher App Key     |
| **PUSHER\_APP\_SECRET** |     Pusher application secret     | Yes (if using Pusher) |      None     |    Valid Pusher App Secret   |
|   **PUSHER\_USE\_SSL**  | Enables SSL for Pusher connection |           No          |     False     |     Boolean (True/False)     |
|   **PUSHER\_CLUSTER**   |           Pusher cluster          |           No          |      None     |   Valid Pusher cluster name  |

### OpenAI


  OpenAI configuration is used for integrating with OpenAI services. These
  settings are important if you're utilizing OpenAI capabilities within Keep for
  tasks such as natural language processing or AI-assisted operations.


|               Env var              |                                                                   Purpose                                                                  | Required |    Default Value    |                            Valid options                           | Backend/Frontend |
| :--------------------------------: | :----------------------------------------------------------------------------------------------------------------------------------------: | :------: | :-----------------: | :----------------------------------------------------------------: | :--------------: |
|        **OPENAI\_API\_KEY**        |                                                         API key for OpenAI services                                                        |    No    |         None        |                        Valid OpenAI API key                        |       Both       |
|       **OPENAI\_MODEL\_NAME**      |                                                    Model name to use for OpenAI requests                                                   |    No    | "gpt-4o-2024-08-06" |    Valid OpenAI model name (e.g., "gpt-4o", "gpt-4o-mini", ...)    |       Both       |
|   **OPEN\_AI\_ORGANIZATION\_ID**   |                                                     Organization ID for OpenAI services                                                    |    No    |         None        |                    Valid OpenAI organization ID                    |       Both       |
|        **OPENAI\_BASE\_URL**       |                                             Base URL for OpenAI API (useful for LiteLLM proxy)                                             |    No    |         None        | Valid URL (e.g., "[http://localhost:4000](http://localhost:4000)") |       Both       |
| **KEEP\_AI\_DISABLE\_TEMPERATURE** | Omit the `temperature` parameter from AI requests (some models, e.g. reasoning models, only accept the default and reject explicit values) |    No    |        false        |                            true / false                            |      Backend     |


  For various different LLM based features, we also require to set these
  environment variables for Keep's frontend too.


### Posthog


  Posthog configuration controls Keep's integration with the Posthog analytics
  platform. These settings are useful for tracking usage patterns and gathering
  insights about how your Keep instance is being used.


|        Env var        |            Purpose            | Required |                    Default Value                   |     Valid options     |
| :-------------------: | :---------------------------: | :------: | :------------------------------------------------: | :-------------------: |
| **POSTHOG\_API\_KEY** | API key for PostHog analytics |    No    | "phc\_muk9qE3TfZsX3SZ9XxX52kCGJBclrjhkP9JxAQcm1PZ" | Valid PostHog API key |
| **POSTHOG\_DISABLED** |  Disables PostHog integration |    No    |                       "false"                      |   "true" or "false"   |

### Sentry


  Sentry configuration controls Keep's integration with Sentry for error
  monitoring and reporting. These settings are important for maintaining the
  stability and reliability of your Keep instance.


|        Env var       |           Purpose           | Required | Default Value |   Valid options   |
| :------------------: | :-------------------------: | :------: | :-----------: | :---------------: |
| **SENTRY\_DISABLED** | Disables Sentry integration |    No    |    "false"    | "true" or "false" |

### Ngrok


  Ngrok configuration enables secure tunneling to your Keep instance. These
  settings are particularly useful for development or when you need to expose
  your local Keep instance to the internet securely.


|         Env var        |             Purpose            | Required | Default Value |      Valid options     |
| :--------------------: | :----------------------------: | :------: | :-----------: | :--------------------: |
|     **USE\_NGROK**     |   Enables ngrok for tunneling  |    No    |    "false"    |    "true" or "false"   |
| **NGROK\_AUTH\_TOKEN** | Authentication token for ngrok |    No    |      None     | Valid ngrok auth token |
|    **NGROK\_DOMAIN**   |     Custom domain for ngrok    |    No    |      None     |    Valid domain name   |

### Elasticsearch


  Elasticsearch configuration controls Keep's integration with Elasticsearch for
  advanced search capabilities. These settings are important if you're using
  Elasticsearch to enhance Keep's search functionality and performance.


|           Env var          |                   Purpose                   |           Required           | Default Value |         Valid options         |
| :------------------------: | :-----------------------------------------: | :--------------------------: | :-----------: | :---------------------------: |
|    **ELASTIC\_ENABLED**    |      Enables Elasticsearch integration      |              No              |    "false"    |       "true" or "false"       |
|    **ELASTIC\_API\_KEY**   |          API key for Elasticsearch          | Yes (if using Elasticsearch) |      None     |  Valid Elasticsearch API key  |
|     **ELASTIC\_HOSTS**     | Comma-separated list of Elasticsearch hosts | Yes (if using Elasticsearch) |      None     | Valid Elasticsearch host URLs |
|      **ELASTIC\_USER**     |    Username for Elasticsearch basic auth    |              No              |      None     |         Valid username        |
|    **ELASTIC\_PASSWORD**   |    Password for Elasticsearch basic auth    |              No              |      None     |         Valid password        |
| **ELASTIC\_INDEX\_SUFFIX** |     Suffix for Elasticsearch index names    |    Yes (for single tenant)   |      None     |        Any valid string       |

### Redis


  Redis configuration specifies the connection details for Keep's Redis
  instance. Redis is used for various caching and queueing purposes, making
  these settings important for optimizing Keep's performance and scalability.


|       Env var       |        Purpose        | Required | Default Value |         Valid options        |
| :-----------------: | :-------------------: | :------: | :-----------: | :--------------------------: |
|      **REDIS**      |     Redis enabled     |    No    |     false     |         true or false        |
|   **REDIS\_HOST**   | Redis server hostname |    No    |  "localhost"  | Valid hostname or IP address |
|   **REDIS\_PORT**   |   Redis server port   |    No    |      6379     |       Valid port number      |
|    **REDIS\_DB**    |  Redis database slot  |    No    |       0       |        Valid DB number       |
| **REDIS\_USERNAME** |     Redis username    |    No    |      None     |     Valid username string    |
| **REDIS\_PASSWORD** |     Redis password    |    No    |      None     |     Valid password string    |

### Redis Sentinel


  Redis sentinel configuration specifies the connection details for Keep's Redis sentinel
  instance. Redis sentinel is used when you have a redis cluster and it acts as a broker.


|               Env var              |           Purpose           | Required |   Default Value   |                Valid options                |
| :--------------------------------: | :-------------------------: | :------: | :---------------: | :-----------------------------------------: |
|              **REDIS**             |        Redis enabled        |    No    |       false       |                true or false                |
|     **REDIS\_SENTINEL\_HOSTS**     |   Redis sentinel server(s)  |    No    | "localhost:26379" | "host1:port1,host2:port2" (comma-separated) |
| **REDIS\_SENTINEL\_SERVICE\_NAME** | Redis sentinel service name |    No    |     "mymaster"    |          Valid service name string          |
|            **REDIS\_DB**           |     Redis database slot     |    No    |         0         |               Valid DB number               |
|         **REDIS\_USERNAME**        |        Redis username       |    No    |        None       |            Valid username string            |
|         **REDIS\_PASSWORD**        |        Redis password       |    No    |        None       |            Valid password string            |

### ARQ


  ARQ (Asynchronous Task Queue) configuration controls Keep's background task
  processing. These settings are crucial for managing how Keep handles
  long-running or scheduled tasks, ensuring efficient resource utilization and
  responsiveness.


|             Env var            |                       Purpose                       | Required | Default Value |     Valid options    |
| :----------------------------: | :-------------------------------------------------: | :------: | :-----------: | :------------------: |
| **ARQ\_BACKGROUND\_FUNCTIONS** | Comma-separated list of background functions to run |    No    |      None     | Valid function names |
|      **ARQ\_KEEP\_RESULT**     |      Duration to keep job results (in seconds)      |    No    |      3600     |   Positive integer   |
|        **ARQ\_EXPIRES**        |       Default job expiration time (in seconds)      |    No    |      3600     |   Positive integer   |
|      **ARQ\_EXPIRES\_AI**      |         AI job expiration time (in seconds)         |    No    |    3600000    |   Positive integer   |

### Rate Limiting


  Rate limiting configuration controls how many requests can be made to Keep's
  API endpoints within a specified time period. This helps prevent abuse and
  ensures system stability.


|            Env var           |                Purpose                | Required | Default Value |                            Valid options                            |
| :--------------------------: | :-----------------------------------: | :------: | :-----------: | :-----------------------------------------------------------------: |
|    **KEEP\_USE\_LIMITER**    |   Enables or disables rate limiting   |    No    |    "false"    |                          "true" or "false"                          |
| **KEEP\_LIMIT\_CONCURRENCY** | Sets the rate limit for API endpoints |    No    |  "100/minute" | Format: "/" where interval can be "second", "minute", "hour", "day" |


  Currently, rate limiting is applied to the following endpoints:

  * POST `/alerts/event` - Generic event ingestion endpoint
  * POST `/alerts/{provider_type}` - Provider-specific event ingestion endpoints

  These endpoints are rate-limited according to the `KEEP_LIMIT_CONCURRENCY` setting when `KEEP_USE_LIMITER` is enabled.


### Maintenance Windows


  The strategy enables the ability to manage how the alerts are handled
  in case of a match with the Maintenance Windows Rules.


|              Env var              |                   Purpose                   | Required | Default Value |               Valid options              |
| :-------------------------------: | :-----------------------------------------: | :------: | :-----------: | :--------------------------------------: |
| **MAINTENANCE\_WINDOW\_STRATEGY** |             Choose the strategy             |    No    |   "default"   | "default" or "recover\_previous\_status" |
|     **WATCHER\_LAPSED\_TIME**     | Time in seconds to execute the alert review |    No    |       60      |          Valid positive integer          |

## Frontend Environment Variables


  Frontend configuration variables control the behavior and features of Keep's
  user interface. These settings are crucial for customizing the frontend's
  appearance, functionality, and integration with the backend services.


### General

| Env var                               | Purpose                                                             | Required | Default Value                                                                         | Valid options        |
| ------------------------------------- | ------------------------------------------------------------------- | -------- | ------------------------------------------------------------------------------------- | -------------------- |
| **API\_URL**                          | Specifies the URL of the Keep backend API                           | Yes      | None                                                                                  | Valid URL            |
| **AUTH\_SESSION\_TIMEOUT**            | Specifies user session timeout in seconds. Default is 30 days.      | No       | 2592000                                                                               | Value in seconds     |
| **KEEP\_HIDE\_SENSITIVE\_FIELDS**     | Hides sensitive fields                                              | No       | None                                                                                  | "true", "false"      |
| **HIDE\_NAVBAR\_CORRELATION**         | Hides the correlation page from the navigation bar in the UI        | No       | None                                                                                  | "true"               |
| **HIDE\_NAVBAR\_WORKFLOWS**           | Hides the workflows page from the navigation bar in the UI          | No       | None                                                                                  | "true"               |
| **HIDE\_NAVBAR\_SERVICE\_TOPOLOGY**   | Hides the service topology page from the navigation bar in the UI   | No       | None                                                                                  | "true"               |
| **HIDE\_NAVBAR\_MAPPING**             | Hides the mapping page from the navigation bar in the UI            | No       | None                                                                                  | "true"               |
| **HIDE\_NAVBAR\_EXTRACTION**          | Hides the extraction page from the navigation bar in the UI         | No       | None                                                                                  | "true"               |
| **HIDE\_NAVBAR\_MAINTENANCE\_WINDOW** | Hides the maintenance window page from the navigation bar in the UI | No       | None                                                                                  | "true"               |
| **HIDE\_NAVBAR\_AI\_PLUGINS**         | Hides the AI plugins page from the navigation bar in the UI         | No       | None                                                                                  | "true"               |
| **KEEP\_WF\_LIST\_EXTENDED\_INFO**    | Use a list instead a button to show the complete execution list     | No       | "true"                                                                                | "true", "false"      |
| **ALERT\_SIDEBAR\_FIELDS**            | Specifies a list of fields to show in the sidebar.                  | No       | service,source,description,message,fingerprint,url,incidents,timeline,relatedServices | Valid list of fields |

### Authentication


  Authentication configuration determines how Keep verifies user identities and
  manages access control. These settings are essential for securing your Keep
  instance and integrating with various authentication providers.


|        Env var       |              Purpose              | Required | Default Value |                              Valid options                             |
| :------------------: | :-------------------------------: | :------: | :-----------: | :--------------------------------------------------------------------: |
|    **AUTH\_TYPE**    | Specifies the authentication type |    No    |    "NOAUTH"   | "AUTH0", "KEYCLOAK", "DB", "NOAUTH", "OAUTH2PROXY", "OKTA", "ONELOGIN" |
|   **NEXTAUTH\_URL**  |  URL for NextAuth authentication  |    Yes   |      None     |                                Valid URL                               |
| **NEXTAUTH\_SECRET** |      Secret key for NextAuth      |    Yes   |      None     |                          Strong secret string                          |

### Posthog

|      Env var      |                 Purpose                | Required | Default Value |     Valid options     |
| :---------------: | :------------------------------------: | :------: | :-----------: | :-------------------: |
|  **POSTHOG\_KEY** | PostHog API key for frontend analytics |    No    |      None     | Valid PostHog API key |
| **POSTHOG\_HOST** |   PostHog Host for frontend analytics  |    No    |      None     |   Valid PostHog Host  |

### Pusher


  Pusher configuration is essential for enabling real-time updates and
  communication in Keep's frontend. These settings allow the frontend to
  establish a WebSocket connection with the Pusher server, facilitating instant
  updates and notifications.


|        Env var       |            Purpose            |         Required        | Default Value |         Valid options        |
| :------------------: | :---------------------------: | :---------------------: | :-----------: | :--------------------------: |
| **PUSHER\_DISABLED** |  Disables Pusher integration  |            No           |    "false"    |       "true" or "false"      |
|   **PUSHER\_HOST**   | Hostname of the Pusher server |            No           |  "localhost"  | Valid hostname or IP address |
|   **PUSHER\_PORT**   |   Port of the Pusher server   |            No           |      6001     |       Valid port number      |
| **PUSHER\_APP\_KEY** |     Pusher application key    | Yes (if Pusher enabled) |  "keepappkey" |     Valid Pusher App Key     |
|  **PUSHER\_CLUSTER** |         Pusher cluster        |            No           |      None     |   Valid Pusher cluster name  |


# Docker
Source: https://docs.keephq.dev/deployment/docker



### Spin up Keep with docker-compose latest images

The easiest way to start keep is is with docker-compose:

```shell theme={null}
curl https://raw.githubusercontent.com/keephq/keep/main/start.sh | sh
```

```bash start.sh theme={null}
#!/bin/bash
# Keep install script for docker compose
set -e

echo "Creating state directory."
mkdir -p state
test -e state || echo "Unable to create folder"
echo "Changing directory ownership to non-privileged user."
chown -R 999:999 state || echo "Unable to change directory ownership, changing permissions instead." && chmod -R 0777 state
which curl &> /dev/null || echo "curl not installed"
curl https://raw.githubusercontent.com/keephq/keep/main/docker-compose.yml --output docker-compose.yml
curl https://raw.githubusercontent.com/keephq/keep/main/docker-compose.common.yml --output docker-compose.common.yml

docker compose up -d
```

The docker-compose.yml contains 3 services:

* [keep-backend](https://console.cloud.google.com/artifacts/docker/keephq/us-central1/keep/keep-api?project=keephq) - a fastapi service that as the API server.
* [keep-frontend](https://console.cloud.google.com/artifacts/docker/keephq/us-central1/keep/keep-ui?project=keephq) - a nextjs app that serves as Keep UI interface.
* [keep-websocket-server](https://docs.soketi.app/getting-started/installation/docker) - Soketi (a pusher compatible websocket server) for real time alerting.

### Reinstall Keep with the option to refresh from scratch

`Caution:` This usage context will refresh from the beginning and Keep's data and settings will be erased. Even other containers on this host are also erased. So please consider when using the steps below.

For cases where you need to test many different options or simply want to reinstall Keep from scratch using docker compose without spending a lot of time, that is, without repeating the steps of installing docker, downloading the installer.. .. run the commands according to the previous instructions.

Follow these steps

#### Step1: Stop, Clear container, network, volume, image.

In the directory containing the docker compose file you downloaded, say `/root/`

```
docker-compose down

docker-compose down --rmi all

docker-compose down -v

docker system prune -a --volumes
```

#### Step2: Clear Config db, config file in state folder.

```
rm -rf state/*

```

#### Step 3: Run again

```
docker compose up -d
```


# AWS ECS
Source: https://docs.keephq.dev/deployment/ecs



## Step 1: Login to AWS Console

* Open your web browser and navigate to the AWS Management Console.
* Log in using your AWS account credentials.

## Step 2: Navigate to ECS

* Click on the "Services" dropdown menu in the top left corner.
* Select "ECS" from the list of services.

## Step 3: Create 3 Task Definitions

* In the ECS dashboard, navigate to the "Task Definitions" section in the left sidebar.
  Task Definition
* Click on "Create new Task Definition".

  Create new task definition

  ### Task Definition 1 (Frontend - KeepUI):

  * Task Definition Family: keep-frontend
    Task Definition Family
  * Configure your container definitions as below:
    * Infrastructure Requirements:
      * Launch Type: AWS Fargate
      * OS, Architecture, Network mode: Linux/X86\_64
      * Task Size:
        * CPU: 1 vCPU
        * Memory: 2 GB
      * Task Role and Task Execution Role are optional if you plan on using secrets manager for example then create a task execution role to allow access to the secret manager you created.
        Infrastructure Requirements
    * Container Details:
      * Name: keep-frontend
      * Image URI: us-central1-docker.pkg.dev/keephq/keep/keep-api:latest
      * Ports Mapping:
        * Container Port: 3000
        * Protocol: TCP
          Container Details
      * Environment Variables: (This can be static or you can use parameter store or secrets manager)
        * DATABASE\_CONNECTION\_STRING
        * AUTH\_TYPE
        * KEEP\_JWT\_SECRET
        * KEEP\_DEFAULT\_USERNAME
        * KEEP\_DEFAULT\_PASSWORD
        * SECRET\_MANAGER\_TYPE
        * SECRET\_MANAGER\_DIRECTORY
        * USE\_NGROK
        * KEEP\_API\_URL
          (The below variable is optional if you don't want to use websocket)
        * PUSHER\_DISABLED
          (The below variables are optional if you want to use websocket)
        * PUSHER\_APP\_ID
        * PUSHER\_APP\_KEY
        * PUSHER\_APP\_SECRET
        * PUSHER\_HOST
        * PUSHER\_PORT
          Environment Variables
    * Review and create your task definition.

  ### Task Definition 2 (Backend - keepAPI):

  * Configure your container definitions as below:
    * Task Definition Family: keep-frontend
      Task Definition Family
    * Infrastructure Requirements:
      * Launch Type: AWS Fargate
      * OS, Architecture, Network mode: Linux/X86\_64
      * Task Size:
        * CPU: 1 vCPU
        * Memory: 2 GB
      * Task Role and Task Execution Role are optional if you plan on using secrets manager for example then create a task execution role to allow access to the secret manager you created.
        Infrastructure Requirements
    * Container Details:
      * Name: keep-backend
      * Image URI: us-central1-docker.pkg.dev/keephq/keep/keep-api:latest
      * Ports Mapping:
        * Container Port: 8080
        * Protocol: TCP
          Container Details
      * Environment Variables: (This can be static or you can use parameter store or secrets manager)
        * DATABASE\_CONNECTION\_STRING
        * AUTH\_TYPE
        * KEEP\_JWT\_SECRET
        * KEEP\_DEFAULT\_USERNAME
        * KEEP\_DEFAULT\_PASSWORD
        * SECRET\_MANAGER\_TYPE
        * SECRET\_MANAGER\_DIRECTORY
        * USE\_NGROK
        * KEEP\_API\_URL
          (The below variable is optional if you don't want to use websocket)
        * PUSHER\_DISABLED
          (The below variables are optional if you want to use websocket)
        * PUSHER\_APP\_ID
        * PUSHER\_APP\_KEY
        * PUSHER\_APP\_SECRET
        * PUSHER\_HOST
        * PUSHER\_PORT
          Environment Variables
    * Storage:
      * Volume Name: keep-efs
      * Configuration Type: Configure at task definition creation
      * Volume type: EFS
      * Storage configurations:
        * File system ID: Select an existing EFS filesystem or create a new one
        * Root Directory: /
          Volume Configuration
      * Container mount points:
        * Container: select the container you just created
        * Source volume: keep-efs
        * Container path: /app
        * Make sure that Readonly is not selected
          Container Mount
    * Review and create your task definition.

  ### Task Definition 3 (Websocket): (This step is optional if you want to have automatic refresh of the alerts feed)

  * Configure your container definitions as below:
    * Task Definition Family: keep-frontend
      Task Definition Family
    * Infrastructure Requirements:
      * Launch Type: AWS Fargate
      * OS, Architecture, Network mode: Linux/X86\_64
      * Task Size:
        * CPU: 0.25 vCPU
        * Memory: 1 GB
      * Task Role and Task Execution Role are optional if you plan on using secrets manager for example then create a task execution role to allow access to the secret manager you created.
        Infrastructure Requirements
    * Container Details:
      * Name: keep-websocket
      * Image URI: quay.io/soketi/soketi:1.4-16-debian
      * Ports Mapping:
        * Container Port: 6001
        * Protocol: TCP
          Container Details
      * Environment Variables: (This can be static or you can use parameter store or secrets manager)
        * SOKETI\_DEBUG
        * SOKETI\_DEFAULT\_APP\_ID
        * SOKETI\_DEFAULT\_APP\_KEY
        * SOKETI\_DEFAULT\_APP\_SECRET
        * SOKETI\_USER\_AUTHENTICATION\_TIMEOUT
          Environment Variables
  * Review and create your task definition.

## Step 4: Create Keep Service

* In the ECS dashboard, navigate to the "Clusters" section in the left sidebar.
* Select the cluster you want to deploy your service to.
* Click on the "Create" button next to "Services".
* Configure your service settings.
* Review and create your service.

## Step 5: Monitor Your Service

* Once your service is created, monitor its status in the ECS dashboard.
* You can view task status, service events, and other metrics to ensure your service is running correctly.


# Architecture
Source: https://docs.keephq.dev/deployment/kubernetes/architecture



## High Level Architecture

Keep architecture composes of two main components:

1. **Keep API** - A FastAPI-based backend server that handles business logic and API endpoints.
2. **Keep Frontend** -  A Next.js-based frontend interface for user interaction.
3. **Websocket Server** - A Soketi server for real-time updates without page refreshes.
4. **Database Server** - A database used to store and manage persistent data. Supported databases include SQLite, PostgreSQL, MySQL, and SQL Server.

## Kubernetes Architecture

Keep uses a single unified NGINX ingress controller to route traffic to all components (frontend, backend, and websocket). The ingress handles path-based routing:

By default:

* `/` routed to **Frontend** (configurable via `global.ingress.frontendPrefix`)
* `/v2` routed to **Backend** (configurable via `global.ingress.backendPrefix`)
* `/websocket` routed to **WebSocket** (configurable via `global.ingress.websocketPrefix`)

### General Components

Keep uses kubernetes secret manager to store secrets such as integrations credentials.

| Kubernetes Resource |                                                       Purpose                                                      | Required/Optional |                                                                   Source                                                                   |
| :-----------------: | :----------------------------------------------------------------------------------------------------------------: | :---------------: | :----------------------------------------------------------------------------------------------------------------------------------------: |
|    ServiceAccount   | Provides an identity for processes that run in a Pod. Used mainly for Keep API to access kubernetes secret manager |      Required     |              [serviceaccount.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/serviceaccount.yaml)              |
|         Role        |                            Defines permissions for the ServiceAccount to manage secrets                            |      Required     |         [role-secret-manager.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/role-secret-manager.yaml)         |
|     RoleBinding     |                                     Associates the Role with the ServiceAccount                                    |      Required     | [role-binding-secret-manager.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/role-binding-secret-manager.yaml) |
| Secret Deletion Job |                           Cleans up Keep-related secrets when the Helm release is deleted                          |      Required     |           [delete-secret-job.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/delete-secret-job.yaml)           |

### Ingress Component

|  Kubernetes Resource |                     Purpose                     | Required/Optional |                                                     Source                                                     |
| :------------------: | :---------------------------------------------: | :---------------: | :------------------------------------------------------------------------------------------------------------: |
| Shared NGINX Ingress | Routes all external traffic via one entry point |      Optional     | [nginx-ingress.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/nginx-ingress.yaml) |

### Frontend Components

|        Kubernetes Resource       |                            Purpose                            | Required/Optional |                                                        Source                                                        |
| :------------------------------: | :-----------------------------------------------------------: | :---------------: | :------------------------------------------------------------------------------------------------------------------: |
|        Frontend Deployment       |          Manages the frontend application containers          |      Required     |         [frontend.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/frontend.yaml)         |
|         Frontend Service         |       Exposes the frontend deployment within the cluster      |      Required     | [frontend-service.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/frontend-service.yaml) |
|    Frontend Route (OpenShift)    | Exposes the frontend service to external traffic on OpenShift |      Optional     |   [frontend-route.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/frontend-route.yaml)   |
| Frontend HorizontalPodAutoscaler |        Automatically scales the number of frontend pods       |      Optional     |     [frontend-hpa.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/frontend-hpa.yaml)     |

#### Backend Components

|       Kubernetes Resource       |                            Purpose                           |       Required/Optional       |                                                       Source                                                       |
| :-----------------------------: | :----------------------------------------------------------: | :---------------------------: | :----------------------------------------------------------------------------------------------------------------: |
|        Backend Deployment       |          Manages the backend application containers          | Required (if backend enabled) |         [backend.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/backend.yaml)         |
|         Backend Service         |       Exposes the backend deployment within the cluster      | Required (if backend enabled) | [backend-service.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/backend-service.yaml) |
|    Backend Route (OpenShift)    | Exposes the backend service to external traffic on OpenShift |            Optional           |   [backend-route.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/backend-route.yaml)   |
| Backend HorizontalPodAutoscaler |        Automatically scales the number of backend pods       |            Optional           |     [backend-hpa.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/backend-hpa.yaml)     |

#### Database Components

Database components are optional. You can spin up Keep with your own database.

|       Kubernetes Resource      |                          Purpose                         |         Required/Optional        |                                                  Source                                                  |
| :----------------------------: | :------------------------------------------------------: | :------------------------------: | :------------------------------------------------------------------------------------------------------: |
|       Database Deployment      | Manages the database containers (e.g. MySQL or Postgres) |             Optional             |         [db.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/db.yaml)         |
|        Database Service        |    Exposes the database deployment within the cluster    | Required (if deployment enabled) | [db-service.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/db-service.yaml) |
|    Database PersistentVolume   |       Provides persistent storage for the database       |             Optional             |      [db-pv.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/db-pv.yaml)      |
| Database PersistentVolumeClaim |      Claims the persistent storage for the database      |             Optional             |     [db-pvc.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/db-pvc.yaml)     |

#### WebSocket Components

WebSocket components are optional. You can spin up Keep with your own *Pusher compatible* WebSocket server.

|        Kubernetes Resource        |                             Purpose                            |        Required/Optional        |                                                                Source                                                                |
| :-------------------------------: | :------------------------------------------------------------: | :-----------------------------: | :----------------------------------------------------------------------------------------------------------------------------------: |
|        WebSocket Deployment       |        Manages the WebSocket server containers (Soketi)        |             Optional            |         [websocket-server.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/websocket-server.yaml)         |
|         WebSocket Service         |       Exposes the WebSocket deployment within the cluster      | Required (if WebSocket enabled) | [websocket-server-service.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/websocket-server-service.yaml) |
|    WebSocket Route (OpenShift)    | Exposes the WebSocket service to external traffic on OpenShift |             Optional            |   [websocket-server-route.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/websocket-server-route.yaml)   |
| WebSocket HorizontalPodAutoscaler |    Automatically scales the number of WebSocket server pods    |             Optional            |     [websocket-server-hpa.yaml](https://github.com/keephq/helm-charts/blob/main/charts/keep/templates/websocket-server-hpa.yaml)     |

These tables provide a comprehensive overview of the Kubernetes resources used in the Keep architecture, organized by component type. Each table describes the purpose of each resource, indicates whether it's required or optional, and provides a direct link to the source template in the Keep Helm charts GitHub repository.

### Kubernetes Configuration

This sections covers only kubernetes-specific configuration. To learn about Keep-specific configuration, controlled by environment variables, see [Keep Configuration](/deployment/configuration)

Each of these components can be customized via the `values.yaml` file in the Helm chart.

Below are key configurations that can be adjusted for each component.

#### 1. Frontend Configuration

```yaml theme={null}
frontend:
  enabled: true                 # Enable or disable the frontend deployment.
  replicaCount: 1               # Number of frontend replicas.
  image:
    repository: us-central1-docker.pkg.dev/keephq/keep/keep-ui
    pullPolicy: Always          # Image pull policy (Always, IfNotPresent).
    tag: latest
  serviceAccount:
    create: true                # Create a new service account.
    name: ""                    # Service account name (empty for default).
  podAnnotations: {}            # Annotations for frontend pods.
  podSecurityContext: {}        # Security context for the frontend pods.
  securityContext: {}           # Security context for the containers.
  service:
    type: ClusterIP              # Service type (ClusterIP, NodePort, LoadBalancer).
    port: 3000                  # Port on which the frontend service is exposed.
```

#### 2. Backend Configuration

```yaml theme={null}
backend:
  enabled: true                # Enable or disable the backend deployment.
  replicaCount: 1              # Number of backend replicas.
  image:
    repository: us-central1-docker.pkg.dev/keephq/keep/keep-api
    pullPolicy: Always         # Image pull policy (Always, IfNotPresent).
  serviceAccount:
    create: true               # Create a new service account.
    name: ""                   # Service account name (empty for default).
  podAnnotations: {}           # Annotations for backend pods.
  podSecurityContext: {}       # Security context for backend pods.
  securityContext: {}          # Security context for containers.
  service:
    type: ClusterIP      # Service type (ClusterIP, NodePort, LoadBalancer).
    port: 8080           # Port on which the backend API is exposed.
```

#### 3. WebSocket Server Configuration

Keep uses Soketi as its websocket server. To learn how to configure it, please see [Soketi docs](https://github.com/soketi/charts/tree/master/charts/soketi).

#### 4. Database Configuration

Keep supports plenty of database (e.g. postgresql, mysql, sqlite, etc). It is out of scope to describe here how to deploy all of them to k8s. If you have specific questions - [contact us](https://slack.keephq.dev) and we will be happy to help.


# Installation
Source: https://docs.keephq.dev/deployment/kubernetes/installation




  The recommended way to install Keep on Kubernetes is via Helm Chart. 

  Follow these steps to set it up.


# Prerequisites

## Helm CLI

See the [Helm documentation](https://helm.sh/docs/intro/install/) for instructions about installing helm.

## Ingress Controller (Optional)


  You can skip this step if:

  1. You already have **ingress-nginx** installed.
  2. You don't need to expose Keep to the internet/network.


### Overview

An ingress controller is essential for managing external access to services in your Kubernetes cluster. It acts as a smart router and load balancer, allowing you to expose multiple services through a single entry point while handling SSL termination and routing rules.

**Keep works best with both** [ingress-nginx](https://github.com/kubernetes/ingress-nginx) **and** [HAProxy Ingress](https://haproxy-ingress.github.io/) **controllers, but you can customize the helm chart for other ingress controllers too.**

### Nginx Ingress Controller

#### Check ingress-nginx Installed

You check if you already have ingress-nginx installed:

```bash theme={null}
# By default, the ingress-nginx will be installed under the ingress-nginx namespace
kubectl -n ingress-nginx get pods
NAME                                       READY   STATUS    RESTARTS   AGE
ingress-nginx-controller-d49697d5f-hjhbj   1/1     Running   0          4h19m

# Or check for the ingress class
kubectl get ingressclass
NAME    CONTROLLER             PARAMETERS   AGE
nginx   k8s.io/ingress-nginx          4h19m

```

#### Install ingress-nginx


  To read about more installation options, see [ingress-nginx installation docs](https://kubernetes.github.io/ingress-nginx/deploy/).



  Since ingress-nginx 4.12, you'll need to add

  ```
  --set controller.config.annotations-risk-level=Critical
  ```

  See [https://github.com/kubernetes/ingress-nginx/issues/12618#issuecomment-2566084202](https://github.com/kubernetes/ingress-nginx/issues/12618#issuecomment-2566084202)


```bash theme={null}
# simplest way to install
# we set snippet-annotations to true to allow rewrites
#   see https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#allow-snippet-annotations
helm upgrade --install ingress-nginx ingress-nginx \
  --repo https://kubernetes.github.io/ingress-nginx \
  --set controller.config.allow-snippet-annotations=true \
  --set controller.config.annotations-risk-level=Critical \
  --namespace ingress-nginx --create-namespace
```

Verify installation:

```bash theme={null}
kubectl get ingressclass
NAME    CONTROLLER             PARAMETERS   AGE
nginx   k8s.io/ingress-nginx          4h19m
```

Verify if snippet annotations are enabled:

```bash theme={null}
kubectl get configmap -n ingress-nginx ingress-nginx-controller -o yaml | grep allow-snippet-annotations
allow-snippet-annotations: "true"
```

### HAProxy Ingress Controller

#### Install ingress-haproxy


  To read about more installation options, see [haproxy-ingress installation docs](https://haproxy-ingress.github.io/docs/getting-started/).


```bash theme={null}
# simplest way to install
helm upgrade --install haproxy-ingress haproxy-ingress \
  --repo https://haproxy-ingress.github.io/charts \
  --namespace ingress-haproxy --create-namespace
```

Verify installation:

```bash theme={null}
kubectl get ingressclass
NAME      CONTROLLER                      PARAMETERS   AGE
haproxy   haproxy-ingress.github.io/controller                4h19m
```

Verify if controller is running:

```bash theme={null}
kubectl get pods -n ingress-haproxy -l app.kubernetes.io/instance=haproxy-ingress
NAME                                READY   STATUS    RESTARTS   AGE
haproxy-ingress-controller-x4n2z   1/1     Running   0          4h19m
```

## Installation

### With Ingress-NGINX (Recommended)

```bash theme={null}
# Add the Helm repository
helm repo add keephq https://keephq.github.io/helm-charts

# Install Keep with ingress enabled
helm install keep keephq/keep -n keep --create-namespace
```

### With Ingress-HAProxy (Recommended)

```bash theme={null}
# Add the Helm repository
helm repo add keephq https://keephq.github.io/helm-charts

# Install Keep with ingress enabled
helm install keep keephq/keep -n keep --create-namespace --set global.ingress.className=haproxy
```

### Without Ingress (Not Recommended)

```bash theme={null}
# Add the Helm repository
helm repo add keephq https://keephq.github.io/helm-charts

# Install Keep without ingress enabled.
# You won't be able to access Keep from the network.
helm install keep keephq/keep -n keep --create-namespace \
  --set global.ingress.enabled=false
```

## Accessing Keep

### Ingress

If you installed Keep with ingress, you should be able to access Keep.

```bash theme={null}
kubectl -n keep get ingress
NAME           CLASS   HOSTS   ADDRESS        PORTS   AGE
keep-ingress   nginx   *       X.X.X.X        80      4h16m
```

Keep is available at [http://X.X.X.X](http://X.X.X.X) :)

### Without Ingress (Port-Forwarding)

Use the following commands to access Keep locally without ingress:

```bash theme={null}
# Forward the UI
kubectl port-forward svc/keep-frontend 3000:3000 -n keep &

# Forward the Backend
kubectl port-forward svc/keep-backend 8080:8080 -n keep &

# Forward WebSocket server (optional)
kubectl port-forward svc/keep-websocket 6001:6001 -n keep &
```

Keep is available at [http://localhost:3000](http://localhost:3000) :)

## Configuring HTTPS

### Prerequisites

1. Domain Name: Example - keep.yourcompany.com
2. TLS Certificate: Private key (tls.key) and certificate (tls.crt)

### Create the TLS Secret

Assuming:

* `tls.crt` contains the certificate.
* `tls.key` contains the private key.

```bash theme={null}
# create the secret with kubectl
kubectl create secret tls keep-tls --cert=./tls.crt --key=./tls.key -n keep
```

### Update Helm Values for TLS

```bash theme={null}
helm upgrade -n keep keep keephq/keep \
  --set "global.ingress.hosts[0].host=keep.example.com" \
  --set "global.ingress.tls[0].hosts[0]=keep.example.com" \
  --set "global.ingress.tls[0].secretName=keep-tls"
```

Alternatively, update your `values.yaml`:

```bash theme={null}
...
global:
  ingress:
    hosts:
      - host: keep.example.com
    tls:
      - hosts:
          - keep.example.com
        secretName: keep-tls
...
```

## Uninstallation

To remove Keep and clean up:

```bash theme={null}
helm uninstall keep -n keep
kubectl delete namespace keep
```


# Openshift
Source: https://docs.keephq.dev/deployment/kubernetes/openshift



Keep's Helm Chart also supports Openshift installation.

Simply follow the Kubernetes set-up guide, but make sure to modify the following lines under frontend(/backend).route in the values.yaml file as follows:

```
enabled: true
host: 
path:  # should be / for default
tls: 
wildcardPolicy: 
```


# Overview
Source: https://docs.keephq.dev/deployment/kubernetes/overview



 If you need help deploying Keep on Kubernetes or have any feedback or suggestions, feel free to open a ticket in our [GitHub repo](https://github.com/keephq/keep) or say hello in our [Slack](https://slack.keephq.dev). 

Keep is designed as a Kubernetes-native application.

We maintain an opinionated, batteries-included Helm chart, but you can customize it as needed.

## Next steps

* Install Keep on [Kubernetes](/deployment/kubernetes/installation).
* Keep's [Helm Chart](https://github.com/keephq/helm-charts).
* Keep with [Kubernetes Secret Manager](/deployment/secret-store#kubernetes-secret-manager)
* Deep dive to Keep's kubernetes [Architecture](/deployment/kubernetes/architecture).
* Install Keep on [OpenShift](/deployment/kubernetes/openshift).


# Running Keep with LiteLLM
Source: https://docs.keephq.dev/deployment/local-llm/keep-with-litellm




  This guide is for users who want to run Keep with locally hosted LLM models.
  If you encounter any issues, please talk to us at our (Slack
  community)\[[https://slack.keephq.dev](https://slack.keephq.dev)].


## Overview

This guide will help you set up Keep with LiteLLM, a versatile tool that supports over 100 LLM providers. LiteLLM acts as a proxy that adheres to OpenAI standards, allowing seamless integration with Keep. By following this guide, you can easily configure Keep to work with various LLM providers using LiteLLM.

### Motivation

Incorporating LiteLLM with Keep allows organizations to run local models in on-premises and air-gapped environments. This setup is particularly beneficial for leveraging AIOps capabilities while ensuring that sensitive data does not leave the premises. By using LiteLLM as a proxy, you can seamlessly integrate with Keep and access a wide range of LLM providers without compromising data security. This approach is ideal for organizations that prioritize data privacy and need to comply with strict regulatory requirements.

## Prerequisites

### Running LiteLLM locally

1. Ensure you have Python and pip installed on your system.
2. Install LiteLLM by running the following command:

```bash theme={null}
pip install litellm
```

3. Start LiteLLM with your desired model. For example, to use the HuggingFace model:

```bash theme={null}
litellm --model huggingface/bigcode/starcoder
```

This will start the proxy server on `http://0.0.0.0:4000`.

### Running LiteLLM with Docker

To run LiteLLM using Docker, you can use the following command:

```bash theme={null}
docker run -p 4000:4000 litellm/litellm --model huggingface/bigcode/starcoder
```

This command will start the LiteLLM proxy in a Docker container, exposing it on port 4000.

## Configuration

|             Env var            |                   Purpose                   | Required | Default Value |                            Valid options                           |
| :----------------------------: | :-----------------------------------------: | :------: | :-----------: | :----------------------------------------------------------------: |
| **OPEN\_AI\_ORGANIZATION\_ID** | Organization ID for OpenAI/LiteLLM services |    Yes   |      None     |                    Valid organization ID string                    |
|     **OPEN\_AI\_API\_KEY**     |     API key for OpenAI/LiteLLM services     |    Yes   |      None     |                        Valid API key string                        |
|      **OPENAI\_BASE\_URL**     |        Base URL for the LiteLLM proxy       |    Yes   |      None     | Valid URL (e.g., "[http://localhost:4000](http://localhost:4000)") |


  These environment variables should be set on both Keep **frontend** and
  **backend**.


## Additional Resources

* [LiteLLM Documentation](https://docs.litellm.ai/)

By following these steps, you can leverage the power of multiple LLM providers with Keep, using LiteLLM as a flexible and powerful proxy.


# Monitoring
Source: https://docs.keephq.dev/deployment/monitoring



# Healthchecks

Keep's Backend healthcheck url:

```
{BACKEND_API_URL}/healthcheck
```

Keep's Frontend healthcheck url:

```
{FRONTEND_URL}/api/healthcheck
```

# Prometheus Metrics

(TBD)

> Please note that /api/metrics are not designed for production instance's health monitoring, but for usage monitoring by a specific tenant.


# Dashboard Provisioning
Source: https://docs.keephq.dev/deployment/provision/dashboard



Provisioning dashboards in Keep allows you to configure and manage visual representations of your data. This section will guide you through the steps required to set up and provision dashboards.

### Dashboard Provisioning Overview

Dashboards in Keep are configured using JSON strings that define the layout, data sources, and visual components. These configurations can be managed through environment variables or configuration files.

### Environment Variables

To provision dashboards, you need to set the following environment variable:

| Environment Variable | Purpose                                         |
| -------------------- | ----------------------------------------------- |
| `KEEP_DASHBOARDS`    | JSON string containing dashboard configurations |

### Example Configuration

Here is an example of how to set the `KEEP_DASHBOARDS` environment variable (dumped from the database):

```json theme={null}
[
  {
    "dashboard_name": "My Dashboard",
    "dashboard_config": {
      "layout": [
        {
          "i": "w-1728223503577",
          "x": 0,
          "y": 0,
          "w": 3,
          "h": 3,
          "minW": 2,
          "minH": 2,
          "static": false
        }
      ],
      "widget_data": [
        {
          "i": "w-1728223503577",
          "x": 0,
          "y": 0,
          "w": 3,
          "h": 3,
          "minW": 2,
          "minH": 2,
          "static": false,
          "thresholds": [
            { "value": 0, "color": "#22c55e" },
            { "value": 20, "color": "#ef4444" }
          ],
          "preset": {
            "id": "11111111-1111-1111-1111-111111111111",
            "name": "feed",
            "options": [
              { "label": "CEL", "value": "(!deleted && !dismissed)" },
              {
                "label": "SQL",
                "value": {
                  "sql": "(deleted=false AND dismissed=false)",
                  "params": {}
                }
              }
            ],
            "created_by": null,
            "is_private": false,
            "is_noisy": false,
            "should_do_noise_now": false,
            "alerts_count": 98,
            "static": true,
            "tags": []
          },
          "name": "Test"
        }
      ]
    }
  }
]
```

Please read more at [https://github.com/react-grid-layout/react-grid-layout](https://github.com/react-grid-layout/react-grid-layout) for more information on the layout configuration options.


# Mapping Rule Provisioning
Source: https://docs.keephq.dev/deployment/provision/mapping



For any questions or issues related to mapping rule provisioning, please join our [Slack](https://slack.keephq.dev) community.

Mapping rule provisioning in Keep allows you to manage CSV-style alert enrichment rules in version control rather than the UI. This is useful when you want to track mapping changes alongside the rest of your infrastructure-as-code.

### Configuring Mapping Rules

To provision mapping rules, follow these steps:

1. Set the `KEEP_MAPPINGS_DIRECTORY` environment variable to the path of your mapping configuration directory.
2. Create one YAML manifest per mapping rule in that directory.

Example directory structure:

```
/path/to/mappings/
├── prometheus-by-namespace.yaml
├── cloudwatch-by-team.yaml
└── service-topology.yaml
```

### Manifest format

Each YAML manifest describes one mapping rule. Fields mirror the fields accepted by the REST `POST /mapping` endpoint:

```yaml theme={null}
name: example-prometheus-mapping
description: optional description
priority: 0
type: csv
matchers:
  - [namespace]
rows:
  - { namespace: monitoring, team: platform }
  - { namespace: default, team: platform }
```

| Field               | Required      | Notes                                                                      |
| ------------------- | ------------- | -------------------------------------------------------------------------- |
| `name`              | yes           | Lookup key — must be unique across the tenant                              |
| `description`       | no            | Human-readable description                                                 |
| `priority`          | no            | Integer, default `0`. Higher = evaluated first                             |
| `type`              | no            | `csv` (default) or `topology`                                              |
| `matchers`          | yes           | List of attribute groups. Within a list: AND. Between lists: OR            |
| `rows`              | yes for `csv` | List of `{key: value}` dicts; rows match against incoming alert attributes |
| `is_multi_level`    | no            | Default `false`                                                            |
| `new_property_name` | no            | Required if `is_multi_level` is `true`                                     |
| `prefix_to_remove`  | no            | Optional, used with multi-level mappings                                   |

### Update Provisioned Mapping Rules

On every restart, Keep reads the `KEEP_MAPPINGS_DIRECTORY` environment variable and determines which mapping rules need to be added, removed, or updated.

The high-level provisioning mechanism:

1. Keep reads the `KEEP_MAPPINGS_DIRECTORY` value.
2. Keep lists all `.yaml`/`.yml` files under the directory (other files are skipped).
3. For each manifest: lookup an existing rule by `name`. If found, update it and mark `is_provisioned=True`. If not, create a new provisioned rule.
4. Provisioned rules whose source file is no longer present in the directory are deprovisioned (deleted).
5. UI-created rules (`is_provisioned=False`) whose name does not appear in any manifest are untouched.

### Adoption of existing UI rules

If a mapping rule already exists in the UI with the same `name` as one of your manifests, the next provisioning run **adopts** it: `is_provisioned` flips to `True`, the rule's `provisioned_file` is recorded, and its content is overwritten from the manifest. The database id is preserved, so any external references to that rule (URLs, dashboards) continue to work.

Fields not present in the manifest schema (`disabled`, `override`, `condition`) are reset to their model defaults on adoption — the manifest is the source of truth, so a rule that was disabled via the UI will be re-enabled when adopted.

### Removing all provisioned mapping rules

If `KEEP_MAPPINGS_DIRECTORY` is unset on a Keep instance that previously had provisioned rules, all of them are deprovisioned (deleted) on the next restart. UI-only rules are unaffected.

### Per-manifest failures

A malformed manifest (invalid YAML, missing required fields, validation errors) is logged and skipped. Other manifests in the directory still process normally. Each successful manifest is committed in its own transaction, so a later failure does not roll back earlier work.


# Overview
Source: https://docs.keephq.dev/deployment/provision/overview



Keep supports various deployment and provisioning strategies to accommodate different environments and use cases, from development setups to production deployments.

### Provisioning Options

Keep offers four main provisioning options:

1. [**Provider Provisioning**](/deployment/provision/provider) - Set up and manage data providers with their deduplication rules for Keep.
2. [**Workflow Provisioning**](/deployment/provision/workflow) - Configure and manage workflows within Keep.
3. [**Dashboard Provisioning**](/deployment/provision/dashboard) - Configure and manage dashboards within Keep.
4. [**Mapping Rule Provisioning**](/deployment/provision/mapping) - Configure and manage CSV-style alert enrichment rules within Keep.

Choosing the right provisioning strategy depends on your specific use case, deployment environment, and scalability requirements. You can read more about each provisioning option in their respective sections.

### How To Configure Provisioning


  Some provisioning options require additional environment variables. These will
  be covered in detail on the specific provisioning pages.


Provisioning in Keep is controlled through environment variables and configuration files. The main environment variables for provisioning are:

| Provisioning Type | Environment Variable       | Purpose                                                                 |
| ----------------- | -------------------------- | ----------------------------------------------------------------------- |
| **Provider**      | `KEEP_PROVIDERS`           | JSON string containing provider configurations with deduplication rules |
| **Workflow**      | `KEEP_WORKFLOW`            | One workflow to provision right from the env variable.                  |
| **Workflows**     | `KEEP_WORKFLOWS_DIRECTORY` | Directory path containing workflow configuration files                  |
| **Dashboard**     | `KEEP_DASHBOARDS`          | JSON string containing dashboard configurations                         |
| **Mapping Rules** | `KEEP_MAPPINGS_DIRECTORY`  | Directory path containing mapping rule YAML manifests                   |

Hint: use the script to get 1-liner from the workflow file for KEEP\_WORKFLOW:

```
Use `cat workflow_file.yaml | awk '{printf "%s\\n", $0}' | tr -d '\n'; echo` to get the workflow in 1-string format.
```

For more details on each provisioning strategy, including setup instructions and implications, refer to the respective sections.


# Providers Provisioning
Source: https://docs.keephq.dev/deployment/provision/provider



For any questions or issues related to provider provisioning, please join our [Slack](https://slack.keephq.dev) community.

Provider provisioning in Keep allows you to set up and manage data providers dynamically. This feature enables you to configure various data sources that Keep can interact with, such as monitoring systems, databases, or other services.

### Configuring Providers

To provision providers and deduplication rules for them, we can configure via the environment variable. This can be done in two ways:

1. Using `KEEP_PROVIDERS` environment variable which either contains a JSON string or a path to a JSON file that contains the providers configurations.
2. Using `KEEP_PROVIDERS_DIRECTORY` environment variable which contains a path to a directory that contains the providers configurations (configured via YAML files). This is the recommended approach.


  Keep does not allow to use both `KEEP_PROVIDERS` and `KEEP_PROVIDERS_DIRECTORY` environment variables at the same time.



  Keep can automatically install webhooks for providers that support them. This behavior depends on the configuration and the provisioning method used.


Please note: Deduplication rules are not mandatory for provider distribution.

### Providers provisioning using KEEP\_PROVIDERS

Providers provisioning JSON example:

```json theme={null}
{
  "keepVictoriaMetrics": {
    "type": "victoriametrics",
    "authentication": {
      "VMAlertHost": "http://localhost",
      "VMAlertPort": 1234
    },
    "install_webhook": true,
    "deduplication_rules": {
      "deduplication rule name example 1": {
        "description": "deduplication rule name example 1",
        "fingerprint_fields": ["fingerprint", "source", "service"],
        "full_deduplication": true,
        "ignore_fields": ["name", "lastReceived"]
      },
      "deduplication rule name example 2": {
        "description": "deduplication rule name example 2",
        "fingerprint_fields": ["fingerprint", "source", "service"],
        "full_deduplication": false,
      }
    }
  },
  "keepClickhouse1": {
    "type": "clickhouse",
    "authentication": {
      "host": "http://localhost",
      "port": 1234,
      "username": "keep",
      "password": "keep",
      "database": "keep-db"
    }
  }
}
```

Spin up Keep with this `KEEP_PROVIDERS` value:

```json theme={null}
# ENV
KEEP_PROVIDERS={"keepVictoriaMetrics":{"type":"victoriametrics","authentication":{"VMAlertHost":"http://localhost","VMAlertPort": 1234},"install_webhook":true},"keepClickhouse1":{"type":"clickhouse","authentication":{"host":"http://localhost","port":"4321","username":"keep","password":"1234","database":"keepdb"}}}
```

By default, when provisioning using `KEEP_PROVIDERS`, webhooks are automatically installed for providers that support them unless the `install_webhook` flag is set to `false`.

### Providers provisioning using KEEP\_PROVIDERS\_DIRECTORY

Specify the path to the directory containing the providers configurations:

```bash theme={null}
# ENV
KEEP_PROVIDERS_DIRECTORY=/path/to/providers
```

The directory should contain YAML files with the providers configurations.

Example of a provider configuration YAML file:

```yaml theme={null}
name: keepVictoriaMetrics
type: victoriametrics
authentication:
  VMAlertHost: http://localhost
  VMAlertPort: 1234
install_webhook: false
deduplication_rules:
  deduplication_rule_name_example_1:
    description: deduplication rule name example 1
    fingerprint_fields:
      - fingerprint
      - source
      - service
    full_deduplication: true
    ignore_fields:
      - name
      - lastReceived
```

The `install_webhook` field controls whether Keep sets up webhooks automatically for that provider. By default, when provisioning using `KEEP_PROVIDERS_DIRECTORY`, webhook installation is disabled unless explicitly set to `true`.

### Supported Providers

Keep supports a wide range of provider types. Each provider type has its own specific configuration requirements.
To see the full list of supported providers and their detailed configuration options, please refer to our comprehensive provider documentation.

### Update Provisioned Providers

#### Using KEEP\_PROVIDERS

Provider configurations can be updated dynamically by changing the `KEEP_PROVIDERS` environment variable.

On every restart, Keep reads this environment variable and determines which providers need to be added or removed.

This process allows for flexible management of data sources without requiring manual intervention. By simply updating the `KEEP_PROVIDERS` variable and restarting the application, you can efficiently add new providers, remove existing ones, or modify their configurations.

The high-level provisioning mechanism:

1. Keep reads the `KEEP_PROVIDERS` value.
2. Keep checks if there are any provisioned providers that are no longer in the `KEEP_PROVIDERS` value, and deletes them.
3. Keep installs all providers from the `KEEP_PROVIDERS` value.

#### Using KEEP\_PROVIDERS\_DIRECTORY

Provider configurations can be updated dynamically by changing the YAML files in the `KEEP_PROVIDERS_DIRECTORY` directory.

On every restart, Keep reads the YAML files in the `KEEP_PROVIDERS_DIRECTORY` directory and determines which providers need to be added or removed.

The high-level provisioning mechanism:

1. Keep reads the YAML files in the `KEEP_PROVIDERS_DIRECTORY` directory.
2. Keep checks if there are any provisioned providers that are no longer in the YAML files, and deletes them.
3. Keep installs all providers from the YAML files.


# Workflow Provisioning
Source: https://docs.keephq.dev/deployment/provision/workflow



For any questions or issues related to workflow provisioning, please join our [Slack](https://slack.keephq.dev) community.

Workflow provisioning in Keep allows you to set up and manage workflows dynamically. This feature enables you to configure various automated processes and tasks within your Keep deployment.

### Configuring Workflows

To provision workflows, follow these steps:

1. Set the `KEEP_WORKFLOWS_DIRECTORY` environment variable to the path of your workflow configuration directory.
2. Create workflow configuration files in the specified directory.

Example directory structure:

```
/path/to/workflows/
├── workflow1.yaml
├── workflow2.yaml
└── workflow3.yaml
```

### Update Provisioned Workflows

On every restart, Keep reads the `KEEP_WORKFLOWS_DIRECTORY` environment variable and determines which workflows need to be added, removed, or updated.

This process allows for flexible management of workflows without requiring manual intervention. By simply updating the workflow files in the `KEEP_WORKFLOWS_DIRECTORY` and restarting the application, you can efficiently add new workflows, remove existing ones, or modify their configurations.

The high-level provisioning mechanism:

1. Keep reads the `KEEP_WORKFLOWS_DIRECTORY` value.
2. Keep lists all workflow files under the `KEEP_WORKFLOWS_DIRECTORY` directory.
3. Keep compares the current workflow files with the previously provisioned workflows:
   * New workflow files are provisioned.
   * Missing workflow files are deprovisioned.
   * Updated workflow files are re-provisioned with the new configuration.
4. Keep updates its internal state to reflect the current set of provisioned workflows.


# Secret Store
Source: https://docs.keephq.dev/deployment/secret-store



## Overview


  Secret Manager selection is crucial for securing your application. Different
  modes can be set up depending on the deployment type. Our system supports four
  primary secret manager types.


## Secret Manager Factory

The `SecretManagerFactory` is a utility class used to create instances of different types of secret managers. It leverages the Factory design pattern to abstract the creation logic based on the type of secret manager required. The factory supports creating instances of File, GCP, Kubernetes, and Vault Secret Managers.

The `SECRET_MANAGER_TYPE` environment variable plays a crucial role in the SecretManagerFactory for determining the default type of secret manager to be instantiated when no specific type is provided in the method call.

**Functionality**:

**Default Secret Manager**: If the `SECRET_MANAGER_TYPE` environment variable is set, its value dictates the default type of secret manager that the factory will create.
The value of this variable should correspond to one of the types defined in SecretManagerTypes enum (`FILE`, `AWS`, `GCP`, `K8S`, `VAULT`, `DB`).

**Example Configuration**:

Setting `SECRET_MANAGER_TYPE=GCP` in the environment will make the factory create instances of GcpSecretManager by default.
If `SECRET_MANAGER_TYPE` is not set or is set to `FILE`, the factory defaults to creating instances of FileSecretManager.
This environment variable provides flexibility and ease of configuration, allowing different secret managers to be used in different environments or scenarios without code changes.

## File Secret Manager

The `FileSecretManager` is a concrete implementation of the BaseSecretManager for managing secrets stored in the file system. It uses a specified directory (defaulting to ./) to read, write, and delete secret files.

Configuration:

Set the environment variable `SECRET_MANAGER_DIRECTORY` to specify the directory where secrets are stored. If not set, defaults to the current directory (./).

Usage:

* Secrets are stored as files in the specified directory.
* Reading a secret involves fetching content from a file.
* Writing a secret creates or updates a file with the given content.
* Deleting a secret removes the corresponding file.

## AWS Secret Manager

The `AwsSecretManager` integrates with Amazon Web Services' Secrets Manager service for secure secret management. It provides a robust solution for storing and managing secrets in AWS environments.

Configuration:

Required environment variables:

* `AWS_REGION`: The AWS region where your secrets are stored
* For local development:
  * `AWS_ACCESS_KEY_ID`: Your AWS access key
  * `AWS_SECRET_ACCESS_KEY`: Your AWS secret access key
    Optional:
* `AWS_KMS_KEY_ID`: The KMS key ID to use for encrypting secrets
* `AWS_SECRET_MANAGER_TAGS`: Comma-separated list of tags to add to the secret in AWS Secrets Manager, e.g. `key=value,key2=value2`
* `AWS_SECRET_ROTATION_ENABLED`: Set to `true` to enable automatic rotation of secrets (default: `false`)
* `AWS_SECRET_ROTATION_DAYS`: Number of days between automatic rotations (default: `30`)
* `AWS_SECRET_ROTATION_LAMBDA_ARN`: ARN of the Lambda function to use for secret rotation, required if rotation is enabled

Usage:

* Manages secrets using AWS Secrets Manager service
* Supports creating, updating, reading, and deleting secrets
* Can automatically configure secret rotation policies when creating new secrets

### AWS Secret Rotation

Secret rotation is a security best practice that automatically updates secrets at regular intervals. When enabled, Keep will configure newly created secrets with a rotation schedule.

To use secret rotation:

1. Create a Lambda function for rotating your secrets (AWS provides blueprints for common rotation scenarios)
2. Set `AWS_SECRET_ROTATION_ENABLED=true` in your environment
3. Set `AWS_SECRET_ROTATION_LAMBDA_ARN` to the ARN of your rotation Lambda function
4. Optionally set `AWS_SECRET_ROTATION_DAYS` to customize the rotation interval

Example Lambda ARN format: `arn:aws:lambda:region:account-id:function:function-name`

Note: Different secret types (database credentials, API keys, etc.) require different rotation logic. Make sure your Lambda function is appropriate for the type of secrets you're storing.

## Kubernetes Secret Manager

### Overview

The `KubernetesSecretManager` interfaces with Kubernetes' native secrets system.

It manages secrets within a specified Kubernetes namespace and is designed to operate within a Kubernetes cluster.

### Configuration

* `SECRET_MANAGER_TYPE=k8s`
* `K8S_NAMESPACE=keep` - environment variable to specify the Kubernetes namespace. Defaults to `.metadata.namespace` if not set. Assumes Kubernetes configurations (like service account tokens) are properly set up when running within a cluster.
* `K8S_VERIFY_SSL_CERT=true` - environment variable to specify whether to verify the SSL certificate of the Kubernetes API. Defaults to `true`.

Usage:

* Secrets are stored as Kubernetes Secret objects.
* Provides functionalities to create, retrieve, and delete Kubernetes secrets.
* Handles base64 encoding and decoding as required by Kubernetes.

### Environment Variables From Secrets

The Kubernetes Secret Manager integration allows Keep to fetch environment variables from Kubernetes Secrets.

For sensitive environment variables, such as `DATABASE_CONNECTION_STRING`, it is recommended to store as a secret:

#### Creating Database Connection Secret

```bash theme={null}
# Create the base64 encoded string without newline
CONNECTION_STRING_B64=$(echo -n "mysql+pymysql://user:password@host:3306/dbname" | base64)

# Create the Kubernetes secret
kubectl create secret generic keep-db-secret \
  --namespace=keep \
  --from-literal=connection_string=$(echo -n "mysql+pymysql://user:password@host:3306/dbname" | base64)

# Or using a YAML file:
cat <If you are using Keep and have performance issues, we will be more than happy to help you. Just join our [slack](https://slack.keepqh.dev) and shoot a message on the **#help** channel.

## Overview

Spec and stress testing are crucial to ensuring the robust performance and scalability of Keep.
This documentation outlines the key areas of focus for testing Keep under different load conditions, considering both the simplicity of setup for smaller environments and the scalability mechanisms for larger deployments.

Keep was initially designed to be user-friendly for setups handling less than 10,000 alerts. However, as alert volumes increase, users can leverage advanced features such as Elasticsearch for document storage and Redis + ARQ for queue-based alert ingestion. While these advanced configurations are not fully documented here, they are supported and can be discussed further in our Slack community.

## How To Reproduce

To reproduce the stress testing scenarios mentioned above, please refer to the [STRESS.md](https://github.com/keephq/keep/blob/main/STRESS.md) file in Keep's repository. This document provides step-by-step instructions on how to set up, run, and measure the performance of Keep under different load conditions.

## Performance Testing

### Factors Affecting Specifications

The primary parameters that affect the specification requirements for Keep are:

1. **Alerts Volume**: The rate at which alerts are ingested into the system.
2. **Total Alerts**: The cumulative number of alerts stored in the system.
3. **Number of Workflows**: How many automation run as a result of alert.

### Main Components:

* **Keep Backend** - API and business logic. A container that serves FastAPI on top of gunicorn.
* **Keep Frontend** - Web app. A container that serves the react app.
* **Database** - Stores the alerts and any other operational data.
* **Elasticsearch** (opt out by default) - Stores alerts as document for better search performance.
* **Redis** (opt out by default) - Used, together with ARQ, as an alerts queue.

### Testing Scenarios:

* **Low Volume (\< 10,000 total alerts, hundreds of alerts per day)**:
  * **Setup**: Use a standard relational database (e.g., MySQL, PostgreSQL) with default configurations.
  * **Expectations**: Keep should handle queries and alert ingestion with minimal resource usage.

* **Medium Volume (10,000 - 100,000 total alerts, thousands of alerts per day)**:
  * **Setup**: Scale the database to larger instances or clusters. Adjust best practices to the DB (e.g. increasing innodb\_buffer\_pool\_size)
  * **Expectations**: CPU and RAM usage should increase proportionally but remain within acceptable limits.

3. **High Volume (100,000 - 1,000,000 total alerts, >five thousands of alerts per day)**:
   * **Setup**: Deploy Keep with Elasticsearch for storing alerts as documents.
   * **Expectations**: The system should maintain performance levels despite the large alert volume, with increased resource usage managed through scaling strategies.
4. **Very High Volume (> 1,000,000 total alerts, tens of thousands of alerts per day)**:
   * **Setup**: Deploy Keep with Elasticsearch for storing alerts as documents.
   * **Setup #2**: Deploy Keep with Redis and with ARQ to use Redis as a queue.

## Recommended Specifications by Alert Volume

| **Number of Alerts**  | **Keep Backend**  | **Keep Database**                              | **Redis**        | **Elasticsearch**            |
| --------------------- | ----------------- | ---------------------------------------------- | ---------------- | ---------------------------- |
| **\< 10,000**         | 1 vCPUs, 2GB RAM  | 2 vCPUs, 8GB RAM                               | Not required     | Not required                 |
| **10,000 - 100,000**  | 4 vCPUs, 8GB RAM  | 8 vCPUs, 32GB RAM, optimized indexing          | Not required     | Not required                 |
| **100,000 - 500,000** | 8 vCPUs, 16GB RAM | 8 vCPUs, 32GB RAM, advanced indexing           | 4 vCPUs, 8GB RAM | 8 vCPUs, 32GB RAM, 2-3 nodes |
| **> 500,000**         | 8 vCPUs, 16GB RAM | 8 vCPUs, 32GB RAM, advanced indexing, sharding | 4 vCPUs, 8GB RAM | 8 vCPUs, 32GB RAM, 2-3 nodes |

## Performance by Operation Type, Load, and Specification

| **Operation Type** | **Load**                 | **Specification**         | **Execution Time** |
| ------------------ | ------------------------ | ------------------------- | ------------------ |
| Digest Alert       | 100 alerts per minute    | 4 vCPUs, 8GB RAM          | \~0.5 seconds      |
| Digest Alert       | 500 alerts per minute    | 8 vCPUs, 16GB RAM         | \~1 second         |
| Digest Alert       | 1,000 alerts per minute  | 16 vCPUs, 32GB RAM        | \~1.5 seconds      |
| Run Workflow       | 10 workflows per minute  | 4 vCPUs, 8GB RAM          | \~1 second         |
| Run Workflow       | 50 workflows per minute  | 8 vCPUs, 16GB RAM         | \~2 seconds        |
| Run Workflow       | 100 workflows per minute | 16 vCPUs, 32GB RAM        | \~3 seconds        |
| Ingest via Queue   | 100 alerts per minute    | 4 vCPUs, 8GB RAM, Redis   | \~0.3 seconds      |
| Ingest via Queue   | 500 alerts per minute    | 8 vCPUs, 16GB RAM, Redis  | \~0.8 seconds      |
| Ingest via Queue   | 1,000 alerts per minute  | 16 vCPUs, 32GB RAM, Redis | \~1.2 seconds      |

### Table Explanation:

* **Operation Type**: The specific operation being tested (e.g., digesting alerts, running workflows).
* **Load**: The number of operations per minute being processed (e.g., number of alerts per minute).
* **Specification**: The CPU, RAM, and additional services used for the operation.
* **Execution Time**: Approximate time taken to complete the operation under the given load and specification.

## Fine Tuning

As any deployment has its own characteristics, such as the balance between volume vs. total count of alerts or volume vs. number of workflows, Keep can be fine-tuned with the following parameters:

1. **Number of Workers**: Adjust the number of Gunicorn workers to handle API requests more efficiently. You can also start additional API servers to distribute the load.
2. **Distinguish Between API Server Workers and Digesting Alerts Workers**: Separate the workers dedicated to handling API requests from those responsible for digesting alerts, ensuring that each set of tasks is optimized according to its specific needs.
3. **Add More RAM to the Database**: Increasing the RAM allocated to your database can help manage larger datasets and improve query performance, particularly when dealing with high volumes of alerts.
4. **Optimize Database Configuration**: Keep was mainly tested on MySQL and PostgreSQL. Different database may have different fine tuning mechanisms.
5. **Horizontal Scaling**: Consider deploying additional instances of the API and database services to distribute the load more effectively.

## FAQ

### 1. How do I estimate the spec I need for Keep?

To estimate the specifications required for Keep, consider both the number of alerts per minute and the total number of alerts you expect to handle. Refer to the **Recommended Specifications by Alert Volume** table above to match your expected load with the appropriate resources.

### 2. How do I know if I need Elasticsearch?

Elasticsearch is typically needed when you are dealing with more than 50,000 total alerts or if you require advanced search and query capabilities that are not efficiently handled by a traditional relational database. If your system’s performance degrades significantly as alert volume increases, it may be time to consider Elasticsearch.

### 3. How do I know if I need Redis?

Redis is recommended when your alert ingestion rate exceeds 1,000 alerts per minute or when you notice that the API is becoming a bottleneck due to high ingestion rates. Redis, combined with ARQ (Asynchronous Redis Queue), can help manage and distribute the load more effectively.

### 4. What should I do if Keep's performance is still inadequate?

If you have scaled according to the recommendations and are still facing performance issues, consider:

* **Optimizing your database configuration**: Indexing, sharding, and query optimization can make a significant difference.
* **Horizontal scaling**: Distribute the load across multiple instances of the API and database services.
* **Reach out to our Slack community**: For personalized support, reach out to us on Slack, and we’ll help you troubleshoot and optimize your Keep deployment.

For any additional questions or tailored advice, feel free to join our Slack community where our team and other users are available to assist you.


# Keep with an external URL
Source: https://docs.keephq.dev/development/external-url



## Introduction

Several features in Keep necessitate an external URL that is accessible from the internet. This is particularly crucial for functionalities like Webhook Integration when installing providers. Keep uses its API URL to establish itself as a webhook connector during this process.

When an alert is triggered, the corresponding Provider attempts to activate the webhook, delivering the alert payload. Consequently, the webhook must be accessible over the internet for this process to work effectively.

## Utilizing NGROK for External Accessibility

Keep supports the use of NGROK to create an accessible external URL. By starting Keep with the environment variable USE\_NGROK=true, Keep will automatically initiate an NGROK tunnel and utilize this URL for webhook installations.


  While `USE_NGROK` is convenient for development or testing, it's important to note that each restart of Keep results in a new NGROK URL. This change in the URL means that providers configured with the old URL will no longer be able to communicate with Keep.

  For production environments, it's advisable to either:

  * Expose Keep with a permanent, internet-accessible URL.
  * Set up a static NGROK tunnel.

  Subsequently, configure Keep to use this stable URL by setting the KEEP\_API\_URL environment variable.



# Getting started
Source: https://docs.keephq.dev/development/getting-started



### Docker-compose dev images

You can use `docker-compose.dev.yaml` to start Keep in a development mode.

First, clone the Keep repo:

```
git clone https://github.com/keephq/keep.git && cd keep
```

Next, run

```
docker compose -f docker-compose.dev.yml up
```

### Install Keep CLI

First, clone Keep repository:

```shell theme={null}
git clone https://github.com/keephq/keep.git && cd keep
```

Install Keep CLI

```shell theme={null}
poetry install
```

To access the Keep CLI activate the environment, and access from shell.

```shell theme={null}
poetry shell
```

From now on, Keep should be installed locally and accessible from your CLI, test it by executing:

```
keep version
```

## Enable Auto Completion

**Keep's CLI supports shell auto-completion, which can make your life a whole lot easier 😌**

If you're using zsh

```shell title=~/.zshrc theme={null}
eval "$(_KEEP_COMPLETE=zsh_source keep)"
```

If you're using bash

```bash title=~/.bashrc theme={null}
eval "$(_KEEP_COMPLETE=bash_source keep)"
```

> Using eval means that the command is invoked and evaluated every time a shell is started, which can delay shell responsiveness. To speed it up, write the generated script to a file, then source that.

### Testing

Run unittests:

```bash theme={null}
poetry run coverage run --branch -m pytest --ignore=tests/e2e_tests/
```

Run E2E tests (run Keep locally before):

```bash theme={null}
poetry run playwright install;
poetry run coverage run --branch -m pytest -s tests/e2e_tests/
```

### Migrations

Migrations are automatically executed on a server startup. To create a migration:

```bash theme={null}
alembic -c keep/alembic.ini revision --autogenerate -m "Your message"
```

Hint: make sure your models are imported at `./api/models/db/migrations/env.py` for autogenerator to pick them up.

## VS Code (or Cursor)

Run Keep from your VS Code (or Cursor) after cloning the repo by adding this configurations to your `.vscode/launch.json`:

```json theme={null}
{
    "version": "0.2.0",
    "configurations": [
      {
        "name": "Keep Backend",
        "type": "debugpy",
        "request": "launch",
        "program": "keep/cli/cli.py",
        "console": "integratedTerminal",
        "justMyCode": false,
        "python": "venv/bin/python",
        "args": ["--json", "api","--multi-tenant"],
        "env": {
            "PYDEVD_DISABLE_FILE_VALIDATION": "1",
            "PYTHONPATH": "${workspaceFolder}/",
            "PUSHER_APP_ID": "1",
            "SECRET_MANAGER_DIRECTORY": "./state/",
            "PUSHER_HOST": "localhost",
            "PUSHER_PORT": "6001",
            "PUSHER_APP_KEY": "keepappkey",
            "PUSHER_APP_SECRET": "keepappsecret",
            "LOG_FORMAT": "dev_terminal",
        }
      },
      {
        "name": "Keep Simulate Alerts",
        "type": "debugpy",
        "request": "launch",
        "program": "scripts/simulate_alerts.py",
        "console": "integratedTerminal",
        "justMyCode": false,
        "python": "venv/bin/python",
        "env": {
          "PYDEVD_DISABLE_FILE_VALIDATION": "1",
          "PYTHONPATH": "${workspaceFolder}/",
          "KEEP_API_URL": "http://localhost:8080",
          "KEEP_API_KEY": "some-api-key"
        }
      },
      {
        "name": "Keep Frontend",
        "type": "node-terminal",
        "request": "launch",
        "command": "npm run dev",
        "cwd": "${workspaceFolder}/keep-ui",
      }
    ]
}
```

Install dependencies:

```
python3.11 -m venv venv;
source venv/bin/activate;
pip install poetry;
poetry install;
cd keep-ui && npm i && cd ..;
```

Set frontend envs:

```
cp keep-ui/.env.local.example keep-ui/.env.local;
echo "\n\n\n\nNEXTAUTH_SECRET="$(openssl rand -hex 32) >> keep-ui/.env.local;
```

Launch Pusher ([soketi](https://soketi.app/)) container in parallel:

```bash theme={null}
docker run -d -p 6001:6001 -p 9601:9601 -e SOKETI_USER_AUTHENTICATION_TIMEOUT=3000 -e SOKETI_DEFAULT_APP_KEY=keepappkey -e SOKETI_DEFAULT_APP_SECRET=keepappsecret -e SOKETI_DEFAULT_APP_ID=1 quay.io/soketi/soketi:1.4-16-debian
```

## VS Code (or Cursor) + Docker

For this guide to work, the [VS Code Docker](https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-docker) extension is required.
In air-gapped environments, you might consider building the container on an internet-connected computer, exporting the image using docker save, transferring it with docker load in the air-gapped environment, and then using the run configuration.

In cases where you want to develop Keep but are unable to run it directly on your local laptop (e.g., with Windows), or if you lack access to all of its dependencies (e.g., in air-gapped environments), you can still accomplish this using VS Code (or Cursor) and Docker.

To achieve this, follow these steps:

1. Clone Keep and open it with VS Code (or Cursor)
2. Create a tasks.json file to build and run the Keep API and Keep UI containers.
3. Create a launch.json configuration to start the containers and attach a debugger to them.
4. Profit.

### Clone Keep and open it with VS Code (or Cursor)

```
git clone https://github.com/keephq/keep.git && cd keep
code .
```

### Create tasks.json

#### including building the containers

```
{
    "version": "2.0.0",
    "tasks": [
        // The API and UI containers needs to be in the same docker network
        {
            "label": "docker-create-network",
            "type": "shell",
            "command": "docker network create keep-network || true",
            "problemMatcher": []
        },
        // Build the api container
        {
            "label": "docker-build-api-dev",
            "type": "docker-build",
            "dockerBuild": {
                "context": "${workspaceFolder}",
                "dockerfile": "${workspaceFolder}/Docker/Dockerfile.dev.api",
                "tag": "keep-api-dev:latest"
            }
        },
        // Run the api container
        {
            "label": "docker-run-api-dev",
            "type": "docker-run",
            "dependsOn": [
                "docker-build-api-dev", "docker-create-network"
            ],
            "python": {
                "args": [
                    "api"
                ],
                "file": "./keep/cli/cli.py"
            },
            "dockerRun": {
                "network": "keep-network",
                "image": "keep-api-dev:latest",
                "containerName": "keep-api",
                "ports": [
                    {
                        "containerPort": 8080,
                        "hostPort": 8080
                    }
                ],
                "env": {
                    "DEBUG": "1",
                    "SECRET_MANAGER_TYPE": "FILE",
                    "USE_NGROK": "false",
                    "AUTH_TYPE": "DB"
                },
                "volumes": [
                    {
                        "containerPath": "/app",
                        "localPath": "${workspaceFolder}"
                    }
                ]
            }
        },
        // Build the UI container
        {
            "label": "docker-build-ui",
            "type": "docker-build",
            "dockerBuild": {
                "context": "${workspaceFolder}",
                "dockerfile": "${workspaceFolder}/Docker/Dockerfile.dev.ui",
                "tag": "keep-ui-dev:latest"
            }
        },
        // Run the UI container
        {
            "type": "docker-run",
            "label": "docker-run-ui",
            "dependsOn": [
                "docker-build-ui", "docker-create-network"
            ],
            "dockerRun": {
                "network": "keep-network",
                "image": "keep-ui-dev:latest",
                "containerName": "keep-ui",
                "env": {
                    // Uncomment for fully debug
                    // "DEBUG": "*",
                    "NODE_ENV": "development",
                    "API_URL": "http://keep-api:8080",
                    "AUTH_TYPE": "DB",
                },
                "volumes": [
                    {
                        "containerPath": "/app",
                        "localPath": "${workspaceFolder}/keep-ui"
                    }
                ],
                "ports": [
                    {
                        "containerPort": 9229,
                        "hostPort": 9229
                    },
                    {
                        "containerPort": 3000,
                        "hostPort": 3000
                    }
                ],
                "command": "npm run dev",
            },
            "node": {
                "package": "${workspaceFolder}/keep-ui/package.json",
                "enableDebugging": true
            }
        }
    ]
}

```

#### without building the containers

To start Keep without building the containers, you'll need to have `keep-api-dev` and `keep-ui-dev` images loaded into your docker.

```
{
    "version": "2.0.0",
    "tasks": [
        # The API and the UI needs to be in the same docker network
        {
            "label": "docker-create-network",
            "type": "shell",
            "command": "docker network create keep-network || true",
            "problemMatcher": []
        },
        # Run the API container
        {
            "label": "docker-run-api-dev",
            "type": "docker-run",
            "dependsOn": [
                "docker-create-network"
            ],
            "python": {
                "args": [
                    "api"
                ],
                "file": "./keep/cli/cli.py"
            },
            "dockerRun": {
                "network": "keep-network",
                "image": "keep-api-dev:latest",
                "containerName": "keep-api",
                "ports": [
                    {
                        "containerPort": 8080,
                        "hostPort": 8080
                    }
                ],
                "env": {
                    "DEBUG": "1",
                    "SECRET_MANAGER_TYPE": "FILE",
                    "USE_NGROK": "false",
                    "AUTH_TYPE": "DB"
                },
                "volumes": [
                    {
                        "containerPath": "/app",
                        "localPath": "${workspaceFolder}"
                    }
                ]
            }
        },
        # Run the UI container
        {
            "type": "docker-run",
            "label": "docker-run-ui",
            "dependsOn": [
                "docker-create-network"
            ],
            "dockerRun": {
                "network": "keep-network",
                "image": "keep-ui-dev:latest",
                "containerName": "keep-ui",
                "env": {
                    // Uncomment for fully debug
                    // "DEBUG": "*",
                    "NODE_ENV": "development",
                    "API_URL": "http://keep-api:8080",
                    "AUTH_TYPE": "DB"
                },
                "volumes": [
                    {
                        "containerPath": "/app",
                        "localPath": "${workspaceFolder}/keep-ui"
                    }
                ],
                "ports": [
                    {
                        "containerPort": 9229,
                        "hostPort": 9229
                    },
                    {
                        "containerPort": 3000,
                        "hostPort": 3000
                    }
                ],
                "command": "npm run dev",
            },
            "node": {
                "package": "${workspaceFolder}/keep-ui/package.json",
                "enableDebugging": true
            }
        }
    ]
}
```

### Create launch.json

```
{
        "name": "Docker: Keep API",
        "type": "docker",
        "request": "launch",
        "preLaunchTask": "docker-run-api-dev",
        "removeContainerAfterDebug": true,
        "containerName": "keep-api",
        "python": {
          "pathMappings": [
            {
              "localRoot": "${workspaceFolder}",
              "remoteRoot": "/app"
            }
          ],
          "module": "keep.cli.cli"
        }
      },
      {
        "name": "Docker: Keep UI",
        "type": "docker",
        "request": "launch",
        "removeContainerAfterDebug": true,
        "preLaunchTask": "docker-run-ui",
        "containerName": "keep-api",
        "platform": "node",
        "node": {
          "package": "${workspaceFolder}/keep-ui/package.json",
          "localRoot": "${workspaceFolder}/keep-ui"
        }
      },
```


# Facets
Source: https://docs.keephq.dev/incidents/facets



Faceted search is a powerful mechanism for enhancing search functionality, allowing users to filter and refine search results dynamically using multiple dimensions or "facets." These facets are predefined categories or attributes of the data. In Keep, the Incidents page supports faceted search by incident attributes.

### Predefined Incident Facets

These are predefined Incident facets that can be used to filter incidents:

* **Status**: Filter by Incident status
* **Severity**: Filter by Incident severity
* **Assignee**: Filter by Incident assignee
* **Source**: Filter by alert source
* **Service**: Filter by the service the Incident relates to

### Custom Facets Creation

Keep also supports custom facets creation. Here is how to do this:

1. Click the "Add facet" button in the filtering panel.
2. Enter the Facet name. This is the name that will be displayed in the filter panel.
3. Enter the Facet property path the facet will filter by.
4. Click "Create".


  


### Supported Properties to create Facets for

Incident supports facets by direct Incident fields and also by Alert's data linked to the Incident. Here is a list of properties you can create facets for:

* **name**: Incident name
* **summary**: Incident summary
* **creation\_time**: Incident creation time
* **start\_time**: Incident start time
* **end\_time**: Incident end time
* **last\_seen\_time**: Incident last seen time
* **is\_predicted**: Whether the Incident is predicted
* **is\_candidate**: Whether the Incident is candidate
* **alerts\_count**: Number of alerts associated with the Incident
* **merged\_at**: When the Incident was merged
* **merged\_by**: Who merged the Incident
* **hasLinkedIncident**: Whether the Incident has past incident linked
* **alert.**\*: Refers to alert properties in the Incident. Examples: alert.labels.monitor, alert.monitor, etc.


# Overview
Source: https://docs.keephq.dev/incidents/overview



Keep's incident management system provides a comprehensive solution for handling, tracking, and resolving operational incidents. This system helps teams effectively manage incidents from detection through resolution, ensuring minimal downtime and efficient collaboration.


  


### (1) Incident Severity

Displays the severity of the incident, helping teams prioritize and focus on the most critical issues.

### (2) Incident Name

The unique name or identifier of the incident for easy reference and tracking.

### (3) Incident Summary (+ AI Summary)

A brief overview of the incident, optionally enhanced with AI-generated summaries to provide deeper insights.

### (4) Link Similar Incidents

Connects related incidents for better visibility into recurring or interconnected issues.

### (5) Involved Services

Lists the services affected by the incident, allowing teams to understand the scope of the impact.

### (6) Affected Environments

Specifies the environments (e.g., production, staging) impacted by the incident.

### (7) Run Workflow

Quickly initiate workflows to address the incident, such as creating tickets, notifying teams, or executing remediation steps.


  


### (8) Edit Incident

Allows modification of incident details, such as severity, name, or involved services, to keep information up-to-date.


  


### (9) Incident Status

Indicates the current status of the incident (e.g., open, resolved, acknowledged).

### (10) Incident Last Seen At

Records the most recent timestamp when the incident was observed, providing context for its activity.

### (11) Incident Started At

Indicates when the incident was first detected, helping establish timelines for resolution.

### (12) Incident Assignee

Displays the individual or team responsible for resolving the incident, promoting accountability.

### (13) Incident Group By Value

Groups incidents based on a specific attribute, such as service, environment, or severity, for better organization.

### (14) Incident Related Alerts

Lists all alerts linked to the incident, offering a complete view of its underlying causes.

### (15) Incident Activity

Tracks all activities and updates related to the incident, enabling detailed audits and reviews.


  


### (16) Incident Timeline

Provides a chronological view of the incident's lifecycle, including updates, actions, and status changes.


  


### (17) Incident Topology

Visualizes the relationships between affected components, services, and infrastructure in a topology map.


  


### (18) Incident Workflows

Lists workflows associated with the incident, showing actions taken or available options for resolution.

### (19) Incident Chat with AI (Incident Copilot)

Engage with AI-powered chat for guidance, insights, or recommended actions related to the incident.


  


### (20) Incident Alert List

Displays a detailed list of alerts contributing to the incident, with metadata for each alert.

### (21) Incident Alert Link

Provides quick access to the original monitoring tool for a specific alert.

### (22) Incident Alert Status

Shows the current status of each alert, such as acknowledged, resolved, or firing.

### (23) Incident Correlation Type

Indicates how the incident was correlated: manually, via AI, or by rule-based logic.

### (24) Incident Alert Unlink

Enables unlinking specific alerts from the incident if they are found to be unrelated.

***


# AI Correlation
Source: https://docs.keephq.dev/overview/ai-correlation




  Keep Cloud: ✅ 

  Keep Enterprise On-Premises: ✅ 

  Keep Open Source: ⛔️


Keep's AI correlation engine provides a distinctive approach to fully AI-driven alert correlation.
By using historical alert data as its training dataset, the system intelligently classifies new alerts and assigns them to appropriate incidents.

The AI correlator runs on cycles, each iteration cycle completes in 5-15 minutes:

1. Model trained based on historical data.
2. Model is evaluated.
3. All unassigned alerts are clustered and added to incidents when their confidence score exceeds the threshold.

Configuration UI:


  


Incident with alerts correlated by AI:


  


Check the demo on a playground: [https://playground.keephq.dev/ai](https://playground.keephq.dev/ai)

To activate the feature for your on-premises tenant, please [talk to us](https://www.keephq.dev/meet-keep).

## Frequent questions:

**Model used:** proprietary model developed and hosted by Keep.

**Training dataset:** tenant's alerts and incidents.

**Privacy:** tenant's data is used only for training of the model for the same tenant. Data is not mixed between tenants for training.


# AI in Workflows
Source: https://docs.keephq.dev/overview/ai-in-workflows




  Keep Cloud: ✅ 

  Keep Enterprise On-Premises: ✅ 

  Keep Open Source: âœ