> ## Documentation Index
> Fetch the complete documentation index at: https://docs.keephq.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Running Keep with LiteLLM

<Info>
  This guide is for users who want to run Keep with locally hosted LLM models.
  If you encounter any issues, please talk to us at our (Slack
  community)\[[https://slack.keephq.dev](https://slack.keephq.dev)].
</Info>

## Overview

This guide will help you set up Keep with LiteLLM, a versatile tool that supports over 100 LLM providers. LiteLLM acts as a proxy that adheres to OpenAI standards, allowing seamless integration with Keep. By following this guide, you can easily configure Keep to work with various LLM providers using LiteLLM.

### Motivation

Incorporating LiteLLM with Keep allows organizations to run local models in on-premises and air-gapped environments. This setup is particularly beneficial for leveraging AIOps capabilities while ensuring that sensitive data does not leave the premises. By using LiteLLM as a proxy, you can seamlessly integrate with Keep and access a wide range of LLM providers without compromising data security. This approach is ideal for organizations that prioritize data privacy and need to comply with strict regulatory requirements.

## Prerequisites

### Running LiteLLM locally

1. Ensure you have Python and pip installed on your system.
2. Install LiteLLM by running the following command:

```bash theme={null}
pip install litellm
```

3. Start LiteLLM with your desired model. For example, to use the HuggingFace model:

```bash theme={null}
litellm --model huggingface/bigcode/starcoder
```

This will start the proxy server on `http://0.0.0.0:4000`.

### Running LiteLLM with Docker

To run LiteLLM using Docker, you can use the following command:

```bash theme={null}
docker run -p 4000:4000 litellm/litellm --model huggingface/bigcode/starcoder
```

This command will start the LiteLLM proxy in a Docker container, exposing it on port 4000.

## Configuration

|             Env var            |                   Purpose                   | Required | Default Value |                            Valid options                           |
| :----------------------------: | :-----------------------------------------: | :------: | :-----------: | :----------------------------------------------------------------: |
| **OPEN\_AI\_ORGANIZATION\_ID** | Organization ID for OpenAI/LiteLLM services |    Yes   |      None     |                    Valid organization ID string                    |
|     **OPEN\_AI\_API\_KEY**     |     API key for OpenAI/LiteLLM services     |    Yes   |      None     |                        Valid API key string                        |
|      **OPENAI\_BASE\_URL**     |        Base URL for the LiteLLM proxy       |    Yes   |      None     | Valid URL (e.g., "[http://localhost:4000](http://localhost:4000)") |

<Note>
  These environment variables should be set on both Keep **frontend** and
  **backend**.
</Note>

## Additional Resources

* [LiteLLM Documentation](https://docs.litellm.ai/)

By following these steps, you can leverage the power of multiple LLM providers with Keep, using LiteLLM as a flexible and powerful proxy.