Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.keephq.dev/llms.txt

Use this file to discover all available pages before exploring further.

The vLLM Provider supports querying language models deployed with vLLM for prompt-based interactions.

​
Authentication

This provider requires authentication.
  • api_url: vLLM API endpoint URL (required: True, sensitive: False)
  • api_key: Optional API key if your vLLM deployment requires authentication (required: False, sensitive: True)

​
In workflows

This provider can be used in workflows. As “step” to query data, example: 
steps:
    - name: Query vllm
      provider: vllm
      config: "{{ provider.my_provider_name }}"
      with:
        prompt: {value}  
        temperature: {value}  
        model: {value}  
        max_tokens: {value}  
        structured_output_format: {value}
Check the following workflow example:

​
Connecting with the Provider

To connect to a vLLM deployment:
  1. Deploy your vLLM instance or obtain the API endpoint of an existing deployment
  2. Configure the API URL in your provider configuration
  3. If your deployment requires authentication, configure the API key