llm

llmwrappersllms

CLI by Simon Willison for interacting with language models.

Using LLM with LiteLLM

As per the documentation, seek out your config directory and, create a extra-openai-models.yaml file in it:

touch "$(dirname "$(llm logs path)")/extra-openai-models.yaml"

Add your LiteLLM models and endpoints to the yaml file. Provide a key name for your api key but not your actual API key.

- model_id: gemma3:27b
  model_name: gemma3:27b
  api_base: https://litellm.mysite.example/v1
  api_key_name: litellm
- model_id: gemma3:12b
  model_name: gemma3:12b
  api_base: https://litellm.mysite.example/v1
  api_key_name: litellm

Use the LLM cli to set your api key:

 llm keys set litellm