docs

Set Up and Configure Plural AI

Configure Plural AI and mix models to optimize cost and performance

The current state of generative AI is a sprawl of vendors offering products with different specialties and price points, and an optimal AI setup commonly uses models from multiple vendors or multiple models from the same vendor. Plural provides several knobs designed to make that degree of customization seamless and compatible with a GitOps workflow.

Set Up Plural AI

Plural AI can be configured through Settings → AI settings in the Plural Console or with the DeploymentSettings CRD. If you installed Plural with plural up, the resource is already defined at bootstrap/settings.yaml.

For a basic OpenAI configuration, create a credential secret and reference it from DeploymentSettings:

yaml
apiVersion: v1
kind: Secret
metadata:
  name: ai-config
  namespace: plrl-deploy-operator
stringData:
  openai: <access-token>
---
apiVersion: deployments.plural.sh/v1alpha1
kind: DeploymentSettings
metadata:
  name: global
  namespace: plrl-deploy-operator
spec:
  ai:
    enabled: true
    provider: OPENAI
    openAI:
      tokenSecretRef:
        name: ai-config
        key: openai

Provider credentials are always read from secrets in the plrl-deploy-operator namespace. Use additional keys in the same secret when configuring other providers:

yaml
apiVersion: v1
kind: Secret
metadata:
  name: ai-config
  namespace: plrl-deploy-operator
stringData:
  openai: <access-token>
  anthropic: <access-token>
  azure: <access-token>
  vertex: <service-account-json>
Warning:

Never commit provider credentials to Git in plain text. Encrypt the secret with your organization's preferred secret-management workflow before storing it in a GitOps repository.

The full DeploymentSettings schema is available in the Management API reference.

Provider Selection within Plural AI

Edit on GitHub