Build an AI app with the Ploy AI gateway, no provider keys

The first version of an AI feature is a fetch to a model API with a key from an environment variable. The second version has three provider keys, a retry wrapper, a streaming branch, and a spreadsheet of who is paying for what. Most of that is plumbing, and none of it is your product.

Ploy ships an AI gateway as part of the platform. Add ai: true to ploy.yaml and the worker gets an env.AI binding plus an OpenAI-compatible base URL and a per-deployment token. There are no provider keys in your project, the same code runs under ploy dev, and if you self-host Ploy the gateway comes with it. This guide builds an AI app with the Ploy AI gateway three ways: the binding, an OpenAI-compatible SDK, and the Vercel AI SDK in Next.js.

Enable the gateway

kind: dynamic
build: pnpm build
out: dist
ai: true

kind: dynamic
build: pnpm build
out: dist
ai: true

That injects three things into the worker:

• env.AI, with env.AI.run(model, inputs, options?)
• PLOY_AI_URL, an OpenAI-compatible base URL
• PLOY_AI_TOKEN, a token Ploy manages per deployment

Pass a model id such as glm-4.6v-flash or anthropic/claude-sonnet-5, or "auto" to let the gateway pick.

1. The binding: env.AI.run()

The smallest useful AI endpoint is a worker that takes a prompt and returns the model's answer:

interface ChatCompletion {
  choices: { message: { role: "assistant"; content: string } }[];
  usage: { prompt_tokens: number; completion_tokens: number };
}

export default {
  async fetch(request, env) {
    const url = new URL(request.url);
    const prompt = url.searchParams.get("prompt") ?? "Reply with OK.";
    const model = url.searchParams.get("model") ?? "auto";

    const data = await env.AI.run<ChatCompletion>(model, {
      messages: [
        { role: "system", content: "Answer in one short paragraph." },
        { role: "user", content: prompt },
      ],
    });

    return Response.json({
      answer: data.choices[0].message.content,
      usage: data.usage,
    });
  },
} satisfies Ploy;

interface ChatCompletion {
  choices: { message: { role: "assistant"; content: string } }[];
  usage: { prompt_tokens: number; completion_tokens: number };
}

export default {
  async fetch(request, env) {
    const url = new URL(request.url);
    const prompt = url.searchParams.get("prompt") ?? "Reply with OK.";
    const model = url.searchParams.get("model") ?? "auto";

    const data = await env.AI.run<ChatCompletion>(model, {
      messages: [
        { role: "system", content: "Answer in one short paragraph." },
        { role: "user", content: prompt },
      ],
    });

    return Response.json({
      answer: data.choices[0].message.content,
      usage: data.usage,
    });
  },
} satisfies Ploy;

Requests and responses follow the OpenAI chat completions shape, so messages, temperature, max_tokens, and usage all mean what you expect. Token counts come back on every response, which is how you meter per user without a second service.

Streaming

Pass stream: true and run() resolves to a ReadableStream you can return as server-sent events:

const stream = await env.AI.run("auto", {
  prompt: "Write one short sentence about the sea.",
  stream: true,
});

return new Response(stream, {
  headers: {
    "Content-Type": "text/event-stream; charset=utf-8",
    "Cache-Control": "no-cache",
  },
});

const stream = await env.AI.run("auto", {
  prompt: "Write one short sentence about the sea.",
  stream: true,
});

return new Response(stream, {
  headers: {
    "Content-Type": "text/event-stream; charset=utf-8",
    "Cache-Control": "no-cache",
  },
});

Test it with curl -N https://your-app.ploy.app/ai. This is examples/ai-streaming in the Ploy repository.

2. Any OpenAI-compatible SDK

Libraries that take a baseURL and an apiKey work unchanged with the raw variables:

import OpenAI from "openai";

export default {
  async fetch(request, env) {
    const client = new OpenAI({
      apiKey: env.PLOY_AI_TOKEN,
      baseURL: env.PLOY_AI_URL,
    });

    const response = await client.chat.completions.create({
      model: "auto",
      messages: [{ role: "user", content: "Write a haiku about deploys." }],
    });

    return Response.json(response);
  },
} satisfies Ploy;

import OpenAI from "openai";

export default {
  async fetch(request, env) {
    const client = new OpenAI({
      apiKey: env.PLOY_AI_TOKEN,
      baseURL: env.PLOY_AI_URL,
    });

    const response = await client.chat.completions.create({
      model: "auto",
      messages: [{ role: "user", content: "Write a haiku about deploys." }],
    });

    return Response.json(response);
  },
} satisfies Ploy;

The same pattern works for LangChain's ChatOpenAI (apiKey plus configuration.baseURL), which is how examples/ai-langchain builds an agent with tools.

3. Next.js with the Vercel AI SDK

In a Next.js project with ai: true, Ploy registers itself as the AI SDK's default provider during the build and during ploy dev. A bare model id resolves through the Ploy gateway, with no provider setup and no AI_GATEWAY_API_KEY:

import { convertToModelMessages, streamText } from "ai";

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5",
    messages: await convertToModelMessages(messages),
  });

  return result.toUIMessageStreamResponse();
}

import { convertToModelMessages, streamText } from "ai";

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5",
    messages: await convertToModelMessages(messages),
  });

  return result.toUIMessageStreamResponse();
}

This needs ai v5 or later. Ploy generates ploy-ai-provider.js and an instrumentation.js import next to your app directory on each build; commit them or ignore them. If you already register a provider yourself, Ploy leaves it alone.

An app that already uses the AI SDK with provider/model ids moves to Ploy without changing the route.

Add memory, jobs, and scheduling

The gateway is one binding among several, which is where an AI app on Ploy gets interesting:

kind: dynamic
build: pnpm build
out: dist
ai: true
db:
  DB: default
queue:
  EMBED: embed
cron:
  NIGHTLY_SUMMARY: "0 2 * * *"

kind: dynamic
build: pnpm build
out: dist
ai: true
db:
  DB: default
queue:
  EMBED: embed
cron:
  NIGHTLY_SUMMARY: "0 2 * * *"

Store conversations in the SQLite database, push long-running generation onto a queue so the HTTP request returns immediately, and run a nightly summarization from the cron scheduled handler, which receives the same env.AI. For multi-step pipelines that must finish, a durable workflow retries each step; see Background jobs with queues and durable workflows.

Local development

ploy dev routes env.AI through an organization-scoped development sandbox, so the same code works on your machine. If your checkout matches a deployed project, Ploy picks the organization automatically; otherwise set PLOY_DEV_AI_PROJECT_ID to any project in the right organization, or set PLOY_DEV_AI_URL and PLOY_DEV_AI_TOKEN to point at a gateway of your own.

Frequently Asked Questions

Which models can I call?

The gateway is OpenAI-compatible and takes a model id per request; the docs examples use ids such as glm-4.6v-flash and anthropic/claude-sonnet-5, and "auto" lets the gateway choose.

Do I ever handle provider API keys?

No. PLOY_AI_TOKEN is managed by Ploy per deployment. Your repository and environment variables never contain a provider key.

Does the gateway work when self-hosting?

Yes. The gateway is part of the platform. The sandbox provider docs show how to expose its public URL for workloads that run outside your network.

Can I use it from a plain worker and from Next.js in the same project?

Yes. env.AI and the raw variables are available in workers and in the app/ploy.ts handlers of a Next.js app, and the AI SDK default provider covers Next.js routes.

Get started

• Build an AI app on Ploy free
• Read the AI integration docs
• Next: Build a Telegram bot on Ploy