Paid customer API key & production usage guide

This guide explains how Research and Enterprise customers use their Terradev API key, activate hot-swap serving, and call the OpenAI-compatible endpoints for an adapter.

1. Quickstart: prompt to adapter

The simplest automation flow is: send a prompt, wait for the adapter to be generated, then either download the weights or serve it through an OpenAI-compatible gateway.

1.1 Generate an adapter from a prompt

ADAPTER_ID=$(curl -s -X POST https://terradev.cloud/api/adapters/generate \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{"base_model":"meta-llama/Llama-3.1-8B","text":"A helpful coding assistant"}' \
  | jq -r '.adapter_id')

1.2 Poll until the adapter is ready

while true; do
  STATUS=$(curl -s "https://terradev.cloud/api/adapters/${ADAPTER_ID}" \
    -H "Authorization: Bearer <your-api-key>" | jq -r '.status')
  echo "Status: $STATUS"
  [ "$STATUS" = "ready" ] && break
  [ "$STATUS" = "failed" ] && { echo "Generation failed"; exit 1; }
  sleep 5
done

1.3 Download the LoRA weights

curl -s "https://terradev.cloud/api/adapters/${ADAPTER_ID}/download" \
  -H "Authorization: Bearer <your-api-key>" \
  -o adapter.safetensors

You can now load the .safetensors file into any PEFT/LoRA inference pipeline.

1.4 Or serve the adapter with OpenAI-compatible endpoints

Skip the download and start a hot-swap serving instance instead:

curl -s -X POST "https://terradev.cloud/api/adapters/${ADAPTER_ID}/serve" \
  -H "Authorization: Bearer <your-api-key>"

Poll GET /api/adapters/${ADAPTER_ID} until status is hot and lorax_instance is set, then call the chat endpoint as described in 6. Call the OpenAI-compatible gateway.

2. Get an API key

1. Sign up or log in at https://terradev.cloud.
2. Subscribe to a paid tier (Research or Enterprise) if you have not already.
3. Go to /dashboard.
4. Click Generate API key in the Production API key card.
5. Copy the key immediately — it is the only time it is shown in full.

You can also regenerate the key from the same card or by calling:

curl -X POST https://terradev.cloud/api/auth/api-key \
  -H "Cookie: session=<your-session-cookie>"

3. Authenticate requests

Every paid-tier request must include your API key in one of these headers:

X-API-Key: <your-api-key>
# or
Authorization: Bearer <your-api-key>

This key is used both for the Terradev backend and for the dedicated vLLM/SGLang gateway that is spun up when an adapter is active.

4. List your adapters

curl https://terradev.cloud/api/adapters \
  -H "Authorization: Bearer <your-api-key>"

Each adapter has:

• id — the adapter identifier used as the OpenAI model name.
• status — generating, ready, hot, failed.
• lorax_instance — the active OpenAI-compatible gateway URL (set only when status is hot).
• gateway_url — same value, kept for backward compatibility.
• rank — LoRA rank when known.
• base_model — base model the adapter was trained on.

5. Activate hot-swap serving

Only paid adapters can be served. Click Activate or toggle Hot-swap in the dashboard, or call:

curl -X POST "https://terradev.cloud/api/adapters/<adapter-id>/serve" \
  -H "Authorization: Bearer <your-api-key>"

The backend will:

1. Pick a GPU instance that fits the base model (T4 → A10G → A100 → H100).
2. Start a vLLM or SGLang pod with the base model.
3. Load your LoRA adapter dynamically.
4. Return the gateway URL when healthy.

Note: Adapter generation runs on CPU-only hypernetworks. GPUs are used only for hot-swap serving/inference (/serve), not for generation. GPU instance selection is the fallback order the backend uses when it provisions an EC2 GPU instance. This requires GPU_POOL_ENABLED=true and valid AWS credentials. If the GPU pool is disabled, the serving container runs on the same host as the API instead of provisioning a separate GPU.

Cold-start time depends on the GPU type and base model size, typically 2–10 minutes. Poll GET /adapters/<adapter-id> until status is hot and lorax_instance is set.

6. Call the OpenAI-compatible gateway

When an adapter is hot, the dashboard shows a dropdown with these endpoints:

The full URL is shown in the dashboard and returned as lorax_instance.

cURL example

curl "<lorax_instance>/v1/chat/completions" \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<adapter-id>",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Hello!"}
    ]
  }'

Python example

from openai import OpenAI

client = OpenAI(
    base_url="<lorax_instance>/v1",
    api_key="<your-api-key>",
)

response = client.chat.completions.create(
    model="<adapter-id>",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello!"},
    ],
)
print(response.choices[0].message.content)

Replace <lorax_instance> and <adapter-id> with the values from the dashboard.

7. Import a PEFT adapter from Hugging Face

Research and Enterprise customers can import a LoRA directly from Hugging Face:

curl -X POST https://terradev.cloud/api/adapters/import \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "hf_repo_id": "organization/adapter-name",
    "hf_token": "hf_...",
    "subfolder": "",
    "name": "My imported adapter",
    "base_model": "meta-llama/Llama-3.1-8B-Instruct"
  }'

• hf_token is optional. If omitted, the server uses its own HF_TOKEN.
• base_model is optional if adapter_config.json contains base_model_name_or_path.
• The imported adapter counts against your tier adapter cap (Research 10, Enterprise 50).

8. Stop serving

To stop the GPU instance and stop billing:

curl -X POST "https://terradev.cloud/api/adapters/<adapter-id>/stop" \
  -H "Authorization: Bearer <your-api-key>"

Or toggle Hot-swap off in the dashboard.

9. Production operator checklist

If you are running the backend yourself:

1. Set a strong random SESSION_SECRET and API_KEY env vars if applicable.
2. Export HF_TOKEN so gated base models and PEFT imports work.
3. Choose a serving engine:

   SERVING_ENGINE=vllm
   SERVING_IMAGE=vllm/vllm-openai:v0.11.2
   

4. Configure GPU pool fallback order:

   GPU_POOL_ENABLED=true
   GPU_INSTANCE_SEQUENCE=g4dn.xlarge,g5.xlarge,p4d.24xlarge,p5.4xlarge
   

5. Ensure the API host has Docker, the NVIDIA container runtime, and AWS credentials for EC2 provisioning.
6. Put the API behind HTTPS (e.g. Caddy or a reverse proxy). The dashboard only calls /api/* on the same origin.
7. For pricing, serving costs, and fallback behavior, see aws/GPU_POOL.md and aws/SERVING.md.

10. Limits by tier

Need help?

For dedicated Enterprise support, use the contact card on the dashboard or email support@terradev.cloud.