Gateway System

Gateway System

Gateways let website owners provide AI access to their visitors automatically. Users don’t need extensions or API keys - AI features just work when they visit your site.

Overview

As a website owner, create a gateway to provide AI to your visitors:

• Configure your API keys once at gateway.webllm.org
• Get a private key to generate access tokens
• Embed tokens in your website
• Visitors automatically use your gateway (no setup required)

Set limits per token:

• Domain restrictions - Your domains only
• IP quotas - Rate limit per visitor IP
• Geo-blocking - Allow/block countries
• Cost limits - Budget caps
• Expiration - Time-based validity

How It Works

Website Owner:
1. Create gateway at gateway.webllm.org
   → Get private key (sk-webllm-gateway-...)

2. Configure your API keys (OpenAI, Anthropic, etc.)
   → Keys stored securely in gateway

3. Generate access token with limits
   → Embed in your website code

Visitor:
4. Visit your website
   → WebLLM automatically detects and uses your gateway token
   → AI features work immediately, no setup needed

Private Key (sk-webllm-gateway-...)

• Your admin credential
• Generate access tokens for your site
• Never expose in client code

Access Tokens (wlm-{gatewayId}.{jwt})

• Embedded in your website
• Automatically used by visitors
• Limits enforced per visitor

Why Provide a Gateway

Mobile Support - iOS/Android visitors can use AI features

Zero Friction - Visitors don’t need extensions or API keys

Cost Control - Set budgets, IP quotas, and limits

Your API Keys - You control which AI providers to use

Setup for Website Owners

1. Create Gateway (Console)

Visit gateway.webllm.org:

1. Create Gateway

   • Get gateway ID: abc123xyz
   • Get private key: sk-webllm-gateway-... (save securely!)

2. Add Your API Keys

   • OpenAI, Anthropic, OpenRouter, etc.
   • Set fallback order

2. Generate Access Token (Server)

Generate a token for your website:

// server.js or build script - Never in browser!
import { generateGatewayToken } from 'webllm';

const token = await generateGatewayToken({
  secretKey: process.env.GATEWAY_SECRET_KEY,

  // Limit total requests per month
  requests: { limit: 10000, period: 'month' },

  // Rate limit per visitor IP
  ipQuota: { limit: 100, period: 'day' },

  // Budget cap
  cost: { limit: 50.00, currency: 'USD' },

  // Your domains only
  domains: ['mysite.com', '*.mysite.com'],

  // Geographic restrictions
  allowedCountries: ['US', 'CA', 'GB'],

  // Token validity
  expiresIn: 90 * 24 * 60 * 60 * 1000 // 90 days
});

// Returns: wlm-abc123xyz.eyJhbGc...

3. Embed in Website

// Your website code
import { WebLLMClient } from 'webllm';

const client = new WebLLMClient({
  gatewayToken: 'wlm-abc123xyz.eyJhbGc...' // From step 2
});

// Visitors automatically use your gateway
const result = await client.generateText({
  prompt: 'Hello!'
});

Visitors see AI features working immediately. WebLLM detects your gateway token and uses it automatically.

Access Control Options

Request Quota

requests: { limit: 1000, period: 'month' | 'day' | 'hour' }

IP Quota (Rate Limiting)

ipQuota: { limit: 100, period: 'day' } // Max 100 req/day per IP

Cost Limit

cost: { limit: 10.00, currency: 'USD' } // Stop at $10

Domain Restrictions

domains: ['myapp.com', '*.myapp.com'] // Specific domains
domains: [] // Any domain

Geo-Blocking

allowedCountries: ['US', 'CA', 'GB'] // Only these countries
// OR
blockedCountries: ['CN', 'RU'] // Block these countries

Expiration

expiresIn: 30 * 24 * 60 * 60 * 1000 // 30 days in ms

Common Patterns for Website Owners

Production Site with Budget Control

{
  requests: { limit: 100000, period: 'month' },
  ipQuota: { limit: 100, period: 'day' }, // Prevent single visitor abuse
  cost: { limit: 500.00, currency: 'USD' },
  domains: ['mysite.com'],
  expiresIn: 90 * 24 * 60 * 60 * 1000
}

Beta/Demo Site with Free Tier

{
  requests: { limit: 1000, period: 'day' },
  ipQuota: { limit: 20, period: 'hour' }, // 20 requests/hour per visitor
  cost: { limit: 10.00, currency: 'USD' },
  domains: ['beta.mysite.com'],
  expiresIn: 30 * 24 * 60 * 60 * 1000
}

Regional Site (US/Canada Only)

{
  requests: { limit: 50000, period: 'month' },
  allowedCountries: ['US', 'CA'],
  domains: ['mysite.com'],
  expiresIn: 90 * 24 * 60 * 60 * 1000
}

Security Best Practices

✅ Always:

• Store private key in environment variables
• Generate tokens in server/build script, not browser
• Restrict to your domains
• Set IP quotas to prevent visitor abuse
• Add cost limits to control budget

❌ Never:

• Hardcode private key in client code
• Commit private key to git
• Skip limits for public sites

Starting Free (Website Owners)

OpenRouter - 100 requests/day with free models, no credit card needed

Setup:

1. Sign up at OpenRouter
2. Add API key to gateway console
3. Generate token with conservative limits
4. Start offering AI features to visitors

Monitoring (In Console)

View in the gateway console:

• Token usage and remaining quota
• Cost per token
• IP-based patterns
• Geographic distribution
• Error rates

Visitor Experience: Extension vs Gateway

	Extension Installed	Gateway (No Extension)
Setup	One-time install	None - just works
Privacy	Local (uses their API keys)	Uses website’s gateway
Mobile	Not available	Works
Cost	Visitor’s API keys	Covered by website
Preference	Visitor’s choice of model	Website’s provider

Progressive Enhancement: WebLLM automatically tries extension first (if installed), then falls back to website’s gateway. Website owners don’t need special code - it just works.

FAQ

Do visitors need to do anything? No. They just visit your site and AI features work automatically.

What if a visitor has the extension? WebLLM uses their extension first (better privacy). Gateway is fallback.

How much does it cost? You pay your AI provider costs. Gateway infrastructure is free.

Can I change limits after creating the token? Generate a new token with updated limits and deploy.

What happens when limits are reached? Gateway returns 429 error. You can monitor in console.

IP quota vs request quota?

• Request quota: Total across all visitors
• IP quota: Per visitor (prevents abuse from single IP)

Can visitors see my API keys? No. Keys stay in the gateway. Visitors only see the access token.