SDK Getting Started

Installation

Choose your preferred integration approach:

WebLLM
Vercel AI SDK

Use WebLLM’s native API for a lightweight, direct integration.

npm install webllm

Use WebLLM with the popular Vercel AI SDK for a standardized AI interface.

npm install webllm-ai-provider webllm ai

Quick Start

1. Wait for Extension Readiness

Ensure the WebLLM extension is ready before making requests:

WebLLM
Vercel AI SDK

import { webLlmReady, generateText } from 'webllm';

// Wait for extension to be ready
await webLlmReady();

// Now safe to use - WebLLM routes to user's best provider
const result = await generateText({
  prompt: 'Write a haiku about coding'
});

console.log(result.text);

import { webLlmReady } from 'webllm';
import { generateText } from 'ai';
import { webllm } from 'webllm-ai-provider';

// Wait for extension to be ready
await webLlmReady();

// Now safe to use - WebLLM routes to user's best provider
const result = await generateText({
  model: webllm({ task: 'creative' }),
  prompt: 'Write a haiku about coding'
});

console.log(result.text);

2. Prompt for Installation (Recommended)

For the best user experience, use promptInstall() to guide users through installation:

WebLLM
Vercel AI SDK

import { promptInstall, generateText } from 'webllm';

try {
  // Show installation modal if extension not available
  await promptInstall();

  // Extension is now ready - WebLLM chooses best provider
  const result = await generateText({
    prompt: 'Hello, world!'
  });

  console.log(result.text);
} catch (error) {
  // User cancelled installation or browser not supported
  console.error('Cannot use WebLLM:', error.message);
}

import { promptInstall } from 'webllm';
import { generateText } from 'ai';
import { webllm } from 'webllm-ai-provider';

try {
  // Show installation modal if extension not available
  await promptInstall();

  // Extension is now ready - WebLLM chooses best provider
  const result = await generateText({
    model: webllm({ task: 'general' }),
    prompt: 'Hello, world!'
  });

  console.log(result.text);
} catch (error) {
  // User cancelled installation or browser not supported
  console.error('Cannot use WebLLM:', error.message);
}

3. Manual Availability Check

You can also manually check if the extension is installed:

import { isAvailable, getBrowserInfo } from 'webllm';

if (isAvailable()) {
  console.log('WebLLM is available!');
} else {
  const browserInfo = getBrowserInfo();
  if (browserInfo.isSupported) {
    console.log('Extension not installed. Install from:', browserInfo.installUrl);
  } else {
    console.log('Not supported:', browserInfo.reason);
  }
}

Common Tasks

Simple Text Generation

Generate text without specifying providers - WebLLM intelligently routes to the best available option:

WebLLM
Vercel AI SDK

import { generateText } from 'webllm';

const result = await generateText({
  task: 'creative',
  hints: {
    speed: 'fast',
    quality: 'standard'
  },
  prompt: 'Write a haiku about coding',
  temperature: 0.7,
  maxTokens: 100
});

console.log(result.text);
console.log('Tokens:', result.usage.totalTokens);

import { generateText } from 'ai';
import { webllm } from 'webllm-ai-provider';

const result = await generateText({
  model: webllm({
    task: 'creative',
    hints: {
      speed: 'fast',
      quality: 'standard'
    }
  }),
  prompt: 'Write a haiku about coding',
  temperature: 0.7,
  maxTokens: 100
});

console.log(result.text);
console.log('Tokens:', result.usage.totalTokens);

Chat Conversations

Build multi-turn conversations:

WebLLM
Vercel AI SDK

import { generateText } from 'webllm';

const result = await generateText({
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'Explain quantum computing in simple terms' }
  ],
  temperature: 0.7
});

console.log(result.text);

OpenAI SDK Style (Alternative):

import { webllm } from 'webllm';

const completion = await webllm.chat.completions.create({
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'Explain quantum computing in simple terms' }
  ],
  temperature: 0.7
});

console.log(completion.choices[0].message.content);

import { generateText } from 'ai';
import { webllm } from 'webllm-ai-provider';

const result = await generateText({
  model: webllm({ task: 'qa' }),
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'Explain quantum computing in simple terms' }
  ],
  temperature: 0.7
});

console.log(result.text);

Task-Specific Routing

Guide WebLLM to select the best provider for your specific task:

WebLLM
Vercel AI SDK

import { generateText } from 'webllm';

// Translation task - prioritize accuracy
const translation = await generateText({
  task: 'translation',
  hints: {
    quality: 'high',
    capabilities: {
      multilingual: true
    }
  },
  system: 'You are a professional translator specializing in Spanish.',
  prompt: 'Translate to Spanish: Hello, how are you?'
});

console.log(translation.text);
// "Hola, ¿cómo estás?"

import { generateText } from 'ai';
import { webllm } from 'webllm-ai-provider';

// Translation task - prioritize accuracy
const translation = await generateText({
  model: webllm({
    task: 'translation',
    hints: {
      quality: 'high',
      capabilities: {
        multilingual: true
      }
    }
  }),
  messages: [
    {
      role: 'system',
      content: 'You are a professional translator specializing in Spanish.'
    },
    { role: 'user', content: 'Translate to Spanish: Hello, how are you?' }
  ]
});

console.log(translation.text);
// "Hola, ¿cómo estás?"

Multi-turn Conversations

Maintain conversation context across multiple exchanges:

WebLLM
Vercel AI SDK

import { generateText } from 'webllm';

const result = await generateText({
  messages: [
    { role: 'user', content: 'What is the capital of France?' },
    { role: 'assistant', content: 'The capital of France is Paris.' },
    { role: 'user', content: 'What is its population?' }
  ]
});

console.log(result.text);
// "Paris has a population of approximately 2.2 million people..."

import { generateText } from 'ai';
import { webllm } from 'webllm-ai-provider';

const result = await generateText({
  model: webllm({ task: 'qa' }),
  messages: [
    { role: 'user', content: 'What is the capital of France?' },
    { role: 'assistant', content: 'The capital of France is Paris.' },
    { role: 'user', content: 'What is its population?' }
  ]
});

console.log(result.text);
// "Paris has a population of approximately 2.2 million people..."

Generation Parameters

Control the output with various parameters:

WebLLM
Vercel AI SDK

import { generateText } from 'webllm';

const result = await generateText({
  task: 'creative',
  hints: {
    quality: 'high'
  },
  prompt: 'Write a creative story about AI',
  temperature: 0.9,           // Higher = more creative (0.0-1.0)
  maxTokens: 500,             // Maximum length
  topP: 0.95,                 // Nucleus sampling
  frequencyPenalty: 0.5,      // Reduce repetition
  presencePenalty: 0.5,       // Encourage new topics
  stopSequences: ['THE END']  // Stop at these strings
});

console.log(result.text);

import { generateText } from 'ai';
import { webllm } from 'webllm-ai-provider';

const result = await generateText({
  model: webllm({
    task: 'creative',
    hints: {
      quality: 'high'
    }
  }),
  prompt: 'Write a creative story about AI',
  temperature: 0.9,           // Higher = more creative (0.0-1.0)
  maxTokens: 500,             // Maximum length
  topP: 0.95,                 // Nucleus sampling
  frequencyPenalty: 0.5,      // Reduce repetition
  presencePenalty: 0.5,       // Encourage new topics
  stop: ['THE END']           // Stop at these strings
});

console.log(result.text);

Error Handling

Always handle errors gracefully:

import { promptInstall, generateText, isAvailable } from 'webllm';

try {
  // First, ensure extension is ready
  if (!isAvailable()) {
    await promptInstall();
  }

  // Now make the request
  const result = await generateText({
    prompt: 'Explain machine learning'
  });

  console.log(result.text);
} catch (error) {
  if (error.message.includes('Installation cancelled')) {
    console.log('User cancelled installation');
  } else if (error.message.includes('not supported')) {
    console.log('Browser not supported');
  } else {
    console.error('Request failed:', error.message);
  }
}

Browser Compatibility

Check which browsers are supported:

import { getBrowserInfo } from 'webllm';

const browserInfo = getBrowserInfo();

console.log('Browser:', browserInfo.browserName);
console.log('Supported:', browserInfo.isSupported);

if (!browserInfo.isSupported) {
  console.log('Reason:', browserInfo.reason);
}

if (browserInfo.installUrl) {
  console.log('Install from:', browserInfo.installUrl);
}

Supported Browsers:

Chrome/Chromium ✅
Microsoft Edge ✅
Firefox ⏳ (Coming soon)
Safari ❌ (Not supported)
Mobile browsers ❌ (Not supported)

Graceful Degradation

Provide fallback experiences when extension is not available:

import { isAvailable, generateText } from 'webllm';

async function enhanceContent(text) {
  if (isAvailable()) {
    try {
      const result = await generateText({
        prompt: `Summarize this text: ${text}`,
        maxTokens: 200
      });
      return result.text;
    } catch (error) {
      console.warn('LLM enhancement failed:', error);
      // Fall through to fallback
    }
  }

  // Fallback: Simple truncation
  return text.slice(0, 200) + '...';
}

Advanced Usage

Using webLlmReady() for Progressive Enhancement

Wait for the extension to become available:

import { webLlmReady, generateText } from 'webllm';

async function initializeApp() {
  try {
    // Wait up to 5 seconds for extension
    await webLlmReady(5000);
    console.log('AI features enabled!');

    const result = await generateText({
      prompt: 'Hello, AI!'
    });
    console.log(result.text);
  } catch (error) {
    console.log('AI features not available');
    // Continue without AI features
  }
}

initializeApp();

Intelligent Provider Routing

WebLLM automatically selects the best provider based on user configuration and your preferences. You don’t need to specify exact models or providers - just provide hints about your task:

WebLLM
Vercel AI SDK

import { generateText } from 'webllm';

// Complex reasoning task - prefer capable models
const deepAnalysis = await generateText({
  task: 'qa',
  hints: {
    quality: 'best',
    capabilities: {
      reasoning: true,
      longContext: true
    }
  },
  prompt: 'Explain quantum computing and its implications for cryptography'
});

// Quick factual response - prefer speed
const quickAnswer = await generateText({
  task: 'qa',
  hints: {
    speed: 'fastest',
    quality: 'draft'
  },
  prompt: 'What is 2+2?'
});

// Let WebLLM choose based on user's defaults
const balanced = await generateText({
  task: 'qa',
  prompt: 'Explain quantum computing'
});

import { generateText } from 'ai';
import { webllm } from 'webllm-ai-provider';

// Complex reasoning task - prefer capable models
const deepAnalysis = await generateText({
  model: webllm({
    task: 'qa',
    hints: {
      quality: 'best',
      capabilities: {
        reasoning: true,
        longContext: true
      }
    }
  }),
  prompt: 'Explain quantum computing and its implications for cryptography'
});

// Quick factual response - prefer speed
const quickAnswer = await generateText({
  model: webllm({
    task: 'qa',
    hints: {
      speed: 'fastest',
      quality: 'draft'
    }
  }),
  prompt: 'What is 2+2?'
});

// Let WebLLM choose based on user's defaults
const balanced = await generateText({
  model: webllm({ task: 'qa' }),
  prompt: 'Explain quantum computing'
});

Note: WebLLM never requires specific models. The user configures their available providers (Anthropic, OpenAI, local models, etc.), and WebLLM intelligently routes each request to the best match based on your preferences and their configuration.

Framework Integration

React with Installation Prompt

import { useState, useEffect } from 'react';
import { promptInstall, generateText, isAvailable } from 'webllm';

function AiFeature() {
  const [result, setResult] = useState('');
  const [loading, setLoading] = useState(false);
  const [error, setError] = useState(null);

  async function handleGenerate() {
    setLoading(true);
    setError(null);

    try {
      // Prompt for installation if needed
      if (!isAvailable()) {
        await promptInstall();
      }

      // Generate text
      const response = await generateText({
        prompt: 'Write a haiku about React',
        maxTokens: 100
      });

      setResult(response.text);
    } catch (err) {
      setError(err.message);
    } finally {
      setLoading(false);
    }
  }

  return (
    <div>
      <button onClick={handleGenerate} disabled={loading}>
        {loading ? 'Generating...' : 'Generate Haiku'}
      </button>
      {error && <p style={{ color: 'red' }}>{error}</p>}
      {result && <pre>{result}</pre>}
    </div>
  );
}

React with webLlmReady

import { useState, useEffect } from 'react';
import { webLlmReady, generateText } from 'webllm';

function App() {
  const [ready, setReady] = useState(false);
  const [loading, setLoading] = useState(true);

  useEffect(() => {
    webLlmReady(5000)
      .then(() => {
        setReady(true);
        setLoading(false);
      })
      .catch(() => {
        setReady(false);
        setLoading(false);
      });
  }, []);

  if (loading) return <div>Checking for WebLLM...</div>;
  if (!ready) return <div>WebLLM not available</div>;

  return <AiFeatures />;
}

Vue with Installation Prompt

<template>
  <div>
    <button @click="generate" :disabled="loading">
      {{ loading ? 'Generating...' : 'Generate Text' }}
    </button>
    <p v-if="error" class="error">{{ error }}</p>
    <pre v-if="result">{{ result }}</pre>
  </div>
</template>

<script setup>
import { ref } from 'vue';
import { promptInstall, generateText, isAvailable } from 'webllm';

const loading = ref(false);
const result = ref('');
const error = ref(null);

async function generate() {
  loading.value = true;
  error.value = null;

  try {
    // Prompt for installation if needed
    if (!isAvailable()) {
      await promptInstall();
    }

    // Generate text
    const response = await generateText({
      prompt: 'Explain Vue.js in one sentence',
      maxTokens: 50
    });

    result.value = response.text;
  } catch (err) {
    error.value = err.message;
  } finally {
    loading.value = false;
  }
}
</script>

<style scoped>
.error {
  color: red;
}
</style>

Best Practices

1. Use `promptInstall()` for Best UX

Prefer promptInstall() over manual availability checks:

// ✅ Good - Guides user through installation
await promptInstall();
const result = await generateText({ prompt: 'Hello' });

// ❌ Less ideal - User doesn't know what to do
if (!isAvailable()) {
  alert('Please install WebLLM extension');
}

2. Use `webLlmReady()` for Progressive Enhancement

For non-critical features, use webLlmReady() with timeout:

try {
  await webLlmReady(3000); // Wait 3 seconds max
  // Enable AI features
} catch {
  // Continue without AI features
}

3. Check Browser Compatibility Early

Avoid showing install prompts on unsupported browsers:

import { getBrowserInfo, promptInstall } from 'webllm';

const browserInfo = getBrowserInfo();

if (!browserInfo.isSupported) {
  console.log(`WebLLM not supported: ${browserInfo.reason}`);
  // Show appropriate message
} else {
  await promptInstall();
}

4. Provide Fallback Experiences

Always have a non-AI fallback:

async function enhanceText(text) {
  try {
    if (!isAvailable()) {
      await promptInstall();
    }
    const result = await generateText({ prompt: text });
    return result.text;
  } catch {
    // Fallback to simple processing
    return text.toUpperCase();
  }
}

5. Handle Installation Cancellation

Users might cancel the installation prompt:

try {
  await promptInstall();
} catch (error) {
  if (error.message.includes('cancelled')) {
    console.log('User cancelled - show alternative UI');
  }
}

6. Show Loading States

AI requests can take time:

setLoading(true);
try {
  const result = await generateText({ prompt: 'Long task...' });
  setResult(result.text);
} finally {
  setLoading(false); // Always hide loading state
}

7. Test Without Extension

Ensure your app works when extension is not installed:

// This should never crash
async function safeGenerate(prompt) {
  try {
    await promptInstall();
    return await generateText({ prompt });
  } catch {
    return null; // Graceful degradation
  }
}

Installation Flow Examples

One-Click Experience

import { promptInstall, generateText } from 'webllm';

button.onclick = async () => {
  try {
    await promptInstall(); // Shows modal only if needed
    const result = await generateText({
      prompt: 'Write a poem'
    });
    display(result.text);
  } catch (error) {
    showError('AI features require WebLLM extension');
  }
};

Silent Progressive Enhancement

import { webLlmReady, generateText } from 'webllm';

async function init() {
  try {
    // Try to connect silently (5 second timeout)
    await webLlmReady(5000);

    // Extension available - enable AI features
    enableAiButton();
  } catch {
    // Extension not available - hide AI features
    hideAiButton();
  }
}

init();

Check Once, Use Many Times

import { promptInstall, generateText } from 'webllm';

let extensionReady = false;

async function ensureReady() {
  if (!extensionReady) {
    await promptInstall();
    extensionReady = true;
  }
}

// Use in multiple places
async function feature1() {
  await ensureReady();
  return generateText({ prompt: 'Task 1' });
}

async function feature2() {
  await ensureReady();
  return generateText({ prompt: 'Task 2' });
}

Next Steps

Explore the Client Library API
Learn about Vercel AI Provider
Read the API Specification