🤖 AI SDK Dart
A Dart/Flutter SDK inspired by Vercel AI SDK — provider-agnostic APIs for text generation, streaming, structured output, tool use, embeddings, image generation, speech, and more.
What is this?
AI SDK Dart brings Vercel AI SDK concepts to Dart and Flutter with Dart types, streams, and lifecycle primitives. Core workflows share a provider-neutral API; provider-specific capabilities remain explicit.
What's new in 3.0
Version 3.0 was reviewed against Vercel AI SDK 7.0.111 / provider 4.0.17. See the v2-to-v3 migration guide for the full upgrade path and the contract comparison for the implementation-by-implementation audit.
instructions— the canonical top-level instruction, retained across steps;systemremains a deprecated fallback. System-role input messages are rejected by default unlessallowSystemInMessages: true.- Canonical
stream+ callbacks —result.streamis now the exhaustive typed event stream (the old provider-part stream moved toproviderStream);fullStreamis a deprecated alias, not a second producer.onStart,onStepStart,onToolExecutionStart,onToolExecutionEnd,onStepEnd, andonEndare the canonical lifecycle callbacks (onFinish/onStepFinishremain fallbacks). - Aggregate results +
finalStep—usage,content,toolCalls,sources, andfilesaggregate every step;text, structuredoutput,reasoning,reasoningText, andfinalStepdescribe the last step only.responseMessagesholds just the turns generated by that call — append it once to your own history withresult.responseMessages.map(ModelMessage.fromProvider). - Context, approval, and concurrency —
toolWithContextbinds typed application context to a tool executor without sending it to the provider;approvalPolicymoves approval to the request/agent boundary; a restored approval is bound to the exact call ID, arguments, and policy revision.maxToolConcurrencyadmits more than one tool call at a time (serial by default). - Cancellation and deadlines — a shared
CancellationToken(abortSignal) reaches every adapter and closes real transport work.TimeoutConfigurationdistinguishes total/step/first-chunk/chunk/tool deadlines forgenerateText,streamText, andToolLoopAgent. Settimeout: const TimeoutConfiguration(total: Duration(seconds: 30))for a total deadline. Object, embedding, media, and reranking functions acceptDuration. BodyInclusionPolicy— request/response bodies and raw provider chunks are omitted by default; opt in per call when an integration needs to retain them.- OpenAI Responses, Files and Batch —
openai.responses(id)with hosted tools (web search, file search, code interpreter, image generation, remote MCP), plus Files and Batch lifecycle clients. - Reasoning controls — one
reasoningoption mapped to OpenAI reasoning effort, Anthropic thinking and Gemini thinking budgets; signed reasoning is replayed throughresponseMessages. - Provider file references — canonical file parts distinguish bytes, a
URL, and an opaque
DataContentProviderReference(namespace + ID) that an unrelated provider must reject rather than reinterpret as a URL. - New companion packages —
ai_sdk_conversation(immutable persisted snapshots + codec),ai_sdk_remote(trusted-backend UI-message-stream transport),ai_sdk_json_schema(optional local schema validation before decoding), andai_sdk_telemetry(OTLP/HTTP metrics export) — see the package table below.ai_sdk_realtimetracks the same contracts as an unpublished preview. - MCP modern protocol —
MCPProtocolMode.modern, host-owned auth discovery (MCPAuthConfiguration/MCPAuthDiscovery), typed progress, and explicitretryOnTransportFailurenow that a losttools/callreply is no longer retried automatically by default.
Model IDs remain open strings. The dated capability catalog
separates upstream catalog information from protocol fixtures and live
evidence; a model appearing in that catalog does not establish every SDK
feature works with that model. Live-provider qualification (real API
canaries, device audio for ai_sdk_realtime) is tracked separately and
remains in progress — see docs/provider-canaries.md.
Upgrading from 2.x or earlier? The 2.0 migration guide covers the V3→V4 provider-seam rename first; apply it before the 3.0 guide above.
Screenshots
Flutter Chat App (examples/flutter_chat)
Advanced App (examples/advanced_app)
✨ Features
🗣️ Text Generation & Streaming
generateText— single-turn or multi-step text generation with full result envelopestreamText— real-time token streaming with typed event taxonomysmoothStreamtransform — configurable chunk-size smoothing;delayInMsoption adds per-chunk delay for UX pacing- Multi-step agentic loops with
maxSteps,prepareStep, andstopConditions timeoutacceptsTimeoutConfigurationfor text generation, streaming, and agents; object, embedding, media, and reranking functions accept a totalDurationdeadline- Canonical callbacks:
onStart,onStepStart,onToolExecutionStart,onToolExecutionEnd,onStepEnd,onEnd(plusonChunk,onError,onAbort); deprecatedonFinish/onStepFinishremain fallbacks
🧩 Structured Output
Output.object(schema)— parse model output into a typed Dart objectOutput.array(schema)— parse model output into a typed Dart listOutput.choice(options)— constrain output to a fixed set of string valuesOutput.json()— raw JSON without schema validation- Final structured output requires complete JSON; use
extractJsonMiddlewarewhen a model needs code-fence normalization
🔧 Type-Safe Tools & Multi-Step Agents
tool<Input, Output>()— fully typed tool definitions with JSON schemadynamicTool()— tools with unknown input type for dynamic use cases- Tool choice:
auto,required,none, or specific tool - Tool approval workflow with
needsApproval - Multi-step agentic loops with automatic tool result injection
onInputStart,onInputDelta,onInputAvailablelifecycle hooks
🖼️ Multimodal
generateImage— image generation (gpt-image-1 / DALL·E via OpenAI)generateSpeech— text-to-speech audio synthesistranscribe— speech-to-text transcription- Image inputs in prompts (multimodal vision)
🧮 Embeddings & Cosine Similarity
embed()— single value embedding with usage trackingembedMany()— batch embedding for multiple values with configurable chunk sizecosineSimilarity()— built-in similarity computationwrapEmbeddingModel()— composable middleware pipeline for embedding models
🧱 Middleware System
wrapLanguageModel(model: ..., middleware: ...)— composable middleware pipelineextractReasoningMiddleware— strips<think>tags intoReasoningPartextractJsonMiddleware— strips```json ```fencessimulateStreamingMiddleware— converts non-streaming models to streamingdefaultSettingsMiddleware— applies default temperature/top-p/etc.addToolInputExamplesMiddleware— enriches tool descriptions with exampleswrapEmbeddingModel/wrapImageModel— the same composable middleware pattern for embedding and image models
🌐 Provider Registry
createProviderRegistry— map provider aliases to model factoriescustomProvider()— lightweight on-the-fly provider construction without a full registry- Resolve models by
'provider:modelId'string at runtime - Supports 6 model categories: language, embedding, image, speech, transcription, rerank
- Mix providers in a single registry for multi-provider apps
📱 Flutter UI Controllers & Widgets
ChatController— multi-turn streaming chat with message historyCompletionController— single-turn text completion with statusObjectStreamController— streaming typed JSON object updates- 19 prebuilt, themeable Material widgets —
AiChatScaffold, message list/bubbles, composer, streaming text, typing indicator, tool-call & approval cards, reasoning, citations, usage, and more
🔌 MCP Client (Model Context Protocol)
MCPClient— connect to MCP servers, discover tools, invoke themStreamableHttpClientTransport— MCP Streamable HTTP transport (2025-06-18) for remote serversStdioMCPTransport— stdio process transport (native platforms)- Web-safe —
dart:iois isolated behind conditional imports, so the client runs on Flutter web - Discovered tools are directly compatible with
generateText/streamText
🚨 Typed Errors
- Sealed
AiSdkErrorhierarchy —AiApiCallError,AiNoObjectGeneratedError, andAiRetryErrorfor exhausted retryable failures - Provider API errors are typed — a non-2xx response throws
AiApiCallErrorcarrying the provider'smessage,type,code,statusCode, raw body, and anisRetryableflag, consistently across every provider
🧪 Conformance Suite
- Comprehensive Dart and Flutter tests across every package and both example apps
- A 99% line-coverage gate for published package libraries enforced in CI
- Provider wire-format conformance tests for every provider (plus a typed-error conformance test per provider)
MockEmbeddingModelV2testing utility for embedding model conformance
📦 Packages
ai_sdk_providerandai_sdk_openai_compatibleare transitive dependencies — you do not need to add them directly.
🚀 Quick Start
Local development and trusted Dart servers
dart pub add ai_sdk_dart ai_sdk_openai
import 'dart:io';
import 'package:ai_sdk_dart/ai_sdk_dart.dart';
import 'package:ai_sdk_openai/ai_sdk_openai.dart';
Future<void> main() async {
final apiKey = Platform.environment['OPENAI_API_KEY'];
if (apiKey == null || apiKey.isEmpty) {
throw StateError('Set OPENAI_API_KEY before running this example.');
}
final provider = OpenAIProvider(apiKey: apiKey);
try {
final result = await generateText(
model: provider('gpt-4.1-mini'),
prompt: 'Say hello from AI SDK Dart!',
);
print(result.text);
} finally {
provider.dispose();
}
}
For server and CLI apps, prefer reading credentials from your runtime
environment and passing apiKey: yourself, as shown above. The convenience
factories like openai('...'), anthropic('...'), and google('...') read
compile-time defines such as OPENAI_API_KEY, so they are best paired with
dart run --define=... or Flutter --dart-define=....
Production Flutter through a trusted backend
The optional v3 remote package sends application messages to your backend. The backend owns provider credentials, authorization, and tool execution. Use an application session token to authenticate the client.
import 'package:ai_sdk_conversation/ai_sdk_conversation.dart';
import 'package:ai_sdk_remote/ai_sdk_remote.dart';
Stream<Conversation> sendChatTurn({
required Uri endpoint,
required Conversation conversation,
required Future<String> Function() sessionToken,
}) async* {
final transport = RemoteConversationTransport(
endpoint: endpoint,
authHeaders: () async => {
'Authorization': 'Bearer ${await sessionToken()}',
},
);
try {
yield* transport.send(conversation);
} finally {
transport.dispose();
}
}
A Flutter StreamBuilder<Conversation> can render these snapshots. Cancelling
the subscription cancels its active request and runs the cleanup above. Store
snapshots with ConversationCodec; decoding a snapshot does not execute tools.
The backend must implement the pinned UI-message stream protocol. The reference backend and runnable Dart client demonstrate the request and response flow without provider credentials. The adapter does not retry or resume a disconnected turn automatically.
Streaming
import 'dart:io';
final result = await streamText(
model: openai('gpt-4.1-mini'),
prompt: 'Count from 1 to 5.',
);
await for (final chunk in result.textStream) {
stdout.write(chunk);
}
Structured Output
final result = await generateText<Map<String, dynamic>>(
model: openai('gpt-4.1-mini'),
prompt: 'Return the capital and currency of Japan as JSON.',
output: Output.object(
schema: Schema<Map<String, dynamic>>(
jsonSchema: const {
'type': 'object',
'properties': {
'capital': {'type': 'string'},
'currency': {'type': 'string'},
},
},
fromJson: (json) => json,
),
),
);
print(result.output); // {capital: Tokyo, currency: JPY}
Type-Safe Tools
final result = await generateText(
model: openai('gpt-4.1-mini'),
prompt: 'What is the weather in Paris?',
maxSteps: 5,
tools: {
'getWeather': tool<Map<String, dynamic>, String>(
description: 'Get current weather for a city.',
inputSchema: Schema(
jsonSchema: const {
'type': 'object',
'properties': {'city': {'type': 'string'}},
},
fromJson: (json) => json,
),
execute: (input, _) async => 'Sunny, 18°C',
),
},
);
print(result.text);
Error handling
try {
final result = await generateText(
model: openai('gpt-4.1-mini'),
prompt: 'Hello',
);
} on AiApiCallError catch (e) {
// Typed provider error — message, status, and retryability are all available.
print('${e.statusCode}: ${e.message} (retryable: ${e.isRetryable})');
}
Flutter Chat UI
dart pub add ai_sdk_dart ai_sdk_openai ai_sdk_flutter_ui
import 'package:ai_sdk_dart/ai_sdk_dart.dart';
import 'package:ai_sdk_openai/ai_sdk_openai.dart';
import 'package:ai_sdk_flutter_ui/ai_sdk_flutter_ui.dart';
final agent = ToolLoopAgent(
model: openai('gpt-4.1-mini'),
instructions: 'You are a helpful assistant.',
);
final chat = ChatController();
// In your widget — a complete chat surface:
AiChatScaffold(controller: chat, agent: agent);
Do not ship long-lived provider API keys inside distributed browser, mobile, or desktop clients. Use a trusted proxy or backend-minted short-lived credentials instead. The Flutter examples below use
--dart-definefor local development and smoke testing, not as a production secret-distribution strategy.
🤖 Providers
🛠️ Flutter UI
The ai_sdk_flutter_ui package provides three reactive controllers plus a library of 19 prebuilt,
themeable Material widgets — so you can wire up a full chat UI in a few lines, or drop down to the
controllers and render everything yourself.
Drop-in chat UI
import 'package:ai_sdk_dart/ai_sdk_dart.dart';
import 'package:ai_sdk_flutter_ui/ai_sdk_flutter_ui.dart';
import 'package:ai_sdk_openai/ai_sdk_openai.dart';
final agent = ToolLoopAgent(model: openai('gpt-4.1-mini'));
final chat = ChatController();
// A complete message list + composer, wired to the controller + agent:
AiChatScaffold(controller: chat, agent: agent);
Other widgets — ChatMessageList, ChatMessageBubble, ChatComposer, StreamingTextView,
TypingIndicator, ToolCallCard, ToolApprovalCard, ReasoningView, SourceCitations,
UsageView, PromptSuggestions, ObjectStreamView, and more — can be composed à la carte. They
read only the controllers' public state, so they work with any state-management approach.
ChatController — Multi-turn streaming chat
final agent = ToolLoopAgent(model: openai('gpt-4.1-mini'));
final chat = ChatController();
// In your widget:
ListenableBuilder(
listenable: chat,
builder: (context, _) {
return Column(
children: [
for (final msg in chat.messages)
Text('${msg.role}: ${msg.content}'),
if (chat.isLoading) const CircularProgressIndicator(),
],
);
},
);
// Send a message:
await chat.sendMessage(agent: agent, text: 'What is the capital of France?');
CompletionController — Single-turn completion
final completion = CompletionController(
agent: ToolLoopAgent(model: openai('gpt-4.1-mini')),
);
await completion.complete('Write a haiku about Dart.');
print(completion.completion);
ObjectStreamController — Streaming typed JSON
final controller = ObjectStreamController<Map<String, dynamic>>(
model: openai('gpt-4.1-mini'),
schema: Schema<Map<String, dynamic>>(
jsonSchema: const {'type': 'object'},
fromJson: (json) => json,
),
);
await controller.submit('Describe Japan as a JSON object.');
print(controller.value); // Partial updates arrive in real-time
🔌 MCP Support
Connect to any Model Context Protocol server and use its tools directly in your AI calls:
import 'package:ai_sdk_dart/ai_sdk_dart.dart';
import 'package:ai_sdk_mcp/ai_sdk_mcp.dart';
import 'package:ai_sdk_openai/ai_sdk_openai.dart';
final client = MCPClient(
transport: StreamableHttpClientTransport(
url: Uri.parse('http://localhost:3000/mcp'),
headers: {'Authorization': 'Bearer <short-lived-token>'},
),
);
await client.initialize();
final tools = await client.tools(); // Returns a ToolSet
final result = await generateText(
model: openai('gpt-4.1-mini'),
prompt: 'What files are in the project?',
tools: tools,
maxSteps: 5,
);
For stdio-based MCP servers (local processes):
final client = MCPClient(
transport: StdioMCPTransport(
command: 'npx',
args: ['-y', '@modelcontextprotocol/server-filesystem', '/path/to/dir'],
),
);
StreamableHttpClientTransport speaks the MCP Streamable HTTP transport
(2025-06-18) against a single endpoint. It negotiates the protocol version
during initialize(), sends notifications/initialized, accepts JSON or SSE
responses to each POST, starts the optional GET SSE listener for
server-pushed notifications, reconnects that listener with Last-Event-ID, and
sends DELETE on shutdown when the server assigned Mcp-Session-Id. Put
required auth or routing headers in headers, but avoid embedding long-lived
secrets in shipped browser or mobile clients. The HTTP transport is web-safe —
dart:io is only pulled in by StdioMCPTransport on native platforms, behind
a conditional import — so the client also runs on Flutter web.
The client also has an explicit MCPProtocolMode.modern strategy for
protocol 2026-07-28, including per-request metadata, MRTR input-required
results, typed progress, and subscriptions/listen. MCP Tasks, cache-hint
interpretation, and arbitrary partial tool-output streams are outside the
current package surface; progress notifications do not represent partial
results.
🗺️ Roadmap
✅ Implemented
- ✅
generateText— full result envelope (text, steps, usage, reasoning, sources, files) - ✅
streamText— complete event taxonomy (20 typed event types),onAbortcallback - ✅
generateObject/ structured output (object, array, choice, json) with native JSON schema - ✅
embed/embedMany+cosineSimilarity,wrapEmbeddingModel - ✅
generateImage(OpenAI gpt-image-1 / DALL·E) - ✅
generateSpeech(OpenAI TTS) - ✅
transcribe(OpenAI Whisper) - ✅
rerank - ✅
timeoutparameter on all core functions - ✅
customProvider()for lightweight on-the-fly provider construction - ✅ Middleware system — 5 built-in language-model middlewares, plus embedding & image model middleware
- ✅ Provider registry (
createProviderRegistry) — 6 model categories - ✅ Multi-step agentic loops with tool approval
- ✅ Flutter UI controllers (Chat, Completion, ObjectStream) + 19 prebuilt Material widgets
- ✅ MCP client (Streamable HTTP + stdio transports, prompts, resources, web-safe)
- ✅ Typed provider API errors (
AiApiCallErrorwith status / type / code / body) across all providers - ✅ OpenAI (with reasoning options), Anthropic (with thinking options), Google providers
- ✅ Cohere, Mistral, Groq, Ollama, Azure OpenAI providers — all with tools + multimodal
- ✅ Comprehensive tests with a 99% line-coverage gate for published package libraries
🔜 Planned
- 🔜 Versioned MCP Tasks/polling adapter and richer extension support
- 🔜 Richer attachment widgets (file/image pickers, audio capture)
- 🔜 Dart Edge / Cloudflare Workers support
- 🔜 WebSocket transport for MCP
🤝 Contributing
Contributions are welcome! Please open an issue first to discuss changes before submitting a PR.
- 🐛 Bug reports — use the Bug Report template
- 💡 Feature requests — use the Feature Request template
- 💬 Questions & discussions — use GitHub Discussions
Running tests
fvm flutter pub get
make test
make analyze
Run make benchmark for the structured-stream throughput benchmark.
Or run a smaller set of pinned toolchain smoke checks directly:
fvm dart analyze .
fvm dart test packages/ai_sdk_dart/test/
fvm dart test packages/ai_sdk_openai/test/
fvm dart test packages/ai_sdk_anthropic/test/
fvm dart test packages/ai_sdk_google/test/
fvm flutter test examples/flutter_chat/
fvm flutter test examples/advanced_app/
Runnable examples
CLI and server examples can read credentials from the process environment. The
repo make targets forward those values to the provider factories as
compile-time defines when needed:
OPENAI_API_KEY=sk-... make run-basic
OPENAI_API_KEY=sk-... make run-mcp
Equivalent direct Dart invocation:
OPENAI_API_KEY=sk-... \
fvm dart run --define=OPENAI_API_KEY=sk-... examples/basic/lib/main.dart
Flutter example apps are different: they read compile-time defines from
String.fromEnvironment, so pass keys with --dart-define:
cd examples/flutter_chat && fvm flutter run \
--dart-define=OPENAI_API_KEY=sk-...
cd examples/advanced_app && fvm flutter run \
--dart-define=OPENAI_API_KEY=sk-... \
--dart-define=ANTHROPIC_API_KEY=sk-ant-... \
--dart-define=GOOGLE_API_KEY=AIza...
The Flutter commands above compile the keys into the client app. Use them for local demos only. Production apps should call a trusted backend or fetch short-lived provider credentials instead of embedding long-lived secrets.
Development
Managed with the Dart pub workspace and the repository Makefile:
fvm flutter pub get
make analyze
make test
See docs/v6-parity-matrix.md for a feature-by-feature parity matrix against Vercel AI SDK v6.