Digital.ai Testing — MCP Server
Talk to your Digital.ai Testing device farm from your AI assistant. This MCP (Model Context Protocol) server connects AI clients such as Claude, GitHub Copilot and Cursor to Digital.ai Testing, so you can ask in plain language to find devices, install apps, explore an app on a real device, generate tests, run test suites, investigate failures and compare performance.
It provides 210 tools, 2 resources and 7 prompts across 27 capability areas: device management, test execution (including Espresso, XCUITest and Maestro test runs), app lifecycle, reporting and root-cause analysis, analytics, performance, project administration, interactive mobile inspection, interactive browser inspection, and more.
Quick Start
1. Check that Node.js 22 or later is installed (npm comes with it). If it isn't, install the LTS version from https://nodejs.org first.
node --version
npm --version
2. Install the MCP server.
npm install -g digital-ai-testing-mcp
3. Add it to your AI client with your tenant URL and access key. Claude Desktop is shown here; see Connecting AI Clients for VS Code, JetBrains, Copilot and Cursor.
{
"mcpServers": {
"digital-ai-testing": {
"command": "digital-ai-testing-mcp",
"args": [],
"env": {
"DIGITAL_AI_BASE_URL": "https://your-tenant.experitest.com",
"DIGITAL_AI_ACCESS_KEY": "your-access-key",
"MCP_DEPLOYMENT_MODE": "local"
}
}
}
}
4. Restart the client and ask: "Show me the overall health of the device farm."
Prefer a container? See Option B — Pull from GitHub Container Registry.
Table of Contents
Get started
- Quick Start
- Installation — npm · Docker image · from source · Upgrading
- Connecting AI Clients — Claude Desktop · Claude Code (VS Code) · Claude Code (JetBrains) · GitHub Copilot · Cursor · Docker configs
- Access Keys — access levels, finding your key, multiple profiles
- Configuration — Toolsets · Debug mode
Use it
- What You Can Do — example prompts · tasks at a glance · all capability areas
- Creating Tests — mobile · Test Orchestrator · web · validating a test
- Test Results and Root Cause Analysis
- Performance Comparison Reports
- Workflow Reference — POC lifecycle · Project lifecycle
- Safety Guards
Help and reference
- Troubleshooting
- Reference — response format · maxResults · list filters · local file transfer (Docker)
- Known Limitations
- Development
- License
Installation
What you need:
- A Digital.ai Testing account with a valid access key.
- Node.js 22+ for Option A (recommended — no Docker needed), or
- Docker for Option B or Option C (container-based install).
Option A — Install via npm (Recommended)
No Docker required — the server runs directly under Node.js.
npm install -g digital-ai-testing-mcp
This puts a digital-ai-testing-mcp command on your PATH. Use it as the "command" in your AI client configuration. You don't need an image name or a .env file: settings go in the client's env block (see Configuration).
To run without a global install, use "command": "npx" with "args": ["-y", "digital-ai-testing-mcp"] instead — npx downloads and runs the latest published version.
Set MCP_DEPLOYMENT_MODE=local alongside your credentials (the client examples below already include it). It tells the server that its filesystem is your own machine, so tool descriptions and a few upload/download behaviors drop the Docker-specific caveats and path warnings.
Option B — Pull from GitHub Container Registry
No cloning or building required. Use this if you'd rather run the server in a container than install it with npm.
Step 1 — Pull the image
docker pull ghcr.io/digitalai-opensource/digital-ai-testing-mcp:latest
Step 2 — Set up your .env
curl -O https://raw.githubusercontent.com/digitalai-opensource/digital-ai-testing-mcp/main/.env.example
cp .env.example .env
# Edit .env — set DIGITAL_AI_BASE_URL and DIGITAL_AI_ACCESS_KEY
Step 3 — Connect your client using ghcr.io/digitalai-opensource/digital-ai-testing-mcp:latest as the image name — see Docker client configuration.
Images are published when a GitHub Release is created — not on every commit. Each release is also tagged with its version (e.g.
2.0.3and2.0) if you want to pin one. Iflatestis not available yet, use Option C to build from source.
With Docker, tools that read or write local files work inside the container, not on your machine — see Local File Transfer.
Option C — Build from source
Needed only for local development or to modify the server.
git clone https://github.com/digitalai-opensource/digital-ai-testing-mcp
cd digital-ai-testing-mcp
cp .env.example .env
# Edit .env — set DIGITAL_AI_BASE_URL and DIGITAL_AI_ACCESS_KEY
docker build -t digital-ai-testing-mcp:latest .
Use digital-ai-testing-mcp:latest as the image name in your Docker client configuration.
Upgrading
To confirm the upgrade, ask your AI to call get_server_info, which reports the running version.
Connecting AI Clients
The examples below use the npm install (Option A): credentials go directly in the client's env block. If you run the server in Docker (Options B/C), use the Docker client configuration instead.
Claude Desktop
Find your config file:
- macOS:
~/Library/Application Support/Claude/claude_desktop_config.json - Windows: Open Claude Desktop → Settings → Developer → Edit Config (the path varies by install method — Windows Store vs. direct installer)
{
"mcpServers": {
"digital-ai-testing": {
"command": "digital-ai-testing-mcp",
"args": [],
"env": {
"DIGITAL_AI_BASE_URL": "https://your-tenant.experitest.com",
"DIGITAL_AI_ACCESS_KEY": "your-access-key",
"MCP_DEPLOYMENT_MODE": "local"
}
}
}
}
Restart Claude Desktop after editing — the tools appear automatically.
Claude Code (VS Code)
- Open the Claude Code extension panel
- Go to Settings → MCP Servers
- Add a new server with command
digital-ai-testing-mcp, no args, andDIGITAL_AI_BASE_URL/DIGITAL_AI_ACCESS_KEY/MCP_DEPLOYMENT_MODE=localset in the server's environment variables
Using Docker? Add the Docker command from Docker client configuration instead.
Claude Code (JetBrains / Android Studio)
Install the Claude Code plugin from the JetBrains Marketplace, then open your project and run this command from the project root:
claude mcp add digital-ai-testing --env DIGITAL_AI_BASE_URL=https://your-tenant.experitest.com --env DIGITAL_AI_ACCESS_KEY=your-access-key --env MCP_DEPLOYMENT_MODE=local -- digital-ai-testing-mcp
This stores the server configuration in ~/.claude.json, scoped to the current project. Alternatively, use the Claude Code panel: Settings → MCP Servers and add the same command used for Claude Desktop.
Restart the Claude Code panel after adding the server — the tools appear automatically.
Using Docker? See Docker client configuration — and on Windows, use forward slashes in the .env path.
GitHub Copilot (VS Code)
GitHub Copilot supports MCP tools in Agent mode only. Register the server in one of two scopes:
- User settings — available in all workspaces. Open the user MCP configuration (
Ctrl+Shift+P→ "MCP: Open User Configuration") and add the sameserversentry shown for the workspace file below. (Older VS Code versions used an"mcp": { "servers": { … } }block in the usersettings.jsoninstead.) - Workspace settings — scoped to one project. Create
.vscode/mcp.json:
{
"servers": {
"digital-ai-testing": {
"command": "digital-ai-testing-mcp",
"args": [],
"env": {
"DIGITAL_AI_BASE_URL": "https://your-tenant.experitest.com",
"DIGITAL_AI_ACCESS_KEY": "your-access-key",
"MCP_DEPLOYMENT_MODE": "local"
}
}
}
}
Committing .vscode/mcp.json to source control shares the server configuration with the entire team automatically.
To use the tools: open Copilot Chat (Ctrl+Alt+I), switch the mode dropdown to Agent, and type your request.
GitHub Copilot on the web (
github.com/copilot) does not support external MCP servers. VS Code is required.
Using Docker? See Docker client configuration.
Cursor
Cursor supports MCP in Agent mode. Add the server in Cursor Settings → MCP (or Cursor Settings → Features → MCP):
{
"mcpServers": {
"digital-ai-testing": {
"command": "digital-ai-testing-mcp",
"args": [],
"env": {
"DIGITAL_AI_BASE_URL": "https://your-tenant.experitest.com",
"DIGITAL_AI_ACCESS_KEY": "your-access-key",
"MCP_DEPLOYMENT_MODE": "local"
}
}
}
}
Alternatively, create a .cursor/mcp.json file in your project root with the same mcpServers object — this scopes the server to that workspace and can be committed to share it with your team.
Cursor is available on macOS, Windows, and Linux. It is the recommended option for iOS developers on macOS, where Xcode is the primary IDE but does not natively support MCP.
Using Docker? See Docker client configuration.
Docker client configuration
For Option B or Option C, the client starts the container instead of the digital-ai-testing-mcp command, and credentials come from your .env file. In every example:
- Replace
/ABSOLUTE/PATH/TO/.envwith the full path to your.envfile. - Built from source (Option C)? Replace
ghcr.io/digitalai-opensource/digital-ai-testing-mcp:latestwithdigital-ai-testing-mcp:latest.
Claude Desktop and Cursor (mcpServers format):
{
"mcpServers": {
"digital-ai-testing": {
"command": "docker",
"args": [
"run", "--rm", "-i",
"--env-file", "/ABSOLUTE/PATH/TO/.env",
"ghcr.io/digitalai-opensource/digital-ai-testing-mcp:latest"
]
}
}
}
GitHub Copilot — in either configuration file, replace the "digital-ai-testing" entry's command/args/env with:
{
"command": "docker",
"args": [
"run", "--rm", "-i",
"--env-file", "/ABSOLUTE/PATH/TO/.env",
"ghcr.io/digitalai-opensource/digital-ai-testing-mcp:latest"
]
}
Claude Code (JetBrains / Android Studio, or VS Code) — from the project root, or add the same command under Settings → MCP Servers:
# macOS / Linux
claude mcp add digital-ai-testing -- docker run --rm -i --env-file /absolute/path/to/.env ghcr.io/digitalai-opensource/digital-ai-testing-mcp:latest
# Windows (use forward slashes — backslashes are stripped by Claude Code)
claude mcp add digital-ai-testing -- docker run --rm -i --env-file C:/projects/digital-ai-testing-mcp/.env ghcr.io/digitalai-opensource/digital-ai-testing-mcp:latest
Windows users on Docker: always use forward slashes (
C:/path/to/.env), never backslashes — whether you register via the CLI command above or through the Claude Code panel. Claude Code silently strips backslashes when it writes the configuration to~/.claude.json, which breaks the path and causes a cryptic-32000reconnection error (how to fix it).
Access Keys
Your Digital.ai access key decides what the MCP server can do on your behalf. There are three access levels:
You can't tell the access level from the key's format. Older keys are long
eyJ…tokens; since Digital.ai 24.11, newly generated keys use a shorteraut_1_…token — for Cloud Admins too. So the server asks the API for your role (GET /api/v1/users/my-account-info→Admin/ProjectAdmin/User) and never guesses from the key's prefix.get_server_infoshows the level detected for the active profile, andlist_environmentsshows it for every profile. Per-tool role requirements are in docs/tools.md.
If you call a Cloud Admin tool with a project-level role, the server returns a plain-language explanation — and, if a Cloud Admin profile is configured, a ready-to-use switch_environment(...) command.
Finding your key
- Log in to the Digital.ai Testing portal
- Click your name or avatar in the top-right corner
- Select Access Key
The key shown is tied to the project currently selected in the portal. If you belong to multiple projects, each has a separate key.
Using multiple keys
Set up named profiles to switch between keys (for example, Cloud Admin and a project) without editing files. With the npm install, add the variables to the env block of your AI client's configuration; with Docker, add them to your .env file:
# Default connection (typically Cloud Admin credentials — either key format works)
DIGITAL_AI_BASE_URL=https://your-tenant.experitest.com
DIGITAL_AI_ACCESS_KEY=...your-cloud-admin-key...
# Project-scoped profiles
DAI_PROFILE_QA_URL=https://your-tenant.experitest.com
DAI_PROFILE_QA_KEY=...your-qa-key...
DAI_PROFILE_STAGING_URL=https://your-tenant.experitest.com
DAI_PROFILE_STAGING_KEY=...your-staging-key...
Then ask your AI:
- "What environments are configured?" →
list_environments - "Switch to QA" →
switch_environment("qa") - "Switch back to Cloud Admin" →
switch_environment("cloud admin")(a role alias — resolves to whichever profile the API reports as Cloud Admin, even if it is not the one nameddefault)
The switch takes effect immediately — no restart needed.
Security: Never commit
.envto source control. Each team member should use their own key, obtained from the portal under their own account.
Configuration
Set these as environment variables: in the client's env block (npm install) or in your .env file (Docker).
Additional DAI_PROFILE_{NAME}_URL / DAI_PROFILE_{NAME}_KEY pairs configure named profiles for multi-project or multi-environment use. See Using multiple keys and .env.example for examples.
Toolsets (reducing context size)
All 210 tool definitions together take up roughly 70K tokens of your AI's context. If your AI client is short on context — or runs several MCP servers — set MCP_TOOLSETS to the areas you use, e.g. MCP_TOOLSETS=reporting (about 26K tokens), MCP_TOOLSETS=inspection,authoring (about 34K), or MCP_TOOLSETS=core (about 17K — only the always-loaded core in full).
Nothing becomes unavailable. Tools outside the chosen toolsets are still listed, with a one-line description. When the AI calls one, the server loads that whole toolset and replies with the tool's full description and parameter guidance, then the AI calls it again — so it always sees the detailed usage guidance before the tool runs. That first call executes nothing (it is marked read-only for clients that honour tool annotations). The AI can also load toolsets up front with enable_toolset, whose reply lists a typed signature for every tool it loaded — so even a client that never refreshes its tool list has the exact parameter names and types. Leave MCP_TOOLSETS unset (the default) to load everything in full.
Always loaded in full: environment and connection tools (get_server_info, switch_environment, …), enable_toolset, and validate_test_script.
Debug mode (remediation notes)
Set MCP_DEBUG_MODE=true while evaluating or developing the MCP to collect feedback for improving it. With the default false, nothing changes.
When it is on:
-
The AI records notes. The server asks the AI client to call
record_remediation_note(a tool that exists only in debug mode) whenever it:- hits an unexpected error;
- wastes calls because guidance was unclear;
- is corrected or stopped by you;
- gives up;
- finds a better path than the one recommended;
- spots a way to make things cleaner or cheaper.
It records each note once the situation is resolved or abandoned, not in the middle of your task. Expected outcomes, such as deliberate negative tests or a safety check that worked as intended, are not recorded: they show nothing to improve.
-
The server logs what it sees. Every tool call is logged with its duration, response size and outcome (ok, error or guard), with flags for retries and repeated calls. This is objective data the AI can't under-report.
-
The AI is told up front. The debug-mode instructions come first in what the server sends at connect time, and
get_server_infoandcheck_connectivityshow "Debug mode: ON". -
Errors get a reminder. The first time a tool returns a given error or guard, the response starts with a one-line reminder to record a note if it was unexpected (in JSON responses, as a
_debugModefield).
Debug mode costs extra tokens (the instructions and the notes). Leave it off for normal use, and don't use debug sessions as token baselines.
Where the files go: a remediation/ folder in the root of the project the AI is working in, so each note sits next to the app and tests it's about.
With the npm install, the server finds the project in this order, and records the step it used as location: in each file:
MCP_REMEDIATION_DIR, if set (files go directly into that folder);- the workspace folder the AI client reports;
./remediationin the folder the server was started in, if that folder is a project (has a.git,package.json,pom.xml, …).
Otherwise nothing is written: events stay in memory and each note is returned to the AI client to save, as in Docker mode. Notes are never written to your home folder, a drive root or a system folder.
The remediation/ folder contains its own .gitignore (*), so it is never committed to the project it lives in. There is one file per session; the <id> suffix keeps parallel sessions apart, and each file records the machine and user, so files collected from several machines can be merged. Keys, tokens, emails and signed-URL parameters are redacted before anything is written or returned.
To share notes, send the remediation/ folder (or several, from different machines) to the MCP maintainers — they are analysed in batches with internal tooling that is not part of this repository.
What You Can Do
Example prompts
Once connected, talk to the server in plain language — no tool names needed:
- "Show me the overall health of the device farm — how many devices are available, reserved, and offline?"
- "Find an available Android phone running at least version 13 in the US2 region"
- "Did all tests in the QA project pass today? I need a go/no-go for the release."
- "Install the latest build of com.mycompany.app on all available Android devices"
- "Find an available Android phone, then generate a Java JUnit5 test boilerplate"
- "Show the pass/fail breakdown by OS — are Android and iOS failing at different rates?"
- "Did the latest release introduce a CPU regression? Compare version 10553 vs 10554 on Android."
- "Set up a new POC for Acme Corp — 6 devices in US2, 3 iOS and 3 Android, for the next 4 weeks"
→ docs/examples.md has 200+ prompts organized by scenario: farm health, device groups, regression sign-off, test analytics, performance, usage reports, interactive inspection, POC lifecycle, and more.
Tasks at a glance
Interactive inspection
An inspection session opens a live connection to a real device or browser so the AI can see the screen and interact with it — useful for exploring an app and capturing real element IDs before writing a test.
- Mobile (tools) — Android and iOS, on both Appium Grid and Appium Server projects: screenshots, element discovery, the full gesture set, keys, app/device control (including background/resume, mid-session install from the repository and a webview-cache reset), hybrid-app webview contexts (
webview_context), the live device log, HAR and device-log capture into the file repository, network toggles and NV network conditions, camera image injection, device diagnostics (battery, screen, dark mode, iOS accessibility audit), Android Auto / CarPlay projection, audio play/record on devices with an audio line, and mock biometric authentication. The session shares a watch-only live view URL so you can follow along. iOS behaves differently in a few places (limitation 29), and Appium Grid projects support fewer commands (limitation 18). - Browser (tools) — navigate, discover elements (including inside Shadow DOM), verify CSS selectors and generate browser-neutral Selenium scripts.
- Remote debug (tools) —
get_remote_debug_commandmakes a cloud device appear as if it were plugged into your machine (over ADB for Android, usbmuxd for iOS), so Android Studio, Xcode and command-line ADB can use it — for example for shell commands or file push. Install the app before connecting (see Troubleshooting).
Example prompts: Interactive Inspection, Android Auto / CarPlay, Web Browser Inspection.
Test runs (Espresso, XCUITest, Maestro)
The platform runs these suites itself: execute_test_run starts a run, get_test_run_status follows it, cancel_test_run stops it, and get_test_run_command gives a command to start it from your own machine. generate_maestro_flow builds a Maestro flow from structured steps. Maestro is Android-only. Espresso and Maestro runs are verified; XCUITest uses the same API but is not yet verified end to end. See Test Runs and example prompts.
Appium and Selenium tests are different: you run them from your own Appium or Selenium client (IDE, CI scripts), and the server manages the devices, apps, reservations and results they use.
Usage reports
Cloud Admins can export the platform's usage-report CSVs — device reservations, users/devices/browser usage, user statistics, license usage — or have summarize_usage_report aggregate one in memory and return just the summary, with no file written. Usage data is kept for 2 years (platform 25.9+); earlier periods come back empty or partial. A large unfiltered export runs only after you explicitly agree (confirmLargeExport, see Safety Guards). See Usage Reports, limitation 21 and example prompts.
All capability areas
210 tools across 27 capability areas. The complete per-tool reference — descriptions, filters, access requirements and usage notes — is in docs/tools.md.
Creating Tests
Two ways to create a test
The server supports two ways of creating a test, for both mobile and web, and the AI is instructed to pick between them based on context:
Autonomous — get_test_boilerplate (mobile) / get_web_test_boilerplate (web)
- When: your intent is specific — a standard flow ("create a login test") or step-level detail — AND the selectors are known (app source in the workspace, or a prior inspection session).
- Your involvement: none required — the AI writes the test from the boilerplate, the source and best practices.
- Output: in an IDE, a local test automation project; in a chat-only client, portable project files shown inline.
Interactive — start_inspection_session / start_browser_inspection_session, or the collaborative_test_creation / collaborative_web_test_creation prompts
- When: the request is vague ("I want a test for app X"), the AI has no access to the app's source (chat-only, or an IDE without this app's code), or you want to watch or drive on a live device or browser.
- Your involvement: collaborative — the AI builds the test step by step with you (browser sessions relay screenshots; mobile sessions share a live device view URL).
- Output: the same, plus selectors verified against the real app or page along the way.
A mix is often best: even in autonomous mode, the AI can open a short inspection session — with no involvement from you — to capture or check element IDs or CSS selectors against the real build. The AI is explicitly instructed to never make up selectors: when in doubt it checks live, and when the mode itself is unclear it asks.
Stronger guarantees: the server sends this policy to the AI automatically (in its MCP
instructions), and both boilerplate tools refuse to produce code for a real app or URL without verified selectors. But not every client passes serverinstructionson to the AI, and no MCP server can stop an AI from writing a test file by hand — so you can add a client-side reinforcement for a guarantee that doesn't depend on the client. See Recommended agent guardrails.
Mobile app tests (get_test_boilerplate)
Generates a complete, pre-configured Appium test project. The Digital.ai server URL and access key are filled in from the active connection profile — switch profiles with switch_environment first if you want the script to carry a project-scoped key instead of your admin key.
Recommended workflow for building a new test:
get_application_info— confirm package name, launch activity, and app IDfind_available_device— select a healthy device and read itsregioninstall_application— install the app on that device before starting a sessionstart_inspection_session— capture real element IDs withget_element_tree/find_elementsget_test_boilerplate(appId, region, …)— generates code while the inspection session is live, or withconfirmSelectorsVerified: truevalidate_test_script— before saving or presenting the test- Run the test, then
stop_inspection_sessionandrelease_device— explicit cleanup
Inspection gate: when you target a real app (
appId/packageName/bundleIdentifier), this tool returns no code — just a structured{ status: "blocked", reason: "no_verified_selectors" }error — UNLESS a live inspection session exists in the MCP server, or you setconfirmSelectorsVerified: true(for selectors already captured another way: rdb/UIAutomator,open_mobile_studio, or authoritative source). The built-in ExperiBank demo (no app identifiers) is never gated. This way you can never receive a placeholder scaffold that could be passed off as a finished test.
Finding element locators: use an inspection session first — start_inspection_session → get_element_tree / find_elements gives the AI the element tree, screenshots and gestures, and is the only way to capture verified selectors inside the MCP. open_mobile_studio (the platform's browser-based UI Inspector) is a visual alternative that needs no local tooling. For direct device access (ADB shell commands and file push on Android, Xcode on iOS), use get_remote_debug_command to connect the device locally.
⚠️ The generated script is a finished product, not a discovery tool — don't run it to find out what's on screen. Running it without known element selectors creates Incomplete sessions visible to the whole team. Discover selectors with an inspection session (or
open_mobile_studio) first.
Generated files. projectType controls the Java output layout only (it is ignored for Node.js, Python and Robot Framework): standalone-gradle produces src/test/java/ with both build.gradle and pom.xml; standalone-maven produces pom.xml only; android-gradle-submodule puts the files under e2e-tests/ (with e2e-tests/build.gradle) for embedding in an existing Android Studio project.
All boilerplate uses the ExperiBank demo app as its starting point. ExperiBank is available on most farm devices, so you can verify connectivity before switching to your own app.
Locators and waits. Every generated step goes through an explicit-wait helper (find() in Java/Python/Node, Wait And Click / Wait And Input in Robot) that polls up to 20 seconds for the element; no template sets an implicit wait, and adding one would make every miss wait both timeouts. Locators follow a fixed ladder — Android: resource-id, then accessibility id (content-desc), then an attribute XPath; iOS: accessibility id (the element's name: identifier, else label — the same match as a name XPath without walking the tree), then -ios predicate string (Appium Server only), then an attribute XPath. Never positional XPath ([1], [2]) and never fixed sleeps; validate_test_script warns on both. The full table is in docs/tools.md.
Parallel runs. Every template already creates one session per test method, which is the shape that survives parallel execution. The response's parallelNote says how to turn it on per framework (pytest-xdist, JUnit 5 junit-platform.properties, a TestNG parallel="methods" suite, WebdriverIO maxInstances, pabot) and reminds you to check the project's maxDevelopmentLicense and the number of free matching devices first — a worker that finds no device waits for newSessionWaitTimeout and then fails.
Performance transactions: pass includePerformanceTransactions: true to wrap the test body in startPerformanceTransaction / endPerformanceTransaction calls. The start argument is the NV network profile (defaults to "Monitor" — observe without throttling); the end argument is the transaction name. The platform records CPU, memory, battery and Speed Index metrics; results appear in the reporter Transactions tab about 1 minute after endPerformanceTransaction and can be queried with list_transactions. Prerequisite: an NV server must be ONLINE and tunnel-connected in the device's region (list_nv_servers). A throttling profile turns on network shaping immediately; "Monitor" is pass-through and carries no ANR (app-not-responding) risk.
Accessibility scanning: pass includeAxeScan: true to add a Deque Axe DevTools Mobile accessibility scan (a mobile: axeScan executeScript call). It sets the required appium:automationName capability automatically (AxeUiAutomator2 for Android, AxeXCUITest for iOS) and pins appiumVersion to 2.16.2 (the Axe driver is unavailable on Appium Server 3.3.0+; don't raise it). The key comes from AXE_DEVTOOLS_API_KEY; if that's unset, a placeholder is written.
Both flags can be combined — the Axe scan runs inside the performance transaction.
Test Orchestrator (default for Java on Appium Server)
For Java (JUnit 5 / TestNG) on an Appium Server project, the generated project is set up for the Digital.ai Test Orchestrator (the Smart Agent): failed tests are retried automatically, each attempt is reported separately, the real pass/fail is synced to Reporter, and runs are grouped by Build ID — with no change to the test code. Everything else (Appium Grid projects, Python, Node.js and Robot Framework) gets the classic boilerplate. Pass orchestration: "off" to get the classic boilerplate for Java too.
get_test_boilerplateadds conditional agent wiring tobuild.gradle/pom.xml, anorchestrator/config.template.yml(retries default to 2; 0 for placeholder or performance tests), plusorchestrator/.gitignoreandlib/.gitignoreso the rendered key and the agent JAR can never be committed. No root.gitignoreis written (it would overwrite yours).install_test_orchestrator_agent(projectDir)downloads the agent on demand, verifies its SHA-256 and places it atlib/smart-agent.jar. Under Docker it returns a download command (bash and PowerShell) that does the same and deletes the file on a checksum mismatch. If the download fails, the response gives the URL, the checksum and the target path for a manual install.- Set
DIGITAL_AI_ACCESS_KEYand rungradle test/mvn test. The agent attaches only when the JAR is present and the key is set; the key is rendered into the git-ignoredorchestrator/rendered/at build time, never into a tracked file, and Java 17 is required only in that case. Otherwise the project builds and runs exactly like the classic boilerplate (Gradle prints why the agent was not attached). SetBUILD_IDto group a CI build's results.
Requirements: a Java 17 or 21 JDK with a current truststore, JUnit 5 or TestNG 7+, Appium java-client 8–11 (the generated projects pin 11.0.0). See Limitations for known issues.
The agent JAR is not shipped with this server. By default it is downloaded from the Digital.ai Test Orchestrator sample repository, pinned to a fixed commit and verified by SHA-256 (see resources/test-orchestrator/agent.json). To use a different location — for example an internal mirror — set TEST_ORCHESTRATOR_JAR_URL and TEST_ORCHESTRATOR_JAR_SHA256 (both required; a URL without a checksum is refused).
Web tests (get_web_test_boilerplate)
Generates a complete, pre-configured Selenium test project for browser testing. The Digital.ai server URL and access key are filled in from the active connection profile.
Recommended workflow for building a new web test:
list_available_browsers— confirm which browsers and OS combinations are available in your regionstart_browser_inspection_session— open a live browser session on the target URLget_page_dom— extract the rendered DOM, including shadow trees; note element IDs anddata-testidvaluesfind_web_elements/tap_element/type_into_element— interact with elements to verify selectors worktake_inspection_screenshot— relay screenshots at each key step for verificationstop_browser_inspection_sessionget_web_test_boilerplate(confirmSelectorsVerified: true)— generate the verified script (once the session is stopped, passshadowDomSupport: "always"ifget_page_domreported shadow DOM)
Inspection gate: when you target a real URL (
urlprovided), this tool returns no code — just a structured{ status: "blocked", reason: "no_verified_selectors" }error — UNLESS a live browser inspection session exists in the MCP server, or you setconfirmSelectorsVerified: true(for selectors captured in a priorstart_browser_inspection_sessionsession). This mirrors the mobile gate.
Browser-neutral by default: when targetBrowser is omitted, the generated script reads the browser name at runtime from the BROWSER environment variable, so the same script runs on Chrome, Firefox, Microsoft Edge, Safari and Opera without changes (Robot Framework: every browser except Opera, which SeleniumLibrary cannot drive) — just change the variable or your CI matrix. Specify targetBrowser only when you need browser-specific options (e.g. Safari-only session setup).
Shadow DOM support: with always, the generated script includes a shadowQuery(host, cssSelector) helper; with auto (the default), it is included only while a browser inspection session is open in the MCP server; never omits it. The helper reaches into shadow roots using executeScript. Use it for apps built with Web Components, Lit, Polymer or other shadow-DOM frameworks. get_page_dom detects shadow DOM automatically during inspection and reports hasShadowDom: true when present.
Check a test before you deliver it (validate_test_script)
A final check for the case the boilerplate gate can't catch: a test written by hand with guessed selectors. Pass the full scriptContent before presenting or saving any test. It looks for unreplaced <…> placeholder selectors, the deliberate scaffold fail-guard, placeholder or made-up credentials, and known fabricated resource IDs. If it finds a serious problem it returns isError with a fail verdict, so a test that can't work isn't delivered as finished. A pass is necessary but not sufficient: it confirms the obvious placeholders are gone, not that the selectors are real — a live inspection is still the authoritative source.
Test Results and Root Cause Analysis
The reporting tools search test reports, summarize pass/fail results and explain failures. Field names, filter properties and ID types are in the Test reporting schema; more prompts are in Root Cause Analysis and Functional Test Analytics.
Two things to know:
- The list doesn't include failure details.
list_test_reportsreturns summary records; callget_test_reporton a test to get its cause, error category and stack trace. - Filter by date with
startDate/endDate, not with astart_timefilter (the platform rejects that).
Common queries:
# Recent failures
list_test_reports
filter: [{"property":"status","operator":"=","value":"Failed"}]
sort: [{"property":"start_time","descending":true}]
limit: 20
# Date range
list_test_reports
startDate: "2026-05-01T00:00:00Z"
endDate: "2026-05-15T23:59:59Z"
# Pass/fail by OS
get_grouped_test_reports
groupBy: ["device.os"]
pivotBy: ["status"]
# Project summary
get_project_test_summary
startDate: "2026-05-01T00:00:00Z"
projectName: "My Project"
Root cause analysis of a failure:
# 1. Find the failures
list_test_reports
filter: [{"property":"status","operator":"=","value":"Failed"}]
# 2. Diagnose — structured cause without any download
get_test_report(testId: 69)
→ cause, errorCategory, errorClassification, errorDetail (stack trace)
# 2b. Platform root-cause analysis (read-only)
get_root_cause_analysis(uuid: "…") # or reportUrl / testId
# 3. Full Appium log as text, when the structured fields aren't enough
get_test_log(uuid: "…", logType: "appium")
# 4. Pull the binary artifacts (session video .mp4, full ZIP) to YOUR machine.
# With the npm install, call download_test_video / download_test_attachments
# directly; under Docker/remote, generate a local command instead:
get_test_attachments_download_command(uuid: "…", localPlatform: "macos")
→ a curl command you run locally; the file never transits the container
Tests missing from the results? See Troubleshooting — a project with its own reporter needs projectName.
Performance Comparison Reports
Compare performance (Speed Index, CPU, memory, battery, duration) between two sets of conditions — app v1 vs v2, device A vs device B, OS version X vs Y, region to region, two network profiles, or two automation scripts. Works with all access levels — Cloud Admin sees all projects; project-level keys (Project Admin and Project User) see only their own project's transactions.
The headline metric is reported three ways every time — trimmed mean, median, and raw mean — so a small or noisy sample can't hide behind a single number.
Why the confound check matters: comparing two transactions with Speed Index 1015 vs 1000 looks like a 1.5% version delta — but if they ran on different device models, different OS versions, and different projects, and one has no CPU/memory telemetry,
assess_comparison_confoundsreturns confounded and names each cause, so the delta is never misread as a version regression.
The performance_comparison_report prompt runs the full workflow end to end: define the axis → scrub confounds → negotiate sample size → require explicit plan confirmation with a time estimate → run the series with outlier-driven re-runs → report the delta with root-cause reasoning. True host/background interference can't be observed directly; the report approximates device quietness (idle, healthy status, no concurrent reservation) and states that limit explicitly.
To record new performance samples from a generated test, see performance transactions in Creating Tests. More prompts: Performance Testing Analytics.
Workflow Reference
Six workflow tools set up and wind down POC and project environments. They don't make changes themselves: each returns a guided, numbered plan that the AI then carries out with the individual tools (create_device_group, create_user, …), pausing for your confirmation at the points the plan specifies. The plans use Cloud Admin tools (user, group and project administration), so run them with a Cloud Admin profile.
POC Lifecycle
Three tools cover the full POC lifecycle. All derive the project name ("<Customer> POC") and device tag ("<customername>poc") from customerName — pass the same value to all three tools.
create_poc — Onboard a new POC environment
Collects all parameters up front, checks that the app binaries are in the repository, presents a confirmation summary to the operator (Step 0), then runs 10 steps in order:
- Create (or reuse) a device group named
<Customer> POC - Locate the Default device group
- Select available phones from the Default group — region-matched, conflict-tag-free, phones only (tablets excluded). Presents the selection to the operator for confirmation before proceeding.
- Add selected devices to the POC group
- Tag each device with the derived POC tag (e.g.
acmecorppoc) - Create (or reuse) a project; record the Salesforce URL and end date in project notes
- Remove devices from the Default group — done only after the project exists, so a project-creation failure never strands devices
- Locate the Default project
- Create users, assign to the POC project, remove from Default, and tag each account with the POC tag. Cloud Admin access is never granted through this workflow.
- Assign ExperiBank (or the app identified by
androidPackageName/iosBundleId) to the POC project — if a binary isn't in the repository, the summary reports it as a manual upload and the workflow continues
If any step fails, the workflow stops, reports which steps completed with all created resource IDs, and offers to resume (call it again — existing resources are detected and reused) or unwind (delete_poc). Steps 1 and 6 check for an existing group/project by name before creating, so it is safe to re-run after a partial failure.
Required parameters:
Optional parameters:
close_poc — Wind down without deleting
Reverses the device and user changes from create_poc while leaving the project and device group intact:
- Locate the POC project and device group by derived name
- List devices in the POC group
- Remove the POC tag from each device (all other tags are preserved)
- Move devices out of the POC group and back into the Default group
- Process users by POC tag: delete accounts that have no other project memberships; revoke POC access only for users who also belong to other projects
Only customerName is required.
The Digital.ai REST API has no endpoint to lock or disable a user. Deleting the account via
delete_useris the only way the API offers to remove access for users created only for the POC.
delete_poc — Full teardown
Performs the same wind-down as close_poc, then permanently deletes the device group and the project. Requires confirmDeletion: true.
Before any destructive action, the workflow gathers a full inventory (project ID, group ID, device list, user breakdown) and presents it for explicit confirmation. Unlike close_poc, user processing is confirmed once for the whole batch via this inventory rather than user by user.
Best practice: Run close_poc first — it winds down the POC while preserving the project and device group. Note that close_poc is not fully reversible: device moves and tag removal can be undone, but it permanently deletes POC-only user accounts (each deletion requires per-user operator confirmation). Use delete_poc only once the project data is confirmed no longer needed. Calling delete_poc without confirmDeletion: true returns a safe explanation and suggests close_poc as the alternative.
Project Lifecycle
Three tools mirror the POC lifecycle for general project management. They follow the same rule for users who belong to several projects: those users are removed from this project only, never deleted from the platform.
setup_project — Provision a new project environment
If you call it with only a project name (no region, deviceCount or users), the workflow first asks whether to create a simple project record only, or a full environment (device group, device allocation, user provisioning, app assignment) — and for a full environment, collects those parameters. If any of them are supplied, it goes straight to the full setup and asks you to confirm the parameters.
Full setup steps:
- Create (or reuse) the project with the specified name and automation type; record
projectMemoin project notes - Create (or reuse) a device group and link it to the project
- Pre-flight check that the target app exists in the repository
- Select available devices matching the target OS and region — presented to the operator for confirmation
- Add the selected devices to the project group
- Tag each device with a project-derived tag
- Remove devices from other groups — only when
isolateDevices: true; by default existing group links (including Default) are kept intact - Create users, assign them to the project with the specified roles, remove them from Default, and tag each account
- Assign the specified application to the project
Required parameters:
Optional parameters:
close_project_resources — Release resources without deleting
Winds down the project environment while leaving the project record intact:
- Locate the project and its associated device group
- List the devices in the project group
- Remove the project-derived tag from each device
- Return devices to the Default group
- Process the users carrying the project tag (applied by
setup_project): delete accounts with no other project memberships; revoke project access only for multi-project users (each action individually confirmed by the operator)
Only projectName is required. Active sessions and installed apps on the devices are not touched — release sessions with release_orphaned_sessions and remove apps with uninstall_application separately if needed.
teardown_project — Full project deletion
Performs all close_project_resources steps, then permanently deletes the device group and the project. Requires confirmDeletion: true.
The workflow presents a full inventory summary before any irreversible action; as with delete_poc, user processing is confirmed once for the whole batch via this inventory rather than user by user. Users who belong to other projects are protected in the same way.
Best practice: Run close_project_resources first, then confirm the project is no longer needed before calling teardown_project.
Safety Guards
All destructive operations require confirmDeletion: true. Without it:
⚠️ Safety guard triggered. "Delete user 42" is a destructive operation that cannot be undone. To confirm you want to proceed, include
confirmDeletion: truein your request. No changes were made.
The first call describes what will be deleted. The second call — with confirmDeletion: true — carries it out. This prevents accidental data loss when an AI assistant misreads your intent. release_orphaned_sessions works the same way: without confirmDeletion: true (or with dryRun: true) it only previews the devices it would release.
Two other actions have their own confirmation flags:
confirmPublicShare—share_test_reportcreates a public link anyone can open without logging in, and it cannot be revoked early (see limitation 22). Without the flag it returns a preview only.confirmLargeExport— usage-report tools stop before an unfiltered export spanning more than 31 days (it can reach hundreds of MB) and ask. Only you should approve such an export, not the AI on its own (see limitation 21).
More checks run before any request leaves the server:
- Permission check first: report-deletion tools (
delete_test_reports*,cleanup_inspection_sessions,cleanup_browser_inspection_sessions) check your detected access level first. A project-level role is refused up front when its project'sallowUsersDeleteTestssetting is readable and false, with a message telling you to have a Cloud Admin enable it (or switch to a Cloud Admin profile) — instead of an unexplained 403. - Path validation: tools that read or write local files require an absolute path and refuse path traversal (
..). Upload tools also refuse credential-file names (.env*, SSH private keys), so a misdirected request cannot publish secrets to the cloud repository.
Troubleshooting
Common symptoms
Failed to reconnect to digital-ai-testing: -32000
On Windows with Docker, this usually means the --env-file path was stored with its backslashes stripped. Open ~/.claude.json, find the mcpServers entry for this project, and check that the env file path looks right (e.g. C:/projects/digital-ai-testing-mcp/.env). If it shows something like C:projectsdigital-ai-testing-mcp.env, the backslashes were removed. Fix the path in the file directly (use forward slashes), then run /mcp in Claude Code to reconnect.
Workflow tools failing?
Run this sequence:
1. get_server_info — confirm tool count (expect 210; 211 with MCP_DEBUG_MODE=true) and active profile
2. check_workflow_readiness — which dependency tools are present or missing
3. check_connectivity — confirm the backend API is reachable
check_workflow_readiness returns a structured report:
{
"allWorkflowsReady": true,
"registeredToolCount": 210,
"workflows": {
"create_poc": { "ready": true, "missingRead": [], "missingWrite": [] },
"setup_project": { "ready": true, "missingRead": [], "missingWrite": [] }
}
}
If ready is false, missingRead and missingWrite list exactly which tools are absent. The most common cause is an outdated install — see Upgrading.
The server also logs a readiness check at startup (visible in your terminal, or docker logs/your AI client's MCP server logs if running in a container): Workflow readiness: all workflows ready ✓, or ⚠️ DEGRADED: create_poc — missing: ... when tools are absent.
Reference
Response Format
All data-returning tools accept an outputFormat parameter:
The default is "json" because the main consumers are AI agents that need to pick out IDs for follow-up calls. An agent can turn JSON into readable output for you; it can't reliably pull IDs out of prose.
# Example: pass device group IDs without string manipulation
list_device_groups { outputFormat: "json" }
→ {"deviceGroups":[{"id":"7","name":"Default"},{"id":"12","name":"Acme Corp POC"}]}
get_devices_in_group { groupId: "7" }
→ {"devices":[{"id":"abc123","name":"iPhone 14 Pro",...}]}
add_devices_to_group { groupId: "12", deviceIds: ["abc123"] }
Tools that only make changes (create, delete, install, etc.) return simple success messages and have no outputFormat parameter. One exception to the json default: get_page_dom defaults to human (a readable element list); pass outputFormat: "json" to chain its results.
Understanding maxResults
The Digital.ai API returns complete datasets — most endpoints have no server-side paging. The MCP caps list responses at maxResults (default: 50, max: 500) to keep responses a manageable size for AI assistants. list_test_reports uses limit (same default and maximum) plus page, and find_elements details at most 20 matches by default (see limitation 19).
Use filters to narrow results rather than raising maxResults. When results are cut off, you will see:
⚠️ Showing 50 of 312 results. Use filters to narrow results, or increase maxResults (max 500) to see more.
List Filters & Sorting
The list tools with sortBy also accept sortOrder: "asc" | "desc" (default "asc", except list_applications, which defaults to "desc", newest first).
list_test_reports sorting is done by the platform for every access level. If the platform ever refuses it for a credential, the server automatically falls back to scanning up to 5,000 records itself, so "newest first" results stay correct, just slower on large report sets.
Local File Transfer (Docker / remote deployments only)
If you installed the npm package (Option A) and run the server on your own machine, skip this section — the download_*/upload_* tools already read and write your local files directly.
The rest of this section applies only to Docker or remote deployments (Option B or C). There, any tool that reads or writes a local file path works on the MCP server's own filesystem — which, in the published Docker image or a remote deployment, is not your machine. A download_* tool writes into the container; an upload_* tool reads from it. Neither is reachable from your local shell unless you mount a shared folder into the container.
For those deployments, every file-transfer tool has a command-generator counterpart that produces a curl / PowerShell command you run locally, so the file moves directly between your machine and the platform — never through the container: