# Apps API
Source: https://docs.autosana.ai/api-apps
List apps and their builds, and manage default Chrome extension dependencies
Use the Apps API to discover app and build UUIDs, then configure the default
Chrome extensions loaded with a web app. All requests require an API key in the
`X-API-Key` header. See [API Reference](/api-reference) for authentication.
## List Apps
**GET** `/api/v1/apps` — Returns `200 OK`
Optional platform filter: `ios`, `android`, `web`, or `chrome-extension`.
Optional. Only apps with exactly this bundle ID.
Optional. Only apps with this `app_id`: a mobile app's custom app ID when it has one, otherwise its bundle ID. This is not the UUID in `id`.
Optional. Only apps in the environment with this name. An unknown name returns `400`.
```bash theme={null}
curl "https://backend.autosana.ai/api/v1/apps?platform=chrome-extension" \
-H "X-API-Key: YOUR_API_KEY"
```
```json theme={null}
{
"apps": [
{
"id": "extension-app-uuid",
"name": "MetaMask",
"platform": "chrome-extension",
"bundle_id": "nkbihfbeogaeaoehlefnkodbefgpgknn",
"app_id": "nkbihfbeogaeaoehlefnkodbefgpgknn",
"ios_keychain_access_group_remapping_enabled": false,
"active_build": {
"id": "extension-build-uuid"
}
}
],
"count": 1
}
```
`id` is the app's UUID, which the other endpoints on this page take in their
path. `app_id` is the ID you pass as `app_id` when you upload or run: a mobile
app's custom app ID when it has one, otherwise its bundle ID. For a web app it
is the app ID. When a mobile app's `app_id` equals its bundle ID, passing it is
the same as leaving `app_id` out, so it never creates a second app.
`active_build` is `null` when an app has no active build. `bundle_id` and
`app_id` may also be `null`. `ios_keychain_access_group_remapping_enabled` is an app-level iOS
preference for future `.ipa` uploads; it does not indicate whether the active
build was instrumented.
To find the app for one release version, combine the filters, then use the
returned `id` to [list its builds](#list-app-builds):
```bash theme={null}
curl "https://backend.autosana.ai/api/v1/apps?platform=android&bundle_id=com.example.app&app_id=v7.12.0" \
-H "X-API-Key: YOUR_API_KEY"
```
## List App Builds
List all of an app's builds, newest first, with the commit and branch each one
was built from. Filter by `commit_sha` or `branch_name` to find a specific build,
then pass its `id` as `app_build_id` when you [run flows](/api-runs#run-flows).
**GET** `/api/v1/apps/{app_id}/builds` — Returns `200 OK`
The app's UUID: the `id` returned by [List Apps](#list-apps), not its `app_id` or bundle ID. Anything else returns `404`.
Optional. A full 40-character commit SHA, or a prefix of at least 7
characters. Matching is case-insensitive.
Optional. Exact branch name, such as `main` or `feature/login`. Combine it
with `limit=1` to get the latest build on a branch.
Optional. Return only the newest builds, from `1` to `1000` of them. Leave it
out to get every matching build.
```bash theme={null}
curl "https://backend.autosana.ai/api/v1/apps/APP_UUID/builds?commit_sha=0123456" \
-H "X-API-Key: YOUR_API_KEY"
```
```json theme={null}
{
"app_id": "app-uuid",
"builds": [
{
"id": "build-uuid",
"commit_sha": "0123456789abcdef0123456789abcdef01234567",
"branch_name": "main",
"created_at": "2026-10-05T18:00:00+00:00"
}
],
"count": 1
}
```
`builds` is empty when nothing matches. When both filters are set, a build
must match both. A commit can have more than one build if it was uploaded
again, so take the first result for the most recent one.
`commit_sha` and `branch_name` are `null` for builds uploaded without git
metadata, and deleted builds are not listed. An invalid `commit_sha` or `limit`
returns `422`.
## List App Dependencies
List the Chrome extensions attached by default to a web app.
**GET** `/api/v1/apps/{app_id}/dependencies` — Returns `200 OK`
```bash theme={null}
curl "https://backend.autosana.ai/api/v1/apps/WEB_APP_UUID/dependencies" \
-H "X-API-Key: YOUR_API_KEY"
```
```json theme={null}
{
"app_id": "web-app-uuid",
"dependencies": [
{
"app": {
"id": "extension-app-uuid",
"name": "MetaMask",
"platform": "chrome-extension",
"bundle_id": "nkbihfbeogaeaoehlefnkodbefgpgknn",
"app_id": "nkbihfbeogaeaoehlefnkodbefgpgknn",
"ios_keychain_access_group_remapping_enabled": false,
"active_build": {
"id": "extension-build-uuid"
}
}
}
],
"count": 1
}
```
## Attach a Dependency
Attach a Chrome extension app to a web app. Future runs inherit the
extension's active build unless the run request supplies an explicit
`dependencies` override.
**POST** `/api/v1/apps/{app_id}/dependencies` — Returns `201 Created`
```bash theme={null}
curl -X POST \
"https://backend.autosana.ai/api/v1/apps/WEB_APP_UUID/dependencies" \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"dependency_app_id": "EXTENSION_APP_UUID"}'
```
```json theme={null}
{
"app_id": "web-app-uuid",
"dependency": {
"app": {
"id": "extension-app-uuid",
"name": "MetaMask",
"platform": "chrome-extension",
"bundle_id": "nkbihfbeogaeaoehlefnkodbefgpgknn",
"app_id": "nkbihfbeogaeaoehlefnkodbefgpgknn",
"ios_keychain_access_group_remapping_enabled": false,
"active_build": {
"id": "extension-build-uuid"
}
}
}
}
```
The primary app must have platform `web`, and the dependency must have platform
`chrome-extension`. Both apps must belong to the API key's organization.
## Detach a Dependency
**DELETE** `/api/v1/apps/{app_id}/dependencies/{dependency_app_id}` — Returns `200 OK`
```bash theme={null}
curl -X DELETE \
"https://backend.autosana.ai/api/v1/apps/WEB_APP_UUID/dependencies/EXTENSION_APP_UUID" \
-H "X-API-Key: YOUR_API_KEY"
```
```json theme={null}
{
"app_id": "web-app-uuid",
"dependency_app_id": "extension-app-uuid",
"detached": true
}
```
Detaching changes future runs only. Historical run results retain the exact
extension build UUIDs they used. To override defaults for one run, including
running with no extensions, see [Run Flows](/api-runs#run-flows).
# App Build Upload API
Source: https://docs.autosana.ai/api-ci
Upload mobile builds or web app URLs programmatically via the API
These endpoints allow you to upload mobile app builds or web app URLs programmatically. For most use cases, we recommend using our [GitHub Action](/ci-cd-integration) instead.
All requests require an API key in the `X-API-Key` header. See [API Reference](/api-reference) for authentication details.
***
## Endpoints Overview
| Endpoint | Platform | Description |
| - | - | - |
| [Start Upload](#start-upload) | iOS, Android | Get a presigned URL to upload a build file |
| [Confirm Upload](#confirm-upload) | iOS, Android | Finalize a mobile build upload |
## Start Upload
Initiate an app build upload and get a presigned URL for uploading your build file.
**POST** `/api/ci/start-upload` — Returns `200 OK`
### Request Body
Your app's bundle identifier (e.g., `com.company.app`)
Target platform: `ios` or `android`
Name of the build file (e.g., `app-release.apk` or `MyApp.zip`)
Display name for your app. Used when the app is created, and updates the name if the app already exists with a different one.
Custom identifier that keeps several apps with the same bundle ID in one environment, for example one app per release version (`v7.12.0`). The app is created the first time an `app_id` is used. Up to 64 characters: lowercase letters, numbers, dots, hyphens, and underscores. It must start and end with a letter or number, with no two separators in a row. If omitted, the build goes to the app identified by bundle ID alone. Passing your bundle ID as `app_id` is the same as leaving it out, and the format rule does not apply to it.
Environment name to associate this app with (e.g., `staging`, `production`). Apps with the same bundle ID but different environments are treated as separate apps. Must match an existing environment in your organization. If omitted, the app is assigned to your default environment.
### Example Request
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/ci/start-upload \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"bundle_id": "com.company.app",
"platform": "android",
"filename": "app-release.apk"
}'
```
### Response Fields
Presigned URL for uploading your build file via PUT request. Valid for 1 hour.
Storage path of the uploaded file. Pass this to the confirm-upload endpoint.
### Example Response
```json theme={null}
{
"upload_url": "https://storage.supabase.co/...",
"file_path": "app-uuid/android/app-release-20250115T103000.apk"
}
```
After receiving the response, upload your build file to the `upload_url` using a PUT request:
```bash theme={null}
curl -X PUT "UPLOAD_URL_FROM_RESPONSE" \
-H "Content-Type: application/octet-stream" \
--data-binary @./path/to/your/app.apk
```
***
## Confirm Upload
After uploading your build file, call this endpoint to finalize the upload and trigger any configured automations.
**POST** `/api/ci/confirm-upload` — Returns `200 OK`
### Request Body
Your app's bundle identifier (must match the one used in start-upload)
Target platform: `ios` or `android`
The `file_path` returned from the start-upload response
Custom app identifier. Must match the value used in start-upload. Leaving it out and passing your bundle ID count as the same value. A mismatch is rejected.
Display name for your app (e.g., "My Android App"). If the app already exists and the name differs, it will be updated.
Environment name (must match the value used in start-upload). Used to look up the correct app when multiple apps share the same bundle ID.
Git commit SHA for tracking which commit this build came from.
Git branch name. Displayed in the Autosana UI for build identification.
Repository name in `org/repo` format (e.g., `myorg/myrepo`). Required for [GitHub Bot](/github-integration) integration — links this build to your repository so the bot can find it when processing PRs.
Key-value variables to attach to this build. Available in flow instructions via `${env:KEY}`. Accepts a string (`"KEY1=VALUE1,KEY2=VALUE2"`) or a JSON object (`{"KEY1": "VALUE1"}`). See [Build Variables](/variables#build-variables).
iOS `.ipa` only. Persists the app preference and instruments the IPA so Team-ID-prefixed keychain access groups keep working after cloud device re-signing. When omitted, the app's saved preference is used.
### Example Request
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/ci/confirm-upload \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"bundle_id": "com.company.app",
"platform": "ios",
"uploaded_file_path": "app-uuid/ios/MyApp-20250115T103000.ipa",
"name": "My iOS App",
"environment": "staging",
"commit_sha": "abc123def456",
"branch_name": "feature/new-login",
"repo_full_name": "myorg/myrepo",
"enable_ios_keychain_access_group_remapping": true,
"variables": "PR_NUMBER=42,BRANCH=feature/new-login"
}'
```
Android / non-IPA uploads omit `enable_ios_keychain_access_group_remapping`:
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/ci/confirm-upload \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"bundle_id": "com.company.app",
"platform": "android",
"uploaded_file_path": "app-uuid/android/app-release-20250115T103000.apk",
"name": "My Android App",
"environment": "staging",
"variables": "PR_NUMBER=42,BRANCH=feature/new-login"
}'
```
### Response Fields
Result status: `success` or `error`
Human-readable description of the result
Number of automations triggered by this upload (based on your [Automations](/automations) configuration).
### Example Response
```json theme={null}
{
"status": "success",
"message": "App build uploaded successfully",
"triggered_flows": 3
}
```
***
## Upload Workflow
Here's the complete workflow for uploading a build via the API:
Call `/api/ci/start-upload` with your app details to get a presigned upload URL.
PUT your build file (`.apk` for Android, `.zip` for iOS) to the presigned URL.
Call `/api/ci/confirm-upload` to finalize the upload and trigger automations.
### Build Requirements
Upload a `.zip` file containing a simulator-compatible `.app` bundle. See [App Build Guide](/app-build-guide) for details.
Upload a universal `.apk` file (not AAB). The APK must be compatible with `x86_64` emulators.
***
## Example: Upload from EAS Build Hooks
If you use Expo EAS, you can upload to Autosana directly from an [EAS build hook](https://docs.expo.dev/build-reference/npm-hooks/) instead of using the GitHub Action. This avoids keeping a GitHub Actions runner idle while waiting for the EAS build to finish, saving CI minutes.
Create an `eas-build-on-success.sh` file in your project root (next to `package.json`). EAS automatically runs this script when a build completes successfully.
Replace `com.example.myapp` with your bundle ID, `MyApp.app` with your `.app` bundle name, and `"My App"` with your app's display name.
```bash eas-build-on-success.sh theme={null}
#!/usr/bin/env bash
set -euo pipefail
upload_to_autosana() {
if [[ -z "${AUTOSANA_API_KEY:-}" ]]; then
echo "AUTOSANA_API_KEY is not set, skipping upload."
return 0
fi
local platform="$EAS_BUILD_PLATFORM"
local commit_sha="${EAS_BUILD_GIT_COMMIT_HASH:-unknown}"
local branch_name="${EAS_BUILD_GIT_COMMIT_BRANCH:-}"
local bundle_id="com.example.myapp"
local app_name="My App"
local repo_full_name="myorg/myrepo" # TODO: Replace with your org/repo
if [[ "$platform" == "ios" ]]; then
local app_path
app_path=$(find . -name "MyApp.app" -type d 2>/dev/null | head -1)
if [[ -z "$app_path" ]]; then
echo "Error: Could not find .app bundle"
return 1
fi
local filename="app-simulator.zip"
local artifact="/tmp/${filename}"
pushd "$(dirname "$app_path")"
zip -r "$artifact" "$(basename "$app_path")"
popd
elif [[ "$platform" == "android" ]]; then
local artifact
artifact=$(find . -name "*.apk" -type f 2>/dev/null | head -1)
if [[ -z "$artifact" ]]; then
echo "Error: Could not find .apk file"
return 1
fi
local filename=$(basename "$artifact")
else
echo "Unsupported platform: $platform"
return 1
fi
local start_response
start_response=$(curl -sf -X POST https://backend.autosana.ai/api/ci/start-upload \
-H "X-API-Key: $AUTOSANA_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"bundle_id\": \"$bundle_id\",
\"platform\": \"$platform\",
\"filename\": \"$filename\"
}")
local upload_url file_path
upload_url=$(echo "$start_response" | jq -r '.upload_url')
file_path=$(echo "$start_response" | jq -r '.file_path')
if [[ -z "$upload_url" || "$upload_url" == "null" ]]; then
echo "Error: Failed to get presigned URL"
echo "Response: $start_response"
return 1
fi
if ! curl -sf -X PUT "$upload_url" \
-H "Content-Type: application/octet-stream" \
--data-binary "@$artifact"; then
echo "Error: Failed to upload file to presigned URL"
return 1
fi
if ! curl -sf -X POST https://backend.autosana.ai/api/ci/confirm-upload \
-H "X-API-Key: $AUTOSANA_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"bundle_id\": \"$bundle_id\",
\"platform\": \"$platform\",
\"uploaded_file_path\": \"$file_path\",
\"name\": \"$app_name\",
\"commit_sha\": \"$commit_sha\",
\"branch_name\": \"$branch_name\",
\"repo_full_name\": \"$repo_full_name\"
}"; then
echo "Error: Failed to confirm upload"
return 1
fi
echo "Uploaded to Autosana successfully"
}
upload_to_autosana || echo "Warning: Autosana upload failed, continuing..."
```
Make the script executable and commit it to your repo:
```bash theme={null}
chmod +x eas-build-on-success.sh
```
Then add your API key as an EAS secret:
```bash theme={null}
eas secret:create --name AUTOSANA_API_KEY --value your-api-key-here --scope project
```
When you trigger a build (`eas build --platform ios --profile preview-simulator --non-interactive`), EAS runs the hook automatically after the build succeeds — no `--wait` flag or GitHub runner required.
## Upload Web Build
Register a web app URL for testing. This endpoint creates or updates the app and creates a new build record.
**POST** `/api/ci/upload-web-build` — Returns `200 OK`
### Request Body
Unique identifier for your web app. Must be lowercase alphanumeric with hyphens only (e.g., `my-web-app`). This identifier is used to track your web app across deployments.
The URL to test. Must start with `http://` or `https://`. This is typically your preview deployment URL.
Display name for your web app (e.g., "My Web App"). If the app already exists and the name differs, it will be updated.
Environment name to associate this app with (e.g., `staging`, `production`). Apps with the same app ID but different environments are treated as separate apps. Must match an existing environment in your organization. If omitted, the app is assigned to your default environment.
Git commit SHA for tracking which commit this build came from.
Git branch name. Displayed in the Autosana UI for build identification.
Repository name in `org/repo` format. Required for GitHub Check integration.
Key-value variables to attach to this build. Available in flow instructions via `${env:KEY}`. Accepts a string (`"KEY1=VALUE1,KEY2=VALUE2"`) or a JSON object (`{"KEY1": "VALUE1"}`). See [Build Variables](/variables#build-variables).
### Example Request
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/ci/upload-web-build \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"app_id": "my-web-app",
"url": "https://my-app-preview.vercel.app",
"name": "My Web App",
"commit_sha": "abc123def456",
"branch_name": "feature/new-login",
"repo_full_name": "myorg/myrepo",
"variables": {"PR_NUMBER": "42", "DEPLOY_URL": "https://pr-42.preview.app"}
}'
```
### Response Fields
Result status: `success` or `error`
Human-readable description of the result
The internal app ID (UUID) for the registered web app
The internal build ID (UUID) for this registration
Number of automations triggered (if no GitHub metadata provided)
Status of GitHub check runs: `pending` (if GitHub metadata provided)
### Example Response
```json theme={null}
{
"status": "success",
"message": "Web URL registered successfully",
"app_id": "550e8400-e29b-41d4-a716-446655440000",
"build_id": "6ba7b810-9dad-11d1-80b4-00c04fd430c8",
"triggered_flows": 2
}
```
### app\_id Format Requirements
The `app_id` parameter must follow these rules:
* Lowercase letters and numbers only
* Hyphens allowed between words
* No spaces, underscores, or special characters
* Cannot start or end with a hyphen
* Maximum 64 characters
**Valid examples:** `my-web-app`, `staging`, `preview-app-123`, `frontend`
**Invalid examples:** `My-Web-App`, `my_web_app`, `my web app`, `-my-app`
## Upload an Extension Zip
Use the normal two-step upload endpoints with
`"platform": "chrome-extension"`. The build must be a `.zip` of an unpacked
Manifest V3 extension. Extension apps are organization-wide and do not take an
`environment`.
```bash theme={null}
# 1. Request an upload URL
curl -X POST https://backend.autosana.ai/api/ci/start-upload \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"bundle_id": "my-wallet-extension",
"platform": "chrome-extension",
"filename": "extension.zip",
"name": "My Wallet Extension"
}'
# 2. PUT extension.zip to the returned upload_url
# 3. Confirm the upload
curl -X POST https://backend.autosana.ai/api/ci/confirm-upload \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"bundle_id": "my-wallet-extension",
"platform": "chrome-extension",
"uploaded_file_path": "FILE_PATH_FROM_STEP_1",
"name": "My Wallet Extension"
}'
```
You can also snapshot a public Chrome Web Store extension without uploading a
file:
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/ci/upload-extension-from-store \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"store_url": "https://chromewebstore.google.com/detail/metamask/nkbihfbeogaeaoehlefnkodbefgpgknn",
"name": "MetaMask"
}'
```
Limits: 500MB compressed, 2GB expanded, and 25,000 archive entries. Encrypted
archives, symlinks, unsafe paths, and Manifest V2 are rejected.
# Environment Variables API
Source: https://docs.autosana.ai/api-env-vars
List environments and manage environment variables programmatically
These endpoints let you browse [environments](/environments) and manage their environment variables programmatically. Use them to bulk-import config from another secrets manager, keep credentials in sync with CI, or build dashboards on top of Autosana.
All requests require an API key in the `X-API-Key` header. See [API Reference](/api-reference) for authentication and the standard `401` / `403` / `429` / `500` error shapes shared by every endpoint on this page.
**Secret values are write-only through the API.** When `is_secret=true`, the value is stored encrypted in vault and is never returned by any read endpoint — list and get responses always render secret values as `"***"`. There is no API path to retrieve a decrypted secret.
During test runs, secrets are never sent to the AI agent. The agent works with the `${env:KEY}` placeholder and the real value is injected only at the moment keystrokes are sent to your app, so run history records the placeholder, not the value. See [Secret Variables](/environments#secret-variables).
Environments themselves (create / rename / delete) are managed from the [dashboard](https://autosana.ai/settings?tab=environments). The API surface here covers env vars and read access to the environments that contain them.
***
## List Environments
Returns every environment in your organization with its env vars nested. Plaintext values come back in full; secret values are masked as `"***"`.
**GET** `/api/v1/environments` — Returns `200 OK`
### Example Request
```bash theme={null}
curl -X GET https://backend.autosana.ai/api/v1/environments \
-H "X-API-Key: YOUR_API_KEY"
```
### Response Fields
List of environments.
Unique identifier (UUID).
Environment name (e.g. "Staging", "Production").
Environment variables, alphabetically by `key`.
Unique identifier (UUID).
UUID of the parent environment.
Variable name (referenced from flows as `${env:KEY}`).
Plaintext value, or `"***"` placeholder when `is_secret=true`. **Never** the decrypted secret.
Whether the value is stored encrypted in vault.
Optional human-readable note.
ISO 8601 timestamp.
ISO 8601 timestamp.
Total number of environments returned.
#### Example Response
```json theme={null}
{
"environments": [
{
"id": "770e8400-e29b-41d4-a716-446655440099",
"name": "Staging",
"env_vars": [
{
"id": "880e8400-e29b-41d4-a716-446655440010",
"environment_id": "770e8400-e29b-41d4-a716-446655440099",
"key": "API_TOKEN",
"value": "***",
"is_secret": true,
"description": null,
"created_at": "2026-01-15T10:30:00Z",
"updated_at": "2026-01-15T10:30:00Z"
},
{
"id": "880e8400-e29b-41d4-a716-446655440011",
"environment_id": "770e8400-e29b-41d4-a716-446655440099",
"key": "API_URL",
"value": "https://staging.api.example.com",
"is_secret": false,
"description": "Staging API base URL",
"created_at": "2026-01-15T10:31:00Z",
"updated_at": "2026-01-15T10:31:00Z"
}
]
}
],
"count": 1
}
```
***
## Create Env Var
**POST** `/api/v1/env-vars` — Returns `201 Created`. Returns `409 Conflict` if an env var with the same `key` already exists on this environment.
### Request Body
UUID of the environment this variable belongs to.
Variable name (1–255 characters). Referenced from flows and curl hooks as `${env:KEY}`.
Variable value — must be a non-empty string. Empty values are rejected with `422` because they're indistinguishable from "unset" at consumption time. For secrets, this is written to vault and never returned by any read endpoint.
Optional human-readable note.
Defaults to `false`. When `true`, the value is stored encrypted in vault and is never readable back through the API.
### Example Request (plaintext)
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/v1/env-vars \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"environment_id": "770e8400-e29b-41d4-a716-446655440099",
"key": "API_URL",
"value": "https://staging.api.example.com",
"description": "Staging API base URL"
}'
```
### Example Request (secret)
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/v1/env-vars \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"environment_id": "770e8400-e29b-41d4-a716-446655440099",
"key": "API_TOKEN",
"value": "abc123...super-secret",
"is_secret": true
}'
```
#### Example Response
```json theme={null}
{
"id": "880e8400-e29b-41d4-a716-446655440010",
"environment_id": "770e8400-e29b-41d4-a716-446655440099",
"key": "API_TOKEN",
"value": "***",
"is_secret": true,
"description": null,
"created_at": "2026-01-15T10:30:00Z",
"updated_at": "2026-01-15T10:30:00Z"
}
```
***
## Get Env Var
**GET** `/api/v1/env-vars/{env_var_id}` — Returns `200 OK` with the same shape as items in [List Environments](#list-environments)' `env_vars`, or `404` if the var does not exist. Secret values are returned as `"***"`.
UUID of the env var.
```bash theme={null}
curl -X GET https://backend.autosana.ai/api/v1/env-vars/ENV_VAR_UUID \
-H "X-API-Key: YOUR_API_KEY"
```
***
## Update Env Var
Update an env var. Only provided fields change; omitted fields are unchanged.
**PATCH** `/api/v1/env-vars/{env_var_id}` — Returns `200 OK`. `400` if no recognized fields are provided. `404` if the var does not exist. `409` if renaming `key` collides with another env var in the same environment. `422` for invalid transitions (see below).
**No-op PATCHes don't bump `updated_at`.** If the body resolves to no actual state change — e.g. `{"is_secret": true}` on a row that's already a secret — the endpoint returns the unchanged row without writing to the DB. Don't rely on PATCH to act as a heartbeat.
UUID of the env var.
New variable name (1–255 characters). **Note:** any flows or curl hooks that reference the old name as `${env:OLD_KEY}` must be updated in lockstep — there is no automatic rewrite. Collisions with another env var in the same environment return `409`.
New value. Omit to keep the existing value unchanged. Required when changing `is_secret` in either direction (the API never re-uses the existing value across a secret/plaintext transition). `null` is treated as "no change", not "clear" — there is no way to clear a value without deleting the row.
New description. Pass `""` or `null` to explicitly clear (this is the only field where `null` clears rather than meaning "no change").
Move the var between plaintext and vault-encrypted storage. Switching the flag in either direction requires `value`.
### Secret transition rules
| Existing state | Request | Result |
| - | - | - |
| Plaintext | `value: "new"` | Plaintext value rewritten in place. |
| Plaintext | `is_secret: true`, no `value` | **422** — promoting to secret requires a `value`. |
| Plaintext | `is_secret: true`, `value: "secret-x"` | New vault row written; row repointed to it. |
| Secret | `value: "rotated"` | New vault row written, old one cleaned up after the row is repointed. |
| Secret | `is_secret: false`, no `value` | **422** — demoting to plaintext requires a `value`. |
| Secret | `is_secret: false`, `value: "plain"` | Old vault row cleaned up, plaintext value written. |
### Example Request
```bash theme={null}
# Rotate an existing secret
curl -X PATCH https://backend.autosana.ai/api/v1/env-vars/ENV_VAR_UUID \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{ "value": "new-rotated-secret-value" }'
# Promote a plaintext var to a secret
curl -X PATCH https://backend.autosana.ai/api/v1/env-vars/ENV_VAR_UUID \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{ "is_secret": true, "value": "secret-only-known-at-rotation" }'
```
***
## Delete Env Var
Deletes the env var. If the var is a secret, its vault entry is cleaned up too.
**DELETE** `/api/v1/env-vars/{env_var_id}` — Returns `204 No Content`, or `404` if the var does not exist.
UUID of the env var.
```bash theme={null}
curl -X DELETE https://backend.autosana.ai/api/v1/env-vars/ENV_VAR_UUID \
-H "X-API-Key: YOUR_API_KEY"
```
Deleting a variable will cause any flows or hooks that reference `${env:VARIABLE_NAME}` to fail. Update or remove references before deleting.
# Hooks API
Source: https://docs.autosana.ai/api-hooks
Manage hooks (curl, scripts, app launch args) programmatically
These endpoints let you create, read, update, and delete [hooks](/hooks) — the setup, teardown, and runtime scripts that prepare app state and bridge your backend with the Autosana agent.
Requests authenticate with an API key in the `X-API-Key` header. The [Test Hook](#test-hook) endpoint also accepts a signed-in user's session token. See [API Reference](/api-reference) for authentication and the standard `401` / `403` / `429` / `500` error shapes shared by every endpoint on this page.
This API covers hook CRUD and test execution. To attach a hook as a setup/teardown step on a flow or suite, use the [dashboard](https://autosana.ai/hooks) or the Autosana MCP. Runtime hooks need no attachment — reference them from flow instructions as `${hooks:Hook Name}`.
***
## List Hooks
**GET** `/api/v1/hooks` — Returns `200 OK`
### Query Parameters
Optional. Filter to setup/teardown hooks attached to this flow. Runtime hooks (embedded in flow instructions as `${hooks:NAME}`) are NOT returned. Mutually exclusive with `suite_id`.
Optional. Filter to setup/teardown hooks attached to this suite. Mutually exclusive with `flow_id`.
### Example Request
```bash theme={null}
curl -X GET https://backend.autosana.ai/api/v1/hooks \
-H "X-API-Key: YOUR_API_KEY"
# Filter to a flow's setup/teardown hooks
curl -X GET "https://backend.autosana.ai/api/v1/hooks?flow_id=FLOW_UUID" \
-H "X-API-Key: YOUR_API_KEY"
# Filter to a suite's setup/teardown hooks
curl -X GET "https://backend.autosana.ai/api/v1/hooks?suite_id=SUITE_UUID" \
-H "X-API-Key: YOUR_API_KEY"
```
### Response Fields
List of hook objects.
Unique identifier (UUID).
Hook name. Referenced from flow instructions as `${hooks:NAME}` for runtime hooks.
Full hook script (curl command, Python source, etc.).
One of `curl`, `python`, `javascript`, `typescript`, `bash`, `launch_args`.
Optional human-readable note.
Filename-derived identity (e.g. `seed-db`) used by code-managed tooling to name the exported hook file and to match on adopt-by-slug.
ISO 8601 timestamp.
ISO 8601 timestamp of the last modification. Useful for sync tooling that polls for drift.
Present only when filtering by `flow_id` or `suite_id`: which attachment(s) this hook has on that flow/suite — `setup`, `teardown`, or both (parallel to `run_orders`).
Present only when filtering by `flow_id` or `suite_id`: the execution order for each entry in `hook_types`, parallel by index.
Total number of hooks returned.
#### Example Response
```json theme={null}
{
"hooks": [
{
"id": "660e8400-e29b-41d4-a716-446655440001",
"name": "Create Test User",
"script": "curl -X POST https://api.example.com/users -d '{...}'",
"script_type": "curl",
"description": "Provisions a fresh test user before each flow",
"slug": "create-test-user",
"created_at": "2026-01-15T10:31:00Z",
"updated_at": "2026-01-15T10:31:00Z",
"hook_types": ["setup"],
"run_orders": [1]
}
],
"count": 1
}
```
***
## Create Hook
Create a new hook. Reference it in flow instructions as `${hooks:NAME}` for runtime use, or attach it as a setup/teardown hook from the dashboard.
**POST** `/api/v1/hooks` — Returns `201 Created`
### Request Body
Name of the hook (1–255 characters). Used as the `${hooks:NAME}` reference in flow instructions.
The script content (curl command, Python source, etc.).
One of `curl`, `python`, `javascript`, `typescript`, `bash`, `launch_args`. See [Hook Types](/hooks#hook-types) for what each one runs.
Optional description of what the hook does.
### Example Request
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/v1/hooks \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "Create Test User",
"script": "curl -X POST https://api.example.com/users -H \"Content-Type: application/json\" -d \"{\\\"email\\\":\\\"test@example.com\\\"}\"",
"script_type": "curl",
"description": "Provisions a fresh test user"
}'
```
#### Example Response
```json theme={null}
{
"id": "660e8400-e29b-41d4-a716-446655440001",
"name": "Create Test User",
"script": "curl -X POST https://api.example.com/users ...",
"script_type": "curl",
"description": "Provisions a fresh test user",
"created_at": "2026-01-15T10:31:00Z"
}
```
***
## Get Hook
**GET** `/api/v1/hooks/{hook_id}` — Returns `200 OK` with the same shape as items in [List Hooks](#list-hooks), or `404` if the hook does not exist.
UUID of the hook.
```bash theme={null}
curl -X GET https://backend.autosana.ai/api/v1/hooks/HOOK_UUID \
-H "X-API-Key: YOUR_API_KEY"
```
***
## Update Hook
Update a hook's `script`, `script_type`, and/or `description`. Omitted fields are left unchanged.
**Hook names cannot be updated.** Runtime hooks reference hooks by name in flow instructions (`${hooks:NAME}`), so renaming would silently break those references. To rename safely: create a new hook, update every flow that references the old name to use the new one, then delete the old hook. There is no atomic rename.
**PATCH** `/api/v1/hooks/{hook_id}` — Returns `200 OK`. `400` if no recognized fields are provided. `404` if the hook does not exist.
UUID of the hook.
New script content.
New script type (`curl`, `python`, `javascript`, `typescript`, `bash`, `launch_args`).
New description. Pass `""` or `null` to clear.
```bash theme={null}
curl -X PATCH https://backend.autosana.ai/api/v1/hooks/HOOK_UUID \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{ "script": "curl -X POST https://api.example.com/v2/users" }'
```
***
## Delete Hook
Deletes the hook. Past run history (the rows that referenced this hook when they ran) is preserved.
If the hook is currently attached to one or more flows or suites, the call returns `409 Conflict` with the names of the affected resources. Pass `?force=true` to detach and delete anyway — the setup/teardown attachments are deactivated as part of the delete.
**DELETE** `/api/v1/hooks/{hook_id}` — Returns `204 No Content`. `409` if the hook is in use and `force=false`. `404` if the hook does not exist.
UUID of the hook.
Defaults to `false`. When `true`, deletes the hook even if it's attached to flows or suites — those attachments are deactivated.
`force=true` detaches the hook from **every** flow and suite at once. There's no per-attachment opt-out, no dry-run, and no undo. Inspect the `409` conflict response (without `force`) first to see exactly which flows/suites will be affected.
```bash theme={null}
# Refuses to delete if attached anywhere
curl -X DELETE https://backend.autosana.ai/api/v1/hooks/HOOK_UUID \
-H "X-API-Key: YOUR_API_KEY"
# Force delete + detach
curl -X DELETE "https://backend.autosana.ai/api/v1/hooks/HOOK_UUID?force=true" \
-H "X-API-Key: YOUR_API_KEY"
```
#### Example 409 Response
```json theme={null}
{
"detail": "Hook 'Create Test User' is used in flow(s): Login Flow. Deleting will detach it from these flows/suites. Pass ?force=true to delete anyway."
}
```
***
## Test Hook
Execute the hook in isolation against a chosen environment and return the result. Uses a shorter 60-second timeout for fast feedback.
Authenticate with `X-API-Key` or `Authorization: Bearer `. Both the hook and environment must belong to the authenticated organization. If both credentials are supplied, the API key takes precedence. The dashboard sends your session token automatically, so testing a hook there does not require creating or pasting an API key.
**POST** `/api/v1/hooks/{hook_id}/test` — Returns `200 OK`. `404` if the hook does not exist.
### Request Body
UUID of the environment whose env vars to inject.
Optional. Test a modified script without saving the changes — useful for CI to dry-run a proposed update before persisting it via `PATCH /api/v1/hooks/{id}`.
### Example Request
```bash theme={null}
# Test the saved script
curl -X POST https://backend.autosana.ai/api/v1/hooks/HOOK_UUID/test \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{ "environment_id": "ENV_UUID" }'
# Test a proposed script change without saving
curl -X POST https://backend.autosana.ai/api/v1/hooks/HOOK_UUID/test \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"environment_id": "ENV_UUID",
"script_override": "curl -X POST https://api.example.com/v2/users -d \"{...}\""
}'
```
### Response Fields
Whether the hook executed successfully.
Exit code (for scripts) or HTTP status (for `curl`).
Captured stdout/stderr from the sandbox.
Wall-clock duration in milliseconds.
Category of failure when `success=false` (e.g. `Execution timed out`, `Missing environment variable`).
# Labels API
Source: https://docs.autosana.ai/api-labels
Create, manage, and run tests by label programmatically
Labels organize your flows and suites and let you run groups of tests by name (e.g. `smoke`, `checkout`, `regression`). These endpoints manage labels, attach them to flows and suites, and trigger runs by label — useful for CI pipelines that run a subset of tests per branch or PR.
All requests require an API key in the `X-API-Key` header. See [API Reference](/api-reference) for authentication and the standard `401` / `403` / `429` / `500` error shapes shared by every endpoint on this page.
Labels are also managed in the [dashboard](https://autosana.ai/settings?tab=labels) (Settings → Labels, and inline on any flow or suite). The dashboard uses your signed-in session; this API is for programmatic and CI use with an organization API key.
***
## List Labels
Returns every label in your organization, ordered by name.
**GET** `/api/v1/labels` — Returns `200 OK`
### Example Request
```bash theme={null}
curl -X GET "https://backend.autosana.ai/api/v1/labels" \
-H "X-API-Key: YOUR_API_KEY"
```
### Response Fields
List of label objects.
Unique identifier (UUID).
Label name (unique per organization, case-insensitive).
Hex color used to render the label (e.g. `"#4cb782"`).
ISO 8601 timestamp.
Number of active flows carrying this label.
Number of active suites carrying this label.
Total number of labels returned.
#### Example Response
```json theme={null}
{
"labels": [
{
"id": "aa0e8400-e29b-41d4-a716-446655440001",
"name": "smoke",
"color": "#4cb782",
"created_at": "2026-01-15T10:30:00Z",
"flow_count": 12,
"suite_count": 2
},
{
"id": "aa0e8400-e29b-41d4-a716-446655440002",
"name": "checkout",
"color": "#5e6ad2",
"created_at": "2026-01-16T09:00:00Z",
"flow_count": 4,
"suite_count": 1
}
],
"count": 2
}
```
***
## Create Label
**POST** `/api/v1/labels` — Returns `201 Created`. Returns `409 Conflict` if a label with the same `name` already exists (names are unique per organization, case-insensitive).
### Request Body
Label name (1–255 characters). Must be unique within your organization.
Optional hex color (e.g. `"#eb5757"`). Defaults to a stable color derived from the name.
### Example Request
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/v1/labels \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{ "name": "smoke", "color": "#4cb782" }'
```
#### Example Response
```json theme={null}
{
"id": "aa0e8400-e29b-41d4-a716-446655440001",
"name": "smoke",
"color": "#4cb782",
"created_at": "2026-01-15T10:30:00Z"
}
```
***
## Get Label
**GET** `/api/v1/labels/{label_id}` — Returns `200 OK` with the same shape as items in [List Labels](#list-labels), or `404` if the label does not exist.
UUID of the label.
```bash theme={null}
curl -X GET https://backend.autosana.ai/api/v1/labels/LABEL_UUID \
-H "X-API-Key: YOUR_API_KEY"
```
***
## Update Label
Rename or recolor a label. Only provided fields change; omitted fields are unchanged.
**PATCH** `/api/v1/labels/{label_id}` — Returns `200 OK`. `400` if no recognized fields are provided. `404` if the label does not exist. `409` if renaming collides with another label.
UUID of the label.
New name (1–255 characters).
New hex color.
Renaming a label affects [runs by label](#run-by-label) and the CI `labels` input, which match on name — update any workflows that reference the old name.
```bash theme={null}
curl -X PATCH https://backend.autosana.ai/api/v1/labels/LABEL_UUID \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{ "name": "smoke-tests", "color": "#26b5ce" }'
```
***
## Delete Label
Deletes the label and removes it from every flow and suite it's attached to.
**DELETE** `/api/v1/labels/{label_id}` — Returns `204 No Content`, or `404` if the label does not exist.
UUID of the label.
```bash theme={null}
curl -X DELETE https://backend.autosana.ai/api/v1/labels/LABEL_UUID \
-H "X-API-Key: YOUR_API_KEY"
```
***
## Attach & Detach Labels
Attach or detach a single label to a flow or suite. There is no bulk endpoint — issue one call per (entity, label) pair.
### Attach to a flow
**POST** `/api/v1/flows/{flow_id}/labels/{label_id}` — Returns `201 Created`. `404` if the flow or label does not exist. `409` if the label is already attached.
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/v1/flows/FLOW_UUID/labels/LABEL_UUID \
-H "X-API-Key: YOUR_API_KEY"
```
### Detach from a flow
**DELETE** `/api/v1/flows/{flow_id}/labels/{label_id}` — Returns `204 No Content`. `404` if the label is not attached to the flow.
```bash theme={null}
curl -X DELETE https://backend.autosana.ai/api/v1/flows/FLOW_UUID/labels/LABEL_UUID \
-H "X-API-Key: YOUR_API_KEY"
```
### Attach to / detach from a suite
Same shape, with `/suites/{suite_id}` instead of `/flows/{flow_id}`:
```bash theme={null}
# Attach
curl -X POST https://backend.autosana.ai/api/v1/suites/SUITE_UUID/labels/LABEL_UUID \
-H "X-API-Key: YOUR_API_KEY"
# Detach
curl -X DELETE https://backend.autosana.ai/api/v1/suites/SUITE_UUID/labels/LABEL_UUID \
-H "X-API-Key: YOUR_API_KEY"
```
***
## Run by Label
Trigger a run of everything carrying one or more labels by passing `labels` (label **names**) to the [Runs API](/api-runs#run-flows). Autosana runs the union of the suites and flows that carry any of those labels; a flow already covered by a matched suite runs once. As with any run, provide `app_id` (web) or `bundle_id` + `platform` (mobile). A request whose labels match nothing returns `400`.
**POST** `/api/v1/flows/run` — Returns `200 OK` with a `batch_id` to poll via [Run Status](/api-runs#run-status).
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/v1/flows/run \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"app_id": "your-app-id",
"labels": ["smoke", "checkout"]
}'
```
See the [Runs API](/api-runs) for the full request body, polling, and result shapes.
***
## CI Usage
In CI, the simplest entry point is the [autosana-ci GitHub Action](/ci-cd-integration) — pass a comma-separated `labels` input to run the matching suites and flows after your build is uploaded, instead of listing `suite-ids` / `flow-ids`:
```yaml theme={null}
- uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
bundle-id: com.company.app
platform: android
build-path: app.apk
labels: smoke,checkout # run everything carrying any of these labels
```
See [CI/CD Integration](/ci-cd-integration) for the full workflow setup.
# Members API
Source: https://docs.autosana.ai/api-members
List, invite and remove team members programmatically
These endpoints let you manage your [team](/team) programmatically, so onboarding and offboarding can be automated from your identity or HR tooling.
All requests require an API key in the `X-API-Key` header. See [API Reference](/api-reference) for authentication and the standard `401` / `403` / `429` / `500` error shapes shared by every endpoint on this page.
**An API key acts as an admin.** It can invite admins and members, and remove members. Owners and admins can only be removed from [Settings → Team](https://autosana.ai/settings?tab=team).
Members belong to your company, so they can reach every workspace in it. Removing someone takes away their access to all of them.
***
## List Members
Returns everyone in your company, oldest first.
**GET** `/api/v1/members` — Returns `200 OK`
### Example Request
```bash theme={null}
curl -X GET https://backend.autosana.ai/api/v1/members \
-H "X-API-Key: YOUR_API_KEY"
```
### Response Fields
List of members.
Unique identifier (UUID). Use it to remove the member.
Email address.
Full name.
`owner`, `admin` or `member`.
ISO 8601 timestamp.
Total number of members returned.
#### Example Response
```json theme={null}
{
"members": [
{
"id": "550e8400-e29b-41d4-a716-446655440000",
"email": "jane@example.com",
"name": "Jane Doe",
"role": "member",
"created_at": "2026-01-15T10:30:00Z"
}
],
"count": 1
}
```
***
## Invite Member
Invites someone to your company. New users get an email invitation to set up their account. People who already have an Autosana account are added right away and get an email letting them know.
**POST** `/api/v1/members` — Returns `201 Created`. `400` if they are already a member, or if your organization uses SSO (those users sign in through your identity provider instead). Inviting someone whose invitation is still pending sends them a fresh one.
### Request Body
Email address to invite.
Full name (1–200 characters).
`member` (default) or `admin`.
### Example Request
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/v1/members \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"email": "jane@example.com",
"name": "Jane Doe",
"role": "member"
}'
```
#### Example Response
```json theme={null}
{
"id": "550e8400-e29b-41d4-a716-446655440000",
"email": "jane@example.com",
"name": "Jane Doe",
"role": "member",
"created_at": "2026-01-15T10:30:00Z"
}
```
***
## Remove Member
Removes a member from your company.
**DELETE** `/api/v1/members/{member_id}` — Returns `204 No Content`. `403` if the member is an owner or admin. `404` if the member does not exist.
UUID of the member, from [List Members](#list-members).
```bash theme={null}
curl -X DELETE https://backend.autosana.ai/api/v1/members/MEMBER_UUID \
-H "X-API-Key: YOUR_API_KEY"
```
# API Reference
Source: https://docs.autosana.ai/api-reference
Programmatically manage test suites, flows, and app builds via the Autosana API
The Autosana API allows you to programmatically create test suites, flows, and upload app builds. This is useful for integrating Autosana with your own automation platforms, AI agents, or CI/CD pipelines.
For a simpler CI/CD setup using GitHub Actions, see [CI/CD Integration](/ci-cd-integration). The endpoints below are for custom integrations.
## Base URL
```
https://backend.autosana.ai
```
All API endpoints are relative to this base URL.
***
## Authentication
All API requests require an API key passed in the `X-API-Key` header.
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/v1/suites \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"name": "My Test Suite"}'
```
Your organization's API key. Create and manage it in [Settings → API Keys](https://autosana.ai/settings?tab=api-keys).
***
## Available Endpoints
List apps and their builds, and manage default Chrome extension dependencies
Create and list test suites and flows
Manage labels, attach them to flows/suites, and run by label
Trigger flows, poll for results, and fetch run details
Manage setup, teardown, and runtime hook scripts
List environments and manage their env vars (including secrets)
List, invite and remove team members
Read flow run and AI action usage for a workspace
Upload app builds programmatically
***
## Error Responses
| Status Code | Description |
| - | - |
| **400** | Bad request - missing or invalid parameters |
| **401** | Missing API key |
| **402** | Balance too low (pay as you go): the run did not start |
| **403** | Invalid API key |
| **404** | Resource not found |
| **422** | Validation error - check the response body for details |
| **500** | Server error |
#### Example Error Response
```json theme={null}
{
"detail": "Suite not found"
}
```
***
## Rate Limits
The API is rate-limited to prevent abuse. If you receive a `429 Too Many Requests` response, wait a few seconds before retrying.
***
## Use Cases
Generate test cases from requirements using AI, then push them to Autosana via the API
Automatically create test suites when tickets move to "Ready for QA" status
Keep Autosana tests in sync with your test management system
Migrate existing test cases from spreadsheets or other tools
***
## Need Help?
Learn how to write clear, reliable test instructions
Reach out if you need help with your integration
# Runs API
Source: https://docs.autosana.ai/api-runs
Trigger flow execution, poll for results, retrieve run results, and terminate in-progress runs programmatically
All requests require an API key in the `X-API-Key` header. See [API Reference](/api-reference) for authentication details.
***
## List Devices
Discover the device and OS permutations that can be requested for cloud runs.
**GET** `/api/v1/devices` — Returns `200 OK`
Optional query parameters are `platform` (`android` or `ios`) and `physical`
(`true` or `false`). Each model appears once with its supported `os_versions`.
```bash theme={null}
curl "https://backend.autosana.ai/api/v1/devices?physical=false" \
-H "X-API-Key: YOUR_API_KEY"
```
Example response (abbreviated):
```json theme={null}
{
"devices": [
{
"platform": "android",
"physical": false,
"model": "Pixel 10 Pro",
"os_versions": ["17", "16"]
},
{
"platform": "ios",
"physical": false,
"model": "iPhone 18 Pro",
"os_versions": ["27.0"],
"fast_os_versions": ["27.0"]
},
...
]
}
```
Choose a model and one of its `os_versions`. For iOS simulators,
`fast_os_versions` identifies choices with an optimized startup path. Every
returned combination is supported.
***
## Run Flows
Trigger flow execution. Returns a `batch_id` to poll via [Run Status](#run-status) and a `batch_url` to view or share the entire batch in the dashboard.
**POST** `/api/v1/flows/run` — Returns `200 OK`
### Request Body
App bundle identifier (mobile). Required for mobile runs.
`ios` or `android`. Required with `bundle_id`.
Web app identifier. Required for web runs. For mobile runs, optionally pass it with `bundle_id` and `platform` to pick the app that was [uploaded with that `app_id`](/api-ci#start-upload); omit it to run the app identified by bundle ID alone. Passing your bundle ID as `app_id` is the same as leaving it out.
Optional. Pin the run to a specific build (UUID) instead of the app's active build — useful for testing a particular CI/CD build. The build must belong to the resolved app, otherwise the request returns `404`. To find the build for a commit, see [List App Builds](/api-apps#list-app-builds).
Suite UUIDs to run. At least one of `suite_ids` or `flow_ids` is required.
Optional suite execution override. Explicit `true` or `false` takes precedence
over each suite's saved **Run flows in parallel** setting. Omit it or send
`null` to use the saved suite setting.
Flow UUIDs to run. At least one of `suite_ids`, `flow_ids`, or `labels` is required.
When provided, suite auth instructions and suite-scoped variables are resolved automatically unless `resolve_suite_context` is set to `false`. See [Suite context for individual flows](#suite-context-for-individual-flows) below.
Label **names** to run (e.g. `["smoke", "checkout"]`). Runs the union of the suites and flows carrying any of these labels; a flow already covered by a matched suite runs once. Returns `400` if nothing matches. See [Run by Label](/api-labels#run-by-label).
When running individual flows via `flow_ids`, whether to look up the flow's suite and run auth instructions first. Defaults to `true`. Set to `false` to run the flow without suite auth or suite-scoped variables — useful for unauthenticated smoke tests or flows that handle login themselves.
Optional. When a flow belongs to multiple suites, pass a JSON object mapping each ambiguous `flow_id` to the `suite_id` whose auth instructions and variables should be used, e.g. `{"": ""}`. Ignored when `resolve_suite_context` is `false`.
Key-value variables to attach to the build being run (the pinned `app_build_id` if provided, otherwise the active build) and inject into flow instructions via `${env:KEY}`. Accepts a string (`"KEY1=VALUE1,KEY2=VALUE2"`) or a JSON object (`{"KEY1": "VALUE1"}`). See [Build Variables](/variables#build-variables).
For web apps: which browser to run the flow in. One of:
* `chrome` — real Google Chrome (default). Includes proprietary codecs (H.264, AAC) and Widevine DRM. Use this unless you have a specific reason to pick another browser.
* `chromium` — open-source Chromium. No proprietary codecs, no DRM, deterministic and vendor-neutral.
* `firefox` — Mozilla Firefox stable.
* `edge` — Microsoft Edge stable.
Aliases accepted: `Chrome` / `Google Chrome` → `chrome`; `msedge` / `Microsoft Edge` → `edge`. Ignored for mobile runs.
Optional for web apps. Override the app's default Chrome extensions for this
run. Each entry can be an extension app UUID or an object that pins an exact
build:
```json theme={null}
[
"extension-app-uuid",
{
"app_id": "another-extension-app-uuid",
"app_build_id": "extension-build-uuid"
}
]
```
Omit this field to inherit the web app's attached defaults. Pass `[]` to run
without extensions. When extensions load, `chrome`, `edge`, and `chromium`
run in Chromium; `firefox` is rejected because Chrome extensions are not
supported there.
Optional for mobile apps. Selects the execution target and, when specified,
the exact supported device permutation. Fields are `physical` (defaults to
`false`), `model` (for example,
`"Pixel 10 Pro"`), and `os_version` (for example, `"17"`). Set `model` or
`os_version` to `"latest"` to explicitly request rolling resolution for that
field. This lets you pin one while keeping the other current. Use
[List Devices](#list-devices) to discover valid combinations. Omit it to use
the recommended default. A request with no supported match fails fast with a
clear error. For an iOS simulator, use `physical: false` with a listed iPhone
or iPad model and iOS version. For a real iOS device, use `physical: true`
and an `.ipa` build.
Use instead of `device` to choose each device of a two-device flow. Pass
exactly two entries in Device 1, Device 2 order. A run can mix one-device and
two-device flows: one-device flows use the first entry, and a two-device flow
run with `device` (or with neither field) uses that selection for both
devices. Both entries must select virtual devices
(`physical: false`) or both must select real devices (`physical: true`). Model
and OS can be exact, `"latest"`, or omitted for rolling resolution. Both
devices use the run's app build and platform.
```json theme={null}
[
{ "physical": true, "model": "iPhone 17 Pro", "os_version": "26" },
{ "physical": true, "model": "iPhone 16 Pro", "os_version": "18" }
]
```
See [Multi-Device Testing](/multi-device-testing) for a complete setup and
execution guide.
For example, keep the model and OS current on every run:
```json theme={null}
{
"device": {
"physical": false,
"model": "latest",
"os_version": "latest"
}
}
```
### Suite context for individual flows
When you trigger a single flow via `flow_ids`, Autosana resolves the suite it belongs to and runs that suite's **Auth Instructions** first — the same behavior as running the flow manually from the dashboard.
```bash theme={null}
# Runs auth instructions first, then the flow (default)
curl -X POST https://backend.autosana.ai/api/v1/flows/run \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"app_id": "your-app-id",
"flow_ids": ["660e8400-e29b-41d4-a716-446655440001"]
}'
# Skip suite auth — run only the flow instructions
curl -X POST https://backend.autosana.ai/api/v1/flows/run \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"app_id": "your-app-id",
"flow_ids": ["660e8400-e29b-41d4-a716-446655440001"],
"resolve_suite_context": false
}'
```
If a flow belongs to multiple suites, the request returns `400` unless you pass `suite_overrides` mapping that flow to one of its suite UUIDs, e.g. `{"660e8400-e29b-41d4-a716-446655440001": "550e8400-e29b-41d4-a716-446655440000"}`.
### Example Request
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/v1/flows/run \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"bundle_id": "com.company.app",
"platform": "android",
"suite_ids": ["550e8400-e29b-41d4-a716-446655440000"],
"variables": "PR_NUMBER=42,BRANCH=feature/login"
}'
```
### Example: Run with Chrome extensions
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/v1/flows/run \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"app_id": "your-web-app-uuid",
"flow_ids": ["660e8400-e29b-41d4-a716-446655440001"],
"dependencies": [
"extension-app-uuid",
{
"app_id": "wallet-extension-app-uuid",
"app_build_id": "wallet-extension-build-uuid"
}
]
}'
```
All dependency apps and pinned builds must belong to the same organization as
the primary web app. Duplicate app IDs are loaded once; conflicting build pins
return `400`.
### Example Response
```json theme={null}
{
"status": "success",
"batch_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"batch_url": "https://autosana.ai/runs/batches/a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"flow_group_run_ids": ["f1a2b3c4-d5e6-7890-abcd-ef1234567890"],
"flow_run_count": 5
}
```
***
## Run Status
Poll execution status for a batch triggered by [Run Flows](#run-flows).
**GET** `/api/v1/runs/status` — Returns `200 OK`
### Query Parameters
The `batch_id` from `/api/v1/flows/run`.
### Example Request
```bash theme={null}
curl -X GET "https://backend.autosana.ai/api/v1/runs/status?batch_id=BATCH_ID" \
-H "X-API-Key: YOUR_API_KEY"
```
### Example Response
```json theme={null}
{
"batch_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"batch_url": "https://autosana.ai/runs/batches/a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"is_complete": true,
"timestamp": "2025-01-31T12:34:56.789Z",
"app": {
"id": "com.company.app",
"bundle_id": "com.company.app",
"name": "My App",
"platform": "android",
"url": "https://storage.example.com/builds/app.apk"
},
"git": {
"commit_sha": "abc123",
"branch": "main"
},
"summary": {
"total_groups": 2,
"passed_groups": 1,
"failed_groups": 1,
"total_flows": 10,
"passed_flows": 8,
"failed_flows": 2,
"error_flows": 0,
"terminated_flows": 0,
"skipped_flows": 0
},
"run_groups": [
{
"name": "Login Suite",
"status": "passed",
"url": "https://autosana.ai/runs/groups/group-uuid-1",
"source": "ci",
"actor": { "id": "770e8400-e29b-41d4-a716-446655440077", "name": "CI Key", "type": "api_key" },
"dependency_app_build_ids": ["extension-build-uuid"],
"runs": [
{
"id": "run-uuid-1",
"name": "Login with valid credentials",
"status": "passed",
"url": "https://autosana.ai/runs/flow/run-uuid-1",
"summary": "Successfully logged in and verified the dashboard loaded."
}
]
},
{
"name": "Checkout Suite",
"status": "failed",
"url": "https://autosana.ai/runs/groups/group-uuid-2",
"source": "schedule",
"actor": { "id": "f8d8fb05-6292-4daa-9bf4-b0cc3b40a366", "name": "Autosana", "type": "system" },
"dependency_app_build_ids": [],
"runs": [
{
"id": "run-uuid-2",
"name": "Checkout with expired card",
"status": "failed",
"url": "https://autosana.ai/runs/flow/run-uuid-2"
}
]
}
]
}
```
`app.id` is the app's ID: the bundle ID for a mobile app, or its custom `app_id` when it was [uploaded with one](/api-ci#start-upload). `app.bundle_id` is always the bundle ID, so use it if you match on bundle ID. For a web app both hold the app ID.
Each run group also includes:
* `status`: common values include `creating`, `pending`, `queued`, `running`, `reviewing`, `verifying`, `terminating`, `terminated`, `passed`, `failed`, `error`, and `skipped`. Treat this list as non-exhaustive. Poll until a terminal status rather than switching on every value.
* `source`: how it was triggered (`ci`, `dashboard`, `schedule`, or `mcp`)
* `actor`: who triggered it, as `{ id, name, type }`, where `type` is `user`, `system`, or `api_key`
* `dependency_app_build_ids`: exact Chrome extension build UUIDs loaded for the group
Runs triggered with an API key (this endpoint, MCP, CI) are attributed to the key itself: `type` is `api_key`, `id` is the key's ID, and `name` is the key's name from your dashboard. `source` and `actor` may be `null` for older runs.
***
## Get Run Results
Get full details of a run, including metadata, summary, the raw run recording URL, and all actions with screenshot URLs. Use the `id` from individual runs in the [Run Status](#run-status) response.
**GET** `/api/v1/runs/{run_id}` — Returns `200 OK`
### Path Parameters
UUID of the run. Available in the `runs[].id` field from [Run Status](#run-status).
### Example Request
```bash theme={null}
curl -X GET "https://backend.autosana.ai/api/v1/runs/run-uuid-1" \
-H "X-API-Key: YOUR_API_KEY"
```
### Example Response
```json theme={null}
{
"run_id": "run-uuid-1",
"flow_name": "Login with valid credentials",
"platform": "web",
"status": "passed",
"started_at": "2025-01-31T12:34:50.000000+00:00",
"completed_at": "2025-01-31T12:35:01.000000+00:00",
"source": "ci",
"actor": { "id": "770e8400-e29b-41d4-a716-446655440077", "name": "CI Key", "type": "api_key" },
"dependency_app_build_ids": ["extension-build-uuid"],
"summary": "Successfully logged in and verified the dashboard loaded.",
"issues": [
{
"type": "ux",
"severity": "minor",
"title": "No loading indicator on login",
"description": "After tapping 'Sign In', there is no visual loading state before the dashboard appears.",
"actions": [4,5]
}
],
"url": "https://autosana.ai/runs/flow/run-uuid-1",
"recording_url": "https://storage.example.com/recordings/run-uuid-1.mp4",
"performance_data_url": "https://storage.example.com/perf/run-uuid-1.json",
"device_log_url": "https://storage.example.com/logs/run-uuid-1/device.log?token=...",
"network_log_url": "https://storage.example.com/logs/run-uuid-1/network.jsonl?token=...",
"actions": [
{
"id": "action-uuid-1",
"position": 1,
"type": "tap",
"status": "passed",
"description": "Tap the 'Sign In' button",
"value": null,
"screenshot_url": "https://storage-path/screenshot1.png",
"annotated_screenshot_url": "https://example.com/screenshot1_annotated.png",
"executed_at": "2025-01-31T12:34:56.789000+00:00"
},
{
"id": "action-uuid-2",
"position": 2,
"type": "send_keys",
"status": "passed",
"description": "Type email into the email field",
"value": "user@example.com",
"screenshot_url": "https://storage-path/screenshot2.png",
"annotated_screenshot_url": "https://example.com/screenshot2_annotated.png",
"executed_at": "2025-01-31T12:34:58.789000+00:00"
},
{
"id": "action-uuid-3",
"position": 3,
"type": "pass",
"status": "passed",
"description": "Login completed successfully — dashboard is visible",
"value": null,
"screenshot_url": "https://storage-path/screenshot3.png",
"annotated_screenshot_url": null,
"executed_at": "2025-01-31T12:35:00.789000+00:00"
}
]
}
```
### Artifact URLs
Exact Chrome extension build UUIDs loaded for this run.
Per-device metadata and artifact URLs for a two-device run, ordered by
`device_index`. Each entry includes model and OS metadata and that
device's recording, performance, device-log, and network-log URLs.
Single-device runs return an empty array and continue to use the top-level
artifact fields below.
Public URL for the raw, unannotated MP4 recording of the run. `null` when the run did not produce a recording.
Time-series performance metrics (memory, CPU, web vitals) as JSON.
Device log (Android `logcat` / iOS syslog) or browser console log (web), as plain text or JSONL.
HTTP requests captured during the run, as JSONL — one entry per request with `method`, `url`, `status`, `resource_type`, `duration_ms`, `response_size`, etc. Failed requests (DNS, abort, CORS) appear with `status: 0` and an `error` string.
Produced for **web** and **iOS** runs (iOS from the cloud simulator; apps that pin their TLS certificate aren't captured). `null` when no traffic was captured. See [Network Traffic](/network-traffic).
Any of these may be `null` if the artifact wasn't produced for that run.
Actions from a two-device run include `device_index` (`1` or `2`).
***
## Terminate Run
Terminate an in-progress flow group run. `run_id` may be a flow group run ID from [Run Flows](#run-flows) (`flow_group_run_ids`) or a flow run ID from [Run Status](#run-status) / [Get Run Results](#get-run-results).
Passing a flow run ID terminates the **entire parent group**, including every sibling flow in that suite or batch, not just the one run.
**POST** `/api/v1/runs/terminate` returns `200 OK`
### Request Body
UUID of a flow group run or a flow run.
Pending runs are terminated immediately. Queued or running groups are marked `terminating`. Already-terminal runs return `action: "none"`. Unknown or other-organization IDs return `404`.
To watch termination finish, reuse the `batch_id` from [Run Flows](#run-flows) and poll [Run Status](#run-status) until that group's `status` is terminal (`terminated`, `skipped`, `passed`, `failed`, or `error`). The terminate response does not include `batch_id`. You can also poll [Get Run Results](#get-run-results) with the original flow run ID until that flow reaches any terminal status. Children that never started may be `skipped` rather than `terminated`.
### Example Request
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/v1/runs/terminate \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"run_id": "660e8400-e29b-41d4-a716-446655440001"
}'
```
This example passes a **flow run** ID. The response echoes that ID in `run_id` and the resolved parent group in `flow_group_run_id`.
### Example Response
```json theme={null}
{
"run_id": "660e8400-e29b-41d4-a716-446655440001",
"flow_group_run_id": "f1a2b3c4-d5e6-7890-abcd-ef1234567890",
"status": "terminating",
"message": "Marked for termination (worker will finalize)",
"action": "terminating"
}
```
The `run_id` you sent (flow group run or flow run).
The parent group that was actually terminated. Same as `run_id` when you passed a group ID.
Current group status after the request. Typically `terminated` or `terminating`. When `action` is `none`, this is the group's existing terminal status (`passed`, `failed`, `error`, `terminated`, or `skipped`).
Human-readable outcome, for example `"Marked for termination (worker will finalize)"` or `"Already in terminal state"`.
`terminated` (pending run stopped immediately), `terminating` (worker will finalize), or `none` (already terminal).
***
## Polling
Poll `/api/v1/runs/status` every 10-15 seconds until `is_complete` is `true`.
```bash theme={null}
while true; do
RESPONSE=$(curl -s "https://backend.autosana.ai/api/v1/runs/status?batch_id=$BATCH_ID" \
-H "X-API-Key: YOUR_API_KEY")
if [ "$(echo "$RESPONSE" | jq -r '.is_complete')" = "true" ]; then
echo "$RESPONSE" | jq '.summary'
break
fi
sleep 15
done
```
# Suites & Flows API
Source: https://docs.autosana.ai/api-suites-flows
Create and list test suites and flows programmatically
These endpoints allow you to manage test suites and flows via the API. All requests require an API key in the `X-API-Key` header. See [API Reference](/api-reference) for authentication details.
## Suites
### List Suites
Get all test suites for your organization.
**GET** `/api/v1/suites` — Returns `200 OK`
#### Example Request
```bash theme={null}
curl -X GET https://backend.autosana.ai/api/v1/suites \
-H "X-API-Key: YOUR_API_KEY"
```
#### Response Fields
List of suite objects
Unique identifier (UUID)
Name of the suite
Description of what the suite tests
Free-form **suite-level context** injected into the AI agent's system prompt for every flow run in the suite. See [Suite Context](/suites#suite-context).
UUID of the authentication setup flow, if the suite has auth instructions configured
The authentication/login instructions for the suite's setup flow, if configured
ISO 8601 timestamp of when the suite was created
Who created the suite, as `{ id, name, type }` (`type` is always `user`). `null` for older suites.
Total number of suites returned
#### Example Response
```json theme={null}
{
"suites": [
{
"id": "550e8400-e29b-41d4-a716-446655440000",
"name": "Login Feature Tests",
"description": "Tests for user authentication",
"instructions": "Treat the welcome tour as expected on a fresh login — dismiss it and continue.",
"setup_flow_id": "770e8400-e29b-41d4-a716-446655440099",
"auth_instructions": "Tap Login\nEnter ${env:USERNAME}\nEnter ${env:PASSWORD}\nTap Submit",
"created_at": "2025-01-15T10:30:00Z",
"created_by": { "id": "990e8400-e29b-41d4-a716-446655440088", "name": "Ada Lovelace", "type": "user" }
},
{
"id": "550e8400-e29b-41d4-a716-446655440001",
"name": "Checkout Flow Tests",
"description": null,
"instructions": null,
"setup_flow_id": null,
"auth_instructions": null,
"created_at": "2025-01-14T09:00:00Z",
"created_by": null
}
],
"count": 2
}
```
***
### Create Suite
Create a new test suite to organize your flows.
**POST** `/api/v1/suites` — Returns `201 Created`
#### Request Body
Name of the test suite (1-255 characters)
Optional description of what the suite tests
Optional free-form **suite-level context** that gets injected into the AI agent's system prompt for every flow run in the suite. Use this for product knowledge, conventions, or constraints that apply to all flows in the suite — e.g. *"All flows assume the user is already onboarded; if you see the welcome tour, dismiss it."* This is not executed as a flow; for actions that should run before each flow, use `auth_instructions` instead. See [Suite Context](/suites#suite-context).
Optional login/authentication instructions that run before each flow in the suite. Use `${env:VAR_NAME}` placeholders for sensitive values like credentials — configure these in your [Environment Settings](https://autosana.ai/settings?tab=environments).
#### Example Request
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/v1/suites \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "JIRA-1234: User Authentication",
"description": "Auto-generated tests for login feature",
"instructions": "Treat any \"Beta\" badge in the UI as expected — do not fail because of it.",
"auth_instructions": "Tap Login\nEnter ${env:USERNAME} into the username field\nEnter ${env:PASSWORD} into the password field\nTap Submit"
}'
```
#### Response Fields
Unique identifier (UUID) of the created suite
Name of the suite
Description of what the suite tests
Suite-level context echoed back when `instructions` was provided.
UUID of the authentication setup flow. Present when `auth_instructions` was provided.
The authentication/login instructions for the suite's setup flow. Echoed back when `auth_instructions` was provided.
ISO 8601 timestamp of when the suite was created
Who created the suite, as `{ id, name, type }` (`type` is always `user`). `null` for older suites.
#### Example Response
```json theme={null}
{
"id": "550e8400-e29b-41d4-a716-446655440000",
"name": "JIRA-1234: User Authentication",
"description": "Auto-generated tests for login feature",
"instructions": "Treat any \"Beta\" badge in the UI as expected — do not fail because of it.",
"setup_flow_id": "770e8400-e29b-41d4-a716-446655440099",
"auth_instructions": "Tap Login\nEnter ${env:USERNAME} into the username field\nEnter ${env:PASSWORD} into the password field\nTap Submit",
"created_at": "2025-01-15T10:30:00Z",
"created_by": { "id": "990e8400-e29b-41d4-a716-446655440088", "name": "Ada Lovelace", "type": "user" }
}
```
***
### Get Suite
**GET** `/api/v1/suites/{suite_id}` — Returns `200 OK` (same shape as Create Suite), or `404` if the suite does not exist.
UUID of the suite.
```bash theme={null}
curl -X GET https://backend.autosana.ai/api/v1/suites/SUITE_UUID \
-H "X-API-Key: YOUR_API_KEY"
```
***
### Update Suite
Update any subset of a suite's fields — omitted fields are unchanged. Returns the same shape as Create Suite.
**PATCH** `/api/v1/suites/{suite_id}` — Returns `200 OK`. `400` if no recognized fields are provided. `404` if the suite does not exist.
UUID of the suite.
New name (1-255 characters).
New description. Pass `null` to clear.
New suite-level context. Pass `null` to clear. See [Suite Context](/suites#suite-context).
New login instructions that run before each flow in the suite. Use `${env:VAR_NAME}` placeholders for credentials — configure them in your [Environment Settings](https://autosana.ai/settings?tab=environments). To remove auth entirely, use the dashboard.
```bash theme={null}
curl -X PATCH https://backend.autosana.ai/api/v1/suites/SUITE_UUID \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{ "instructions": "All flows assume the staging tenant; dismiss the welcome tour if it appears." }'
```
***
### Delete Suite
Deletes the suite. Flows inside the suite are **not** deleted — delete them separately if needed.
**DELETE** `/api/v1/suites/{suite_id}` — Returns `204 No Content`, or `404` if the suite does not exist.
UUID of the suite.
```bash theme={null}
curl -X DELETE https://backend.autosana.ai/api/v1/suites/SUITE_UUID \
-H "X-API-Key: YOUR_API_KEY"
```
***
## Flows
### List Flows
Get all test flows for your organization. Optionally filter by suite.
**GET** `/api/v1/flows` — Returns `200 OK`
#### Query Parameters
Optional. Filter flows by suite ID to get only flows in a specific suite (ordered by position).
#### Example Request
```bash theme={null}
# Get all flows
curl -X GET https://backend.autosana.ai/api/v1/flows \
-H "X-API-Key: YOUR_API_KEY"
# Get flows in a specific suite
curl -X GET "https://backend.autosana.ai/api/v1/flows?suite_id=550e8400-e29b-41d4-a716-446655440000" \
-H "X-API-Key: YOUR_API_KEY"
```
#### Response Fields
List of flow objects
Unique identifier (UUID)
Name of the flow
Natural language test instructions
`off`, `standard`, or `aggressive`. `null` inherits the workspace mode. A flow cannot exceed its workspace’s mode.
Number of devices required by the flow: `1` or `2`.
ISO 8601 timestamp of when the flow was created
Who created the flow, as `{ id, name, type }` (`type` is always `user`). `null` for older flows.
Total number of flows returned
#### Example Response
```json theme={null}
{
"flows": [
{
"id": "660e8400-e29b-41d4-a716-446655440001",
"name": "Login with valid credentials",
"instructions": "Enter valid email and password, tap login, verify home screen",
"caching_mode": null,
"device_count": 1,
"created_at": "2025-01-15T10:31:00Z",
"created_by": { "id": "990e8400-e29b-41d4-a716-446655440088", "name": "Ada Lovelace", "type": "user" }
},
{
"id": "660e8400-e29b-41d4-a716-446655440002",
"name": "Login with invalid password",
"instructions": "Enter valid email but wrong password, verify error message",
"caching_mode": null,
"device_count": 1,
"created_at": "2025-01-15T10:32:00Z",
"created_by": null
}
],
"count": 2
}
```
***
### Create Flow
Create a new test flow (test case). Optionally attach it to a suite.
**POST** `/api/v1/flows` — Returns `201 Created`
#### Request Body
Name of the test flow (1-255 characters)
Natural language instructions for the test. See [Writing Effective Flow Instructions](/writing-effective-flow-instructions) for best practices.
Optional UUID of a suite to attach this flow to. The flow will be added to the end of the suite.
`off`, `standard`, or `aggressive`. Omit or pass `null` to inherit the workspace mode. A flow can reduce the workspace mode, never exceed it.
Number of devices the agent controls. Allowed values are `1` and `2`. A flow
attached to a suite must match every member and auth flow in that suite.
#### Example Request
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/v1/flows \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "Login with valid credentials",
"instructions": "1. Tap the Sign In button\n2. Enter test@example.com in the email field\n3. Enter password123 in the password field\n4. Tap the Login button\n5. Verify that the home screen appears with Welcome message",
"suite_id": "550e8400-e29b-41d4-a716-446655440000",
"device_count": 2
}'
```
#### Response Fields
Unique identifier (UUID) of the created flow
Name of the flow
Natural language test instructions
`off`, `standard`, or `aggressive`. Defaults to `null`, which inherits the workspace mode.
Number of devices required by the flow: `1` or `2`.
UUID of the suite this flow belongs to (if attached)
Position of this flow within the suite (0-indexed, if attached)
ISO 8601 timestamp of when the flow was created
Who created the flow, as `{ id, name, type }` (`type` is always `user`). `null` for older flows.
#### Example Response
```json theme={null}
{
"id": "660e8400-e29b-41d4-a716-446655440001",
"name": "Login with valid credentials",
"instructions": "1. Tap the Sign In button\n2. Enter test@example.com in the email field\n3. Enter password123 in the password field\n4. Tap the Login button\n5. Verify that the home screen appears with Welcome message",
"caching_mode": null,
"suite_id": "550e8400-e29b-41d4-a716-446655440000",
"position": 0,
"created_at": "2025-01-15T10:31:00Z",
"created_by": { "id": "990e8400-e29b-41d4-a716-446655440088", "name": "Ada Lovelace", "type": "user" }
}
```
***
### Get Flow
**GET** `/api/v1/flows/{flow_id}` — Returns `200 OK` with the same shape as items in [List Flows](#list-flows), or `404` if the flow does not exist.
UUID of the flow.
```bash theme={null}
curl -X GET https://backend.autosana.ai/api/v1/flows/FLOW_UUID \
-H "X-API-Key: YOUR_API_KEY"
```
***
### Update Flow
Update any subset of a flow's fields — omitted fields are unchanged. Returns the same shape as items in [List Flows](#list-flows).
To move a flow from one suite to another, use [Add Flow to Suite](#add-flow-to-suite) on the target suite and [Remove Flow from Suite](#remove-flow-from-suite) on the source suite. Run history is preserved.
**PATCH** `/api/v1/flows/{flow_id}` — Returns `200 OK`. `400` if no recognized fields are provided. `404` if the flow does not exist.
UUID of the flow.
New name (1-255 characters).
Natural-language test instructions.
New device requirement: `1` or `2`. The update is rejected when it would make
a suite's member or auth flow counts inconsistent.
```bash theme={null}
curl -X PATCH https://backend.autosana.ai/api/v1/flows/FLOW_UUID \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{ "instructions": "1. Tap Sign In\n2. Enter the email\n3. Tap Submit" }'
```
***
### Delete Flow
Deletes the flow. Past runs are preserved.
**DELETE** `/api/v1/flows/{flow_id}` — Returns `204 No Content`, or `404` if the flow does not exist.
UUID of the flow.
```bash theme={null}
curl -X DELETE https://backend.autosana.ai/api/v1/flows/FLOW_UUID \
-H "X-API-Key: YOUR_API_KEY"
```
***
## Suite Membership
Move a flow between suites without losing run history.
### Add Flow to Suite
Attaches an existing flow to the end of a suite.
**POST** `/api/v1/suites/{suite_id}/flows/{flow_id}` — Returns `201 Created`. `404` if the suite or flow does not exist. `409` if the flow is already in the suite.
UUID of the suite.
UUID of the flow to attach.
#### Response Fields
UUID of the suite the flow was attached to.
UUID of the flow that was attached.
0-indexed position the flow was placed at — always the next slot at the end of the suite.
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/v1/suites/SUITE_UUID/flows/FLOW_UUID \
-H "X-API-Key: YOUR_API_KEY"
```
#### Example Response
```json theme={null}
{
"suite_id": "550e8400-e29b-41d4-a716-446655440000",
"flow_id": "660e8400-e29b-41d4-a716-446655440001",
"position": 3
}
```
***
### Remove Flow from Suite
Detaches a flow from a suite. The flow itself is **not** deleted.
**DELETE** `/api/v1/suites/{suite_id}/flows/{flow_id}` — Returns `204 No Content`. `404` if the suite does not exist, or if the flow is not in this suite.
UUID of the suite.
UUID of the flow to detach.
```bash theme={null}
curl -X DELETE https://backend.autosana.ai/api/v1/suites/SUITE_UUID/flows/FLOW_UUID \
-H "X-API-Key: YOUR_API_KEY"
```
***
## Example Workflow
Here's a typical workflow for creating a test suite with multiple test cases:
First, create a suite to organize your test cases. If your app requires login, include `auth_instructions` — these run automatically before each flow:
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/v1/suites \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "Login Feature Tests",
"auth_instructions": "Tap Sign In\nEnter ${env:USERNAME}\nEnter ${env:PASSWORD}\nTap Submit"
}'
```
Save the returned `id` for the next step.
Create test flows and attach them to the suite:
```bash theme={null}
# Positive test case
curl -X POST https://backend.autosana.ai/api/v1/flows \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "Login with valid credentials",
"instructions": "Enter valid email and password, tap login, verify home screen appears",
"suite_id": "SUITE_ID_FROM_STEP_1"
}'
# Negative test case
curl -X POST https://backend.autosana.ai/api/v1/flows \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "Login with invalid password",
"instructions": "Enter valid email but wrong password, tap login, verify error message appears",
"suite_id": "SUITE_ID_FROM_STEP_1"
}'
```
Run your tests from the [Flows page](https://autosana.ai/flows), trigger them via the [Runs API](/api-runs) (auth instructions run automatically for individual flows), or set up [Automations](/automations) to run them on a schedule.
# Usage API
Source: https://docs.autosana.ai/api-usage
Read flow run and AI action usage for a workspace
These endpoints return how many flow runs and AI actions a workspace used in any time range, so you can track usage or bill it on to your own customers.
All requests require an API key in the `X-API-Key` header. See [API Reference](/api-reference) for authentication and the standard `401` / `403` / `429` / `500` error shapes shared by every endpoint on this page.
**An API key reads its own workspace.** To cover several workspaces, call these endpoints once with each workspace's key.
A run counts toward the range it was created in, together with all of its actions. **[Billing & usage](https://autosana.ai/settings?tab=billing)** counts each action when it happens, so the two can differ slightly for runs in progress at either end of a range.
***
## Get Usage Totals
Returns the totals for every flow run your workspace created in the range.
**GET** `/api/v1/usage` — Returns `200 OK`
### Query Parameters
Start of the range, inclusive. An ISO 8601 date or timestamp, read as UTC when it has no offset.
End of the range, exclusive. Must be after `start`.
### Example Request
```bash theme={null}
curl -X GET "https://backend.autosana.ai/api/v1/usage?start=2026-10-01&end=2026-11-01" \
-H "X-API-Key: YOUR_API_KEY"
```
### Response Fields
Start of the range, as an ISO 8601 UTC timestamp.
End of the range, as an ISO 8601 UTC timestamp.
Flow runs created in the range.
Total run time of the finished runs, in seconds.
AI actions taken by those runs: `uncached_action_count` plus `cached_action_count`.
AI actions that were not replayed from cache.
AI actions replayed from cache. These cost 10% of an uncached action. See [Run Caching](/flows#run-caching).
#### Example Response
```json theme={null}
{
"start": "2026-10-01T00:00:00Z",
"end": "2026-11-01T00:00:00Z",
"run_count": 412,
"duration_seconds": 51234.8,
"action_count": 6180,
"uncached_action_count": 1240,
"cached_action_count": 4940
}
```
### Errors
| Status | Description |
| - | - |
| 400 | `end` is not after `start` |
| 422 | `start` or `end` is missing or not a date or timestamp |
***
## List Run Usage
Returns the runs behind the totals, oldest first, each with its own usage and a link to the run in the dashboard.
**GET** `/api/v1/usage/runs` — Returns `200 OK`
### Query Parameters
Start of the range, inclusive. An ISO 8601 date or timestamp, read as UTC when it has no offset.
End of the range, exclusive. Must be after `start`.
Runs per page, from 1 to 500.
The `run_id` of the last run on the previous page. Returns the runs listed after it. Keep `start` and `end` the same while paging: a run outside the range still marks a position, so you get the runs in the range created after it.
### Example Request
```bash theme={null}
curl -X GET "https://backend.autosana.ai/api/v1/usage/runs?start=2026-10-01&end=2026-11-01&limit=100" \
-H "X-API-Key: YOUR_API_KEY"
```
### Response Fields
One page of runs.
Unique identifier (UUID). Pass it to [Get Run Results](/api-runs#get-run-results) for the run's full details.
The flow that ran.
The flow's name.
Run status, such as `passed`, `failed` or `error`. See [Run Status](/api-runs#run-status) for the values.
When the run was created. This places the run in a range.
When the run started executing.
When the run finished.
Seconds from `started_at` to `completed_at`. `null` until the run finishes, or when it never started.
AI actions the run took: `uncached_action_count` plus `cached_action_count`.
AI actions that were not replayed from cache.
AI actions replayed from cache.
Link to the run in the dashboard.
Whether more runs follow this page. If `true`, request the next page with `starting_after` set to the last `run_id` in `runs`.
#### Example Response
```json theme={null}
{
"runs": [
{
"run_id": "660e8400-e29b-41d4-a716-446655440001",
"flow_id": "550e8400-e29b-41d4-a716-446655440000",
"flow_name": "Log in",
"status": "passed",
"created_at": "2026-10-02T08:00:01.123456Z",
"started_at": "2026-10-02T08:01:00Z",
"completed_at": "2026-10-02T08:05:11.400000Z",
"duration_seconds": 251.4,
"action_count": 25,
"uncached_action_count": 5,
"cached_action_count": 20,
"url": "https://autosana.ai/runs/flow/660e8400-e29b-41d4-a716-446655440001"
}
],
"has_more": true
}
```
### Errors
| Status | Description |
| - | - |
| 400 | `end` is not after `start`, or `starting_after` is not a run in this workspace |
| 422 | `start` or `end` is missing or not a date or timestamp, `limit` is outside 1 to 500, or `starting_after` is not a UUID |
# Building Your Mobile App for Our Cloud
Source: https://docs.autosana.ai/app-build-guide
Step-by-step guide to building your mobile app for our virtual device cloud
**Testing a website?** You don't need to build anything - just enter your URL when creating an app. This guide is for mobile apps (iOS and Android) only.
Add our [MCP Server](/mcp-setup) to help you build your app for our cloud.
## Build Requirements
Autosana runs your mobile apps on both our **virtual device cloud** (iOS Simulators and Android Emulators) and **[real devices in the cloud](/real-device-testing)**. The build you upload decides where it can run:
| Platform | Build | Runs on |
| - | - | - |
| iOS | `.app` (compressed as `.zip`) | Simulator only |
| iOS | `.ipa` | Real devices only |
| Android | `.apk` / `.aab` | Emulators **and** real devices |
### iOS Builds
* **Simulator** — a `.app` bundle compressed as `.zip`, built for the simulator SDK (`arm64`). Cheapest and fastest; covers most testing.
* **Real device** — a `.ipa` built for physical hardware (`iphoneos`). Required for [real-device testing](/real-device-testing).
Building on an M1+ Mac will use `arm64` by default. Intel Macs will produce `x86_64` builds, which won't work on our simulators.
The simulator (`.app`) build steps are below; jump to [Real device builds](#real-device-builds-ipa) for `.ipa`.
#### App Clips
* A simulator build of your full app already contains its App Clip, in `YourApp.app/AppClips/`. Upload the full app as usual.
* You can also upload an App Clip on its own: an `.app` whose `Info.plist` contains `NSAppClip`, built for the simulator and zipped like any other `.app`. This is a valid upload.
See [Testing App Clips](/guides-app-clips) to open the App Clip in a test.
### Android Builds
* **Format**: `.apk` or `.aab`
* **Architecture**: a **universal** APK/AAB (recommended) runs on both emulators and real devices. An `x86_64`-only APK runs on emulators but not real devices.
Most build commands create universal APKs that include all architectures by default, so the same Android build works on both emulators and real devices — no separate build needed.
***
Select your framework to see build instructions:
**Important:** React Native apps must be built in Release mode. Debug builds will attempt to connect to Metro bundler and fail to run on the device. Release builds bundle the JavaScript code directly into the app, making them standalone.
## React Native (iOS)
1. Build it with:
```bash theme={null}
npx react-native run-ios --mode Release
```
2. App file should be here:
```bash theme={null}
ios/build/Build/Products/Release-iphonesimulator/[YourApp].app
```
In macOS, it should just be `YourApp` because macOS hides `.app` extensions
3. Go to Finder, compress `YourApp` to a `.zip` and upload/drag it into the Autosana app upload dialog
### React Native with Expo (iOS)
For Expo projects, use EAS Build to create simulator builds:
1. Add this profile to your `eas.json`:
```json theme={null}
{
"build": {
"autosana-simulator": {
"distribution": "internal",
"ios": {
"simulator": true
}
}
}
}
```
2. Run the build command:
```bash theme={null}
eas build --platform ios --profile autosana-simulator
```
3. Once the build completes, EAS will provide a download URL for the build
4. Download and extract it to get the `.app` file
5. Compress the `.app` to a `.zip` and upload it to Autosana
This profile only creates a Simulator artifact; it does not enable APNs. If
the app uses `expo-notifications`, add its config plugin. During Expo Prebuild
or Continuous Native Generation, that plugin adds
`aps-environment=development` to the iOS app. If the app does not use
`expo-notifications`, declare the entitlement under `ios.entitlements` or
configure it in the native Xcode project. Follow [Testing Push Notifications:
Configure an Expo EAS
build](/guides-push-notifications#configure-an-expo-eas-build).
## React Native (Android)
1. Navigate to your Android directory and build the release APK:
```bash theme={null}
cd android && ./gradlew assembleRelease
```
2. The APK will be located at:
```bash theme={null}
android/app/build/outputs/apk/release/app-release.apk
```
3. Upload the `.apk` file to Autosana
### React Native with Expo (Android)
For Expo projects, use EAS Build to create APK builds:
1. Add this profile to your `eas.json`:
```json theme={null}
{
"build": {
"preview": {
"distribution": "internal",
"channel": "preview",
"android": {
"buildType": "apk"
}
}
}
}
```
2. Run the build command:
```bash theme={null}
eas build --platform android --profile preview
```
3. Once the build completes, EAS will provide a download URL for the `.apk` file
4. Download the APK and upload it to Autosana
## Flutter (iOS)
This builds a `.app` for the **simulator**. To test on real iPhones, build a `.ipa` instead — see [Real device builds](#real-device-builds-ipa).
1. Build the app for simulator:
```bash theme={null}
flutter build ios --simulator
```
If your app uses flavors, add the `--flavor` flag:
```bash theme={null}
flutter build ios --simulator --flavor [flavorName]
```
2. The `.app` file will be located at:
```bash theme={null}
build/ios/iphonesimulator/Runner.app
```
3. Compress the `.app` to a `.zip` file
4. Drag or upload the `.zip` file into the Autosana app upload dialog
## Flutter (Android)
1. Build apk in debug mode:
```bash theme={null}
flutter build apk --debug
```
If your app uses flavors, add the `--flavor` flag:
```bash theme={null}
flutter build apk --debug --flavor [flavorName]
```
2. File is located at:
```bash theme={null}
build/app/outputs/flutter-apk/app-debug.apk
```
3. Drag or upload the `.apk` file into the Autosana app upload dialog
## Native iOS (Xcode/SwiftUI)
### Option 1: Using Xcode (Graphical Interface)
1. Open your project in Xcode
2. At the top of Xcode, set:
* The scheme to your app target (e.g., "MyApp")
* The device to a simulator (e.g., "iPhone 16 Pro")
3. In Xcode menu, select **Product > Scheme > Edit Scheme**
4. Under **Run**, change **Build Configuration** to **Release**
5. Press **Command + B** to build the project
6. After the build finishes, the `.app` file will be located at:
```bash theme={null}
~/Library/Developer/Xcode/DerivedData//Build/Products/Release-iphonesimulator/.app
```
7. Right click and open in Finder
8. Right click and compress it to make a `.zip` file
9. Drag or upload the `.zip` file into the Autosana app upload dialog
**Finding the build folder in Finder:**
* Open Finder
* Press **Command + Shift + G**
* Paste the path and press Enter
### Option 2A: Ordinary unsigned Simulator build
1. Open Terminal and navigate to the root of your Xcode project:
```bash theme={null}
cd /path/to/your/project
```
2. Run this command to build the Release version for simulator:
```bash theme={null}
xcodebuild -scheme YourAppScheme \
-sdk iphonesimulator \
-configuration Release \
-destination 'platform=iOS Simulator,OS=latest,name=iPhone 16 Pro' \
-derivedDataPath ./build \
ARCHS=arm64 \
ONLY_ACTIVE_ARCH=NO \
CODE_SIGN_IDENTITY="" \
CODE_SIGNING_REQUIRED=NO \
CODE_SIGNING_ALLOWED=NO \
build
```
3. After the build completes, your `.app` file will be located:
```bash theme={null}
./build/Build/Products/Release-iphonesimulator/YourAppName.app
```
4. Right click and open in Finder
5. Right click and compress it to make a `.zip` file
6. Drag or upload the `.zip` file into the Autosana app upload dialog
This recipe explicitly disables signing. It cannot be used for APNs push
notification testing or any feature that requires a signed entitlement.
### Option 2B: Push-capable signed Simulator build
First enable **Push Notifications** for the app target under **Signing & Capabilities** in Xcode. Then build without the settings that disable signing:
```bash theme={null}
xcodebuild -scheme YourAppScheme \
-sdk iphonesimulator \
-configuration Release \
-destination 'platform=iOS Simulator,OS=latest,name=iPhone 16 Pro' \
-derivedDataPath ./build \
ARCHS=arm64 \
ONLY_ACTIVE_ARCH=NO \
build
```
The signed `.app` is located at:
```bash theme={null}
./build/Build/Products/Release-iphonesimulator/YourAppName.app
```
Before compressing it, confirm that the signed entitlements contain the APNs development environment:
```bash theme={null}
codesign -d --entitlements :- \
./build/Build/Products/Release-iphonesimulator/YourAppName.app
```
The output must contain `aps-environment` with the value `development`. Then run the complete architecture, entitlement, and bundle-ID preflight in [Testing Push Notifications](/guides-push-notifications#verify-the-simulator-artifact), compress the `.app`, and upload the resulting `.zip`.
**Why Release configuration?** Release builds provide production parity with
optimizations enabled, matching what your users experience. Debug builds work,
but they are not recommended because they do not reflect the production
environment.
## Native Android (Kotlin/Java)
### Option 1: Debug Build (Recommended for Quick Testing)
Debug builds work immediately without any signing configuration.
**Using Android Studio:**
1. Open your project in Android Studio
2. From the menu bar, select **Build > Build Bundle(s) / APK(s) > Build APK(s)**
3. Wait for the build to complete
4. Click on **locate** in the notification that appears, or navigate to:
```bash theme={null}
app/build/outputs/apk/debug/app-debug.apk
```
5. Upload the `.apk` file to Autosana
**Using Terminal:**
```bash theme={null}
cd /path/to/your/android/project
./gradlew assembleDebug
```
APK location: `app/build/outputs/apk/debug/app-debug.apk`
### Option 2: Release Build (Production Parity)
Release builds provide better production parity but require signing configuration.
**Step 1: Configure Signing (if not already set up)**
Add this to your `app/build.gradle`:
```gradle theme={null}
android {
buildTypes {
release {
// Use debug keystore for testing (not for Play Store)
signingConfig signingConfigs.debug
minifyEnabled false
}
}
}
```
This uses the debug keystore for convenience. For Play Store releases, you'll need a proper release keystore.
**Step 2: Build the Release APK**
**Using Terminal:**
```bash theme={null}
cd /path/to/your/android/project
./gradlew assembleRelease
```
APK location: `app/build/outputs/apk/release/app-release.apk`
**Using Android Studio:**
1. Open your project in Android Studio
2. From the menu bar, select **Build > Select Build Variant**
3. Change from "debug" to "release"
4. Select **Build > Build Bundle(s) / APK(s) > Build APK(s)**
**Which should I use?**
* **Debug**: Faster builds, works immediately, easier debugging
* **Release**: Production-like optimizations, catches minification issues
***
## Real device builds (`.ipa`)
To run on [real iPhones](/real-device-testing) instead of the simulator, upload an **`.ipa`** built for physical hardware (`iphoneos`) — not a zipped `.app`. The file extension is how we route the build: `.ipa` → real device, `.app` / `.zip` → simulator.
**Signing matters.** Use a **Development** or **Ad Hoc** signed `.ipa`. You do **not** need to register our device UDIDs — builds are automatically re-signed for the test device. Enterprise-signed builds fail to launch on our devices, and App Store builds aren't supported. Re-signing strips some entitlements (e.g. Push Notifications, Apple Pay, App Groups), so features depending on them won't work during tests.
If re-signing breaks Team-ID-prefixed Keychain access groups, expand **Advanced** during IPA upload and enable **Fix Keychain access after re-signing**. Most apps do not need this option.
Android needs no separate build — a universal `.apk` / `.aab` already runs on both emulators and real devices.
Use an EAS profile with internal distribution — note there is **no** `simulator: true`:
```json theme={null}
{
"build": {
"device": {
"distribution": "internal"
}
}
}
```
```bash theme={null}
eas build --platform ios --profile device
```
EAS provides a download URL for the `.ipa`. Upload that file to Autosana.
```bash theme={null}
flutter build ipa --export-method development
```
(The default export method is `app-store`, which won't launch on our devices.)
The `.ipa` is written to:
```bash theme={null}
build/ios/ipa/[YourApp].ipa
```
Upload that `.ipa` to Autosana.
**Using Xcode:**
1. Set the run destination to **Any iOS Device** (not a simulator)
2. **Product > Archive**
3. In the Organizer, click **Distribute App** and choose **Debugging** (development) or **Release Testing** (ad hoc) — not Enterprise or App Store Connect
4. Upload the exported `.ipa` to Autosana
**Using Terminal:**
```bash theme={null}
xcodebuild -scheme YourAppScheme \
-sdk iphoneos \
-configuration Release \
-archivePath ./build/YourApp.xcarchive \
archive
xcodebuild -exportArchive \
-archivePath ./build/YourApp.xcarchive \
-exportPath ./build/ipa \
-exportOptionsPlist ExportOptions.plist
```
With `ExportOptions.plist` setting the signing method to `debugging` (or `release-testing`):
```xml theme={null}
methoddebugging
```
The `.ipa` lands in `./build/ipa/`. Upload it to Autosana.
After uploading your `.ipa`, start a run with **Target: Real device** (or `device.physical: true` via the API). See [Real Device Testing](/real-device-testing) for how to run and pick a device.
# Apps
Source: https://docs.autosana.ai/apps
Upload and manage your apps and websites in Autosana
Before you can run any flows, you need to add your app or website to Autosana.
## What is an App?
In Autosana, an **app** represents something you want to test - this can be a mobile app (iOS or Android) or a website.
### Key Concepts
* **App**: A mobile application or website you want to test
* **Chrome Extension**: An optional dependency included in web flow runs
* **Build**: A specific version of your mobile app (.app for iOS, APK for Android, or a url for a website)
* **Active Build**: The build that will be used when running flows
* **Environment**: Optional grouping to organize apps (e.g., Development, Staging, Production)
## Creating a New App
### Step 1: Navigate to Apps Page
Click on **[Apps](https://autosana.ai/apps)** in the sidebar navigation.
### Step 2: Click "Create New App"
Click the **Create New App** button in the top right corner.
### Step 3: Fill in App Details
**Required Fields:**
* **App Name**: A descriptive name for your app (e.g., "MyApp iOS")
* **Bundle ID**: Your app's unique identifier (e.g., `com.company.appname`)
**Optional Fields:**
* **App ID** (iOS and Android): Expand **App ID (optional)** to set one. Only needed to keep several apps with the same bundle ID in one environment, for example one per release version.
* **Environment**: Assign the app to an environment for better organization
* **Agent Context**: Add special instructions for the agent (e.g., "The app requires biometric authentication on first launch")
[Learn more about finding your Bundle ID →](/bundle-id)
### Step 4: Select Platform and Upload
Choose your app's platform and provide the required build or URL:
**Upload Requirements:**
* For simulator tests, upload a `.zip` containing an `.app` built for iOS Simulator
* For physical-device tests, upload a Development or Ad Hoc signed `.ipa`
* Autosana routes `.zip` / `.app` builds to simulators and `.ipa` builds to real devices
When uploading an IPA whose Team-ID-prefixed Keychain access groups break
after cloud re-signing, expand **Advanced** and enable **Fix Keychain access
after re-signing**. Most apps do not need this option.
[Learn more about simulator and real-device iOS builds →](/app-build-guide)
**Upload Requirements:**
* Drag and drop an `.apk` file
* Ensure the APK includes the `x86_64` architecture (universal APKs are recommended)
* Every Android device includes Google Play services, so no extra setup is needed
[Learn more about building Android apps →](/app-build-guide#android-builds)
**No build file required!**
* Simply enter the URL of your website (e.g., `https://example.com`)
Simply enter your website URL — no build files needed. When triggering a run you can choose between **Chrome** (real Google Chrome — default), **Firefox**, **Edge** (Microsoft Edge), or **Chromium** (open-source, no proprietary codecs / DRM).
[See Web Testing for how each engine differs and when to pick which →](/web-testing)
Add a Manifest V3 extension in either form:
* Paste its Chrome Web Store URL to snapshot the current published version
* Upload a `.zip` containing the unpacked extension directory
Store registration derives the extension name and ID automatically. For a
zip upload, enter a name; the Bundle ID is optional but recommended when CI
will upload future versions of the same extension.
Extension zips may contain `manifest.json` at the archive root or inside one
top-level folder. The compressed archive limit is 500MB. Uploaded extension
archives are stored privately and are only accessed when Autosana runs your tests.
Extensions are organization-wide dependencies, so they do not belong to an
environment and cannot be run directly. Attach them from a web app card or
select them for one run in the Run dialog.
### Step 5: Complete Upload
Click **Upload** and wait for the file to upload. Once complete, your app will appear in the Apps list.
## Managing Builds
Each app can have multiple builds. This allows you to test new versions while keeping previous builds available.
### Uploading a New Build
1. Find your app in the Apps list
2. Click **Upload New Build**
3. Select your new build file
4. Click **Upload**
The new build will automatically become the active build.
### Setting an Active Build
The **active build** is the version used when running flows.
To change the active build:
1. Click **Show build history** on your app card
2. Find the build you want to activate
3. Click **Set as active**
### Downloading Builds
To download a previous build:
1. Click **Show build history**
2. Click the download icon (⬇️) next to any build
### Deleting Builds
To delete a build:
1. Click **Show build history**
2. Click the trash icon (🗑️) next to the build
3. Confirm deletion
If you delete the active build, the next most recent build will automatically become active. If no builds remain, flows cannot run until a new build is uploaded.
## Editing App Details
### Changing App Name
1. Click the pencil icon (✏️) next to your app name
2. Edit the name
3. Press **Enter** or click **Save**
### Adding Agent Context
Agent Context provides special instructions to the agent when running flows on this app.
1. Click **Add Agent Context** (or **Edit Agent Context** if already set)
2. Enter instructions
3. Click **Save**
**Example Agent Context:**
```
This app uses Face ID on first launch.
The test account is test@example.com with password TestPass123.
After login, dismiss the tutorial overlay by tapping "Skip".
```
### Changing Environment
If you use environments to organize your apps:
1. Click the environment dropdown on your app card
2. Select a different environment
## Organizing with Environments
Environments help you organize apps by deployment stage or purpose.
### Creating an Environment
1. Navigate to **[Settings](https://autosana.ai/settings)** → **Environments**
2. Click **Create Environment**
3. Enter a name (e.g., "Staging", "Production", "QA")
4. Click **Create**
### Assigning Apps to Environments
* **During app creation**: Select an environment from the dropdown
* **After creation**: Use the environment dropdown on the app card
* **Via CI/CD**: Pass the `environment` parameter in the [GitHub Action](/ci-cd-integration#step-3-set-up-your-github-workflow-with-the-autosana-github-action) or [Upload API](/api-ci) to automatically assign the app to an environment on upload
Apps grouped by environment appear in separate sections on the Apps page.
If you have release and develop builds that share the same bundle ID, use different environments to keep them as separate apps. See [Assigning Apps to Environments](#assigning-apps-to-environments) above for setup instructions.
To keep several apps with the same bundle ID in one environment, for example one per release version, give each its own App ID. Set it when you create the app, on the app card, or by uploading with an `app-id`. The app card shows it next to the bundle ID. See [Separate apps that share a bundle ID](/ci-cd-integration#separate-apps-that-share-a-bundle-id).
## Troubleshooting
### Common Issues
**iOS:**
* For simulator runs, ensure the `.zip` contains a valid simulator `.app`
* For physical-device runs, upload a Development or Ad Hoc signed `.ipa`
* Do not rename an IPA to `.zip`; the extension determines the target device type
* Verify the bundle ID matches what you entered
**Android:**
* Ensure the APK is properly signed
* Check that the file extension is `.apk`
* Verify the bundle ID matches the package name
**Chrome extensions:**
* Only Manifest V3 extensions are supported
* Upload a `.zip` of the unpacked extension, not a `.crx`
* Keep `manifest.json` at the archive root or inside one top-level folder
* Extension-enabled runs use Chromium
## Next Steps
Now that you've uploaded your app:
* [Create your first flow →](/flows)
* [Set up our CI/CD integration →](/ci-cd-integration)
* [Organize flows with suites →](/suites)
# Automations
Source: https://docs.autosana.ai/automations
Schedule flows to run automatically on intervals, daily, weekly, or on new builds
Automations allow you to schedule flows and suites to run automatically based on triggers like new builds, time intervals, or specific schedules. This ensures continuous testing without manual work.
## What is an Automation?
An **Automation** defines when and how flows or suites should run automatically. Automations can be triggered by:
* **CI/CD Build Uploads**: Run flows whenever a new app build is uploaded
* **Daily Schedule**: Run flows at a specific time every day
* **Weekly Schedule**: Run flows on a specific day and time each week
* **Intervals**: Run flows at a custom interval (as frequent as every minute)
## Creating an Automation
### Step 1: Navigate to Automations
Click on **[Automations](https://autosana.ai/automations)** in the sidebar.
### Step 2: Click "Create Automation"
Click the **Create Automation** button in the top right.
### Step 3: Name Your Automation
Enter a descriptive name (e.g., "Nightly Regression Flows", "Smoke Flows on Build")
### Step 4: Select an App
Choose which app the flows will run on.
### Step 5: Select a Device
For mobile apps, choose the execution target, device model, and OS version. **Latest** is the default for both device and OS, so each run uses the newest compatible option available at that time. Choose a specific model or OS to pin it. If a pinned selection is no longer available, the automation card shows the dispatch failure instead of silently using different hardware.
For a two-device flow or suite, choose separate models and OS versions for
**Device 1** and **Device 2**. An automation can mix one-device and two-device
targets: one-device targets run on Device 1. If a two-device flow later joins a
label that an automation with a single device targets, that device selection is
used for both of its devices.
### Step 6: Select Targets
Choose what to run:
**Option 1: Select Suites (Recommended)**
* Check one or more suites to run
* Suites automatically stay up-to-date when flows are added or removed
* Best for organized, maintainable automations
**Option 2: Select Individual Flows**
* Check specific flows to run
* Creates a fixed list of flows
* Best for one-off or specific flow combinations
Using suites is recommended because your automation automatically includes new flows added to the suite later.
### Step 7: Configure Schedule
Choose when and how often the automation runs:
**CI/CD Build Upload** - Runs when new builds are uploaded via CI/CD
* Best for smoke flows on every build
* No additional configuration needed
**Daily** - Runs once per day
* Set time (e.g., 2:00 AM) and timezone (ET/CT/PT/UTC)
* Best for nightly regression suites
**Weekly** - Runs once per week
* Set day, time, and timezone (ET/CT/PT/UTC)
* Best for comprehensive regression before releases
**Interval** - Runs at a custom interval
* Choose from presets (5 min, 15 min, 30 min, 1 hr, etc.) or set a custom value
* Start immediately or at a specific time in ET/CT/PT/UTC
* Best for continuous monitoring
**"Only Run on New Builds"** (Optional)
* Enable to skip runs if no new build was uploaded since the last run
* Saves resources on unchanged builds
* Useful for daily/weekly automations
### Step 8: Review and Create
Review the summary at the bottom of the dialog:
> "Autosana will run **\[flows/suites]** on **\[app name]** \[trigger schedule]"
Click **Create Schedule** to activate your automation.
## Managing Automations
### Running an Automation Manually
To trigger an automation immediately:
1. Click the play icon (▶️) on the automation card
2. Confirm you want to run it
3. View results in the Runs page
This is useful for testing an automation or running it ad-hoc.
### Enabling/Disabling an Automation
Toggle the switch on the automation card:
* **Enabled (blue)**: Automation will run on schedule
* **Disabled (gray)**: Automation is paused
Disabling an automation keeps the configuration but prevents it from running.
## Automation Triggers in Detail
### CI/CD Trigger
**How it works:**
* Monitors for new app or build uploads via CI/CD API
* Automatically starts when a new build is detected
* Runs immediately upon upload completion
**Use cases:**
* Post-deployment validation
* Continuous integration pipelines
* Automated smoke flows
**Example:**
```
Name: "Smoke Flows on Build"
Trigger: CI/CD app/build upload
Suites: ["Smoke Flows"]
Only run on new builds: N/A (always runs on upload)
```
[Learn how to set up our CI/CD integration →](/ci-cd-integration)
### Daily Schedule
**How it works:**
* Runs once every 24 hours at the specified time
* Uses your selected timezone (ET, CT, PT, or UTC)
* Calculates next run based on current time
**Use cases:**
* Nightly regression suites
* Daily health checks
* Morning or evening validation
**Example:**
```
Name: "Nightly Regression"
Trigger: Daily at 2:00 AM PT
Suites: ["Regression Suite"]
Only run on new builds: Yes
```
### Weekly Schedule
**How it works:**
* Runs once per week on the specified day and time
* Uses your selected timezone (ET, CT, PT, or UTC)
* Ideal for less frequent, comprehensive suites
**Use cases:**
* Weekly regression before releases
* Weekend validation
* Comprehensive flow suites
**Example:**
```
Name: "Weekly Full Regression"
Trigger: Monday at 9:00 AM ET
Suites: ["Full Regression Suite"]
Only run on new builds: No
```
### Interval Schedule
**How it works:**
* Repeats at a custom interval (minimum 1 minute, maximum 1 week)
* Can start immediately or at a specific time in ET, CT, PT, or UTC
* Continues running until disabled
**Use cases:**
* High-frequency monitoring (every 1-15 minutes)
* Continuous validation (every 1-2 hours)
* Scheduled checks (every 12 hours)
**Example:**
```
Name: "Frequent Smoke Check"
Trigger: Every 30 minutes, starting at 8:00 AM PT
Suites: ["Critical Path Flows"]
Only run on new builds: Yes
```
## Best Practices
**Use Suites for Automations**
Automate suites rather than individual flows. When you add a flow to the suite, it's automatically included in all related automations.
**Enable "Only Run on New Builds"**
For daily/weekly automations, enable this option to skip runs when no new build has been uploaded since the last run.
## Next Steps
* [Set up our CI/CD integration →](/ci-cd-integration)
* [Learn about organizing flows with suites →](/suites)
# Finding Your Bundle ID
Source: https://docs.autosana.ai/bundle-id
Learn how to find your Bundle ID for iOS and Application ID for Android
Add our [MCP Server](/mcp-setup) to help you find your bundle ID.
## iOS: Find Your Bundle ID
### App Store Connect
1. Go to [App Store Connect](https://appstoreconnect.apple.com)
2. Click on **My Apps**
3. Select your app
4. In the left menu, click **App Information**
5. Under **General Information**, you'll see your **Bundle ID**
```text Example theme={null}
com.yourcompany.yourapp
```
### Info.plist
1. Open your Flutter or React Native project folder
2. Go to: `ios/Runner/Info.plist`
3. Look for:
```xml theme={null}
CFBundleIdentifier$(PRODUCT_BUNDLE_IDENTIFIER)
```
If it shows `$(PRODUCT_BUNDLE_IDENTIFIER)`, the actual value is defined in Xcode.
### Xcode
1. Open the project in **Xcode**
2. Click the blue project icon in the navigator (top-left)
3. Select the main app target under **Targets**
4. Go to the **General** tab
5. Under the **Identity** section, you'll see **Bundle Identifier**
```text Example theme={null}
com.example.myapp
```
The Bundle ID must be unique for each app on the App Store.
***
## Android: Find Your Application ID
### AndroidManifest.xml
1. Go to: `android/app/src/main/AndroidManifest.xml`
2. Look at the top:
```xml theme={null}
```
3. The value of `package` is your **Application ID**.
### build.gradle (App-level)
1. Go to: `android/app/build.gradle`
2. Find:
```gradle theme={null}
defaultConfig {
applicationId "com.yourcompany.yourapp"
}
```
3. This is the **Application ID** used at build time.
### Google Play Store URL
The easiest way to find an Android app's bundle ID is from its Play Store URL:
```text theme={null}
https://play.google.com/store/apps/details?id=org.wikipedia&hl=en_US
```
The `id` parameter is the Application ID: **org.wikipedia**
### Google Play Console
1. Go to [Google Play Console](https://play.google.com/console)
2. Open your app
3. Click **Setup > App integrity**
4. Under **App signing**, you'll see your **Package name** — that's the Application ID.
The Application ID (Android) and Bundle ID (iOS) are often called "package name" in various places.
# Github Action for App Uploads
Source: https://docs.autosana.ai/ci-cd-integration
Automatically upload mobile builds or register web apps with Autosana
Integrate Autosana into your CI/CD pipeline with our GitHub Action to automatically upload and test your builds with every commit. Supports iOS, Android, web, and Chrome-extension platforms.
Add our [MCP Server](/mcp-setup) to help with setting up your CI/CD pipeline.
## Step 1: Choose Your Platform
Select your platform to see the relevant setup instructions:
Mobile app builds (Expo, Fastlane, native)
Web apps (Vercel, Netlify, custom deployments, etc.)
Manifest V3 extension zip builds
***
## Mobile Setup
### Choose Your Build Tool
React Native with Expo's build service
Native iOS/Android builds with Fastlane
Gradle, Flutter, or custom builds
***
## Step 2: Set Secrets
Go to your repository **Settings → Secrets → Actions** and add these secrets. See the [GitHub Secrets docs](https://docs.github.com/en/actions/security-guides/encrypted-secrets) for help.
### Required for All
Your Autosana API key for uploading builds. Create one with `autosana keys create` or in [Settings → API Keys](https://autosana.ai/settings?tab=api-keys).
### Required for Expo EAS
Your Expo access token for EAS builds. Get it from [Expo account settings](https://expo.dev/accounts/\[account]/settings/access-tokens).
***
## Step 3: Set up your Github Workflow with the Autosana Github Action
Create a `.github/workflows/autosana-ios.yml` and/or `.github/workflows/autosana-android.yml` file in your repository. Examples are provided below to get started.
For more specific build instructions by framework, check out our [App Build Guide](/app-build-guide) for detailed instructions on creating builds for iOS and Android.
Add this step to your existing GitHub workflow after your build step:
```yaml theme={null}
- name: Upload to Autosana
uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
bundle-id: com.your.app # Replace with your app's bundle ID
platform: ios # or 'android'
build-path: ./path/to/your/build.ipa # iOS: .ipa or zipped .app | Android: .apk/.aab
# labels: smoke,critical # Optional: run everything carrying these labels
# suite-ids: uuid1,uuid2 # Optional: trigger suites after upload
# flow-ids: uuid1,uuid2 # Optional: or specify individual flows
# suite-keys: smoke # Optional: run a code-managed suite by its YAML key
# flow-keys: auth/login,checkout # Optional: run code-managed flows by YAML key
# environment: staging # Optional: separate apps by environment
# variables: TEST_ACCOUNT=qa-smoke,CHECKOUT_VARIANT=control # Optional: attach build variables
# enable-ios-keychain-access-group-remapping: true # Optional: IPA re-signing compatibility
```
**Required Parameters:**
* `api-key`: Your Autosana API key (from secrets)
* `bundle-id`: Your app's bundle ID (e.g., `com.company.app`)
* `platform`: Either `ios` or `android`
* `build-path`: Path to your build artifact
**Optional Parameters:**
* `name`: Display name for your app (e.g., "My iOS App"). Updates existing app name if different.
* `app-id`: Custom identifier that keeps several apps with the same bundle ID in one environment (e.g., `v7.12.0`). See [Separate apps that share a bundle ID](#separate-apps-that-share-a-bundle-id) below.
* `labels`: Comma-separated label names to run after upload (e.g., `smoke,critical`). Runs every suite and flow carrying any of the labels — resolved at run time, so CI config never changes as coverage grows. See [Labels API](/api-labels#run-by-label).
* `suite-ids`: Comma-separated suite UUIDs to run after upload (e.g., `uuid1,uuid2`).
* `flow-ids`: Comma-separated flow UUIDs to run after upload (e.g., `uuid1,uuid2`).
* `suite-keys`: Comma-separated [code-managed](/code-managed-flows) suite keys to run from the checked-out commit.
* `flow-keys`: Comma-separated [code-managed](/code-managed-flows) flow keys to run from the checked-out commit.
* `environment`: Name of the environment to associate this app with (e.g., `staging`, `production`). See [Assigning apps to environments](/apps#assigning-apps-to-environments).
* `variables`: Key-value variables to attach to the uploaded build. Available in flow instructions via `${env:KEY}`. Use `KEY1=VALUE1,KEY2=VALUE2`. See [Build Variables](/variables#build-variables).
* `physical-device`: Mobile runs only. Set to `true` to use real hardware; defaults to `false`.
* `device-model`: Mobile runs only. Model from the Autosana device catalog, such as `Pixel 10 Pro`, or `latest`. Two-device flows use it for both devices.
* `os-version`: Mobile runs only. OS version supported by the selected model, such as `17`, or `latest`. Two-device flows use it for both devices.
* `devices`: Mobile runs only. JSON array of exactly two ordered device selections, used instead of `physical-device`, `device-model`, and `os-version` to choose each device of a two-device flow. One-device flows in the same run use the first entry.
* `dependencies`: For direct web runs only, a JSON array overriding the web app's default Chrome extension loadout. Omit it to inherit defaults, use `'[]'` for no extensions, or provide extension app UUIDs and optional build pins. Requires `suite-ids`, `flow-ids`, `flow-keys`, or `labels`. It cannot be combined with `suite-keys`.
* `wait`: Whether to wait for selected tests and fail the job when they fail. Defaults to `true`; set it to `false` for fire-and-forget runs.
* `enable-ios-keychain-access-group-remapping`: iOS `.ipa` only. Set to `true` if cloud re-signing breaks Team-ID-prefixed Keychain access groups. The preference is saved on the app, so future IPA uploads can omit it; set it to `false` to disable.
### Run code-managed tests by key
Use stable YAML keys instead of Autosana UUIDs when your flows and suites are
[managed in code](/code-managed-flows). The Action resolves each key from the
exact checked-out commit:
```yaml theme={null}
- uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
platform: android
bundle-id: com.example.app
build-path: build/app-release.apk
flow-keys: auth/login,checkout
suite-keys: smoke
```
You can combine `flow-keys` with `suite-keys`. Do not combine key selectors
with `flow-ids`, `suite-ids`, or `labels`.
### Select a mobile device
Device inputs apply when any flow, suite, or label selector triggers tests. Use
`latest` to make rolling model and OS selection explicit:
```yaml theme={null}
- uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
platform: android
bundle-id: com.example.app
build-path: build/app-release.apk
suite-ids: "suite-uuid"
physical-device: false
device-model: latest
os-version: latest
```
You can pin either field while leaving the other on `latest`, or pin both to a
supported catalog combination:
```yaml theme={null}
physical-device: false
device-model: Pixel 10 Pro
os-version: "17"
```
Omitting `device-model` and `os-version` keeps the same rolling-Latest behavior.
For the latest available real device, set `physical-device: true` and use
`latest` (or omission) for the model and OS.
### Separate apps that share a bundle ID
By default, every build with the same bundle ID and platform in an environment
goes to one app. Set `app-id` to keep them apart, for example one app per
release version:
```yaml theme={null}
- uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
platform: android
bundle-id: com.example.app
app-id: v7.12.0
build-path: build/app-release.apk
suite-ids: "suite-uuid"
```
The first upload with a new `app-id` creates the app, named after the
`app-id` unless you also set `name`. Later uploads with the same `app-id` add
builds to it, and any tests the step triggers run against that app. All of
these apps share the environment's variables.
`app-id` can be up to 64 characters: lowercase letters, numbers, dots, hyphens,
and underscores. It must start and end with a letter or number, with no two
separators in a row. Automations are set up per app, so a new app starts with none.
Trigger its tests from this step with `suite-ids`, `flow-ids`, or `labels`.
### Github Action Workflow Examples
### Configure eas.json
Add this profile to your `eas.json` for iOS simulator builds:
```json eas.json theme={null}
{
"build": {
"preview-simulator": {
"distribution": "internal",
"ios": {
"simulator": true
}
}
}
}
```
### GitHub Workflow
**Important:** Replace `YOUR_BUNDLE_ID` with your app's bundle ID.
```yaml .github/workflows/autosana-eas-ios.yml theme={null}
name: EAS iOS Simulator Build + Autosana
on:
# Triggers on pushes to main branch (including merges)
push:
branches:
- main
# Allows for manual triggering of the workflow from the GitHub UI.
workflow_dispatch:
jobs:
build-and-test:
runs-on: ubuntu-latest
steps:
# Step 1: Check out the repository's code
- name: Checkout repository
uses: actions/checkout@v4
# Step 2: Set up Node.js
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '18'
cache: 'npm'
# Step 3: Set up Expo and EAS
- name: Set up Expo and EAS
uses: expo/expo-github-action@v8
with:
eas-version: latest
token: ${{ secrets.EXPO_TOKEN }}
# Step 4: Install project dependencies
- name: Install dependencies
run: npm install
# Step 5: Run the EAS Build for an iOS Simulator
- name: Run iOS Simulator Build
run: eas build --platform ios --profile preview-simulator --non-interactive --wait --json > build-info.json
# Step 6: Download the .app artifact from the completed EAS build
- name: Download zipped .app from EAS
run: |
BUILD_URL=$(jq -r '.[0].artifacts.buildUrl' build-info.json)
echo "Downloading zipped .app from $BUILD_URL"
curl -L "$BUILD_URL" -o app.zip
# Step 7: Run the Autosana CI action for testing
- name: Run Autosana CI
uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
bundle-id: YOUR_BUNDLE_ID # TODO: Make sure this matches your app's bundle ID
platform: ios
build-path: app.zip
# labels: smoke,critical # Optional: run everything carrying these labels
# suite-ids: uuid1,uuid2 # Optional: trigger suites after upload
# flow-ids: uuid1,uuid2 # Optional: or specify individual flows
```
### Configure eas.json
Add this profile to your `eas.json` for Android emulator builds:
```json eas.json theme={null}
{
"build": {
"preview": {
"distribution": "internal",
"android": {
"buildType": "apk"
}
}
}
}
```
### GitHub Workflow
**Important:** Replace `YOUR_BUNDLE_ID` with your app's bundle ID.
```yaml .github/workflows/autosana-eas-android.yml theme={null}
name: EAS Android Build + Autosana
on:
push:
branches:
- main
workflow_dispatch:
jobs:
build-and-test:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '18'
cache: 'npm'
- name: Set up Expo and EAS
uses: expo/expo-github-action@v8
with:
eas-version: latest
token: ${{ secrets.EXPO_TOKEN }}
- name: Install dependencies
run: npm install
- name: Run Android Build
run: eas build --platform android --profile preview --non-interactive --wait --json > build-info.json
- name: Download APK from EAS
run: |
BUILD_URL=$(jq -r '.[0].artifacts.buildUrl' build-info.json)
echo "Downloading APK from $BUILD_URL"
curl -L "$BUILD_URL" -o app.apk
- name: Run Autosana CI
uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
bundle-id: YOUR_BUNDLE_ID # TODO: Replace with your bundle ID
platform: android
build-path: app.apk
# labels: smoke,critical # Optional: run everything carrying these labels
# suite-ids: uuid1,uuid2 # Optional: trigger suites after upload
# flow-ids: uuid1,uuid2 # Optional: or specify individual flows
```
**Important:** Replace `com.example.app` with your app's bundle ID.
```yaml .github/workflows/autosana-fastlane-ios.yml theme={null}
name: Fastlane iOS + Autosana
on:
push:
branches: [ main ]
jobs:
build-and-test:
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
- name: Set up Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: '3.0'
bundler-cache: true
- name: Build with Fastlane
run: bundle exec fastlane ios build_for_testing
- uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
bundle-id: com.example.app # TODO: Replace with your bundle ID
platform: ios
build-path: ./build/MyApp.app
# labels: smoke,critical # Optional: run everything carrying these labels
# suite-ids: uuid1,uuid2 # Optional: trigger suites after upload
# flow-ids: uuid1,uuid2 # Optional: or specify individual flows
```
**Important:** Replace `com.example.app` with your app's bundle ID.
```yaml .github/workflows/autosana-fastlane-android.yml theme={null}
name: Fastlane Android + Autosana
on:
push:
branches: [ main ]
jobs:
build-and-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up JDK
uses: actions/setup-java@v3
with:
java-version: '11'
distribution: 'temurin'
- name: Set up Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: '3.0'
bundler-cache: true
- name: Build with Fastlane
run: bundle exec fastlane android build
- uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
bundle-id: com.example.app # TODO: Replace with your bundle ID
platform: android
build-path: ./app/build/outputs/apk/release/app-release.apk
# labels: smoke,critical # Optional: run everything carrying these labels
# suite-ids: uuid1,uuid2 # Optional: trigger suites after upload
# flow-ids: uuid1,uuid2 # Optional: or specify individual flows
```
**Important:**
* Replace `com.example.app` with your app's bundle ID
* Replace the build command with your specific build process
* Update the build path to match your build output
```yaml .github/workflows/autosana-custom-ios.yml theme={null}
name: Build and Upload to Autosana (iOS)
on:
push:
branches: [ main ]
jobs:
build-and-upload:
runs-on: macos-latest # iOS builds require macOS
steps:
- uses: actions/checkout@v4
# TODO: Build your iOS app
# Examples:
# - xcodebuild -workspace App.xcworkspace -scheme App -configuration Release
# - flutter build ios --release --no-codesign
# - react-native run-ios --configuration Release
- uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
bundle-id: com.example.app # TODO: Replace with your app's bundle ID
platform: ios
build-path: build/ios/iphoneos/MyApp.app # TODO: Update path
# labels: smoke,critical # Optional: run everything carrying these labels
# suite-ids: uuid1,uuid2 # Optional: trigger suites after upload
# flow-ids: uuid1,uuid2 # Optional: or specify individual flows
```
**Important:**
* Replace `com.example.app` with your app's bundle ID
* Replace the build command with your specific build process
* Update the build path to match your build output
```yaml .github/workflows/autosana-custom-android.yml theme={null}
name: Build and Upload to Autosana (Android)
on:
push:
branches: [ main ]
jobs:
build-and-upload:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# TODO: Build your Android app
# Examples:
# - ./gradlew assembleRelease
# - flutter build apk --release
# - react-native run-android --variant=release
- uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
bundle-id: com.example.app # TODO: Replace with your app's bundle ID
platform: android
build-path: app/build/outputs/apk/release/app-release.apk # TODO: Update path
# labels: smoke,critical # Optional: run everything carrying these labels
# suite-ids: uuid1,uuid2 # Optional: trigger suites after upload
# flow-ids: uuid1,uuid2 # Optional: or specify individual flows
```
***
## Chrome Extension Setup
Build or package the unpacked Manifest V3 extension directory as a `.zip`, then
upload it like a mobile build:
```yaml theme={null}
- name: Upload extension to Autosana
uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
platform: chrome-extension
bundle-id: my-wallet-extension
name: My Wallet Extension
build-path: ./dist/extension.zip
```
The first upload creates the extension; later uploads with the same
`bundle-id` create new versions and make the newest build active. Extension
apps are organization-wide and do not use `environment`.
The zip must contain `manifest.json` at its root or inside one top-level folder.
Only Manifest V3 is supported. Limits are 500MB compressed, 2GB expanded, and
25,000 entries.
After upload, attach the extension to a web app in the Autosana Apps page.
CI-triggered tests inherit that app's default extensions.
Chrome extension upload steps cannot trigger tests directly. Do not add
flow, suite, or label selectors to a `platform: chrome-extension` step.
Upload and attach the extension first, then run tests in a separate
`platform: web` Action step using the web app's default extensions or a
`dependencies` override.
***
## Web Setup
For web applications, you can register a URL (such as a preview deployment) for testing. This is useful for testing Vercel, Netlify, or other preview deployments on every PR.
To reuse cached navigation across preview deployments, also pass the deployment origin as a build variable named `BASE_URL`. The `url` input selects the launch URL; it does not create this variable automatically. The variable must contain only an HTTP(S) origin (an optional trailing `/` is allowed), without a path, query string, fragment, or credentials. See [Run Caching](/flows#run-caching) for eligibility and live-execution limits. Keep production and preview targets in separate Autosana environments.
### Parameters
**Required Parameters for Web:**
* `api-key`: Your Autosana API key (from secrets)
* `platform`: Must be `web`
* `app-id`: A unique identifier for your web app (lowercase, alphanumeric with hyphens, e.g., `my-web-app`)
* `url`: The URL to test (e.g., your preview deployment URL)
**Optional Parameters:**
* `name`: Display name for your web app (e.g., "My Web App"). Updates existing app name if different.
* `labels`: Comma-separated label names to run after upload (e.g., `smoke,critical`). Runs every suite and flow carrying any of the labels — resolved at run time, so CI config never changes as coverage grows. See [Labels API](/api-labels#run-by-label).
* `suite-ids`: Comma-separated suite UUIDs to run after upload (e.g., `uuid1,uuid2`).
* `flow-ids`: Comma-separated flow UUIDs to run after upload (e.g., `uuid1,uuid2`).
* `suite-keys`: Comma-separated [code-managed](/code-managed-flows) suite keys to run from the checked-out commit.
* `flow-keys`: Comma-separated [code-managed](/code-managed-flows) flow keys to run from the checked-out commit.
* `environment`: Name of the environment to associate this app with (e.g., `staging`, `production`). See [Assigning apps to environments](/apps#assigning-apps-to-environments).
* `variables`: Key-value variables to attach to the registered build. Available in flow instructions via `${env:KEY}`. Use `KEY1=VALUE1,KEY2=VALUE2`. See [Build Variables](/variables#build-variables).
### Basic Usage
```yaml theme={null}
- name: Register Web App with Autosana
uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
platform: web
app-id: my-web-app
url: https://my-app-preview.vercel.app
name: My Web App # Optional: display name for the app
# labels: smoke,critical # Optional: run everything carrying these labels
# suite-ids: uuid1,uuid2 # Optional: trigger suites after upload
# flow-ids: uuid1,uuid2 # Optional: or specify individual flows
# suite-keys: smoke # Optional: run a code-managed suite by its YAML key
# flow-keys: auth/login,checkout # Optional: run code-managed flows by YAML key
# variables: DEPLOY_URL=${{ steps.deploy.outputs.url }},LOGIN_VARIANT=passwordless # Optional: attach build variables
```
### Override Chrome Extensions for a Web Run
The `dependencies` input controls the extension loadout for tests triggered
directly by this Action. It supports ID selectors, labels, and `flow-keys`, but
cannot currently be combined with `suite-keys`:
```yaml theme={null}
# Omit dependencies to inherit the web app's configured defaults.
labels: smoke
# Explicitly run with no extensions.
dependencies: '[]'
# Use one extension's active build and pin another extension to a specific build.
dependencies: >-
["11111111-1111-1111-1111-111111111111",
{"app_id":"22222222-2222-2222-2222-222222222222",
"app_build_id":"33333333-3333-3333-3333-333333333333"}]
```
Each string is an extension app UUID. Object entries use `app_id` and may add
`app_build_id` to pin an exact build. The input must be a valid JSON array and
is rejected for iOS, Android, and Chrome extension uploads.
### Web Workflow Examples
For Vercel deployments using the GitHub integration (OAuth), use the `wait-for-vercel-preview` action to get the preview URL:
```yaml .github/workflows/autosana-web-vercel.yml theme={null}
name: Web App Testing with Autosana
on:
pull_request:
branches: [ main ]
jobs:
test-preview:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
deployments: read
steps:
- name: Wait for Vercel Preview
uses: patrickedqvist/wait-for-vercel-preview@v1.3.2
id: vercel
with:
token: ${{ secrets.GITHUB_TOKEN }}
max_timeout: 600
- name: Register with Autosana
uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
platform: web
app-id: my-web-app # Choose a unique identifier for your app
url: ${{ steps.vercel.outputs.url }}
variables: BASE_URL=${{ steps.vercel.outputs.url }}
# labels: smoke,critical # Optional: run everything carrying these labels
# suite-ids: uuid1,uuid2 # Optional: trigger suites after upload
# flow-ids: uuid1,uuid2 # Optional: or specify individual flows
```
The `wait-for-vercel-preview` action requires the `permissions` block shown above to access deployment information. Without these permissions, you may see "Resource not accessible by integration" errors.
The `app-id` must be lowercase alphanumeric with hyphens only (e.g., `my-web-app`, `staging`, `preview-app-123`). This identifier is used to track your web app across deployments.
If you have a static staging URL or custom deployment:
```yaml .github/workflows/autosana-web-custom.yml theme={null}
name: Web App Testing with Autosana
on:
push:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Register with Autosana
uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
platform: web
app-id: my-staging-app
url: https://staging.myapp.com
# labels: smoke,critical # Optional: run everything carrying these labels
# suite-ids: uuid1,uuid2 # Optional: trigger suites after upload
# flow-ids: uuid1,uuid2 # Optional: or specify individual flows
```
***
## Step 4: What Happens Next?
Once you've set up our CI/CD integration:
Add a flow, suite, or label selector to your action config to trigger tests immediately after upload. For code-managed tests, use `flow-keys` or `suite-keys` so the workflow keeps stable YAML identifiers instead of Autosana UUIDs. The Action waits for all tests to complete and reports results inline, failing the job if any test fails.
Alternatively (or additionally), set when flows run in your [Automations](https://autosana.ai/automations) page to run flows on a schedule or trigger.
Check your test runs in the [Runs](https://autosana.ai/runs) page.
***
## Need Help Setting Up?
If you're using Claude Code or another AI agent, you can add our MCP server to help automate the setup process. The MCP server can:
* Set up your CI/CD pipeline automatically
* Build your app for iOS or Android
* Find your app's bundle ID
* And more!
Learn more at our [MCP Server documentation](https://docs.autosana.ai/mcp-setup).
### Instructions for AI Agents Using the Autosana MCP (humans, you can ignore this!)
If you are an AI agent (such as Claude Code, Cline, or Cursor) using the Autosana MCP server to help set up CI/CD workflows, use the following prompt to guide your workflow generation.
```
Scan the repo to understand the project and how it is built. You should determine things like:
1. The project type (react-native, flutter, android-native, ios-native, etc)
2. Supported platforms (android, ios, or both)
3. Existing Github workflow files, if any
4. The correct build commands and configurations, indicated by the presence of Fastlane, EAS, etc... It can also help to look for .md files that explain the build process.
5. Available environments (development, staging, production, etc) and how they are configured
Create a plan for how to build the app. Then, generate a complete autosana-ios.yml and autosana-android.yml (whatever platforms are applicable) file, in .github/workflows. Use the best approach for the project (Fastlane, EAS, native, etc).
Unless specified otherwise by the user, the file(s) should start with the following:
name: Autosana (platform) App Upload
on:
workflow_dispatch:
push:
branches:
- (primary branch)
pull_request:
branches:
- (primary branch)
The file MUST contain the following Autosana Github Actions script at the end:
- uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
bundle-id: com.example.app # TODO: Replace with your bundle ID
platform: ios/android
build-path: ./build/MyApp.app # TODO: Replace with your build path
Important requirements:
- For Android, the APK must include the `x86_64` architecture; a universal APK is recommended
- For iOS, always build for simulator, not a real device
- Include actual paths based on common conventions for the detected framework
- Prefer the staging environment, if available (and not specified otherwise by the user)
- For EAS iOS builds, you can use a linux runner if doing cloud builds
Finally, once implemented, thoroughly review the script(s) to ensure it is correct before it's run.
It is critical that the script(s) will run successfully.
```
***
## Helpful Resources
Learn how to build your app for our cloud
View the source code and documentation for our GitHub Action
Copy your API key from Settings → Integrations
### Commit detection and PR results
For `deployment` and `deployment_status` events, the action automatically uses the deployed commit, even when the workflow checks out trusted code. PR events continue to use the PR head SHA; other events retain checkout-based detection. For manual or scheduled workflows targeting a different commit from the checkout, pass `commit-sha` explicitly. Optional `branch-name` and `repo-full-name` inputs override the corresponding GitHub metadata. Web tests use the build returned by registration, keeping concurrent preview runs separate.
The action reports its result through the workflow job. To show that result on a different PR, the calling workflow must create and complete a check on the PR's head SHA with `checks: write` permission.
# Running from the CLI
Source: https://docs.autosana.ai/code-managed-cli
Run code-managed flows and suites from the terminal: on your own device, on Autosana's cloud devices, or from a branch
`autosana run` runs the tests in your `.autosana/` folder from the terminal, without opening a pull request. It is shorthand for `autosana flows run`. There are three ways to run, and the flag you pass decides where the tests come from and where they execute.
## Three ways to run
| | `--local` | `--cloud` | Branch (the default) |
| - | - | - | - |
| **Tests come from** | your working copy, uncommitted | your working copy, uncommitted | the branch's committed files on GitHub |
| **Run on** | your simulator, emulator, phone, or dev server | Autosana's cloud devices | Autosana's cloud devices |
| **App build** | whatever is on your device | the branch's newest upload, or one you name | the upload from the same commit, or one you name |
| **Hooks** | not run | run | run |
| **You can select** | one flow or one suite | any flows, suites, labels, or `--all` | flows, suites, or `--all` |
Use `--local` while you iterate on a flow. Use `--cloud` to try an edit on real devices before you commit it. Use a branch run for what CI would do: the committed tests, at a commit, on cloud devices.
## Before you start
[Install the CLI](/install-cli) and log in. Run `autosana login` from inside the repository and the login is stored for that repository, so different repos can use different organizations.
```bash theme={null}
autosana login
autosana whoami # the organization the CLI will run as
```
The organization you are logged in as must own the app you run against. A run that reports `App not found. Upload a build first.` almost always means the login belongs to a different organization than the app. Check `autosana whoami` before checking anything else.
`autosana run` looks in `.autosana/` in the current directory: `--local` and `--cloud` take the tests from there, and every mode reads `config.yaml` from there (a branch run takes its tests from GitHub). Pass `--path` to point at another folder, for example in a monorepo. (`autosana flows validate` takes the folder as its argument instead.)
## Choosing the tests
Name flows by key and suites by folder:
```bash theme={null}
autosana run login --local # one flow, by key (--local or --cloud only)
autosana run --suite checkout --cloud # one suite, by folder name
autosana run --flow login --flow signup # several flows (a branch run needs --flow)
autosana run --cloud --label smoke # flows and suites whose labels: list has this name
autosana run --all --ref my-branch # every suite and standalone flow, here from a branch
```
A flow's key is its path under `.autosana/` without the `.flow.yaml` ending, so `.autosana/checkout/cart.flow.yaml` is `checkout/cart`. A suite's key is its folder, so the folder `.autosana/checkout/` with a `_suite.yaml` inside is the suite `checkout`.
A bare name is always read as a **flow** key. `autosana run checkout` fails with `flow key 'checkout' not found` when `checkout` is a suite folder. Use `--suite checkout`, or name a flow inside it such as `checkout/cart`.
A label matches the `labels:` list in your flow and suite files. Labels set only in the dashboard are not seen. A flow matched by a label runs inside its suite, with the suite's setup, even when only the flow file carries the label. A flow that belongs to a selected suite runs inside that suite; a flow named by its key runs on its own.
## Choosing the app
A run needs to know which of your apps to run against. For a mobile app that is its bundle id and platform. For a web app, `--app-id` takes its slug: a lowercase id such as `my-web-app`, the `app-id` input of the [CI action](/ci-cd-integration), which the [List Apps API](/api-apps#list-apps) returns as the app's `bundle_id`. A branch run accepts only the slug; local and cloud runs also accept the app's UUID.
```bash theme={null}
autosana run login --cloud --bundle-id com.example.app --platform ios
autosana run login --cloud --app-id
```
Rather than type these on every run, commit them once in `.autosana/config.yaml`, next to your flows (key reference in [Files & Schema](/code-managed-files#config-file)):
```yaml theme={null}
apps:
ios:
bundle_id: com.example.app.dev
android:
bundle_id: com.example.app
web:
app_id: my-web-app
default_platform: ios
environment: staging # optional
```
With that file, `autosana run --cloud --suite checkout` needs no app flags: it uses the iOS app because `default_platform` says so. Add `--platform android` and it uses the Android one. `--platform web`, or `default_platform: web`, uses the `app_id`. A file with a single entry needs no `default_platform` at all. A local run picks the entry for whichever platform your `autosana up` session is on.
A value you type beats one from the environment, which beats the file: `--bundle-id` on the command line wins over the `AUTOSANA_BUNDLE_ID` variable, which wins over `config.yaml`. The keys, their rules, and what the file is not (a test, or anything the sync reads) are in [Files & Schema](/code-managed-files#config-file).
Two more things affect the target:
* **Environment.** `--environment ` (or `environment:` in the file) selects the [variable set](/variables) your flows resolve `${env:…}` references against. It is required when one app spans several [environments](/environments).
* **Local runs on mobile** can leave the app out entirely when your organization has a single app for that platform. Cloud and branch runs always need it named, by flag, variable, or file.
## Local runs
Start a session on your device or dev server with `autosana up`, then run against it:
```bash theme={null}
# Mobile: boot a local session, then run the working-copy flow
autosana up --platform ios --detach
autosana run login --local
# Web: connect your dev server, then target its Autosana app
autosana up --platform web --port 3000 --detach
autosana run login --local --app-id
# Suites work locally too
autosana run --suite checkout --local
```
A local run executes your **uncommitted** YAML on the target that `autosana up` connected. Web is inferred from `--app-id`; the mobile platform is taken from your session. Use `--platform` when you have sessions on more than one platform, or `--session-id` when several sessions match. Set-up details for simulators, emulators, physical phones, and dev servers are on the [Local Testing](/local-testing) page.
Local runs execute your flow's **instructions** only. [Hooks](/hooks) are not run locally because they manipulate the test environment rather than the target. Use a suite's `setup_flow` for setup steps that can run on the target.
## Cloud runs
`--cloud` uploads your uncommitted `.autosana/` and runs it on Autosana's devices, so you can try an edit before committing it. Select any number of flows, suites, or labels:
```bash theme={null}
autosana run login --cloud --bundle-id com.example.app --platform ios
autosana run --cloud --suite checkout --label smoke
autosana run --cloud --all --branch feat/new-cart
autosana run login --cloud --app-build-id
```
The first line names the app; the others leave it to `config.yaml` (see [Choosing the app](#choosing-the-app)). The tests always come from your working copy. What `--branch` and `--app-build-id` choose is only the **build**:
| Flags | Which build runs |
| - | - |
| *(none)* | the newest build uploaded from your current git branch |
| `--branch ` | the newest build uploaded from that branch |
| `--app-build-id ` | the build you name, whatever branch it came from |
When no build was uploaded from the branch, the run stops and, if the app has an active build, names it so you can pin it with `--app-build-id`.
Unlike `--local`, a cloud run does execute [hooks](/hooks). Hook scripts in your working copy are uploaded with the flows and take precedence; a hook a flow names that has no file in `.autosana/` is taken from the dashboard.
## Branch runs
With neither `--local` nor `--cloud`, the CLI runs the tests **committed** on a branch. It reads that branch's `.autosana/` from GitHub, so nothing uncommitted takes part:
```bash theme={null}
autosana run --suite checkout --ref my-branch
autosana run --flow login --ref my-branch
autosana run --all --ref my-branch
```
`--ref` defaults to your current branch. Name flows with `--flow` here; the bare `autosana run login` form is for `--local` and `--cloud` only. The repository comes from your `origin` remote, or pass `--repo owner/name`.
By default the run uses the app build made from the same commit as the tests, so every result names a single commit. Two flags change which build runs:
| Flags | Which build runs |
| - | - |
| *(none)* | the build at the same commit as the tests |
| `--build-match branch-latest` | the newest build on that branch |
| `--app-build-id ` | the build you name, whatever commit it came from |
The last row is the one to reach for when you changed only test files: no build exists at your commit, and rebuilding an app that did not change is wasted work. Reach for `--build-match branch-latest` when you want the branch's newest build but do not have its ID to hand, in CI for example.
If the option you choose matches no build, the run is refused and says so. Autosana never quietly falls back to the app's active build, because a result that does not name the build it came from is worse than no result.
`--build-match` needs a branch name. Pass a full commit sha as `--ref` and the flag is refused, because a sha names no branch to take a latest build from. Name the build with `--app-build-id` instead, which works with any ref.
The dashboard runs branches too: the run dialogs on the **Flows** page take a branch and a build, and the build you pick there need not come from that branch.
## What a run prints
Each run that starts is printed as a link you can open, followed by the batch that groups them:
```text theme={null}
✓ Started 1 run at branch dev
https://autosana.ai/runs/groups/
batch
```
The link opens the run in the dashboard. The batch id is what the [Run Status API](/api-runs#run-status) polls, if you want to wait on the result from a script.
## Troubleshooting
### App not found. Upload a build first.
The organization the CLI is logged in as does not own an app with that bundle id and platform. Run `autosana whoami`; if it names the wrong organization, run `autosana login` from inside the repository. If the organization is right, check the bundle id against the [Apps](/apps) page. A web app is looked up by app id, a mobile app by bundle id and platform.
### flow key 'x' not found in the working copy
`x` is not a flow. Either it is a suite folder (use `--suite x`), or the file is not named `.flow.yaml`, or you are in a directory without `.autosana/` (pass `--path`).
### no build was uploaded with branch\_name 'x' for this app
A `--cloud` run found no build uploaded from that branch. Upload one from CI, or pin any build with `--app-build-id`. The error names the app's active build when there is one.
### My suite won't run against a branch
A branch run fails with an error naming one or more flow files, instead of starting:
```text theme={null}
suite 'checkout' would not run all of its tests: the flow file
.autosana/checkout/cart.flow.yaml sits beside .autosana/checkout/_suite.yaml but
could not be used, so the suite would run short
```
Autosana refuses to run a suite when one of its tests is missing. A suite that runs four tests out of five and still reports a pass would hide the fact that a test never ran. This happens when the suite has no `flows:` list: its tests are then every `.flow.yaml` beside its `_suite.yaml`, and on that branch one of those files could not be read.
**Possible Causes:**
* The file has a YAML error
* The file couldn't be downloaded from GitHub
* Two files claim the same key, so both were dropped
**Solutions:**
* Fix or delete every file the error names. It lists them all at once, so you can repair them in a single commit
* Run `autosana flows validate` to reproduce a YAML error or a duplicate key
* If a file failed to download, start the run again. If the same error comes back, check that the file is committed on that branch
* Give the suite an explicit `flows:` list, so its tests are the ones you name rather than whatever the folder contains
Only runs you start against a branch are refused this way. The automatic check on a pull request is unaffected, because suite changes there don't apply until you merge (see [Pull request workflow](/code-managed-sync#pull-request-workflow)).
## Next Steps
* [Learn the file schema →](/code-managed-files)
* [Validate, sync, and preview on PRs →](/code-managed-sync)
* [Set up local devices and dev servers →](/local-testing)
* [Poll a run from a script →](/api-runs#run-status)
# Files & Schema
Source: https://docs.autosana.ai/code-managed-files
The repository layout and the full flow, suite, and hook file reference for code-managed flows
This is the complete file reference for [code-managed flows](/code-managed-flows): where files live under `.autosana/`, and every recognized key in flow files, suite manifests, and hook scripts.
## Repository layout
Everything Autosana manages lives under a single `.autosana/` folder — at your repository root by default, or under a subdirectory you point Autosana at (see [Monorepos](#monorepos)). Flows are `*.flow.yaml` files, a suite is a folder containing a `_suite.yaml` manifest, and hooks are script files under a `hooks/` directory.
```text theme={null}
.autosana/
├── autosana.md # shared repository context for test runs
├── config.yaml # run defaults for the CLI (not a test, never synced)
├── login.flow.yaml # a root flow (folder "")
├── hooks/
│ └── seed-db.py # a hook — slug "seed-db"
└── smoke/ # a suite folder
├── _suite.yaml # the suite manifest (suite key "smoke")
├── checkout.flow.yaml # a flow in the smoke suite
└── hooks/
└── reset-test-env.sh # another hook — slug "reset-test-env"
```
| File type | Location | Purpose |
| - | - | - |
| **Flow** | `.flow.yaml` (anywhere under `.autosana/`) | One test flow |
| **Suite** | `/_suite.yaml` (one per folder) | Groups the flows in that folder |
| **Hook** | `/hooks/.` (direct parent must be `hooks/`) | A setup/teardown script |
| **Context** | `autosana.md` (root of `.autosana/` only) | Shared guidance for runs associated with this repository; see [Repository context](/code-managed-flows#repository-context) |
| **Config** | `config.yaml` (root of `.autosana/` only) | Default app target for `autosana run`; see [Config file](#config-file) |
The sync reads `autosana.md` at the root of `.autosana/`, files that end in `.flow.yaml`, are named exactly `_suite.yaml`, or scripts directly inside a `hooks/` folder with a supported extension. Every other file under `.autosana/` is ignored, including `config.yaml`, which only the CLI reads.
## Flow files
A flow file ends in `.flow.yaml`. The only required keys are `name` and `instructions`, where `instructions` is either a single multiline prompt or a YAML list of steps.
As a multiline prompt:
```yaml theme={null}
key: login
name: Login
instructions: |
Log in as qa+smoke@example.com using the password in ${env:LOGIN_PASSWORD}.
Open the account menu and verify the signed-in email is shown.
Sign out and confirm you land back on the login screen.
```
Or as a list of steps:
```yaml theme={null}
key: checkout
name: Checkout happy path
description: Buys one item as a logged-in user
caching_mode: 'off'
device_count: 2
instructions:
- From the home screen, tap the cart icon
- Tap "Checkout"
- Verify the order confirmation screen appears
```
The recognized top-level keys are:
| Key | Required | Notes |
| - | - | - |
| **name** | Yes | Non-empty string. |
| **instructions** | Yes | A YAML list of non-empty step strings, or a single non-empty prompt string. A list is numbered into a single prompt. |
| **key** | No | Unique flow key. Defaults to the path minus `.autosana/` and `.flow.yaml` (e.g. `smoke/checkout`). |
| **description** | No | Free-text description. |
| **app** | Sometimes | Name of a linked app. Optional when the repo is linked to 0 or 1 apps; **required on every flow once the repo is linked to more than one app**. An unknown name fails the sync. |
| **caching\_mode** | No | `off`, `standard`, or `aggressive`. Omit or use `null` to inherit the workspace mode. A flow can reduce the workspace mode, never exceed it. Quote `'off'` so YAML treats it as a string. |
| **caching** | No | Legacy boolean: `false` means Off; `true` means inherit. Do not combine with `caching_mode`. |
| **device\_count** | No | Number of devices the agent controls. Use `1` (default) or `2`. Every member and setup flow in a suite must use the same value. |
| **setup\_hooks** | No | Ordered list of hook slugs to run before the flow. |
| **teardown\_hooks** | No | Ordered list of hook slugs to run after the flow. |
| **labels** | No | List of label names — see [Labels](#labels). |
| **variables** | No | Map of variable names to literal string values. See [Referencing variables](#referencing-variables). |
Any top-level key that isn't in the table above is a hard error. A typo like `instructoins:` fails validation with a "Did you mean 'instructions'?" suggestion on the [Autosana - Code-Managed Flows check](/code-managed-sync#sync-status-&-validation), so mechanical mistakes are caught before they merge.
**Let the path derive your key.** If you omit `key`, the key comes from the file path (`.autosana/smoke/checkout.flow.yaml` becomes `smoke/checkout`). Set an explicit `key` when you want the key to stay stable even if you move or rename the file — see [Deleting and renaming](/code-managed-sync#deleting-and-renaming).
### Exporting caching settings
CLI 0.9.15 and later preserve each caching mode when exporting flows. Older CLI
versions export an explicit Standard limit as caching Off, because their boolean
format cannot preserve that limit. Upgrade the CLI before exporting to retain
the exact setting.
### Referencing variables
Define flow overrides in `*.flow.yaml` and suite defaults in `_suite.yaml` with `variables`:
```yaml theme={null}
name: Login
variables:
ACCOUNT_ROLE: guest
instructions: Sign in as ${env:ACCOUNT_ROLE}.
```
Flow variables override suite defaults. Hooks read them as environment variables (see [Variables](/variables)).
Values must be literal strings; quote numbers and booleans (`CODE: "00123"`, `ENABLED: "false"`).
Names use letters, digits, and underscores, and cannot start with a digit.
A supplied map replaces that flow's or suite's overrides on sync. Omit `variables` to keep existing values, or use `variables: {}` to clear them.
Keep secrets in [Environments](/environments), since YAML values are committed to your repository.
### Referencing hooks
Attach hooks to a flow with `setup_hooks` and `teardown_hooks`, each an ordered list of hook **slugs**:
```yaml theme={null}
name: Checkout happy path
setup_hooks:
- seed-db
teardown_hooks:
- reset-test-env
instructions:
- Add an item to the cart and check out
- Verify the order confirmation appears
```
A slug resolves against **every active hook in your organization** — not just repo files — so you can also reference hooks authored in the dashboard, including cURL hooks (which can only be created in the dashboard). Find a dashboard hook's slug on the hook in the [Hooks](/hooks) page. See [Hooks as files](#hooks-as-files) for how a repo file's slug is derived.
[Built-in hooks](/guides-otp-testing) don't need to be added from the Hook Library first. Reference **Lease email address** as `lease-email-address` and **Lease phone number** as `lease-phone-number`, and the sync adds the hook to your organization if it isn't there yet. If one of your own hooks already uses that slug, your hook is used instead.
Duplicate slugs within one list are rejected. The same slug may appear in both `setup_hooks` and `teardown_hooks` — that's legal. To reference a hook by name inside your instructions at run time, use `${hooks:Hook Name}` (by display name); see [Hooks](/hooks).
When the list mixes scripts with a launch-args (`.json`) hook, the app starts where that hook sits — see [when launch configuration runs](/hooks#when-it-runs).
### Labels
Attach [labels](/api-labels) to a flow or suite with `labels`, a list of label names:
```yaml theme={null}
name: Checkout happy path
labels: [smoke, critical]
instructions:
- Add an item to the cart and check out
```
Names match case-insensitively against your org's labels; unknown names are auto-created. Omitting `labels` leaves the flow/suite's existing labels untouched — an explicit `labels: []` clears them. Labeled flows and suites dispatch via run-by-label as usual.
## Suites
A suite is a folder containing a `_suite.yaml` manifest. The suite's key is its folder path (a `smoke/` folder gives the suite key `smoke`). The only required key is `name`:
```yaml theme={null}
name: Smoke Tests
description: Critical-path checks
setup_flow: login
parallelize_flows: false
instructions: |
Test account is qa+smoke@example.com.
flows: [login, checkout]
```
The recognized keys are:
| Key | Required | Notes |
| - | - | - |
| **name** | Yes | Non-empty string. |
| **flows** | No | Ordered list of flow-key references. Omit it to include every flow **directly in the suite's own folder** (subfolders are not included), in filename-alphabetical order. |
| **setup\_flow** | No | A single flow-key reference to run first. |
| **parallelize\_flows** | No | Boolean. Runs each member flow in an independent session when `true`; defaults to `false`. An explicit API request override takes precedence for that launch; an omitted or `null` override uses this saved value. |
| **description** | No | Free-text description. |
| **instructions** | No | Suite-level context shared across the suite's flows (e.g. shared test-account info). |
| **setup\_hooks** / **teardown\_hooks** | No | Ordered lists of hook slugs. |
| **labels** | No | List of label names — see [Labels](#labels). |
| **variables** | No | Map of variable names to literal string values. See [Referencing variables](#referencing-variables). |
The **Lease email address** and **Lease phone number** built-in hooks may be listed in a suite's `setup_hooks`. Unlike ordinary suite setup scripts, a lease hook is inherited by every member flow and creates a separate inbox or number for each flow run. The suite's `setup_flow`, when configured, receives its own lease too. This also applies when `parallelize_flows: true`. These built-ins are setup-only and are rejected in `teardown_hooks`. Reference them as `lease-email-address` and `lease-phone-number`; you don't need to add them from the Hook Library first.
References in `flows` and `setup_flow` resolve **sibling-first**: a flow directly in the suite's own folder wins, otherwise the reference is matched against every flow in the repo. Flows in a subfolder are not default members — list them explicitly (or give the subfolder its own `_suite.yaml`). Flow keys and suite keys must each be unique across the whole repo. An unknown reference, or the same flow listed twice, is a validation error.
The `_suite.yaml` file must live inside a folder — it cannot sit at the `.autosana/` root, because a suite *is* a folder. Membership is defined by the suite (its `flows` list or its folder), never declared inside the flow files themselves.
YAML parses bare `no`, `yes`, `on`, and `off` as booleans (the "Norway problem"). Because `setup_flow` and `name` must be strings, `setup_flow: no` fails validation — quote the value if you really mean the string `"no"`.
For how variables and setup flows propagate across a suite's flows, see [Suites](/suites).
## Hooks as files
Hooks live as script files inside a `hooks/` folder — either `.autosana/hooks/` or `/hooks/`. The file extension determines the hook type, and the filename becomes the hook's slug.
| Extension | Hook type |
| - | - |
| `.py` | Python script |
| `.js` | JavaScript script |
| `.ts` | TypeScript script |
| `.sh` | Bash script |
| `.json` | Launch args (must be valid JSON) |
The **slug** is derived from the filename with its final extension removed: lowercased, with runs of non-alphanumeric characters collapsed to `-`. So `Seed DB.sh` becomes slug `seed-db`, and `reset.env.json` becomes `reset-env`. This slug is what you reference in `setup_hooks` / `teardown_hooks`. The **display name** is the humanized filename (`reset-test-env.sh` shows as "Reset Test Env").
**Hook slugs are unique across your whole organization** — the folder a hook lives in has no scoping effect. Two files that slugify the same (`seed-db.py` and `seed_db.sh`), or a slug that collides with a **different repository's** hook, fail the sync with a rename error. A slug that matches an existing **dashboard-authored** hook is *not* an error — that hook is adopted (its run history preserved), the same way flows adopt by name; see [Migrating existing flows](/code-managed-migrate). Rename the file if that isn't what you intend.
The file content **is** the hook script, stored verbatim. Empty or whitespace-only files are rejected. A `.json` launch-args hook must parse as valid JSON, and a filename with no letters or digits (so it produces an empty slug) is rejected.
Script hooks (`.py`, `.js`, `.ts`, `.sh`) read environment variables **natively** — `os.environ.get(...)` in Python, `process.env` in JavaScript/TypeScript, `$VAR` in Bash. Launch-args (`.json`) hooks — and dashboard-authored cURL hooks — use the `${env:KEY}` token instead, since they aren't executed scripts. cURL hooks have no file form; reference an existing dashboard cURL hook by slug, or use a `.sh` hook that runs `curl`. See [Hooks](/hooks) for the full hook model.
## Config file
`config.yaml` at the root of `.autosana/` sets defaults for `autosana run`. It is not a test: the sync never reads it and the CLI never uploads it. How the defaults apply is on [Running from the CLI](/code-managed-cli#choosing-the-app).
```yaml theme={null}
apps:
ios:
bundle_id: com.example.app.dev
android:
bundle_id: com.example.app
web:
app_id: my-web-app
default_platform: ios
environment: staging
```
| Key | Required | Notes |
| - | - | - |
| **apps** | No | Map keyed by `ios`, `android`, or `web`. A mobile entry takes `bundle_id`; the `web` entry takes `app_id`. |
| **default\_platform** | Sometimes | One of `ios`, `android`, `web`. Not needed when `apps` has a single entry, which is the default. With several entries and no `default_platform`, a run must pass `--platform`. |
| **environment** | No | Name of the [environment](/environments) runs resolve variables against. |
Any other key, an unknown platform, or a mobile entry without `bundle_id` (or a web entry without `app_id`) is an error. `autosana flows validate` reports it, and `autosana run` refuses to start until it is fixed.
## Monorepos
By default Autosana looks for `.autosana/` at your repository root. If your definitions live in a subdirectory — common in a monorepo — point Autosana at that folder with the repo's **Root directory** setting.
1. Go to **[Settings > Integrations > GitHub](https://autosana.ai/settings?tab=integrations\&integration=github)**
2. On the connected repo, set **Root directory** to the folder that contains `.autosana/` (for example, `services/mobile` for `services/mobile/.autosana/`)
3. Leave it blank to use the repository root
Flow and suite **keys stay relative to `.autosana/`**, so moving your definitions into a subdirectory doesn't change any keys — a flow at `services/mobile/.autosana/login.flow.yaml` still has the key `login`. The GitHub links on each row point at the full path in your repo.
Saving a new Root directory doesn't re-sync on its own. Click **Resync** on the repository card, or push to the default branch to sync from the new location. Once `.autosana/` has been detected, the nightly resync also checks it for drift.
## Next Steps
* [Validate, sync, and preview on PRs →](/code-managed-sync)
* [Run your tests from the terminal →](/code-managed-cli)
* [Write effective flows →](/flows)
* [Organize flows with Suites →](/suites)
* [Learn about hooks →](/hooks)
* [Manage variables →](/variables)
# Code-Managed Flows
Source: https://docs.autosana.ai/code-managed-flows
Define flows, suites, and hooks as files in your repository, synced via GitHub
Keep your flows, suites, and hooks in version control by defining them as files in your repository's `.autosana/` folder. When you push to your default branch, Autosana syncs those files and materializes them as flows and suites in your dashboard. This is GitOps for your tests: your repository is the source of truth, changes go through pull requests, and every definition is code-reviewed alongside the app it tests.
## How it works
1. You define flows, suites, and hooks as files under `.autosana/` in your repo
2. On a push to your default branch, Autosana fetches `.autosana/`, validates every file, and materializes the changes
3. On a pull request, Autosana previews the proposed flow definitions and posts a GitHub check
4. The dashboard shows the resulting flows and suites as **read-only**; edit them by pushing to the repo
The repository layout and the full flow, suite, hook, and config file reference
Run your tests on your own device, on cloud devices, or from a branch
Validate locally, preview changes on pull requests, and read the sync check
Export the flows you already have and let the sync adopt them
## Enabling code-managed flows
Code-managed flows are powered by the [GitHub Bot](/github-integration), so install the GitHub App first if you haven't.
1. Connect your repository in **[Settings > Integrations > GitHub](https://autosana.ai/settings?tab=integrations\&integration=github)**
2. Commit your flow, suite, and hook files under `.autosana/`
3. Push to the default branch to sync them, or open a pull request to preview the changes
Code-managed flows work automatically for connected repositories. Every push to the default branch reconciles the files, and a nightly resync catches drift in repositories where `.autosana/` has been detected. For a folder below the repository root, set the repo's **Root directory** first (see [Monorepos](/code-managed-files#monorepos)).
Using GitLab? Sync from a CI job instead of the GitHub App: see [GitLab CI](/gitlab-ci).
You need to be an admin of your Autosana organization to install the GitHub App and manage integration settings.
## Read-only in the dashboard
A flow, suite, or hook is code-managed when it originates from a connected repository. Code-managed rows are **read-only** in the dashboard: to change one, edit the YAML (or script) in your repository and push, or open a PR to preview the change first. On the Flows page, use **View** to inspect a flow or suite without editing it, or select the GitHub icon to open its source file directly.
Read-only applies to the **definition**, not execution. You can still run code-managed flows and suites from the dashboard, and from the [terminal](/code-managed-cli). Environment values stay in the dashboard. Flow and suite overrides can be defined in YAML with [`variables`](/code-managed-files#referencing-variables).
Deleting files and pushing archives their corresponding flows, suites, and hooks. Their history remains available; removing a file does not turn its archived definition into a dashboard-editable flow.
## Repository context
Add `.autosana/autosana.md` to share plain Markdown context with the test agent. Use it for app terminology, navigation hints, and conventions that apply across tests. No frontmatter or YAML is required. The file must be UTF-8 text, at most 64 KiB.
```markdown title=".autosana/autosana.md" theme={null}
# App context
A workspace contains projects. Switch workspaces from the account menu.
Dismiss the optional product tour before starting a test.
```
Each flow receives context from the repositories associated with that flow, its suite, and its app build. Each repository is included once per flow; context from an unrelated flow in the same run is not shared. These sources are combined without a precedence order.
When you select a branch or commit for a run, Autosana reads context at the resolved commit for that repository. Otherwise, it uses the latest successfully synced default-branch context. An app build identifies a relevant repository but does not select the build's commit for context. Local working-copy runs can use uncommitted context from the working copy.
Default-branch context becomes available after the repository syncs. Builds can still run without a connected repository, and existing repositories can run while their first context sync completes. An explicit context read failure stops the run with an error.
Missing or empty context files add nothing to the agent's prompt. Each run saves its selected context and source details, so later repository changes do not change the historical snapshot. Edit context in Git, just like code-managed flow definitions.
Open **Settings → Integrations → GitHub** and select **View autosana.md** on a repository to read its latest synced default-branch file and see the source commit. This view also reports missing files and sync failures.
## Migrating existing flows
Already have flows in the dashboard? Export them to files and let the first sync adopt them with their history: see [Migrating](/code-managed-migrate).
## Use with Claude Code
If you use [Claude Code](https://claude.com/claude-code), install the Autosana plugin so Claude knows the `.autosana/` schema, the validate-before-push workflow, and the gotchas. It can then scaffold and fix flows, suites, and hooks for you in one pass, and validate them before you push:
```
/plugin marketplace add Autosana/claude-code-plugin
/plugin install autosana@autosana
```
Invoke it with `/autosana:code-managed-flows`, or just start editing files under `.autosana/` and Claude pulls in the skill automatically. The plugin is open source at [Autosana/claude-code-plugin](https://github.com/Autosana/claude-code-plugin).
## Next Steps
* [Learn the file schema →](/code-managed-files)
* [Run your tests from the terminal →](/code-managed-cli)
* [Validate, sync, and preview on PRs →](/code-managed-sync)
* [Write effective flows →](/flows)
* [Set up the GitHub Bot →](/github-integration)
# Migrating Existing Flows
Source: https://docs.autosana.ai/code-managed-migrate
Export the flows, suites, and hooks you already have in the dashboard to files, and let the sync adopt them
Already have flows in the dashboard? Export them to files with the CLI, commit them, and the first sync adopts them with their run history intact.
## Export
[Install the CLI](/install-cli) and log in, then export:
```bash theme={null}
# Write every dashboard flow, suite, and their hooks to ./.autosana (with explicit keys)
autosana flows export --all
# Or export a single suite and its flows
autosana flows export --suite "Smoke Tests"
```
Export writes each flow (`*.flow.yaml`), suite (`_suite.yaml`), and the flows' **and** suites' own setup/teardown **hooks** (`hooks/.`), adding `setup_hooks`/`teardown_hooks`, `labels`, and `variables` to the flow files and the suite manifests as needed. cURL hooks have no file form, so they're referenced by slug but stay dashboard-authored.
Review the generated files, run `autosana flows validate`, commit them under `.autosana/`, then push to the connected repository's default branch (see [Enabling code-managed flows](/code-managed-flows#enabling-code-managed-flows)).
## How adoption works
On the first sync, anything whose identity **matches** an active dashboard definition is **adopted**: its run history is preserved and it becomes code-managed (read-only), now driven by your files.
* A **flow** or **suite** is adopted when its `name` exactly matches one active dashboard flow or suite.
* A **hook** is adopted when its slug (its filename) matches one active dashboard hook.
Adoption is by **exact, case-sensitive** identity and only on a single match. A YAML flow named the same as an *unrelated* dashboard flow, or a hook file whose slug matches an unrelated dashboard hook, will claim it and make it read-only, so review your exported names before enabling. If two dashboard flows share a name (ambiguous), the sync creates a new flow instead of claiming either. A hook slug that collides with a **different repository's** hook is still a blocking error (see [Hooks as files](/code-managed-files#hooks-as-files)).
## Next Steps
* [Learn the file schema →](/code-managed-files)
* [Validate, sync, and preview on PRs →](/code-managed-sync)
* [Run the exported tests from the terminal →](/code-managed-cli)
# Validation, Syncing & Pull Requests
Source: https://docs.autosana.ai/code-managed-sync
Validate code-managed flows locally, preview changes on pull requests, and read the Autosana - Code-Managed Flows check
[Code-managed flows](/code-managed-flows) sync on every push to your default branch and preview on every pull request. This page covers the workflow around that sync: validating before you push, reading the **Autosana - Code-Managed Flows** check, what applies on a PR versus at merge, and how deletes and renames behave.
## Validating locally
Validate your `.autosana/` folder before you push with the `autosana` CLI ([install it](/install-cli) if you haven't). By default it checks the `.autosana` directory in the current folder; pass a path to validate somewhere else, handy for a monorepo:
```bash theme={null}
autosana flows validate
autosana flows validate services/mobile/.autosana
```
The CLI runs the **flow- and suite-YAML** rules exactly as the sync does — required keys, unknown keys, `instructions` shape, suite references, key uniqueness — and reports each error per file with the offending line. It exits non-zero on errors, so you can wire it into a pre-commit hook or CI step.
It also checks the hook slugs your files reference, including built-in hooks like `lease-email-address`, against your organization's hooks. That needs you to be logged in: run `autosana login` or set `AUTOSANA_API_KEY`.
A few checks still run **only** server-side and appear on the Autosana - Code-Managed Flows check after you push: `app:` name resolution, hook slugs that clash with another repository's hooks, and hook-file contents (empty scripts, invalid launch-args JSON). A green local run doesn't guarantee a green check if a flow references an unknown app.
Beyond validating, you can **run** the tests you are editing, on your own device or on cloud devices, without committing. See [Running from the CLI](/code-managed-cli).
## Sync status & validation
Every sync posts a GitHub check named **Autosana - Code-Managed Flows** on the head commit — on both pushes and pull requests. It concludes as a failure if there are any issues, otherwise success. The summary reports how many files synced (or lists the errors found), and parse errors are attached as inline annotations anchored to the offending line where one is known, with any "Did you mean" hint appended.
This check used to be called **Autosana Flows**. If you list it as a required status check in a branch protection rule, change that rule to `Autosana - Code-Managed Flows` — GitHub matches required checks by exact name, so a rule still naming the old one waits forever for a check that no longer reports.
**One broken file freezes the whole repo's materialization.** The apply step runs only when the entire `.autosana/` folder validates with zero issues. If any file has an error, the check reports everything, but no changes are applied and previously synced flows stay as they were until you fix it. The check makes this loud.
## Pull request behavior
In **Settings → Integrations → GitHub**, expand your repository and choose a **Pull request behavior** under **Code-Managed Testing**:
* **Sync and run changed flows** (default): validate and sync changes, then automatically run changed flows on pull requests.
* **Sync only**: validate and sync changes without automatically running changed flows.
**E2E Autopilot** is independent and can still select and run tests when enabled. Turn it off too if you want no automatic test runs. You can still run tests manually from the dashboard, CLI, or by tagging `@autosana`. Both modes keep syncing on pushes to the default branch.
## Pull request workflow
Open a pull request that changes `.autosana/` and Autosana previews your flow changes without touching mainline:
* Each changed or added `*.flow.yaml` gets a **PR-scoped flow version** pinned to the PR, so the PR's test runs use the proposed definitions
* The **Autosana - Code-Managed Flows** check runs on the PR. For mechanical typos on lines you changed, Autosana also posts inline GitHub *suggestion* comments you can apply in one click
* Suite manifest (`_suite.yaml`) changes are **preview-only** on a PR — a warning notes that suite changes apply when the PR merges
* Removed flow files are ignored during preview; the archival happens on merge
* **Hook files are not previewed.** A PR's runs use the mainline hook scripts, and hook changes (plus their slug-conflict validation) apply on merge. A PR touching *only* hook files posts no check at all
**The PR check validates each changed flow file on its own.** Cross-file checks — repo-wide key uniqueness, suite references, hook slugs and conflicts, and `app:` names — run at merge on the mainline sync, not on the PR. A PR can go green and still fail after merge (for example, a new file whose key duplicates an existing flow). Run `autosana flows validate` before merging to catch cross-file issues early.
**Pull requests from forks are skipped** — no flow versions are minted and no check is posted. Fork-authored YAML never touches your organization; it's validated and applied by the mainline sync only after the PR merges.
When the PR merges, the merge lands as a push to your default branch, and the mainline sync materializes everything for real: new flows go live, PR-created flows are promoted to mainline, and suite, hook, and deletion changes apply.
## Deleting and renaming
Removing a definition is a soft delete. When you delete a `.flow.yaml`, `_suite.yaml`, or hook file and push to your default branch, Autosana **archives** the corresponding flow, suite, or hook — it's deactivated and hidden, but its run history is kept. If a file with the **same key** returns later, the exact same record is reactivated (history intact).
**Renaming or moving a flow file without an explicit `key` splits its history.** Because the key defaults to the file path, moving `login.flow.yaml` to `auth/login.flow.yaml` archives the old flow (`login`) and creates a new one (`auth/login`). Set an explicit `key:` (see [Flow files](/code-managed-files#flow-files)) so the record — and its run history — follows the file.
Deleting a **hook file** while any flow or suite still references its slug fails the whole sync with an "unknown hook" error. Remove the `setup_hooks` / `teardown_hooks` references in the same commit as the hook file.
## Troubleshooting
### My changes didn't appear in the dashboard
**Possible Causes:**
* The repository is not connected to the Autosana GitHub App
* The push wasn't to the default branch
* Your `.autosana/` folder isn't at the repository root and no **Root directory** is set for the repo
* Another file in `.autosana/` has a validation error, so the whole apply step was skipped
* After a **force-push** that rewinds the branch to older-dated commits, the sync may be skipped as stale — use **Resync** in the repository’s GitHub integration settings
**Solutions:**
* Connect the repository in **[Settings > Integrations > GitHub](https://autosana.ai/settings?tab=integrations\&integration=github)**
* Confirm you pushed to the default branch, or use **Resync** in the repository’s GitHub integration settings
* Open the **Autosana - Code-Managed Flows** check on the commit and fix every reported error — apply only runs at zero issues
### The Autosana - Code-Managed Flows check failed
**Possible Causes:**
* An unknown top-level key or a malformed `instructions` value
* A suite reference that doesn't match any flow key, or a duplicate flow/suite key
* A `setup_hooks` / `teardown_hooks` slug that doesn't match any hook, or two hook files whose slugs collide
* An `app:` name that doesn't match a linked app — or an omitted `app:` when the repo is linked to more than one app
* A hook file that's empty, or a `.json` launch-args hook that isn't valid JSON
**Solutions:**
* Read the inline annotations and any "Did you mean" suggestions on the check
* Run `autosana flows validate` locally to reproduce the flow/suite YAML errors before pushing (app and hook errors surface only on the check)
### My PR check passed but nothing applied after merge
**Possible Causes:**
* The PR check validates each changed flow file on its own; cross-file checks (duplicate keys, suite references, hook slugs/conflicts, `app:` names) run at merge
**Solutions:**
* Open the **Autosana - Code-Managed Flows** check on the merge commit and fix the reported cross-file errors
* Run `autosana flows validate` on the whole `.autosana/` folder before merging
### A run from the terminal fails
Errors from `autosana run` (an app that isn't found, a flow key that isn't found, a suite refused because a file on the branch could not be read) are covered in [Running from the CLI](/code-managed-cli#troubleshooting).
### The dashboard won't let me edit a flow
**Possible Causes:**
* The flow is code-managed and therefore read-only in the dashboard
**Solutions:**
* Edit the linked `.autosana/` YAML file in your repository and push, or open a PR to preview the change
## Next Steps
* [Learn the file schema →](/code-managed-files)
* [Run your tests from the terminal →](/code-managed-cli)
* [Set up the GitHub Bot →](/github-integration)
* [Sync from GitLab CI →](/gitlab-ci)
# Code Review
Source: https://docs.autosana.ai/code-review
Automatic AI code review on every pull request
Turn on **Code Review** for a repo and Autosana reviews every PR like a senior engineer — leaving inline comments on the exact lines and an overall summary, plus a GitHub Check.
## Setup
1. Install the GitHub App from **[Settings > Integrations > GitHub](https://autosana.ai/settings?tab=integrations\&integration=github)** (see [GitHub Bot](/github-integration))
2. Toggle **Code Review** on for each repo
No app build or CI setup required — Code Review just reads the code in the PR.
## What it catches
Bugs, security issues, performance problems, missing edge cases, and maintainability concerns — tagged by severity and kept high-signal.
Want tests too? Enable **E2E Autopilot** alongside it (see [GitHub Bot](/github-integration)), or comment `@autosana run e2e tests` on any PR.
# Devin
Source: https://docs.autosana.ai/devin
Connect Devin to Autosana through MCP
Connect Devin so you can list flows, run them, debug runs, and search Autosana docs from a Devin session.
Autosana is a remote HTTP MCP server. In Devin's **Add a custom MCP** form, switch **Transport type** to **HTTP**. Leave **STDIO** — that form is for a local command you run yourself.
Adding a custom MCP in Devin requires permission to manage MCP servers. If you do not see **Add a custom MCP**, ask an organization admin or use **Suggest MCP Integration**.
## Setup
1. Go to **[Settings → Integrations → Devin](https://autosana.ai/settings?tab=integrations\&integration=devin)** and click **Create key and copy**
2. In Devin, open **Customize → MCPs → Add a custom MCP**
3. Set **Transport type** to **HTTP** (not STDIO)
4. Fill in:
```
Server name: Autosana
Transport: HTTP
Server URL: https://mcp.autosana.ai/mcp
Authentication: Auth Header
Header key: x-api-key
Header value:
```
5. Save, then click **Test listing tools** to confirm Devin can see Autosana's tools
6. Start a new session and ask Devin to list or run a flow
The custom MCP form defaults to STDIO (command, arguments, environment variables). Autosana is not a local process. Do not paste `npx`, `mcp-remote`, or a shell command there.
Use **Auth Header** with the header name `x-api-key`. Do not use `Authorization: Bearer`. Adding Autosana in Cursor (`mcp.json`) does not install it in Devin.
See [MCP Server](/mcp-setup) for the tools Devin can call after you connect.
# Dots
Source: https://docs.autosana.ai/dots
Add Autosana in ChatGPT with Developer mode and OAuth
A dot uses plugins installed in ChatGPT. Connect Autosana there. This does not add a separate control inside the dot.
## What you need
* ChatGPT on the web
* Developer mode, on Plus, Pro, Business, Enterprise, or Edu
* An Autosana account. When ChatGPT asks how to authenticate, choose **OAuth** and sign in. Do not use an API key
## Setup
1. In ChatGPT, open **Settings → Security and login** and turn on **Developer mode**
2. Copy the server URL from **[Settings → Integrations → Dots](https://autosana.ai/settings?tab=integrations\&integration=dots)**:
```text theme={null}
https://mcp.autosana.ai/mcp
```
3. Open **ChatGPT Plugins**, select **+**, and create an app named **Autosana** with that server URL, including `/mcp`
4. Choose **OAuth** and approve access in your browser
5. In a chat, open **+**, choose **Developer mode**, and select **Autosana**. Ask it to list or run a flow
Signing in with Codex on your computer does not create this ChatGPT app.
See [MCP Server](/mcp-setup) for the tools available after you connect.
# Environments
Source: https://docs.autosana.ai/environments
Organize apps and manage environment-specific configuration
Environments help you organize your apps by deployment stage (Dev, Staging, Production) and manage environment-specific configuration through environment variables.
## What are Environments?
An **Environment** in Autosana serves two purposes:
1. **App Organization**: Group apps by deployment stage or purpose
2. **Configuration Management**: Store environment-specific variables (API keys, URLs, credentials)
## Creating an Environment
### Step 1: Navigate to Settings
Click your profile icon or navigate to **[Settings](https://autosana.ai/settings)** from the sidebar.
### Step 2: Find the Environments Section
Scroll down to the **Environments** section.
### Step 3: Create Environment
1. Click **Create Environment**
2. Enter a name (e.g., "Development", "Staging", "Production")
3. Click **Create**
Your new environment appears in the list.
## Duplicating an Environment
If you already have an environment with many variables configured, you can duplicate it to quickly create a new one with the same settings.
### Step 1: Find the Source Environment
In the Environments section, locate the environment you want to duplicate.
### Step 2: Click Duplicate
Hover over the environment and click the **copy icon** that appears.
### Step 3: Configure the New Environment
1. Enter a name for the new environment (defaults to "\[Original Name] (copy)")
2. Review the list of variables that will be copied
3. Optionally edit any values before saving
4. Toggle off any variables you don't want to include
5. Click **Duplicate**
All selected variables (including secrets) are copied to the new environment.
Use duplicate when setting up similar environments, like creating a "Sandbox" from an existing "Staging" configuration.
## Managing Environment Variables
Environment variables are key-value pairs that you can reference in hooks and flow instructions. They make your flows and hooks reusable across different environments.
If a web app in this environment is reachable only through Tailscale, configure the [Tailscale integration](/private-network) for the environment before running web flows against it.
### Adding a Variable
1. Find your environment in the Environments section
2. Click **Add Variable** or the **+** icon
3. Enter the **Key** (e.g., `TEST_EMAIL`)
4. Enter the **Value** (e.g., `test@staging.com`)
5. Click **Save** or press Enter
Use UPPERCASE\_WITH\_UNDERSCORES for variable names (e.g., `API_KEY`, `TEST_PASSWORD`) to make them easily identifiable.
### Editing a Variable
1. Find the variable in the environment
2. Click the pencil icon (✏️)
3. Update the key or value
4. Save changes
### Deleting a Variable
1. Find the variable in the environment
2. Click the trash icon (🗑️)
3. Confirm deletion
Deleting a variable will cause any hooks or flows that reference it to fail if they try to use `${env:VARIABLE_NAME}`.
## Using Environment Variables
### In Hooks
Reference variables using `${env:VARIABLE_NAME}` syntax:
```
curl -X POST ${env:API_BASE_URL}/auth/login \
-H "Content-Type: application/json" \
-d '{"email":"${env:TEST_EMAIL}","password":"${env:TEST_PASSWORD}"}'
```
For scripts (Python, JS, Bash), environment variables are injected automatically and accessed via your language's standard method (e.g., `os.environ.get('TEST_EMAIL')` in Python).
[Learn more about using variables in hooks →](/hooks#environment-variables-in-hooks)
### In Flow Instructions
You can also use variables directly in flow instructions:
```
Open the app
Navigate to Settings
Enter "${env:API_KEY}" in the API Key field
Tap Save
```
### Secret Variables
Mark a variable as **secret** when it holds a password, API key, token, or any other sensitive value. Secrets are encrypted at rest and handled differently from plain variables throughout a run:
* **Never sent to the AI agent.** The agent only ever sees the `${env:KEY}` placeholder; the real value is injected at the exact moment the keystrokes are sent to your app.
* **Never stored in run history.** The recorded action keeps the placeholder, so run pages show a key chip, and the API and MCP output return the `${env:PASSWORD}` placeholder instead of the value.
* **Scrubbed from agent-facing output.** If a script result or error message would echo the value, it is replaced with the placeholder before anything is logged or shown.
Screenshots and video capture whatever your app renders on screen. If your app displays a secret in a visible (unmasked) field, it will appear in the recording pixels like it would for any user.
### Variable Scope
Variables are scoped to their environment:
* Apps assigned to "Staging" environment use Staging variables
* Apps assigned to "Production" environment use Production variables
* Apps without an environment don't have access to variables
The `${env:KEY}` syntax also resolves runtime variables — suite variables, flow variables, hook exports, and values set by the agent — in addition to environment variables. Runtime variables take precedence over environment variables when they share the same key. See [Variables](/variables) for details on runtime variables, precedence, and dynamic agent variables.
## Assigning Apps to Environments
### During App Creation
When creating a new app:
1. Select an environment from the **Environment** dropdown
2. Complete the app creation process
### For Existing Apps
1. Navigate to the **[Apps](https://autosana.ai/apps)** page
2. Find your app
3. Click the **Environment** dropdown on the app card
4. Select an environment
## Use Cases
### Development vs Production Credentials
**Development Environment:**
```
TEST_EMAIL = dev-user@example.com
TEST_PASSWORD = DevPass123
API_URL = https://api.dev.example.com
```
**Production Environment:**
```
TEST_EMAIL = prod-user@example.com
TEST_PASSWORD = ProdPass123
API_URL = https://api.example.com
```
**Flow (works in both):**
```
Log in with ${env:TEST_EMAIL} and ${env:TEST_PASSWORD}
Verify API connection to ${env:API_URL}
```
### Feature Flags
**Staging Environment:**
```
FEATURE_NEW_CHECKOUT = true
FEATURE_BETA_UI = true
EXPERIMENTAL_MODE = enabled
```
**Production Environment:**
```
FEATURE_NEW_CHECKOUT = false
FEATURE_BETA_UI = false
EXPERIMENTAL_MODE = disabled
```
**Flow (works in both):**
```
Open settings
Enable experimental mode if ${env:EXPERIMENTAL_MODE} is "enabled"
Verify new checkout is ${env:FEATURE_NEW_CHECKOUT}
```
## Best Practices
**Use Consistent Naming**
Establish a naming convention and stick to it:
* `TEST_EMAIL`, `TEST_PASSWORD` for credentials
* `API_URL`, `BASE_URL` for endpoints
* `FEATURE_*` for feature flags
* `ENV_*` for environment-specific settings
**Use Variables for Anything That Changes**
Not just credentials! Use variables for:
* URLs and endpoints
* Test data (names, addresses, etc.)
* Configuration values
* Feature flags
## Troubleshooting
### Variable Not Replacing in Hook
**Possible Causes:**
* Variable name doesn't match exactly (case-sensitive)
* Typo in variable name
* Wrong syntax (must be `${env:VAR_NAME}`)
* App not assigned to the environment
**Solutions:**
* Check variable name spelling and case
* Use `${env:VARIABLE}` — not `{{VARIABLE}}` or `$VARIABLE`
* Verify app is assigned to the correct environment
### Flow Works in One Environment but Not Another
**Cause:** Missing or different variables
**Solution:**
* Check that all required variables exist in both environments
## Managing Env Vars via API
Environment variables can be created, rotated, and deleted programmatically via the [Environment Variables API](/api-env-vars). Plaintext values come back in full; secret values are returned as `"***"` (write-only through the API).
## Next Steps
* [Use variables in hooks →](/hooks#environment-variables-in-hooks)
* [Organize apps by environment →](/apps#organizing-with-environments)
* [Create reusable hooks →](/hooks)
* [Manage env vars via API →](/api-env-vars)
# Flows
Source: https://docs.autosana.ai/flows
Write E2E tests in natural language
Flows are the heart of Autosana. A **flow** is a test written in natural language that describes what you want your agent to do in your app or website.
## What is a Flow?
A flow is a set of instructions that the agent follows to interact with your app or website. Instead of writing code or recording clicks, you simply describe what you want to test in natural language.
**Example Flow:**
```
Search for "wireless mouse".
Open the top result.
Add it to the cart.
Open the cart.
Verify the cart contains the wireless mouse.
```
The agent interprets these instructions, finds the relevant UI elements, and executes a series of actions, verifying results at each step.
## Creating a Flow
### Method 1: Quick Create
1. Navigate to the **[Flows](https://autosana.ai/flows)** page
2. Click **Create Flow**
3. Enter a flow name (required)
4. Write your instructions in natural language
5. Click **Create Flow**
### Method 2: Create in Suite
Create a flow directly within a suite:
1. Navigate to the **[Flows](https://autosana.ai/flows)** page
2. Expand a suite
3. Click **Create Flow** within the suite
4. Follow the same steps as Quick Create
5. The flow is automatically added to the suite
### Method 3: Define in Your Repo
Keep flows in version control and sync them from your repository — see [Code-Managed Flows](/code-managed-flows).
## Writing Effective Flow Instructions
Use **Save all** in the flow editor to save your edited flows and suites. Saving also refreshes unchanged flows and suites. Changes you make while a save is in progress remain unsaved until you save again.
See [Writing Effective Flow Instructions](/writing-effective-flow-instructions)
for guidance and examples.
## Supported Agent Actions
**Universal Actions** (work on both mobile and web):
| Action | Example Instruction |
| - | - |
| **Enter Text** | "Enter 'Joe Banana' in the name field" |
| **Clear** | "Clear the text from the email field" |
| **Verify** | "Verify that 'Welcome back!' appears on screen" |
| **Wait** | "Wait for the loading indicator to disappear" |
| **Drag and Drop** | "Drag the yellow card to the 'Drop here' area" |
| **Find on Page** | "Find 'Billing' on the page" *(jumps to the first match — Cmd/Ctrl+F on web, native scroll-into-view on mobile — much faster than swiping/scrolling)* |
| **Run Hook** | "Run `${hooks:Generate Test Order}`. Then search for the returned order number" |
| **Set Variable** | "Save the displayed order number as a variable called order\_id" |
| **Get Variable** | "Get the variable order\_id and enter it in the confirmation field" |
**Mobile-Only Actions**:
| Action | Example Instruction |
| - | - |
| **Tap** | "Tap the login button" |
| **Swipe** | "Swipe right on the slider" |
| **Pinch** | "Pinch to zoom in on the map until San Francisco takes up the entire screen" |
| **Long Press** | "Long press on the 'Share' button" |
| **Hide Keyboard** | "Hide the keyboard" |
| **Open Deeplink** | "Open the deeplink 'myapp\://profile/123'" |
| **Open App Clip** | "Open the App Clip at [https://example.com/e/abc123](https://example.com/e/abc123)" *(iOS cloud simulators only — see [Testing App Clips](/guides-app-clips))* |
| **Change Location** | "Change location to longitude -122.4194, latitude 37.7749" *(suite-scoped: persists across all flows in the suite* |
| **Set Device Locale** | "Set the device locale to de\_DE, then verify German pricing is shown" *(changes the device's language & region setting; the app restarts to pick it up; persists until changed)* |
| **Set Orientation** | "Flip the device to landscape" or "Rotate back to portrait" |
| **Toggle Airplane Mode** | "Turn airplane mode on" (Android only) |
| **Set Network Condition** | "Simulate a slow 3G network, then restore full speed" — presets: full, offline, gprs, edge, slow\_3g, fast\_3g, lte (Android only) |
| **Run Appium Script** | `run_appium_script {"command": "mobile: enrollBiometric", "args": {"isEnabled": true}}` |
**Web-Only Actions**:
| Action | Example Instruction |
| - | - |
| **Click** | "Click the login button" |
| **Scroll** | "Scroll down until the settings option appears" |
| **Hover** | "Hover over the dropdown menu" |
| **Right Click** | "Right-click on the image" |
| **Double Click** | "Double-click to edit the text" |
| **Navigate** | "Navigate to [https://example.com/settings](https://example.com/settings)" |
| **Refresh** | "Refresh the page" |
| **Go Back** | "Go back to the previous page" |
| **Go Forward** | "Go forward to the next page" |
| **Hotkey** | "Press Control+c to copy the text" |
| **Run JS** | "Run JS: `document.querySelectorAll('script').length` and verify at least 3 scripts are loaded" |
| **Run Playwright Command** | `run_playwright_command {"target":"page","command":"emulate_media","args":{"color_scheme":"dark"}}` |
| **Read Console Logs** | "Read the browser console logs and verify there are no errors" |
| **Read Network Logs** | "Read the network logs and verify the POST to /api/login returned 200" |
| **Read Cookies** | "Read the browser cookies and verify a session cookie exists" |
| **Set Network Condition** | "Throttle the network to slow 3G, then restore full speed" — presets: full, offline, gprs, edge, slow\_3g, fast\_3g, lte (Chromium-family browsers) |
## Running Flows
### Running a Single Flow
1. Find your flow in the Flows table
2. Click the **Play** button (▶️)
3. Select your app
4. For a mobile app, choose a simulator or emulator model and OS version, or
select a real device
5. Click **Run Flow**
Virtual-device runs default to the latest supported permutation. **Fast launch**
choices usually start sooner; every listed permutation is supported.
To test across two devices, edit the flow, open **Advanced**, and set **Number
of devices** to **2**. Choose both devices when you run it. See
[Multi-Device Testing](/multi-device-testing) for details.
### Running Multiple Flows
Select multiple flows using checkboxes:
1. Check the boxes next to flows you want to run
2. Click **Run Selected** in the floating action bar
3. Select your app
4. Click **Run Flows**
A bulk run can mix one-device and two-device flows. If any selected flow uses
two devices, choose **Device 1** and **Device 2**; one-device flows run on
Device 1.
**Shift-Click Selection**: Hold Shift and click to select a range of flows
quickly
### Auto-Retries
Autosana can automatically re-run a failed flow before reporting it as failed. Configure this in **[Settings → Agent](https://autosana.ai/settings?tab=agent)**.
* **Retries**: Set how many times a failed run is retried (0–5, default 1). A flow is only marked failed after all attempts are exhausted — if any attempt passes, the flow is reported as passed.
* **Intelligent Retries**: On by default. Retries are focused on flows with a track record of passing; flows that are consistently failing aren't retried. Turn it off to retry every failure.
## Managing Flows
### Editing a Flow
1. Click the pencil icon (✏️) next to a flow or click on the flow row
2. Modify the name or instructions
3. Click **Save Changes**
### Viewing Version History
Flows maintain a complete version history:
1. Click **Edit** on a flow
2. Click the **Version History** tab
3. View all previous versions with timestamps
4. Click **Revert** to restore a previous version
### Duplicating a Flow
To create a copy of an existing flow:
1. Click the three dots (**⋯**) next to a flow
2. Select **Duplicate**
3. A copy is created with "(Copy)" appended to the name
4. Edit the duplicate as needed
### Deleting a Flow
1. Click the three dots (**⋯**) next to a flow
2. Select **Delete**
3. Confirm deletion
## Organizing Flows with Suites
Suites help you organize related flows and run them together.
### Adding a Flow to Suites
**Method 1: During Creation**
* When creating a flow, select which suites it should belong to
**Method 2: After Creation**
1. Click the three dots (**⋯**) next to a flow
2. Select **Attach to Suite**
3. Check the suites you want to add the flow to
4. Click **Save**
**Method 3: From Suite View**
1. Expand a suite
2. Click **Add Existing Flows**
3. Select flows to add
4. Click **Add Flows**
### Removing a Flow from a Suite
1. Expand the suite containing the flow
2. Click the three dots (**⋯**) next to the flow
3. Select **Remove from Suite**
4. Confirm removal
## Flow Status and Results
### Status Types
| Status | Meaning |
| - | - |
| **Pending** | Flow hasn't been run yet |
| **Running** | Flow is currently executing |
| **Passed** | Flow completed successfully |
| **Failed** | Flow encountered an error |
### Viewing Results
Cache attempts that stop before sending an action to the device are hidden from the normal step list. They remain available in debug mode. Actual execution failures remain visible.
Click on any status badge to view detailed results:
* **Screenshots**: Visual snapshots at each step
* **Action Log**: Every action performed by the agent
* **Console and Hook Output**: Long output starts as a truncated preview. Select **Show more** to expand it, **Show less** to return to the preview, or the output heading to collapse or reopen it.
* **Errors**: Any issues encountered
* **Timeline**: Duration of each step
* **Device Info**: Device type and OS version used
### Sharing a Run Batch
On the **Runs** page, select the link icon beside a batch's eye icon or inside its results to share the entire batch.
## Advanced Features
### Attaching Files
Add reusable files to your flow instructions:
1. Open a flow's Advanced options and add a file in the Files section.
2. Click the + beside a file to insert its `${file:filename.xyz}` reference in the instructions. You can also type the reference yourself.
3. Files marked **Used in this flow** are prepared when that flow runs. Dimmed **Available** files are saved for reuse but are not prepared for that flow. Open a file's three-dot menu to delete it from the shared library.
Sequential suite flows share a device session, so a file prepared by an earlier flow may still be present for a later flow. Reference the file in each flow that needs it so that flow prepares it explicitly. Parallel suite flows use separate sessions.
### Using Hooks
Hooks are reusable scripts that run before or after your flow:
**Setup Hooks**: Run before the flow (e.g., "Clear app data and restart")
**Teardown Hooks**: Run after the flow (e.g., "Logout and clear cache")
1. Click **Edit** on a flow
2. Click **Add Hook**
3. Select **Setup** or **Teardown**
4. Write the hook instructions
5. Save the flow
You can read more about hooks [here](/hooks).
### Run Caching
When enabled, flows that have previously passed will use cached action sequences for faster execution (5-8x speedup). Run caching is best-effort and enabled by default.
Cached actions cost 90% less than uncached actions (10% of the cost).
On a run’s Details tab, **Action breakdown** shows the recorded cache setting and action counts in the order **Main agent · State cache · Semantic cache**. Hover over or select the counts for labels; keyboard users can focus the counts and press Enter. Other action sources appear as an additional count. If the run did not record its setting, it shows **Not recorded**.
The **Cache rate** tile on [Overview](https://autosana.ai/overview) shows cached actions as a percentage of all actions. It follows the selected date range, app, and workspace scope. Partially cached runs contribute their cached and uncached actions separately. The percentage uses action counts, not billing; periods with no actions show an empty value.
**How it works:**
* After a flow passes successfully, the action sequence is cached
* Subsequent runs replay the cached actions with a smaller, faster model
* If a step cannot be replayed safely (e.g., because of a UI change or a setup-hook value Autosana cannot substitute), the agent handles that step live. Independent earlier steps can still use the cache.
* Replay can resume after live execution when Autosana can verify the required action completed and the app matches a later checkpoint; otherwise the remaining steps run live. Runs that include live steps receive a fresh review.
**Using hooks?** How a hook passes a value to the flow affects whether the flow can be cached. See [Hooks and Run Caching](/hooks#hooks-and-run-caching).
**Preview deployments:** Web runs can reuse cached navigation across deployments when `BASE_URL` or `API_BASE_URL` is supplied as a [build variable](/variables#build-variables), suite/flow variable, or setup-hook export. Setting only the registered web `url` or a variable in Settings → Environments does not opt into this origin reuse. For CI previews, pass the deployment origin as `BASE_URL` through the Action's `variables` input as well as setting `url`; see [Web Setup](/ci-cd-integration#web-setup).
The variable must be an HTTP(S) origin, such as `https://preview.example.com`, with an optional trailing `/` and no other path, query string, fragment, or credentials. For supported cached navigation whose destination matches the recorded `BASE_URL` or `API_BASE_URL` origin, Autosana replaces that leading origin with the current value of the same variable while preserving the path and query. Other destination origins are not rewritten by this mechanism. These named origins qualify without the changing-value learning period required for custom hook exports. Ambiguous values and embedded redirect URLs still require live execution.
Reuse stays within the same app and environment and remains subject to UI checkpoint checks. Keep production and preview targets in separate Autosana environments. Other changing variables still follow their existing caching rules.
Manage it in **[Settings → Agent Features](https://autosana.ai/settings)**.
### Run caching modes
Workspace owners and admins choose **Off**, **Standard**, or **Semantic cache** under **Settings → Agent → Run caching**.
* **Off** runs every step with the live agent.
* **Standard** reuses successful steps when the screen closely matches. Workspaces already using run caching start in this mode.
* **Semantic cache** also finds equivalent controls when the screen changes. It is experimental and requires **UI/UX Agent** reviews.
Selecting Standard or Semantic cache also enables at least one auto-retry if retries were off. If reviews are later disabled, Semantic cache is paused. You can still select Standard or Off.
For dashboard-authored flows, use **Advanced options → Run caching**. New flows use **Use workspace setting**. Choose **Off** to disable caching for a flow, or **Standard** to keep it on Standard even when the workspace uses Semantic cache. A flow cannot exceed its workspace’s mode: choosing Semantic cache in a Standard workspace still uses Standard. Code-managed flow controls remain read-only; set `caching_mode` in YAML instead.
Semantic cache mode tries Standard caching first. When it cannot reuse a step, it compares a successful, reviewed recording with the current screen and finds the control that does the same thing. If the match is unclear, blocked, or missing, the live agent handles the action.
The initial release supports eligible clicks, taps, and text entry on web and iOS. Android, unlabeled image targets, retries, and multi-device runs use the normal agent and Standard cache where supported. Some actions still need the live agent, so Semantic cache does not guarantee a faster run.
Hooks execute again with current inputs and fresh results. Caching never reuses an old hook result. Variable extraction also stays live. Runs using Semantic cache get a fresh UI/UX Agent review, including runs that only record actions for future use. They cannot inherit an earlier pass.
Run details show the recorded **Cache setting** and an **Action breakdown** of Main agent, State cache, and Semantic cache actions. These counts describe the recorded action sources independently of the selected cache setting. Historical runs without a saved setting show **Not recorded**.
## Troubleshooting
### Flow Fails Immediately
**Possible Causes:**
* App build is missing or corrupted
* Bundle ID mismatch
* App crashes on launch
**Solutions:**
* Check that your app has an active build
* Verify the bundle ID is correct
* Test the app build manually first
### Agent Can't Find Elements
**Possible Causes:**
* UI element description is ambiguous
* Element isn't visible on screen
* Element loads after a delay
**Solutions:**
* Disambiguate by purpose or nearby label rather than visual attributes. Prefer "Tap the 'Submit' button under the password field" over "Tap the blue submit button in the bottom right" — describing pixel position or color locks the test to today's UI.
* Add a wait when the element loads asynchronously: "Wait for the order summary to appear, then continue."
## Next Steps
* [Organize flows with Suites →](/suites)
* [Automate flows with Schedules →](/automations)
* [Integrate with our CI/CD →](/ci-cd-integration)
* [Manage flows in your repo →](/code-managed-flows)
# GitHub Bot
Source: https://docs.autosana.ai/github-integration
Analyze PRs, run relevant flows, and get results with videos on your pull requests
Connect Autosana to your repositories. When you open a PR, Autosana analyzes the diff, selects and runs relevant flows, and posts results with videos as a PR comment.
## How it works
1. You open a PR
2. Autosana analyzes the diff and selects relevant flows (editing or creating as needed)
3. Flows run in parallel as your build completes
4. Results are posted as a PR comment with videos and a GitHub Check
## Setup
1. Go to **[Settings > Integrations > GitHub](https://autosana.ai/settings?tab=integrations\&integration=github)** and click **Install GitHub App**
2. Select your GitHub account/organization and choose repositories
3. Toggle **E2E Autopilot** per repo to automatically analyze and test PRs
Each repo also has a **Code Review** toggle for automatic AI code review on PRs — see [Code Review](/code-review). The two toggles are independent; enable both to get a review and test results on the same PR.
### If you aren't an owner of your GitHub organization
Click **Install GitHub App** anyway. GitHub sends your request to an owner of the organization. When the owner approves it, GitHub brings them to Autosana. They sign in with GitHub, see which workspace asked and who asked, and click **Confirm**. They don't need an Autosana account. Your repositories then appear in Settings.
If an owner installed the app straight from GitHub, Autosana gives them a link to send you. Open it, and **Settings > Integrations > GitHub** shows **Finish connecting GitHub**. Click **Link**. If you aren't an owner, Autosana asks one to confirm: click **Copy owner link** and send it to them. Each new copy turns off the previous link.
Settings shows where a connection stands and what to do next:
| Settings shows | What to do |
| - | - |
| Waiting for an owner of your GitHub org to approve | Wait for the owner, or click **Cancel** |
| An owner of *your org* needs to confirm | Click **Copy owner link** and send it to an owner, or click **Cancel** |
| No response from your GitHub owner yet | Click **Retry** |
| The app was removed from *your org* on GitHub | Click **Retry** |
| Finish connecting GitHub | Click **Link** |
You need to be an admin of your Autosana organization to install the GitHub App, link it, retry, copy an owner link, or cancel a request.
Set up the [GitHub Action for App Uploads](/ci-cd-integration) so Autosana can access builds for each PR.
## Code-Managed Flows
Instead of authoring flows in the dashboard, you can define flows, suites, and hooks as files in your repository's `.autosana/` folder. Connect the repo in **[Settings > Integrations > GitHub](https://autosana.ai/settings?tab=integrations\&integration=github)**; Autosana automatically syncs `.autosana/` on every push to your default branch and previews changes on pull requests. Parse errors surface as line annotations on the **Autosana - Code-Managed Flows** GitHub check, and the resulting flows are read-only in the dashboard. See [Code-Managed Flows](/code-managed-flows).
## @autosana
Tag `@autosana` in a PR comment on any connected repository — no toggle needed. Ask questions, request specific tests, re-run failures, or adjust instructions. The agent responds as a PR comment.
## Managing Repositories
Go to **[Settings > Integrations > GitHub](https://autosana.ai/settings?tab=integrations\&integration=github)** and click **Manage Installation**. It opens the app's settings on GitHub, where you can add or remove repositories.
# GitLab CI
Source: https://docs.autosana.ai/gitlab-ci
Sync code-managed flows from a GitLab repository with a CI job
Keep your [code-managed flows](/code-managed-flows) in a GitLab repository, self-hosted or on gitlab.com. A CI job sends the repository's `.autosana/` folder to Autosana. Autosana never connects to your GitLab server.
* **On the default branch**, the job updates your tests in Autosana, as a push to the default branch does on GitHub.
* **In a merge request**, the job checks the files and reports errors. Nothing is saved.
When a file is invalid, the job fails and prints each error with its file and line.
## Set up
Create an API key with `autosana keys create` or in [Settings → API Keys](https://autosana.ai/settings?tab=api-keys). In your GitLab project, open **Settings → CI/CD → Variables** and add `AUTOSANA_API_KEY` with the key as its value.
Mark it **Masked**. Leave **Protect variable** unchecked: GitLab withholds protected variables from merge request pipelines on unprotected branches, so those jobs would fail.
Add this job to `.gitlab-ci.yml`. It needs the `autosana` CLI 0.9.19 or later, which `pip install autosana` installs.
```yaml .gitlab-ci.yml theme={null}
autosana-sync:
image: python:3.12-slim
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
script:
- pip install autosana
- autosana flows sync
```
This job runs in merge request pipelines. If your project's other jobs run only in branch pipelines, each push to a branch with an open merge request starts a second pipeline that holds only this job, and a "Pipelines must succeed" check can pass on it alone. To avoid that, set `workflow:rules` so your project uses merge request pipelines, as in GitLab's [switch between branch pipelines and merge request pipelines](https://docs.gitlab.com/ci/yaml/workflow/#switch-between-branch-pipelines-and-merge-request-pipelines) example.
The first default-branch run connects the project to your Autosana organization. Your tests appear on the **Flows** page, read-only.
Merge requests from forks run in the fork and don't receive your project's variables, so `autosana flows sync` stops with "AUTOSANA\_API\_KEY is not set". Fork changes are checked when they reach your default branch.
## Run a merge request's tests
`autosana flows sync` only checks merge request files. To run them, add a job that runs the merge request's `.autosana/` on Autosana's cloud devices:
```yaml .gitlab-ci.yml theme={null}
autosana-mr-tests:
image: python:3.12-slim
needs: ["autosana-sync"]
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
script:
- pip install autosana
- autosana flows run --cloud --all --branch "$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME"
```
GitLab checks out a commit rather than a branch, so pass `--branch` or `--app-build-id`. `--branch` picks the newest app build uploaded from that branch, and `--app-build-id` picks a build directly. The app comes from `.autosana/config.yaml` or from flags, as in [Cloud runs](/code-managed-cli#cloud-runs).
The job starts the runs and prints a link to each, then a batch id. It does not wait for results. To wait, poll the [Run Status API](/api-runs#run-status) with that batch id.
## Self-hosted GitLab
Add the job from [Set up](#set-up), then check the points below.
Autosana never connects to your GitLab instance. Your runner connects out to Autosana, and the CLI reads only GitLab CI's predefined `CI_*` variables, so the jobs above need GitLab 13.4 or later.
For a runner without Docker Hub or PyPI, the job pulls Python from your internal registry and installs from your PyPI mirror. The hosts are placeholders; use your own:
```yaml .gitlab-ci.yml theme={null}
autosana-sync:
image: registry.example.internal/python:3.12-slim
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
script:
- pip install --index-url https://pypi.example.internal/simple autosana
- autosana flows sync
```
* **Network access.** Runners need outbound HTTPS (port 443) to `backend.autosana.ai`.
* **Proxy.** Set `HTTPS_PROXY`, and `NO_PROXY` if needed, as a CI/CD variable or in the runner's environment. The CLI sends its requests through that proxy.
* **TLS inspection.** If your proxy re-signs HTTPS traffic with an internal certificate authority, set `SSL_CERT_FILE` to a PEM file that holds that CA, or `SSL_CERT_DIR` to a folder of hashed certificates. Either replaces the default trusted CAs, so include any others the job needs.
* **No access to Docker Hub or PyPI.** Use the job above, or an image with `autosana` already installed and no `pip install` line. The job still needs to reach Autosana.
* **Source links.** **Open in GitLab** links in the dashboard use your project's address from `CI_PROJECT_URL`, such as `https://gitlab.example.internal/group/project`. They open only for people who can reach your GitLab, for example on your network or VPN. `http://` addresses work too.
Everything in [Set up](#set-up) applies as on gitlab.com: a masked, unprotected `AUTOSANA_API_KEY`, merge request pipelines, the `workflow:rules` advice, and merge requests from forks.
## What the job reports
| Result | When | Job |
| - | - | - |
| `Synced N files from commit to Autosana.` | A default-branch upload was applied. | passes |
| `N files checked with no errors. Merge request uploads are checked, not saved.` | A merge request upload has no errors. | passes |
| `Autosana already synced a newer commit, so nothing changed.` | A later pipeline synced first. This is normal. If this commit's timestamp is wrong and it must win, re-run with `--force`. | passes |
| `Code-managed tests are turned off for this project in Autosana, so nothing changed.` | Code-managed tests are turned off for this project. Contact Autosana support to turn them back on. | passes |
| `Autosana rejected this upload with N errors. Your tests did not change.` | At least one file is invalid. Each error is printed above it as `path:line error message`. | fails |
One invalid file blocks the whole upload: nothing changes until every file is valid, as on [GitHub](/code-managed-sync#sync-status-&-validation).
### Flags
* `--force` applies this commit even when Autosana already synced a newer one. Use it only when this commit's timestamp is wrong. It works on the default branch only. In a merge request pipeline the CLI refuses it and sends nothing.
* `--allow-empty` sends a `.autosana/` folder that holds no `*.flow.yaml` files. **On the default branch, this removes every test the project synced.** Use it only to delete every test on purpose. Without the flag, the CLI refuses such a folder. A missing `.autosana/` folder is refused even with the flag.
* `--api-url`, or the `AUTOSANA_API_URL` variable, sets the Autosana server. Leave it unset unless Autosana gave you a different URL.
### Exit codes
| Code | Meaning | What to do |
| - | - | - |
| `0` | Synced, checked, already synced, or turned off. | Nothing. |
| `1` | Autosana rejected a file, refused the upload, or could not be reached. | Read the message, then see [Troubleshooting](#troubleshooting). |
| `2` | The CLI stopped before uploading. Nothing was sent. | Fix the setup the message names. See [The job exits with code 2](#the-job-exits-with-code-2). |
## Troubleshooting
### The job exits with code 2
The CLI stopped before sending anything. The message names the fix:
* `runs only inside a GitLab CI job`: run it as a GitLab CI job. To check files locally, run `autosana flows validate`.
* `GitLab CI did not set …`: run the job in a default-branch pipeline or a merge request pipeline. If the message adds "Tag pipelines cannot sync.", remove the job from tag pipelines.
* `AUTOSANA_API_KEY is not set`: add the variable, masked and not protected.
* `The Autosana API key contains spaces or line breaks`: copy the key into the variable again, without them.
* `No .autosana/ folder`, or `holds no flow files`: commit `.autosana/` at the repository root, and check that the job checks out the repository.
* `holds symlinks`, or `.autosana is a symlink`: replace each symlink with the file or folder it points to.
* `Could not read …`, or `File is not valid UTF-8 text`: the job cannot read a file in `.autosana/`. Fix or remove the file the message names.
* `--force applies only to default-branch pipelines.`: remove `--force` from merge request jobs.
* `--api-url must be an http or https URL`: fix `--api-url` or `AUTOSANA_API_URL`.
### Autosana refused the upload (HTTP …)
The job exits with code 1 and prints Autosana's reason.
| HTTP | Meaning | What to do |
| - | - | - |
| 400 | The pipeline is neither on the default branch nor a merge request, a GitLab variable is malformed, or the commit timestamp is more than 5 minutes in the future. | Read the printed reason. Run the job only in the pipelines above. |
| 401, 403 | The API key is missing or invalid. | Check `AUTOSANA_API_KEY`. |
| 404 | The URL the CLI reached is not an Autosana server with GitLab sync. | Check `--api-url` or `AUTOSANA_API_URL`. Leave both unset to use Autosana's server. |
| 409 | Your plan is inactive, or your organization already connects a GitHub repository at the same path. | Renew your plan, or disconnect that GitHub repository. In a merge request, the path conflict is reported as a file error instead. |
| 413 | `.autosana/` holds more than 1,000 files or more than 4,000,000 bytes. | Remove files from `.autosana/`. |
| 5xx, or any other code | Autosana had a problem, or the URL is not Autosana's. | Re-run the job. If it fails again, check `--api-url` or `AUTOSANA_API_URL`. |
If the job says `Autosana answered with an unknown status`, update the CLI. The job above installs the newest version on each run, so re-run it.
### Autosana may have applied this upload
The connection broke after the upload was sent, so the job cannot tell whether it was applied. Re-run the job. Re-sending the same commit is safe.
The CLI already retries once, 5 seconds later, when Autosana is briefly unavailable or the upload never reached it. If it still fails with `Failed to reach …`, nothing was applied: check your runner's [network access](#self-hosted-gitlab) to Autosana.
## Limits
* `.autosana/` must be at the repository root. **Root directory** for monorepos is GitHub-only.
* GitLab-synced tests are read-only in the dashboard, and their source links open GitLab.
* Branch runs need GitHub. The dashboard hides branch choices for GitLab repositories, and `autosana run` without `--local` or `--cloud` reads from GitHub. For a GitLab repository, use `autosana flows run --cloud`.
* Results appear in the job log, not as merge request comments or commit statuses.
* There is no resync button. Re-run the default-branch pipeline instead.
## Next Steps
* [Learn the file schema →](/code-managed-files)
* [Run your tests from the terminal →](/code-managed-cli)
# Grok Bot
Source: https://docs.autosana.ai/grok-bot
Connect Grok Bot to Autosana through MCP
Connect Grok Bot so you can list flows, run them, debug runs, and search Autosana docs from a Grok Bot chat.
Grok Bot has no Settings form for a custom MCP URL. Autosana is not in the Grok Bot marketplace. You add it in chat as a custom MCP. That connector stays on your account — it is not a marketplace listing.
## Setup
1. Go to **[Settings → Integrations → Grok Bot](https://autosana.ai/settings?tab=integrations\&integration=grok-bot)** and click **Create key and copy**
2. In a **Grok Bot** chat, send:
```
Add a custom MCP called Autosana at https://mcp.autosana.ai/mcp with header x-api-key:
```
3. Approve the confirm card when Grok Bot asks to add the connector
4. If it still asks for the key, paste it. The header name is `x-api-key`
5. In the next message, `@` Autosana and ask it to list or run a flow — you can do everything from chat
The same connector is available to your other agents on this account. It may show up as `user-autosana`.
Do not look for an “add custom MCP” screen under Grok Bot Settings (General, Computer, Usage, Updates). **Plugins** is the marketplace for packaged apps, not a paste-URL form. Listing under Plugins → Yours can be flaky — if the chat says the server is connected, you are done.
This is **Grok Bot**, not Grok chat on grok.com and not Grok Build (`grok mcp add`). Adding Autosana in Cursor (`mcp.json`) does not install it in Grok Bot.
See [MCP Server](/mcp-setup) for the tools Grok Bot can call after you connect.
# Testing App Clips
Source: https://docs.autosana.ai/guides-app-clips
Open an iOS App Clip with a URL, the way a QR code or NFC tag would, and test it on cloud simulators.
An App Clip is a small part of your iOS app that people open from a QR code, an NFC tag, or a link, without installing the full app. Autosana opens your App Clip with a URL the same way a scan does, then tests it like any other app.
App Clips work on **iOS cloud simulators** only. Real devices, local runs from the CLI, and Android are not supported.
## Upload your build
Either upload works:
| Upload | What it is |
| - | - |
| Your full app | The simulator build of your app. Xcode puts the App Clip inside it, in `YourApp.app/AppClips/`. Autosana installs the App Clip the first time a test opens it. |
| The App Clip on its own | The App Clip's `.app` (its `Info.plist` contains `NSAppClip`), built for the simulator and zipped like any other `.app`. |
Build either one with the steps in [Building Your Mobile App for Our Cloud](/app-build-guide).
## Open the App Clip in a test
Write a step with the URL that your QR code or NFC tag contains:
```text theme={null}
Open the App Clip at https://example.com/e/abc123
Verify the page shows the event details
```
Before this step, the test starts in the app you uploaded, like any other test: your full app, or your App Clip without a URL. The App Clip then receives the URL as if someone scanned it. A test can open more than one URL, one step each.
## After the App Clip opens
* Closing and reopening the app act on the App Clip. If the App Clip was closed, reopening it opens it again with the most recent URL.
* Logs and performance data come from the App Clip.
* When the test ends, Autosana closes the App Clip. The next test starts in the app you uploaded.
* App Clip data stays between the tests of one suite run, like your app's data.
* In a [multi-device test](/multi-device-testing) on iOS simulators, each device opens its own App Clip.
You can't switch back to the full app in the middle of a test.
## Errors
| Message | What to do |
| - | - |
| App Clips are supported on iOS cloud simulators only | Run the test on an iOS simulator. |
| This build doesn't include an App Clip | Upload a build whose `.app` contains an `AppClips/` folder, or upload the App Clip itself. |
| This build includes more than one App Clip | An app can contain one App Clip. Check the build's `AppClips/` folder. |
| The app isn't installed on the simulator | Run the test again. If it happens again, contact support. |
| Couldn't open the App Clip: iOS doesn't know this App Clip | Check that the URL matches the App Clip and that the App Clip is in the uploaded build. |
| Couldn't open the App Clip: iOS blocked the request | Contact support. |
| Couldn't open the App Clip: the device was busy | Run the test again. |
| Couldn't open the App Clip: the device host was unreachable | Run the test again. |
| App Clip support is unavailable on this host | Contact support. |
| Couldn't open the App Clip: *any other reason* | iOS refused to open the App Clip. Check the URL and that the App Clip is built for the simulator. If the reason mentions the device host, run the test again. |
# Testing In-App Purchases
Source: https://docs.autosana.ai/guides-in-app-purchases
Test RevenueCat purchase flows on virtual devices, or contact us to plan native purchase testing on private devices.
Use RevenueCat Test Store to automate in-app purchase flows without an App Store or Google Play test account. Test purchases update `CustomerInfo`, grant entitlements, and appear as sandbox data in RevenueCat.
To test native Apple or Google purchase flows, see [Native in-app purchases on private devices](#native-in-app-purchases-on-private-devices). Contact Autosana before preparing a build or configuring devices for this workflow.
## RevenueCat
### How it works
RevenueCat Test Store replaces the native store purchase sheet with a test modal. Autosana can select a successful, failed, or cancelled outcome and then verify how your app responds.
Use this workflow to test:
* Paywall navigation and product selection
* Successful purchases and entitlement unlocks
* Failed and cancelled purchases
* Entitlement state after the app is reopened
### Configure RevenueCat Test Store
1. Check that your RevenueCat SDK supports Test Store in the [RevenueCat Test Store requirements](https://www.revenuecat.com/docs/test-and-launch/sandbox/test-store#sdk-version-requirements).
2. In RevenueCat, open **Apps and providers** and create or select a Test Store.
3. In **Product catalog**, create the products used by your test paywall.
4. Attach each product to a package in an offering. Make sure your app fetches that offering.
5. Initialize the RevenueCat SDK with the Test Store API key. Test Store keys begin with `test_`.
Never submit a build configured with a Test Store key to the App Store or Google Play. Use build configuration or environment variables to select the Test Store key only for dedicated test builds.
### Build a standalone test app
Create a development or debug build that:
* Uses the RevenueCat `test_` key
* Includes the app code and assets needed to run without a development server
* Does not depend on Metro, hot reload, or another process on your computer
If your framework's normal debug build connects to a local bundler, create a dedicated standalone testing configuration that remains a development/debug build while bundling the app locally.
See [Building Your Mobile App for Our Cloud](/app-build-guide) for supported artifact formats and platform-specific build requirements.
### Upload the build to Autosana
1. Open **[Apps](https://autosana.ai/apps)** in Autosana.
2. Create or select your iOS or Android app.
3. Upload the standalone test artifact and make it the active build: a zipped `.app` for an iOS Simulator, or an `.apk` or `.aab` for Android.
4. Run the purchase flows on a compatible virtual device.
### Create purchase flows
Create separate flows for outcomes that need a clean starting state. Replace the bracketed text with the screens, plans, and premium state used by your app.
```text theme={null}
Open the app and navigate to [the subscription paywall].
Choose [the monthly plan].
In the RevenueCat Test Store purchase dialog, choose the successful purchase option.
Verify that the purchase completes.
Verify that [the premium screen or feature] is unlocked.
```
```text theme={null}
Open the app and navigate to [the subscription paywall].
Choose [the monthly plan].
In the RevenueCat Test Store purchase dialog, choose the failed purchase option.
Verify that the app shows an error and remains usable.
Verify that [the premium screen or feature] is still locked.
```
```text theme={null}
Open the app and navigate to [the subscription paywall].
Choose [the monthly plan].
Cancel the RevenueCat Test Store purchase dialog.
Verify that the app returns to the paywall and remains usable.
Verify that [the premium screen or feature] is still locked.
```
```text theme={null}
Complete a successful purchase for [the monthly plan].
Close and reopen the app.
Verify that [the premium screen or feature] remains unlocked.
```
### Verify the result in RevenueCat
After a run, open the customer in RevenueCat and enable sandbox data. Confirm that:
* The purchase is associated with the expected App User ID
* A successful purchase activated the expected entitlement
* A failed or cancelled purchase did not activate the entitlement
If you use RevenueCat webhooks or integrations, you can also confirm that the matching sandbox event was delivered.
### Troubleshooting
| Problem | What to check |
| - | - |
| Offerings or packages are empty | Confirm that each Test Store product is attached to a package in the offering fetched by the app. |
| The app exits during RevenueCat configuration | Confirm that the `test_` key is used by a development/debug test build, not a production build. |
| A native store sheet appears or the app reports `BILLING_UNAVAILABLE` | Confirm that this build initializes RevenueCat with the Test Store key instead of an Apple or Google key. |
| The app unlocks content but no purchase appears in RevenueCat | Enable sandbox data and confirm the App User ID used by the test. |
| A test starts with an existing entitlement | Use a fresh test App User ID or reset the customer's Test Store state in RevenueCat before testing a first purchase. |
### Before you ship
Replace the `test_` key with the correct platform-specific RevenueCat key before releasing the app. RevenueCat also recommends completing final purchase validation through Apple or Google platform sandboxes; follow [RevenueCat's sandbox testing guide](https://www.revenuecat.com/docs/test-and-launch/sandbox) for those steps.
## Native in-app purchases on private devices
Native purchase testing exercises the Apple StoreKit or Google Play Billing purchase flow, including the store's purchase UI and sandbox transactions. This also applies to apps that use RevenueCat with its Apple or Google store integration instead of Test Store.
Native purchase testing on AWS Device Farm private devices requires an enterprise contract with Autosana and coordinated onboarding. [Contact us](mailto:founders@autosana.ai) first to confirm feasibility, device availability, commercial terms, and setup for your app. Selecting **Real device** in a standard run does not configure native purchase testing.
### What to share with us
* Your target platform and app bundle ID or package name
* Whether you use StoreKit, Google Play Billing, or RevenueCat
* The scenarios you want to test, such as a first purchase, cancellation, restore, or subscription expiry
* Any required device models or OS versions
We will agree on the device and build configuration, test-account setup, and how purchase state will be prepared between runs. Use dedicated test accounts that your team controls rather than personal accounts. We will agree on sign-in and any secure credential-sharing process during onboarding; do not include passwords in your initial email.
### iOS build and sandbox setup
AWS Device Farm normally re-signs uploaded apps. Private devices allow us to preserve the app's original signing using **Skip app re-signing**, which matters for native Apple capabilities. During onboarding:
1. **We confirm the private devices and signing approach before you build.** If the provisioning profile requires registered devices, we provide the private devices' UDIDs for your Apple Developer account.
2. **You prepare the agreed device build.** For Apple's documented sandbox workflow, use a development-signed build. Register the supplied UDIDs where required, update the agreed provisioning profile, and export the signed `.ipa`. A Simulator `.app` is not suitable for physical devices.
3. **You configure Apple sandbox testing.** Prepare your in-app purchase products and Sandbox Apple Accounts in App Store Connect.
4. **We configure and validate the private-device session.** We coordinate Developer Mode and sandbox sign-in on the private device, preserve app signing, and validate a sandbox purchase before running your automated flows.
An Apple Developer Enterprise account is not a prerequisite. If you already use Enterprise signing, discuss it with us first: installing an Enterprise app and validating native sandbox purchases are separate checks. AWS also requires its support team to trust Enterprise apps on private devices running iOS 18 or later.
See [AWS's private-device signing requirements](https://docs.aws.amazon.com/devicefarm/latest/developerguide/skip-app-re-signing-on-private-devices.html) and [Apple's sandbox setup](https://developer.apple.com/help/app-store-connect/test-in-app-purchases/overview-of-testing-in-sandbox) for the underlying platform requirements.
### Android build and test-account setup
Contact us before preparing an Android build. During onboarding, we agree on the installation path, private-device configuration, and Google account used for purchases.
#### Prepare Google Play and the test account
1. **Confirm the app and product catalog.** Share the package name and the product IDs, subscription base plans, and offers your flows should exercise. Configure them in Play Console and confirm that they are available to the intended tester. If you use RevenueCat, connect the Google products to the appropriate offerings and entitlements, and use the Google store integration in this build instead of a `test_` key.
2. **Add the agreed Google account as a license tester.** Follow [Google's license-testing setup](https://support.google.com/googleplay/android-developer/answer/6062777). We coordinate sign-in on the private device with you.
3. **If distributing through Google Play, grant test-track access too.** Publish the build to an internal or closed testing track, add the tester, and share the opt-in link with us. Track membership and license-testing status are separate settings. See [Google's testing-track setup](https://support.google.com/googleplay/android-developer/answer/9845334).
Joining an internal or closed testing track does not make purchases free. The purchasing account must also be a license tester to use Google's test payment methods. We confirm the test-purchase UI during onboarding before automated purchase flows begin.
#### Choose the build and installation path
We confirm which path fits your app before you upload anything:
| Path | What you provide | What we coordinate |
| - | - | - |
| Google Play test track | The test release, package name, and tester opt-in link | Tester access and installation through Google Play on the agreed private device |
| Sideloaded test build | A standalone APK using the same package name as the app configured in Play Console | License-tester sign-in, installation, and a native billing check |
Google allows license testers to sideload matching-package builds, including debug-signed builds, without uploading every new version to Play. A separate `.debug` package name must correspond to its own configured Play app. See [Google's billing test guide](https://developer.android.com/google/play/billing/test).
For AWS private-device uploads, we coordinate **Skip app re-signing** to preserve your APK's signature. Tell us if the app depends on Play Integrity or certificate-bound services so we can validate those requirements with the selected installation path. See [AWS's signing requirements](https://docs.aws.amazon.com/devicefarm/latest/developerguide/skip-app-re-signing-on-private-devices.html).
#### Validate the first purchase and repeat runs
We verify the purchasing account and test payment method, then check the resulting entitlement with you. Share which success, decline, cancellation, and subscription-state scenarios matter for your app so we can define separate flows and their starting states.
Google test subscriptions run on accelerated schedules. We can discuss Play Billing Lab for repeat trial or subscription-state scenarios during onboarding; any Lab setup must be included in the agreed workflow. See [Google's billing test guide](https://developer.android.com/google/play/billing/test).
Before automating repeated purchases, we agree on how your team will prepare account state and verify backend purchase processing. The flow's assertions should cover both access being granted after a purchase and access changing when the scenario calls for it.
### Keeping purchase tests repeatable
Purchases change the test account's state. Reinstalling the app alone does not establish that an account is eligible for a first purchase or free trial again. During onboarding, we agree on the starting state for each flow and the account reset or rotation process needed between runs.
For each successful purchase, verify both the app's unlocked state and the corresponding sandbox transaction or entitlement in your billing backend or RevenueCat.
# Testing One-time Passwords (OTP)
Source: https://docs.autosana.ai/guides-otp-testing
Let the agent receive and enter a real email or SMS login code using Autosana built-in hooks.
Many apps gate sign-in behind a one-time password — an email or SMS code you have to receive and type back in. Autosana ships **built-in hooks** that handle this for you: each run leases a real inbox (or phone number), the agent enters that address during the flow, the incoming code is captured, and the agent reads it and types it in — no shared test account, no hard-coded code.
## How it works
A built-in lease hook executes as a **flow setup hook**, before the agent starts. You can attach it directly to one flow or add it once to a suite's setup hooks so every member flow inherits it. It:
1. Leases an address for that run alone and exposes it to the flow as an environment variable.
2. Receives the provider's email (or SMS) at that address and stores the code.
3. Lets the agent read the code back and enter it to complete sign-in.
No two runs ever hold the same address at the same time, and a run only ever reads codes that arrive while it holds one — so parallel runs can't read each other's codes. The two hooks get there differently: email addresses are minted fresh for each run and never reused, while phone numbers come from a shared pool and go back when the run ends.
## Built-in hooks
In the dashboard, add these from the **Hook Library**: on the **Hooks** page, click **Add from Library**. [Code-managed flows and the MCP](#code-managed-flows-and-the-mcp) can use them without adding them first.
| Hook | Slug | Provides | Status |
| - | - | - | - |
| **Lease email address** | `lease-email-address` | `AUTOSANA_EMAIL_ADDRESS` | Available |
| **Lease phone number** | `lease-phone-number` | `AUTOSANA_PHONE_NUMBER`, `AUTOSANA_PHONE_NUMBER_NATIONAL` | Available |
## Email OTP
### 1. Add the hook to your organization
On the **Hooks** page, click **Add from Library**, then **Add** on **Lease email address**. This adds it once for your organization; you can then attach it to any flow or suite. Code-managed flows and the MCP add it for you, so you can skip this step there.
### 2. Attach it as a setup hook
For one flow, open the flow, expand **Advanced → Setup Hooks**, and add **Lease email address**. To use email OTP across a suite, edit the suite and add it under **Advanced → Setup Hooks** instead. Each member flow receives its own inbox, including when the suite runs members in parallel.
Email OTP always executes for one flow run. A suite attachment is an inherited default, not one inbox shared by the suite. If the suite has an auth setup flow, that flow receives its own inbox before its instructions run. The hook isn't offered in teardown slots.
### 3. Reference the address in your instructions
Once attached, the hook exposes `AUTOSANA_EMAIL_ADDRESS` for the run. Reference it in your flow steps like any other variable:
```text theme={null}
1. Tap "Continue with email"
2. Sign in with email: $AUTOSANA_EMAIL_ADDRESS
3. Check the inbox for a one-time code
4. Enter the code
Test passes if you reach the app home screen.
```
The agent enters the leased address, waits for the code to arrive at that inbox, reads it, and types it in. You'll see the setup hook and its output at the top of the run's **Agent Actions**:
```text theme={null}
Setup Hooks
Lease email address — Leased email address @… for this run (AUTOSANA_EMAIL_ADDRESS).
```
Keep the sign-in steps in the app's own vocabulary ("Continue with email", "Enter the code") and let the agent handle the rest — you don't need to script polling the inbox or reading the code.
## SMS OTP
**Lease phone number** works the same way for phone-based 2FA. Add it from the Hook Library, then attach it to a flow or suite under **Advanced → Setup Hooks**. A suite attachment leases a separate number for each member flow, so parallel members never share a number. It isn't offered in teardown slots.
It gives you the number in two forms, because sign-in screens ask for it differently:
| Variable | Example | Use it when |
| - | - | - |
| `AUTOSANA_PHONE_NUMBER` | `+17405550123` | The field expects a full international number |
| `AUTOSANA_PHONE_NUMBER_NATIONAL` | `7405550123` | The country code is a separate picker |
Write the steps the way the screen reads:
```text theme={null}
1. Tap "Continue with phone"
2. Enter phone number: $AUTOSANA_PHONE_NUMBER_NATIONAL
3. Tap "Send code"
4. Enter the code that arrives by SMS
5. Tap "Verify"
```
The number is held for the whole run and released when it ends, so a later run never reads your code.
Test numbers are drawn from a shared pool, so runs that need one can queue behind each other at high concurrency. If a run reports that no number was free, it waited and then gave up — rerun it, or get in touch and we'll grow the pool.
## Code-managed flows and the MCP
You don't need the Hook Library to use these hooks from code. In a [code-managed](/code-managed-files#referencing-hooks) flow or suite, list the hook's slug in `setup_hooks`:
```yaml theme={null}
name: Sign in with an email code
setup_hooks:
- lease-email-address
instructions:
- Sign in with email $AUTOSANA_EMAIL_ADDRESS
- Enter the one-time code sent to that inbox
- Verify the app home screen appears
```
If your organization hasn't added the hook yet, the sync adds it.
With the [MCP](/mcp-setup), include the hook in the `hooks` list of `flows_create`, `flows_update`, `suites_create`, or `suites_update`:
```json theme={null}
{"type": "builtin", "builtin_key": "lease_email_inbox", "hook_type": "setup"}
```
Use `lease_phone_number` for **Lease phone number**. The MCP adds the hook to your organization if it isn't there yet, then attaches it.
## Reusing the code
When the agent reads a code from the leased inbox or number, it saves it as `AUTOSANA_OTP_CODE`. Scripts that run later in the run, including teardown hooks, receive it like any other variable. So do later flows in the same suite. If the next message has no code, the variable is cleared.
The code doesn't exist until it arrives, so don't write `${env:AUTOSANA_OTP_CODE}` in the same flow's instructions. The run fails before it starts because the variable is missing.
## Rolling your own
If you'd rather return a magic link or OTP from your own server instead of leasing an inbox, you can do that with a script hook — see [Hooks](/hooks) for returning a code via a cURL request.
# Making Suites, Parallelizable, Isolated, and Repeatable
Source: https://docs.autosana.ai/guides-parallelizable-isolated-tests
Prevent accounts, test data, and sessions from conflicting when tests run concurrently.
Reusing the same credentials is fine for getting started, but it does not scale. Parallel sessions can overwrite each other's account data, and repeated runs can accumulate stale test data.
## Create a unique test account for each session
The recommended approach is:
1. Add an internal endpoint that creates a test account, deterministically seeds the required test data, and returns its credentials.
2. Call that endpoint from a [setup hook](/hooks) and [export the credentials](/hooks#sharing-data-between-hooks).
3. Use the exported credentials in the suite's Auth Instructions.
4. Delete the account and its data in a teardown hook.
When **Run flows in parallel** is enabled, Autosana runs the suite's hooks independently in each session. This gives every flow its own account and data.
If dynamic account creation is not available, use a pool of dedicated automation accounts and reserve a different account for each concurrent session.
## Keep dependent flows sequential
Keep flows sequential when a later flow intentionally uses state created by an earlier flow, such as creating an order and then verifying that same order.
Otherwise, give each flow independent setup and cleanup, then enable [Run flows in parallel](/suites#running-flows-in-parallel).
# Testing Push Notifications
Source: https://docs.autosana.ai/guides-push-notifications
Test real APNs and FCM delivery on Autosana virtual devices.
Autosana can test the complete push-notification journey on iOS Simulators and Android Emulators: your app registers for remote notifications, your backend or managed provider sends a real notification, and the flow verifies both delivery and deep-link behavior.
Use this guide for visible notifications such as alerts, reminders, messages, and order updates.
Push-notification testing on physical iOS and Android devices requires an
enterprise contract. [Contact us](mailto:founders@autosana.ai) for
private-device availability and setup.
## How a push test works
A typical test follows the same path as a real user:
1. Autosana installs and opens your test build.
2. The app requests notification permission and registers for remote notifications.
3. The app associates the resulting token or provider subscription with the test user.
4. The flow performs the action that schedules the notification, or runs a [runtime hook](/hooks#runtime-hooks) that calls your existing test endpoint.
5. The agent presses Home and waits for the notification.
6. The agent verifies the notification, taps it, and checks the destination screen.
Autosana does not need your APNs key or Firebase service-account credentials. Your backend or managed provider remains responsible for sending the notification. If a runtime hook triggers the push, store only the test endpoint URL and its API key in Autosana. Keep the APNs `.p8` file, Key ID, and Team ID—or the equivalent provider credentials—on the backend that sends the notification.
Depending on the integration, your backend may store a raw APNs device token, an FCM or Expo registration token, or a managed-provider subscription.
## Prepare your test environment
Before writing the flow:
1. Choose a test account that can safely receive notifications in your development or staging environment.
2. Register for remote notifications on every launch. After login, make the app send its latest device or registration token to your backend—or refresh its managed-provider subscription—even when the identifier has not changed.
3. Make sure your backend or provider can confirm that this installation registered and associated its current push identifier with the test account after the run began. For direct APNs, also record the expected bundle ID and environment.
4. Configure the build to use the same push provider and backend environment that the flow will exercise.
A dedicated account is not required for sequential tests; you can reuse an
existing automation account. If push tests can overlap, give each run or
worker a separate account or another unique targeting key. Reusing one account
can cause a notification to reach the wrong run when registrations overwrite
one another or multiple registrations remain associated with the same user.
Treat push registrations as temporary. Installing a new build, clearing app
data, or resetting a virtual device can change the token or provider
subscription. Refresh it during every run; do not copy one from an earlier run
into the flow.
An automation-friendly notifications screen makes setup failures visible. Keep
notification authorization and remote-registration status separate. When
authorization is undetermined, offer the system permission prompt. After the
user has made a choice, offer **Open Settings**, and re-check the
authorization and registration state whenever the app returns to the
foreground.
## Platform requirements
Autosana supplies an iOS 16 or later Simulator runtime. Your app's minimum deployment target can be lower than iOS 16.
Every layer below must agree:
| Layer | Required value |
| - | - |
| Artifact architecture | `arm64` iOS Simulator `.app` for Autosana's Apple-silicon worker fleet |
| App capability and signed entitlement | Push Notifications enabled and `aps-environment=development` |
| APNs gateway | `api.sandbox.push.apple.com` when sending directly to APNs |
| APNs credential | A Sandbox-enabled key or certificate, or an older APNs key that supports both Sandbox and Production |
| `apns-topic` | The exact `CFBundleIdentifier` of the installed Simulator app |
| Device registration | A token or provider subscription registered and associated by this installation after the run began |
The `arm64` architecture is an Autosana worker requirement, not an APNs protocol requirement. Apple also supports remote notifications in compatible Simulators running on Intel Macs with a T2 processor, but Autosana's fleet runs on Apple silicon.
Configure the Push Notifications capability in Xcode by selecting the project and app target, opening **Signing & Capabilities**, clicking **+ Capability**, and adding **Push Notifications**. With manual signing, also enable Push Notifications for the explicit App ID in **Certificates, Identifiers & Profiles**, then regenerate its development profile.
Xcode adds `aps-environment=development` to the app target's signed entitlements when you enable the capability and use development signing. It does not go in `Info.plist`. Do not disable code signing or strip the app's signature when producing this build.
In the app's existing notification integration, request permission for user-visible notifications and make sure the app or SDK registers with APNs. Do not add a native registration call when the SDK already manages this step. Use the integration's token or subscription callback to associate its current identifier with the signed-in test account on every launch or sign-in.
### Registration APIs by integration
#### [Native iOS](https://developer.apple.com/documentation/usernotifications/registering-your-app-with-apns)
Request authorization with `UNUserNotificationCenter`, call `registerForRemoteNotifications()`, and send the APNs token received by the app delegate to your backend. Swift and Objective-C use the corresponding UIKit APIs.
#### [React Native Firebase](https://rnfirebase.io/messaging/usage)
Request permission through `@react-native-firebase/messaging`, then associate the FCM token returned by `getToken()` with the test account and update it through `onTokenRefresh()`. The SDK registers with APNs automatically by default; call `registerDeviceForRemoteMessages()` only if your project disables automatic registration.
#### [Expo notifications](https://docs.expo.dev/versions/latest/sdk/notifications/)
Request permission with `requestPermissionsAsync()`, retrieve the native APNs token with `getDevicePushTokenAsync()`, and handle changes with `addPushTokenListener()`. Use `getExpoPushTokenAsync()` instead only when your backend sends through the Expo Push Service.
#### Managed-provider SDKs
Let the provider SDK manage APNs registration, then use its user-identity or subscription API after sign-in. For example, OneSignal associates its automatically created mobile subscription when the app calls `login()` with the test user's external ID. Do not also upload the raw APNs token unless the provider's integration requires it.
### Configure an Expo EAS build
EAS Build and the Expo Push Service are independent. EAS Build syncs the Apple capabilities declared by your project, but running `eas build` alone does not add the push entitlement.
Create a dedicated Autosana Simulator profile in `eas.json`:
```json theme={null}
{
"build": {
"autosana-simulator": {
"ios": {
"simulator": true
}
}
}
}
```
The `ios.simulator` setting controls the target artifact. You only need `developmentClient: true` if you also want Expo development-client tooling. The `aps-environment=development` entitlement below selects the APNs Sandbox environment for this artifact; it does not turn the app into an Expo development client.
If the app uses the `expo-notifications` client library, add its config plugin:
```json theme={null}
{
"expo": {
"plugins": ["expo-notifications"]
}
}
```
The plugin adds the development APNs entitlement even when your backend sends directly through APNs, FCM, or another provider instead of the Expo Push Service.
If the app does not use `expo-notifications`, declare the entitlement directly:
```json theme={null}
{
"expo": {
"ios": {
"entitlements": {
"aps-environment": "development"
}
}
}
}
```
These app-config options apply through Expo Prebuild and Continuous Native Generation. If the repository contains a checked-in native `ios/` directory, add **Push Notifications** to the app target in Xcode or run Prebuild and commit the generated native changes.
Create a new EAS build after changing an entitlement; an over-the-air update cannot change the signed app. See [Expo's direct APNs and FCM guide](https://docs.expo.dev/push-notifications/sending-notifications-custom/) for the complete configuration.
### Verify the Simulator artifact
Run this preflight before compressing the `.app`. Set `EXPECTED_BUNDLE_ID` to the `apns-topic` used by your backend:
```bash theme={null}
#!/usr/bin/env bash
set -euo pipefail
APP_PATH="/path/to/YourApp.app"
EXPECTED_BUNDLE_ID="com.example.app"
EXECUTABLE_NAME=$(/usr/libexec/PlistBuddy \
-c "Print :CFBundleExecutable" "$APP_PATH/Info.plist")
BUNDLE_ID=$(/usr/libexec/PlistBuddy \
-c "Print :CFBundleIdentifier" "$APP_PATH/Info.plist")
ARCHITECTURES=$(lipo -archs "$APP_PATH/$EXECUTABLE_NAME")
APNS_ENVIRONMENT=$(codesign -d --entitlements :- "$APP_PATH" 2>/dev/null \
| plutil -extract aps-environment raw -)
case " $ARCHITECTURES " in
*" arm64 "*) ;;
*) echo "Missing arm64 architecture: $ARCHITECTURES" >&2; exit 1 ;;
esac
test "$APNS_ENVIRONMENT" = "development"
test "$BUNDLE_ID" = "$EXPECTED_BUNDLE_ID"
printf "architectures=%s\naps-environment=%s\nbundle-id=%s\n" \
"$ARCHITECTURES" "$APNS_ENVIRONMENT" "$BUNDLE_ID"
```
A successful check prints an architecture list containing `arm64`, `aps-environment=development`, and the bundle ID your backend uses as `apns-topic`. A failed `codesign` or entitlement check means the uploaded build cannot register with APNs.
**iOS Simulator tokens only work with the APNs sandbox.** If your backend
connects to APNs directly, send these tokens to
`api.sandbox.push.apple.com`. Do not send them to the production endpoint,
`api.push.apple.com`; APNs will reject the token, commonly with a
`BadDeviceToken` response.
If you send through Firebase, Expo, OneSignal, or another push provider,
you normally do not select the APNs hostname yourself. Make sure that
provider is configured to deliver this development build through the APNs
sandbox.
### Configure your push provider
Your provider must preserve the Simulator token's development environment all the way to APNs. Use the matching instructions below; if your provider is not listed, look for its **development**, **sandbox**, or **APNs environment** setting.
#### [Direct APNs](https://developer.apple.com/documentation/usernotifications/sending-notification-requests-to-apns)
Send the Simulator token to `api.sandbox.push.apple.com` and use the app's Simulator bundle ID as the `apns-topic`. Authenticate with a [Sandbox-enabled APNs key](https://developer.apple.com/documentation/usernotifications/establishing-a-token-based-connection-to-apns) or certificate. Older team-scoped keys that support both Sandbox and Production continue to work, but a Production-only key fails against the Sandbox. Do not retry a `BadDeviceToken` response against the production endpoint.
#### [Firebase Cloud Messaging](https://firebase.google.com/docs/cloud-messaging/ios/get-started)
Continue sending the FCM registration token through the normal Firebase Admin SDK or FCM HTTP v1 endpoint. In the Firebase console, make sure the iOS app has a development APNs authentication key or certificate; FCM handles the connection to the APNs sandbox.
#### [Expo](https://docs.expo.dev/push-notifications/push-notifications-setup/)
When using an `ExpoPushToken`, send to the Expo Push Service normally. Install `expo-application` so `expo-notifications` can detect the iOS push environment, or pass `development: true` to `getExpoPushTokenAsync` for the Simulator build. Do not force `development` to `false`. If you retrieve a native APNs token and send it yourself, follow the Direct APNs instructions above.
#### [OneSignal](https://documentation.onesignal.com/docs/en/ios-sdk-setup)
Configure the OneSignal app for the APNs development environment. If you provision it through the OneSignal API, set `apns_env` to `development` because the API defaults this field to `production`; use a `.p8` key enabled for both Sandbox and Production.
#### [Customer.io](https://docs.customer.io/journeys/channels/push/developer-guide/)
In **Workspace Settings > Push > iOS**, enable **Send all push notifications to sandbox**. Customer.io recommends using a separate workspace for the sandbox environment, which also prevents Simulator registrations from mixing with production users.
#### [Braze](https://braze-inc.github.io/braze-swift-sdk/documentation/braze/apns-certificate/)
Use a separate Braze App or App Group for the development build and configure it with the development APNs credential. Braze allows only one active `.p12` certificate per app, so do not replace a live app's production certificate just to test a Simulator build.
#### [Airship](https://docs.airship.com/developer/sdk-integration/apple/installation/advanced-integration/)
Initialize the Simulator build with the Airship development/test app key and secret. Let the current Airship SDK infer the APNs environment from the build, or set `inProduction` to `false` when your integration selects it explicitly.
#### [AWS SNS](https://docs.aws.amazon.com/sns/latest/api/API_CreatePlatformApplication.html)
Create an SNS platform application whose platform is `APNS_SANDBOX`, register the Simulator token against that application's ARN, and send through the resulting endpoint. Do not register the token under an `APNS` production platform application.
#### [Azure Notification Hubs](https://learn.microsoft.com/en-us/azure/notification-hubs/configure-apple-push-notification-service)
Use a separate notification hub configured under **Apple (APNS)** in **Sandbox** mode. Do not switch a shared production hub to sandbox because registrations are tied to the APNs environment.
#### [Iterable](https://support.iterable.com/hc/en-us/articles/360035112332-Push-Notification-Testing-and-Troubleshooting)
Configure the Iterable app for `APNS_SANDBOX` and target the token generated by the Simulator build. A production registration or production push credential cannot be used with that token.
Each Simulator and host Mac combination receives its own token, and the token's length can vary. Store the token as an opaque string without assuming a fixed size or format. Register on every app launch and send the returned token or provider subscription to your backend even when its value matches the previous launch.
If you use Firebase Cloud Messaging on iOS, use Firebase Apple SDK 10.3.0 or later and configure an APNs authentication key in the matching Firebase project. If Firebase method swizzling is disabled, pass the APNs token to Firebase Messaging in your app code.
Upload an `.apk` or `.aab` configured for the Firebase project used by your test environment. The build must:
* Include Firebase Cloud Messaging and the correct Firebase configuration.
* Request `POST_NOTIFICATIONS` at runtime on Android 13 or later.
* Create the notification channel used by the test on Android 8 or later.
* Upload the newest FCM registration token to your test backend.
Unlike APNs, FCM does not have a separate sandbox endpoint. Send the Emulator's registration token through the normal Firebase Admin SDK, FCM HTTP v1 endpoint, or managed provider. Use a development or staging Firebase project when you need to keep test registrations and messages separate from production.
Autosana's Android Emulator image includes Google APIs and Google Play services. If your app enforces Play Integrity or Firebase App Check, configure the provider's supported test or debug path for the Autosana build.
See [Building Your Mobile App for Our Cloud](/app-build-guide) for artifact and architecture requirements.
## Create the flow
### Trigger the notification from the app
Use the normal application workflow whenever an in-app action schedules the notification. For example:
```text theme={null}
Log in as ${env:PUSH_TEST_EMAIL}.
Accept notification permission if the app asks for it.
Enable the order-ready reminder for the test order.
Verify that the app confirms the reminder is scheduled.
Press Home so the app is in the background.
Wait up to 30 seconds for an “Order ready” notification.
Verify that the notification appears.
Tap the notification.
Verify that the app opens the test order.
```
The scheduling action and delay belong to your application. Autosana only needs instructions that describe when to press Home, how long to wait, what notification to expect, and what should happen after it is tapped.
### Trigger the notification with a runtime hook
Use a runtime hook when the application has no convenient in-app trigger. The hook should call a narrow test endpoint that tells your backend to send a named notification to the dedicated test user.
1. Add these values to the app's Autosana environment:
* `PUSH_TEST_URL`
* `PUSH_TEST_API_KEY` as a secret
* `PUSH_TEST_USER_ID`
2. Create a cURL hook named `Trigger Test Push`:
```bash theme={null}
curl --fail-with-body -X POST "${env:PUSH_TEST_URL}" \
-H "Authorization: Bearer ${env:PUSH_TEST_API_KEY}" \
-H "Content-Type: application/json" \
-d '{"user_id":"${env:PUSH_TEST_USER_ID}","template":"order-ready"}'
```
3. Reference the hook at the correct point in the flow:
```text theme={null}
Log in as the push-notification test user.
Accept notification permission if the app asks for it.
Wait until the app confirms that this installation registered for notifications and associated the registration with this account.
Press Home so the app is in the background.
Run ${hooks:Trigger Test Push}.
Wait up to 30 seconds for an “Order ready” notification.
Verify that the notification appears.
Tap the notification.
Verify that the app opens the test order.
```
Runtime hooks run synchronously, so the agent waits for the endpoint response before continuing. A successful endpoint response confirms only that your backend accepted the request; the notification appearing on the virtual device is the end-to-end assertion.
Have the test endpoint return a distinct error when the test user has no
registration confirmed after the current run began. For direct APNs, return
the APNs status, `reason`, `apns-id`, gateway/environment, and `apns-topic`
while showing only a redacted device token. This makes registration,
authentication, routing, and delivery failures easier to distinguish.
## Test the important app states
Create separate flows when your application handles notifications differently in each state:
* **Background:** press Home before sending, then verify and tap the notification.
* **Foreground:** keep the app open and verify the app's foreground presentation or in-app handling.
* **Cold start:** terminate or fully close the app through your normal test workflow, send the notification, and verify the deep-linked destination after tapping it.
* **Notification service extension:** verify any modified title, body, attachment, or category produced by the extension.
Background or silent data delivery is controlled by the operating system and is not guaranteed to run immediately. Use bounded waits, avoid sending notifications repeatedly in a tight loop, and retain physical-device coverage for business-critical background processing.
## Testing notifications on real devices
For device selection, run behavior, and availability, see [Real Device Testing](/real-device-testing). For supported artifacts and standard physical-device build instructions, see [Real device builds](/app-build-guide#real-device-builds-ipa).
Physical push-notification testing uses private devices so the app's original signing and notification entitlements can be preserved. It requires an enterprise contract; [contact us](mailto:founders@autosana.ai) for availability and onboarding.
## Troubleshooting
### Direct APNs responses
APNs returns an HTTP status, an `apns-id` response header, and—for errors—a JSON `reason`. Preserve all three in your backend logs and test-endpoint response.
| Response | Meaning and next check |
| - | - |
| `200` | APNs accepted the request. This does not prove that the operating system delivered or presented the notification; the flow must still observe it. |
| `400 BadDeviceToken` | The token is invalid or does not match the Sandbox environment. Confirm that it came from the current installation and that the request uses `api.sandbox.push.apple.com`. |
| `400 BadTopic` or `400 DeviceTokenNotForTopic` | The `apns-topic` is invalid or does not match the token. Compare it with the installed app's `CFBundleIdentifier`. |
| `403 BadEnvironmentKeyIdInToken` | The APNs key does not support the Sandbox environment. Use a Sandbox-enabled key or an older key that supports both environments. |
| `403 InvalidProviderToken` or `403 ExpiredProviderToken` | Check the JWT signature, Key ID, Team ID, issued-at time, and server clock. Generate a current provider token when it has expired. |
| `410 Unregistered` or `410 ExpiredToken` | The token is no longer active for this topic. Stop sending to it until the app registers again and supplies its current token. |
See [Apple's APNs response reference](https://developer.apple.com/documentation/usernotifications/handling-notification-responses-from-apns) for the complete status and reason list.
### Common problems
| Problem | What to check |
| - | - |
| The permission prompt does not appear | The app may already have permission in the current session, or Android permission may have been granted during setup. Confirm the OS notification setting and run from a clean device state when testing first-run permission behavior. |
| The backend reports no target | Confirm the app logged in as the expected user and finished updating its push registration during this run before the trigger. |
| APNs rejects the request | Use the direct APNs response table above to separate token, topic, credential-environment, and provider-token failures. |
| FCM returns `UNREGISTERED` | The app was reinstalled or its data was cleared. Launch the app and upload the new token before sending again. |
| Android receives a message but displays nothing | Check notification permission, the channel ID and channel settings, and whether the payload is a notification message or a data message. |
| The iOS Simulator never registers | Check the Push Notifications capability, development entitlement, APNs credentials, and that the build targets iOS 16 or later. |
| The hook succeeds but no notification appears | Inspect the APNs or FCM response and confirm that the provider accepted the current token. The hook response and device delivery are separate stages. |
| A silent or background notification is inconsistent | The OS can delay or suppress background work. Reduce test frequency and use a private physical device for critical coverage. |
## Provider references
* [Apple: Xcode 14 Simulator remote-notification support](https://developer.apple.com/documentation/xcode-release-notes/xcode-14-release-notes)
* [Apple: Registering an app with APNs](https://developer.apple.com/documentation/usernotifications/registering-your-app-with-apns)
* [Firebase: Set up FCM for Android](https://firebase.google.com/docs/cloud-messaging/android/get-started)
* [Firebase Apple SDK release notes](https://firebase.google.com/support/release-notes/ios)
# Hermes Agent
Source: https://docs.autosana.ai/hermes-agent
Connect Hermes Agent to Autosana through MCP
Connect Hermes Agent so you can list flows, run them, debug runs, and search Autosana docs from a Hermes session.
Autosana is a remote HTTP MCP server. It is not in the Hermes catalog. Add it in `~/.hermes/config.yaml` with `auth: oauth`. The first time Hermes connects, it opens a browser so you can sign in. No API key goes in the file. Hermes stores the token under `~/.hermes/mcp-tokens/`.
`hermes mcp add --auth header` writes `Authorization: Bearer`. Do not use that. `--auth oauth` writes the same block as below.
## Setup
1. Open **[Settings → Integrations → Hermes Agent](https://autosana.ai/settings?tab=integrations\&integration=hermes-agent)** and copy the config
2. Open `~/.hermes/config.yaml`. If `mcp_servers` is already there, add only the `autosana` entry next to your other servers. If it is not, paste the whole block:
```yaml theme={null}
mcp_servers:
autosana:
url: "https://mcp.autosana.ai/mcp"
auth: oauth
```
Or from a terminal:
```bash theme={null}
hermes mcp add --url https://mcp.autosana.ai/mcp --auth oauth autosana
```
3. Start Hermes, or run `/reload-mcp` if it is already open. Sign in when the browser opens
4. Ask Hermes to list or run a flow
Do not use STDIO (`--command`, `npx`, or `mcp-remote`). Autosana is not a local process. Adding Autosana in Cursor (`mcp.json`) does not install it in Hermes Agent.
See [MCP Server](/mcp-setup) for the tools Hermes Agent can call after you connect.
# Hooks
Source: https://docs.autosana.ai/hooks
Set the state of the app before, during, and after your flows run via scripts, API calls, and app launch configurations
Hooks serve as a bridge between your backend or server-side scripts and the Autosana testing environment. They allow you to communicate data, set the state of the app, set the state of a user, etc.
They are short programs that run in sandboxed environments during the testing process. They can be added to suites and flows, and any output from them (print statements, console.log, echo) is visible to the agent.
A few common examples:
1. Resetting a user's onboarding in the backend via JS code
2. Generating a random email to be used by the Autosana agent via Python code
3. Returning a magic link or OTP code from your server for Autosana to use in the flow via a cURL request
4. Mocking audio or video input by hitting an endpoint on your server via TypeScript code
Possibilities are endless and usually the more hooks the better, as it adds a level of determinism to the state of the test!
**Execution Positions**
There are three hook execution positions:
1. **Setup Hooks**: Run before suites or flows start. Great for providing context.
2. **Runtime Hooks**: Run as an action during the flow — added directly into flow instructions, allows real-time backend interaction during testing
3. **Teardown Hooks**: Run at the very end of suites and flows. Helpful to clean up testing states
## Hook Types
Hooks allow you to configure your test environment in three ways:
1. **Scripts**: Python, JavaScript, TypeScript, or Bash scripts that run server-side
2. **cURL Requests**: Backend API calls for simple HTTP requests
3. **App Launch Configuration**: Configure how your mobile app launches (feature flags, environment settings) - *mobile only*
### Scripts
Server-side scripts that execute in a secure sandbox environment. Choose from Python, JavaScript, TypeScript, or Bash depending on your needs.
If your hook calls a firewalled API, see [Network allowlist](/network-allowlist) for the IPs to allow.
Scripts run in a sandboxed environment and **cannot use external libraries**. Use only built-in/standard libraries (e.g., `urllib` instead of `requests` for Python, built-in `fetch` for JavaScript/TypeScript).
Because hook sandboxes are created from snapshots, pseudo-random generators can repeat values across runs. Avoid `Math.random()` in JavaScript/TypeScript and unseeded `random` in Python when generating emails, usernames, IDs, or other unique test data.
**Common use cases:**
* Complex multi-step API operations
* Data processing and transformation
* Conditional logic based on API responses
* Generating dynamic test data
* Chaining multiple API calls with error handling
#### Generating Random Values
Use cryptographically secure randomness from the language standard library when generating unique test data.
**JavaScript / TypeScript:**
```javascript theme={null}
const crypto = require('crypto');
const suffix = crypto.randomInt(1, 100000001);
const email = `test-${suffix}@example.com`;
const uniqueId = crypto.randomUUID();
console.log(email);
```
**Python:**
```python theme={null}
import secrets
import string
import uuid
suffix = ''.join(secrets.choice(string.ascii_lowercase) for _ in range(8))
email = f"test-{suffix}@example.com"
unique_id = str(uuid.uuid4())
print(email)
```
If you must use Python's `random` module, seed it from OS entropy at the top of your script:
```python theme={null}
import os
import random
random.seed(os.urandom(32))
```
If your flow types, taps or checks a printed value that changes every run, each run executes live. Export the value instead so cached runs can use it. See [Hooks and Run Caching](#hooks-and-run-caching).
### cURL Requests
Backend API calls that execute before (setup) or after (teardown) a flow runs. Best for simple, single HTTP requests.
**Common use cases:**
* Create test user accounts via your API
* Reset database state between flows
* Generate auth tokens or session data
* Configure backend feature flags
* Clean up test data after flows complete
### App Launch Configuration (Mobile Only)
Configure values that are passed to your mobile app when it launches. These values are available to your app code as environment variables (iOS) or intent extras (Android).
App Launch Configuration is only available for mobile apps (iOS and Android). For web testing, use Scripts or cURL Requests to configure your test environment.
**Common use cases:**
* Override feature flag values for specific test scenarios
* Set test environment parameters (staging vs. production mode)
* Configure API timeouts or retry behavior
* Enable debug modes or verbose logging
* Set experiment variants for A/B testing
## Creating a Hook
1. Navigate to **[Hooks](https://autosana.ai/hooks)** in the sidebar
2. Click **Create Hook**
3. Enter a hook name (e.g., "Create Test User" or "Generate Auth Token")
4. Select hook type:
* **Python**: For complex logic and data processing
* **JavaScript**: For Node.js-based operations
* **TypeScript**: For type-safe Node.js operations
* **Bash**: For shell script operations
* **cURL Request**: For simple HTTP API calls
* **App Launch Configuration**: For mobile app settings (mobile only)
5. Enter your script or configuration:
### For Python:
```python theme={null}
import urllib.request
import json
import os
data = json.dumps({
"email": os.environ.get('TEST_EMAIL'),
"password": os.environ.get('TEST_PASSWORD')
}).encode('utf-8')
req = urllib.request.Request(
f"{os.environ.get('API_URL')}/auth/login",
data=data,
headers={'Content-Type': 'application/json'}
)
with urllib.request.urlopen(req) as response:
result = json.loads(response.read().decode('utf-8'))
print(f"Logged in as user: {result['userId']}")
# Export token for subsequent hooks
with open('/tmp/autosana.env', 'w') as f:
f.write(f"AUTH_TOKEN={result['token']}\n")
```
### For JavaScript:
```javascript theme={null}
const response = await fetch(`${process.env.API_URL}/auth/login`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
email: process.env.TEST_EMAIL,
password: process.env.TEST_PASSWORD
})
});
const data = await response.json();
console.log(`Logged in as user: ${data.userId}`);
// Export token for subsequent hooks
const fs = require('fs');
fs.writeFileSync('/tmp/autosana.env', `AUTH_TOKEN=${data.token}\n`);
```
### For TypeScript:
```typescript theme={null}
const response = await fetch(`${process.env.API_URL}/auth/login`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
email: process.env.TEST_EMAIL,
password: process.env.TEST_PASSWORD
})
});
interface LoginResponse {
token: string;
userId: string;
}
const data: LoginResponse = await response.json();
console.log(`Logged in as user: ${data.userId}`);
// Export token for subsequent hooks
const fs = require('fs');
fs.writeFileSync('/tmp/autosana.env', `AUTH_TOKEN=${data.token}\n`);
```
### For Bash:
```bash theme={null}
RESPONSE=$(curl -s -X POST "${API_URL}/auth/login" \
-H "Content-Type: application/json" \
-d "{\"email\":\"${TEST_EMAIL}\",\"password\":\"${TEST_PASSWORD}\"}")
TOKEN=$(echo $RESPONSE | jq -r '.token')
USER_ID=$(echo $RESPONSE | jq -r '.userId')
echo "Logged in as user: $USER_ID"
# Export token for subsequent hooks (append with >>)
echo "AUTH_TOKEN=$TOKEN" >> /tmp/autosana.env
```
### For cURL Requests:
```bash theme={null}
curl -X POST https://api.example.com/users \
-H "Content-Type: application/json" \
-d '{"email":"test@example.com","password":"TestPass123"}'
```
### For App Launch Configuration:
```json theme={null}
{
"testEnvironment": "staging",
"featureFlags": {
"newCheckoutFlow": "enabled",
"darkMode": true
},
"apiTimeout": 30
}
```
6. Click **Create Hook**
Any output from your hooks (print statements, console.log, echo) is passed to the agent as additional context. Use this to provide helpful information about what the hook did!
## Testing Hooks
Before attaching hooks to flows, you can test them to verify they work correctly.
Dashboard tests use your signed-in session. You do not need to create or paste an Autosana API key.
1. Open the hook you want to test
2. Click the **Test Hook** button
3. The hook will execute with a **1-minute timeout**
4. View the output to verify success or debug issues
**What to check:**
* Script executes without errors
* API calls return expected responses
* Environment variables are being read correctly
* Exported values (if any) are formatted correctly
Always test hooks independently before attaching them to flows. This helps isolate issues and speeds up debugging.
## Using Hooks
### Setup & Teardown Hooks
**Attaching Setup & Teardown Hooks**
1. Create a new suite or flow or click **Edit** on an existing one
2. Expand the **Advanced** section
3. Expand the **Setup Hooks** or **Teardown Hooks** section
4. Click the **Add a setup hook...** or **Add a teardown hook...** button
5. Select the hook you want to attach
Hooks attached to a suite are also shown read-only when you expand the suite on the Flows page, so you can see what runs before and after the suite without opening the editor.
### Runtime Hooks
You can execute hooks as actions during flow execution by using the syntax `${hooks:Hook Name}` in the flow instructions.
**Example:**
```
Tap the login button, then ${hooks:Create Test User}, then verify the user appears
```
When the agent encounters `${hooks:Hook Name}`, it will:
1. Look up the hook by name
2. Execute the hook script
3. Continue with the rest of the action
This is useful for:
* Creating data mid-flow (e.g., "add 5 items to cart, then `${hooks:Create Dummy Discount Code}`")
* Triggering backend events at specific points
* Resetting state between actions
**Runtime Hooks** execute synchronously - the flow waits for the hook to complete before continuing.
### Execution Order
1. Setup Hooks execute (if configured)
2. Flow runs on app start
3. Agent runs actions and any Runtime Hooks (if configured)
4. Flow run stops
5. Teardown Hooks execute (if configured)
Steps 1 and 2 overlap when a setup hook is an [App Launch Configuration](#when-it-runs) — the app starts at that hook's position.
## Environment Variables in Hooks
### For cURL Requests
Reference environment variables using `${env:VARIABLE_NAME}` syntax:
```bash theme={null}
curl -X POST ${env:API_URL}/auth/login \
-H "Content-Type: application/json" \
-d '{"email":"${env:TEST_EMAIL}","password":"${env:TEST_PASSWORD}"}'
```
**Environment variables (in Settings → Environments):**
* `API_URL` = `https://staging-api.example.com`
* `TEST_EMAIL` = `test@staging.com`
* `TEST_PASSWORD` = `SecurePass123`
**Executed command:**
```bash theme={null}
curl -X POST https://staging-api.example.com/auth/login \
-H "Content-Type: application/json" \
-d '{"email":"test@staging.com","password":"SecurePass123"}'
```
### For Scripts (Python, JavaScript, TypeScript, Bash)
Environment variables are automatically injected into the script environment. Access them using your language's standard method:
**Python:**
```python theme={null}
import os
api_url = os.environ.get('API_URL')
test_email = os.getenv('TEST_EMAIL')
```
**JavaScript / TypeScript:**
```javascript theme={null}
const apiUrl = process.env.API_URL;
const testEmail = process.env.TEST_EMAIL;
```
**Bash:**
```bash theme={null}
echo $API_URL
echo ${TEST_EMAIL}
```
[Learn more about environment variables →](/environments)
Click **Insert Environment Variable** when creating/editing a hook to browse and select from your available environment variables.
### Run Metadata
Every hook in a run can read details about that run. This is useful for linking your own logs or error reports back to the Autosana run.
| Variable | Value | Available in |
| - | - | - |
| `AUTOSANA_FLOW_RUN_ID` | ID of the flow run | Flow hooks |
| `AUTOSANA_FLOW_ID` | ID of the flow | Flow hooks |
| `AUTOSANA_RUN_GROUP_ID` | ID of the run group, also set when a flow runs on its own | Flow and suite hooks |
| `AUTOSANA_RUN_URL` | Link to the flow run, or to the run group in suite hooks | Flow and suite hooks |
| `AUTOSANA_RUN_GROUP_URL` | Link to the run group | Flow and suite hooks |
| `AUTOSANA_BATCH_ID` | ID of the batch the run was started in | Flow and suite hooks |
| `AUTOSANA_BATCH_URL` | Link to the batch | Flow and suite hooks |
Read them like any other environment variable, or use `${env:AUTOSANA_FLOW_RUN_ID}` in cURL and App Launch Configuration hooks. They can't be overwritten and aren't set in **Test Hook**.
#### Example: Tagging Errors in Your App
Add an App Launch Configuration setup hook to the flow:
```json theme={null}
{
"autosanaRunId": "${env:AUTOSANA_FLOW_RUN_ID}",
"autosanaRunUrl": "${env:AUTOSANA_RUN_URL}"
}
```
Read `autosanaRunId` when your app starts and add it as a tag in your error tracker, for example `Sentry.setTag("autosana_run_id", autosanaRunId)`.
## Sharing Data Between Hooks
Hooks can pass data to subsequent hooks in the same suite. This is useful for scenarios like:
* Generating an auth token in a setup hook and using it in a runtime hook
* Creating a test user and passing the user ID to a teardown hook for cleanup
* Sharing credentials across multiple flows in a suite
### How It Works
1. In your hook, write values to `/tmp/autosana.env` in `KEY=VALUE` format
2. Subsequent hooks can access these values as environment variables
3. Values persist for all subsequent hooks in the suite (across all flows)
### Key Naming Rules
When exporting values, keys must follow these rules:
| Rule | Valid | Invalid |
| - | - | - |
| Alphanumeric + underscore only | `AUTH_TOKEN`, `API_KEY`, `user_id` | `API-KEY`, `user.id` |
| Cannot start with a number | `TOKEN_123`, `_INTERNAL` | `123_TOKEN`, `1ST_KEY` |
### Example: Passing a Token Between Hooks
**Setup Hook (Python) - Creates token:**
```python theme={null}
import urllib.request
import json
import os
data = json.dumps({
"email": "test@example.com",
"password": "secret"
}).encode('utf-8')
req = urllib.request.Request(
f"{os.environ.get('API_URL')}/auth/login",
data=data,
headers={'Content-Type': 'application/json'}
)
with urllib.request.urlopen(req) as response:
result = json.loads(response.read().decode('utf-8'))
token = result['token']
user_id = result['userId']
# Export for subsequent hooks
with open('/tmp/autosana.env', 'w') as f:
f.write(f"AUTH_TOKEN={token}\n")
f.write(f"TEST_USER_ID={user_id}\n")
print(f"Created session for user {user_id}")
```
**Runtime Hook (cURL) - Uses the token:**
```bash theme={null}
curl -X POST ${env:API_URL}/cart/add \
-H "Authorization: Bearer ${env:AUTH_TOKEN}" \
-H "Content-Type: application/json" \
-d '{"productId": "12345", "quantity": 1}'
```
**Teardown Hook (Python) - Cleans up:**
```python theme={null}
import urllib.request
import os
# Access values exported by setup hook
token = os.environ.get('AUTH_TOKEN')
user_id = os.environ.get('TEST_USER_ID')
# Clean up test user
req = urllib.request.Request(
f"{os.environ.get('API_URL')}/users/{user_id}",
method='DELETE',
headers={'Authorization': f'Bearer {token}'}
)
with urllib.request.urlopen(req) as response:
pass # Just need to execute the request
print(f"Cleaned up user {user_id}")
```
### Important Notes
* **Scripts can read and write** exported values
* **cURL hooks can only read** exported values (they cannot write to `/tmp/autosana.env`)
* If multiple hooks export the same key, the **latest value wins**
* Exported values are available to all subsequent hooks in the suite (across all flows)
* Exported values are also available to **subsequent flows** in the suite via `${env:KEY}`
* The agent can also retrieve exported values directly by name using **Get Variable**
Hook exports are part of Autosana's runtime variable system. For a complete overview of how variables work across hooks, flows, and suites, see [Variables](/variables).
## Hooks and Run Caching
[Run Caching](/flows#run-caching) speeds up a flow by replaying what a previous passing run did, instead of working it out again. Whether that replay is possible depends on how your hook passes values to the flow.
If your hook exports a new value every run — a phone number, a test account, a booking reference — Autosana has to learn that the value is meant to change. Until it does, those runs skip the cache. They still pass or fail correctly; they are just slower.
### Export the Values Your Flow Uses
**Do this.** Write the value to `/tmp/autosana.env` under a name, so each run can be given its own:
```python theme={null}
number = generate_phone_number()
with open('/tmp/autosana.env', 'w') as f:
f.write(f"TEST_PHONE={number}\n")
```
Then reference it in your flow instructions as `${env:TEST_PHONE}`, or have the agent fetch it with **Get Variable**.
**Not this.** Printing it is the only record, and there is no name to look it up by:
```python theme={null}
print(f"Generated phone number: {generate_phone_number()}")
```
What a setup hook prints is not part of the cache key. If a recorded run typed, tapped or checked a printed value, a later run replays only when your hook prints the same value; otherwise that run executes live and records again. The exception is a printed copy of an export Autosana already treats as changing (see the Tip below): replay fills in the new value instead. A printed value the flow never uses, such as an ID or timestamp, does not stop runs from replaying, though a short one that happens to match text in a step can occasionally send a run live. Values shorter than 4 characters are not checked, so a recording that used one can replay after it changes.
Use hook output to explain what the hook did. Printing an export's value is fine once Autosana treats that export as changing under the [rules below](#which-values-autosana-treats-as-changing). A printed value your flow uses that is not covered by those rules — such as a short code (a 6-digit OTP) or a differently formatted copy — still makes each run with a new value execute live.
### Which Values Autosana Treats as Changing
For web runs, exported `BASE_URL` and `API_BASE_URL` origins qualify automatically without a changing-value learning period. See [preview-deployment caching](/flows#run-caching) for the required origin format and navigation limits.
For other custom exports, Autosana compares each value against what the same hook exported last time — same flow version, same environment — and treats it as changing only if it actually changed.
| What your hook does with the value | What happens |
| - | - |
| Exports it, it is at least 8 characters, and it differs each run | Each run gets its own value |
| Exports it, and it stays the same (e.g. `API_URL`) | Treated as fixed; caching works normally |
| Exports it, but it is under 8 characters | Left alone — something as short as `4821` would match text that has nothing to do with it |
| Only prints it | If the flow uses it, each run with a new value runs live; if not, caching is unaffected |
The built-in lease hooks automatically recognize `AUTOSANA_EMAIL_ADDRESS`, `AUTOSANA_PHONE_NUMBER`, and `AUTOSANA_PHONE_NUMBER_NATIONAL`, including both exported phone formats. See [OTP testing](/guides-otp-testing) for these hooks. For other custom exports, it takes **three passing runs** to settle: the first records the value, the second notices it changed and rebuilds the cache, the third can use it. Failed runs do not count. This is tracked per flow version and environment, so editing the flow's instructions starts the count over.
### When Only Some Steps Can Be Cached
A setup hook does not automatically turn caching off. For built-in leased identities and custom exports that meet the rules above, cached typing steps and descriptions can use the current run's value.
If a step contains a changed value that Autosana cannot replace reliably — for example, an encoded copy, a phone format other than either exported form, or the exact same value shared by two exports — the agent handles that step live. Earlier independent steps can still use the cache. Autosana resumes caching afterward only when it can confirm the required live action ran and the app matches a later checkpoint; otherwise the remaining steps run live. These partly cached runs still receive a fresh review.
Other printed values work differently: if the recording used a printed value and this run's hook prints a different one, the whole run executes live. A printed copy of an export that meets the rules above is the exception and is handled like the export.
### Runtime Hooks and Caching
A cached run executes each runtime hook again at the same step, so its side effects happen on every run.
* **Exit with a non-zero code to fail a check.** A cached run reads only the exit code, not the output. If the hook fails, the agent finishes the rest of the run live.
* **Values that change each run end the cached part.** If a runtime hook returns a new value each run — such as a sign-in code or order number — and a later step types or taps it, the agent finishes the run live from that hook onward.
### Flows That Are Never Cached
Using the **Set Variable** action turns caching off for a flow, whatever your hooks do.
## Hook Examples
### cURL Examples
#### Create User Account
```bash theme={null}
curl -X POST https://api.example.com/users \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${env:ADMIN_TOKEN}" \
-d '{
"email": "${env:TEST_EMAIL}",
"password": "${env:TEST_PASSWORD}",
"role": "tester"
}'
```
#### Reset Database
```bash theme={null}
curl -X POST https://api.example.com/test/reset \
-H "Authorization: Bearer ${env:ADMIN_TOKEN}" \
-d '{"scope": "test_data"}'
```
#### Generate Auth Token
```bash theme={null}
curl -X POST https://api.example.com/auth/token \
-H "Content-Type: application/json" \
-d '{
"client_id": "${env:CLIENT_ID}",
"client_secret": "${env:CLIENT_SECRET}"
}'
```
#### Delete Test Data (Teardown)
```bash theme={null}
curl -X DELETE https://api.example.com/users/${env:TEST_USER_ID} \
-H "Authorization: Bearer ${env:ADMIN_TOKEN}"
```
### Script Examples
#### Python: Create User and Export Credentials
```python theme={null}
import urllib.request
import urllib.error
import json
import os
import secrets
import string
# Generate random email for test user
random_suffix = ''.join(secrets.choice(string.ascii_lowercase) for _ in range(8))
test_email = f"test_{random_suffix}@example.com"
data = json.dumps({
"email": test_email,
"password": "TestPass123",
"role": "tester"
}).encode('utf-8')
req = urllib.request.Request(
f"{os.environ.get('API_URL')}/users",
data=data,
headers={
'Content-Type': 'application/json',
'Authorization': f"Bearer {os.environ.get('ADMIN_TOKEN')}"
}
)
try:
with urllib.request.urlopen(req) as response:
user_data = json.loads(response.read().decode('utf-8'))
print(f"Created test user: {test_email}")
# Export for use in flow and teardown
with open('/tmp/autosana.env', 'w') as f:
f.write(f"TEST_USER_EMAIL={test_email}\n")
f.write(f"TEST_USER_ID={user_data['id']}\n")
except urllib.error.HTTPError as e:
raise Exception(f"Failed to create user: {e.read().decode('utf-8')}")
```
#### JavaScript: Fetch and Process Data
```javascript theme={null}
const response = await fetch(`${process.env.API_URL}/products`, {
headers: { 'Authorization': `Bearer ${process.env.API_TOKEN}` }
});
const products = await response.json();
// Find a product that's in stock
const availableProduct = products.find(p => p.inStock && p.price < 100);
if (availableProduct) {
console.log(`Found product: ${availableProduct.name} ($${availableProduct.price})`);
// Export for use in flow
const fs = require('fs');
fs.writeFileSync('/tmp/autosana.env', `TEST_PRODUCT_ID=${availableProduct.id}\n`);
} else {
throw new Error('No suitable test product found');
}
```
#### Bash: Quick API Check
```bash theme={null}
# Check if API is healthy before running tests
HEALTH=$(curl -s -o /dev/null -w "%{http_code}" "${API_URL}/health")
if [ "$HEALTH" != "200" ]; then
echo "API health check failed with status: $HEALTH"
exit 1
fi
echo "API is healthy, proceeding with tests"
```
## App Launch Configuration (Mobile Only)
### How It Works
App Launch Configuration hooks pass values to your mobile app when it launches. These values are set **at app startup** and are available throughout your test. This feature is only available for iOS and Android apps.
### When It Runs
Setup hooks all run before the app starts — unless one of them is a launch configuration, in which case the app starts **at that hook's position**. Put a hook that mints a token or seeds a user above it; hooks below it run against the already-running app.
**More than one.** Each applies at its own position, restarting the app if it is already running. Nested objects merge — setting `featureFlags.darkMode` keeps the other `featureFlags` keys — while lists and single values are replaced.
**A restart keeps device data, not screen state.** The app is not reinstalled or reset, so keychain, saved sessions and local databases survive, but the process restarts and the flow begins at your app's launch screen.
**In a suite,** flows share one session — opened by the suite's own launch configuration if it declares one. A flow's launch configuration restarts the app with its values merged over the suite's; a flow with none keeps what the session already has. With **Run flows in parallel**, each flow gets its own session and applies the suite's configuration from scratch.
### Dynamic Values (Environment & Build Variables)
Launch configuration JSON supports `${env:VARIABLE_NAME}` placeholders, resolved when the hook runs — so a placeholder sees values exported by setup hooks above it, as well as your [environment variables](/environments) and **build variables** passed via CI (the `variables` input on the GitHub Action / upload API). This lets you drive launch configuration dynamically — for example, passing an Expo Updates channel from CI:
```json theme={null}
{
"EXPO_CHANNEL": "${env:E2E_CHANNEL}"
}
```
The resolved value is delivered to your app as an iOS environment variable / Android intent extra (see below). Your app is responsible for acting on it — e.g. an Expo app must read the value and call `Updates.setUpdateURLAndRequestHeadersOverride(...)` followed by `Updates.reloadAsync()` to switch channels at runtime.
If a referenced variable is not set — or is set to an empty or whitespace-only value — the run stops with an error naming it, rather than starting your app with a literal `${env:...}` string.
### Accessing Values in Your App
**iOS (Swift):**
```swift theme={null}
// Access simple values (converted to strings)
let environment = ProcessInfo.processInfo.environment
let testEnv = environment["testEnvironment"] // "staging"
let apiTimeout = environment["apiTimeout"] // "30" (as string)
// Nested objects are JSON strings - parse them
if let flagsJSON = environment["featureFlags"],
let flagsData = flagsJSON.data(using: .utf8) {
do {
if let flags = try JSONSerialization.jsonObject(with: flagsData) as? [String: Any] {
let darkMode = flags["darkMode"] as? Bool // true
let checkoutFlow = flags["newCheckoutFlow"] as? String // "enabled"
}
} catch {
print("Failed to parse feature flags: \(error)")
}
}
```
**Android (Kotlin):**
```kotlin theme={null}
import org.json.JSONObject
// Access simple values (type-aware)
val testEnv = intent.extras?.getString("testEnvironment") // "staging"
val apiTimeout = intent.extras?.getInt("apiTimeout") // 30 (as int)
// Nested objects are JSON strings - parse them
val flagsJSON = intent.extras?.getString("featureFlags")
if (flagsJSON != null) {
try {
val flags = JSONObject(flagsJSON)
val darkMode = flags.getBoolean("darkMode") // true
val checkoutFlow = flags.getString("newCheckoutFlow") // "enabled"
} catch (e: Exception) {
Log.e("LaunchArgs", "Failed to parse feature flags", e)
}
}
```
**React Native:**
```javascript theme={null}
import { NativeModules, Platform } from "react-native";
// iOS: values are set as environment variables (available via native bridge)
// Android: values are passed as intent extras
const launchArgs = Platform.OS === "ios"
? NativeModules.ProcessInfo?.environment
: NativeModules.IntentExtras?.getExtras();
// Simple values
console.log(launchArgs.testEnvironment); // "staging"
console.log(launchArgs.apiTimeout); // "30" (string on iOS, number on Android)
// Nested objects come as JSON strings - parse them
if (launchArgs.featureFlags) {
const flags = JSON.parse(launchArgs.featureFlags);
console.log(flags.darkMode); // true
console.log(flags.newCheckoutFlow); // "enabled"
}
```
On iOS, launch configuration values are set as **environment variables** and accessed via `ProcessInfo.processInfo.environment`. On Android, they are passed as **intent extras** and accessed via `intent.extras`. See the platform-specific examples above for details.
### Launch Configuration Examples
#### Override Feature Flags
```json theme={null}
{
"featureFlags": {
"newCheckoutFlow": "treatment_a",
"paymentMethodV2": "enabled",
"showPromotion": false
}
}
```
#### Set Environment and Timeouts
```json theme={null}
{
"testEnvironment": "staging",
"apiBaseUrl": "https://staging-api.example.com",
"requestTimeout": 30,
"retryAttempts": 3
}
```
#### A/B Testing Configuration
```json theme={null}
{
"experiments": {
"checkoutExperiment": "variant_b",
"pricingExperiment": "control"
},
"userId": "test_user_123"
}
```
#### Debug Mode Settings
```json theme={null}
{
"enableDebugMode": true,
"logLevel": "verbose",
"showPerformanceMetrics": true,
"mockPaymentProvider": true
}
```
### When to Use Each Hook Type
| Use Case | Recommended Hook Type |
| - | - |
| Simple single API call | cURL |
| Multiple API calls with logic | Python, JavaScript, or TypeScript |
| Shell operations, piping commands | Bash |
| Configure app at startup | App Launch Configuration |
| Need error handling and retries | Python, JavaScript, or TypeScript |
| Parse and transform API responses | Python, JavaScript, or TypeScript |
## Setup Hooks & Agent Context
**Pro Tip:** When a hook produces output (via print statements, console.log, or echo), the agent automatically receives this data as additional context and can use it during flow execution.
**Example setup hook:**
```python theme={null}
import urllib.request
import json
import os
data = json.dumps({
"email": os.environ.get('TEST_EMAIL'),
"password": os.environ.get('TEST_PASSWORD')
}).encode('utf-8')
req = urllib.request.Request(
f"{os.environ.get('API_URL')}/auth/login",
data=data,
headers={'Content-Type': 'application/json'}
)
with urllib.request.urlopen(req) as response:
result = json.loads(response.read().decode('utf-8'))
# This output is passed to the agent
print(f"Logged in as: {result['email']}")
print(f"User role: {result['role']}")
print(f"Account created: {result['createdAt']}")
```
The agent receives this output and can reference it if needed during the flow execution.
Printing a value that changes every run — a timestamp, a fresh ID — does not affect the run cache unless your flow uses that value. See [Hooks and Run Caching](#hooks-and-run-caching).
## Timeouts
Hooks have different timeout limits depending on the context:
| Context | Timeout |
| - | - |
| **Testing hooks** (via Test button) | 1 minute |
| **Flow execution** (setup, runtime, teardown) | 5 minutes |
If a hook exceeds its timeout, it will be terminated and marked as failed.
Keep hooks focused and fast. If you have long-running operations, consider breaking them into multiple hooks or optimizing your API calls.
## Best Practices
**Use Descriptive Names**
Name hooks after what they do: "Create Premium Test User" instead of "Hook 1"
**Store Secrets in Environment Variables**
Never hardcode API keys or passwords in hooks. Use environment variables like `${env:API_KEY}`.
**Test Hooks Independently**
Use the Test button to verify your hooks work correctly before attaching them to flows.
**Use Teardown Hooks for Cleanup**
Always clean up test data created by setup hooks to avoid polluting your backend.
For concurrent runs, follow [Making Suites, Parallelizable, Isolated, and
Repeatable](/guides-parallelizable-isolated-tests) so one session cannot change
or delete another session's data.
**Keep Hooks Simple**
Each hook should do one thing. Create separate hooks for different setup tasks.
**Use Scripts for Complex Logic**
When you need conditionals, loops, error handling, or data processing, use Python/JavaScript instead of complex cURL commands.
**Export Only What You Need**
When sharing data between hooks, only export the values that subsequent hooks actually need.
**Add Helpful Print Statements**
Output from your hooks is passed to the agent. Add print statements to provide context about what happened.
## Hooks vs Suite Auth Instructions
| Feature | Hooks | Suite Auth Instructions |
| - | - | - |
| **Type** | Scripts, cURL, or Launch args | Natural language instructions |
| **Purpose** | Setup/cleanup backend state | Navigate app to starting point |
| **Execution** | Before/after each flow | Once at suite start |
| **Reusability** | Can be used across many flows | Specific to one suite |
**Use hooks when:** You need to configure backend state, create test data, or call APIs
**Use suite auth instructions when:** You need to start the suite from a specific state (e.g., logged in with a certain test account)
## Troubleshooting
### Script Issues
**Script times out**
* Hooks have a 5-minute timeout during flow execution (1 minute when testing)
* Break long operations into multiple hooks
* Check for infinite loops or slow API endpoints
**Environment variable not found**
* Verify the variable exists in Settings → Environments
* Check spelling and case (variable names are case-sensitive)
* For scripts, use the correct access method for your language
**Random values are the same every run**
* Hook sandboxes are created from snapshots, so pseudo-random generators may repeat their seed across runs
* In JavaScript/TypeScript, use Node's built-in `crypto` module (`crypto.randomInt`, `crypto.randomUUID`) instead of `Math.random()`
* In Python, prefer `secrets` or `uuid`; if you must use `random`, call `random.seed(os.urandom(32))` before generating values
**Exported values not available in next hook**
* Verify you wrote to `/tmp/autosana.env` (exact path)
* Check key naming rules (alphanumeric + underscore, can't start with number)
* Ensure the format is `KEY=VALUE` with one per line
### cURL Request Issues
**Hook fails to execute**
* Verify the curl command works in your terminal first
* Check that environment variables are defined
* Ensure API endpoints are accessible from Autosana's infrastructure
**Environment variables not replacing**
* Syntax must be `${env:VARIABLE_NAME}` (not `{{VARIABLE_NAME}}` or `$VARIABLE_NAME`)
* Variable names are case-sensitive
* Variables must exist in Settings → Environments
### App Launch Configuration Issues
**Configuration not appearing in app**
* Verify JSON is valid (use a JSON validator)
* Check you're accessing values correctly for your platform (ProcessInfo for iOS, intent.extras for Android)
* Ensure the hook is attached as a **setup hook** (launch configuration doesn't work in teardown)
**Values have wrong type**
* On iOS, all values become strings (including numbers and booleans)
* On Android, primitives keep their types (int, float, boolean, string)
* Nested objects are JSON-serialized as strings on both platforms
* Use the examples in this guide to parse nested structures correctly
### Data Sharing Issues
**Hook can't read value from previous hook**
* Ensure the previous hook successfully wrote to `/tmp/autosana.env`
* Check that the previous hook completed without errors
* Verify the key name matches exactly (case-sensitive)
**Invalid key name warning**
* Keys must contain only letters, numbers, and underscores
* Keys cannot start with a number
* Examples: `AUTH_TOKEN` ✓, `123_KEY` ✗, `API-KEY` ✗
## Managing Hooks via API
Hooks can be created, updated, and deleted programmatically via the [Hooks API](/api-hooks). Useful for syncing hook definitions from your repo, bulk-importing from another system, or wiring up custom tooling.
Hooks can also live as script files in your repository and be referenced from code-managed flows — see [Hooks as files](/code-managed-files#hooks-as-files).
## Next Steps
* [Learn about environment variables →](/environments)
* [Organize flows with suites →](/suites)
* [Create your first flow →](/flows)
* [Manage hooks via API →](/api-hooks)
* [Define hooks in your repo →](/code-managed-files#hooks-as-files)
# Install the CLI
Source: https://docs.autosana.ai/install-cli
Install the autosana command-line tool
The `autosana` CLI connects local simulators, emulators, physical devices, and dev servers to Autosana so you can run flows against a build on your own machine. It also validates your [code-managed flow files](/code-managed-flows) before you push (`autosana flows validate`) and runs them from the terminal (`autosana run`).
## Quick install
The fastest way — installs [uv](https://docs.astral.sh/uv/) if it's missing, then the CLI. Zero prerequisites:
```bash theme={null}
curl -fsSL https://get.autosana.ai | sh
```
Upgrade any time with `autosana update`.
## Other install methods
**pipx:**
```bash theme={null}
pipx install autosana
```
**uv:**
```bash theme={null}
uv tool install autosana
```
**pip** (into an environment you manage):
```bash theme={null}
pip install -U autosana
```
### Local testing dependencies
Everything above is a lightweight install that covers the `flows` commands and local web testing. Capturing screenshots from a simulator or physical device needs the optional **`local`** extra, which pulls in Pillow:
```bash theme={null}
uv tool install "autosana[local]" # or: pip install "autosana[local]"
```
Add `[local]` only when running flows against a simulator, emulator, or physical device. Web local runs do not need it.
## Verify
```bash theme={null}
autosana --version
```
If `autosana` isn't found after installing with uv, add uv's tool directory to your PATH and restart your shell:
```bash theme={null}
uv tool update-shell
```
## Set up prerequisites
Web local testing needs cloudflared and a running dev server. Mobile local testing also needs Node.js and Appium. `autosana doctor` checks the CLI dependencies and can install missing tools; `autosana up --platform web` checks that the dev server is reachable when the session starts.
```bash theme={null}
autosana doctor --platform web # check disk space and cloudflared
autosana doctor --platform web --fix # install cloudflared if needed
autosana doctor --platform ios --fix # install mobile prerequisites
```
## Log in
Authenticate the CLI against your Autosana organization — it opens the dashboard for a one-click approval, no key copying:
```bash theme={null}
autosana login # approve in the browser tab that opens
autosana whoami # show which org / project scope the CLI is using
autosana logout # remove the stored credential
autosana keys create # create an API key, e.g. for CI
```
`AUTOSANA_API_KEY` still works for CI and scripting, and always takes precedence over a stored login.
## Upgrading
These docs describe the current release. If a command or flag from the docs is missing, upgrade:
```bash theme={null}
autosana update
```
It upgrades the CLI the same way you installed it. On a version without `autosana update`, re-run the installer instead:
```bash theme={null}
curl -fsSL https://get.autosana.ai | sh # or: pipx upgrade autosana
```
## Next steps
* [Run flows locally →](/local-testing)
* [Run code-managed flows from the terminal →](/code-managed-cli)
* [Write effective flows →](/flows)
# Intro(doc)tion
Source: https://docs.autosana.ai/introduction
Autosana is the platform where you can write end-to-end tests using natural language for **mobile apps** and **websites**. Spend your time shipping features & fixing bugs, not fixing tests (or worse, testing manually).
## Dashboard navigation
Use the sidebar to switch between Overview, Issues, Runs, Flows, Apps, Hooks, Automations, and Settings. Page headings and main content keep a consistent left margin across sections. On desktop, choose **Keep sidebar open** to pin the sidebar; page content moves beside it.
Get your first flow running in 5 minutes
Test sites on Chrome, Firefox, Edge, or Chromium
Automate testing in your deployment pipeline
Write tests in natural language that just work
Keep your tests as files in your repo, synced via GitHub
Schedule tests to run automatically
## How It Works
Upload your iOS (.app) or Android (.apk) build, or simply enter a website URL
Describe what you want to test: "Log in with [test@example.com](mailto:test@example.com) and verify the home screen loads"
The agent executes your flow and provides detailed results with screenshots
## Key Features
Forget about brittle XPath or CSS selectors. Just describe what you want to do.
Tests adapt to UI changes automatically without manual updates.
Write tests like "Tap the login button" instead of `driver.findElement(...)`.
See exactly what happened with screenshots at every step.
## Need Help?
Email us at [founders@autosana.ai](mailto:founders@autosana.ai)
Schedule a 30-minute demo with our team
Jump into the dashboard
# Issues
Source: https://docs.autosana.ai/issues
App bugs, UX suggestions, and unclear instructions flagged in your flow runs
**Issues** flag the cause of a test failure, a UI/UX improvement, or an unclear test instruction, and cite exactly where in the run or instructions it occurred.
Individual issues are grouped in the background into **Issue Groups**, shown on the [Issues](https://autosana.ai/issues) and [Overview](https://autosana.ai/overview) pages.
## Individual Issues
Issues are generated per flow run. View them on a flow run directly, or through the occurrences in an issue group. A flow run shows *what* happened; an issue group shows *where* similar issues happened across your runs.
Each issue has the following properties:
| Property | Description |
| - | - |
| **Title** | A concise identifier for the issue. |
| **Description** | A clear explanation of the issue. |
| **Severity** (`critical`, `major`, `minor`) | The impact level of the issue. Critical issues cause a test to fail; major and minor issues are flagged but don't fail the test. |
| **Type** (`functional`, `ui`, `ux`, `performance`, `state`, `instructions`) | The testing facet the issue falls under. Instruction issues flag a problem in the test itself rather than your app, and show in blue instead of the usual red, orange, and yellow (critical instruction issues stay red). |
| **Instruction highlighting** | The instruction line(s) the issue refers to. |
| **Action references** | The action(s) where the issue occurred. These need not be consecutive. |
In Steps, issue cards display three or more consecutive steps as a range, such as **24–29**. Pairs and isolated steps stay comma-separated.
## Issue Groups
An **Issue Group** collects individual issues that describe the same underlying problem, even when they occur on different runs, flows, apps, or devices. Grouping is semantic, so issues match on meaning rather than exact wording, and runs in the background: new issues join existing groups automatically, duplicate groups merge, and overly broad groups split.
Each group falls into one of three categories:
* **Critical**: App bugs responsible for test failures. Any group that contains a critical issue.
* **UI/UX**: Visual, usability, and performance improvements that didn't fail the test.
* **Instruction**: Instructions the agent found ambiguous, contradictory, or impossible on the current build. These point to the test, not your app.
A group shows:
* **Occurrences**: The total times the problem has been seen.
* **Affected flows, apps, devices, and builds**: The places where it occurs.
* **Severity breakdown**: The count of critical, major, and minor issues in the group.
* **First and last seen**: The times when the problem started and when it last occurred.
* **Evidence**: The runs where it occurred, each with a screenshot from the exact action.
Use **Copy link** on any issue group to easily share it.
### Date range
The Overview page defaults to the past 30 days, including today, in your timezone.
Its date picker filters both activity stats and issue groups. Groups are included
when their **last seen** date falls within the selected range, including older
groups that have recurred recently. Select **All time**
to include older groups. The Issues page has its own date filter.
### Triage
Every group has a status. Use the status toggle on the Issues page to filter by **New**, **Dismissed**, **Closed**, or **All**.
| Action | Result |
| - | - |
| **Mark as complete** | Dismisses the group. It returns if the problem happens again. |
| **Don't show again** | Closes the group. It stays hidden even if the problem recurs. |
| **Restore / Reopen** | Returns a dismissed or closed group to the board. |
Statuses are shared across your workspace.
### Create a Jira or Linear ticket
When Jira or Linear is connected, select **Create ticket** from an issue group
or from an individual issue on a flow run. Review the provider, destination,
issue type, title, and description before creating the ticket. Autosana
prefills the editor from the existing issue analysis and evidence; you can edit
the title and description without changing the Autosana issue.
Autosana shows the external ticket key after creation and prevents a second
current ticket for the same issue group. A pending result shows that creation is
still in progress. If the result is unknown, check Jira or Linear before
retrying. Completed and failed attempts remain in the issue group's ticket
history.
Workspace admins can optionally choose Jira or Linear as the single automatic
provider under **Settings > Integrations**. Automatic creation applies to new
critical issue groups and still creates at most one current ticket.
Ticket behavior follows issue triage:
| Issue event | Ticket behavior |
| - | - |
| **Mark as complete**, then the issue returns while its external ticket is open | The group returns and Autosana adds one recurrence comment with current evidence. |
| The issue returns after its external ticket is completed | Autosana creates or offers a linked regression ticket, depending on the automatic-provider setting. |
| **Don't show again** | The group stays hidden and never creates an automatic ticket, even if the issue occurs again. |
| **Restore / Reopen** | The group returns to Autosana; creating an external ticket still requires the normal manual review or automatic eligibility. |
If Autosana is still grouping a new flow-run issue, the ticket action shows
**Issue analysis in progress** and becomes available when grouping finishes.
## Next Steps
* [Learn how flows are reviewed](/flows)
* [Get notified when runs complete](/notifications)
* [Track performance metrics on every run](/performance-monitoring)
# Jira Cloud
Source: https://docs.autosana.ai/jira-integration
Connect Jira Cloud and choose where Autosana creates tickets
Autosana supports Jira Cloud through Atlassian OAuth. Jira Server and Jira Data
Center are not supported.
## Connect Jira Cloud
You must be an Autosana workspace admin or owner to install and configure Jira.
1. Go to **[Settings > Integrations > Jira](https://autosana.ai/settings?tab=integrations\&integration=jira)**
2. Click **Connect Jira** and approve the Atlassian authorization request
3. If your Atlassian account can access multiple sites, choose the Jira site
4. Select the default project and issue type
Autosana requests these Atlassian scopes:
* `read:jira-work` to read projects, issue types, and ticket status
* `write:jira-work` to create tickets and add recurrence comments
* `manage:jira-webhook` to register ticket-status webhooks
* `offline_access` to refresh the connection without asking you to sign in again
The selected project and issue type are defaults. You can choose a different
available destination in the ticket review dialog before creating a ticket.
## Create and review tickets
Select **Create ticket** on an Autosana issue group or an individual issue in a
flow run. The review dialog includes:
* Jira or Linear as the provider
* the destination project
* the Jira issue type
* an editable title and description with Autosana evidence links
Autosana shows the Jira key after creation and keeps completed and failed
attempts in the issue group's ticket history. It blocks another ticket while a
ticket is open, pending, or has an unknown creation result.
If the same issue returns after its Jira ticket was completed, Autosana offers a
linked regression ticket. The previous provider, project, and issue type are
reused only while they remain available; otherwise you must select a new
destination.
## Automatic tickets
Automatic ticket creation is **Off** for new connections. Workspace admins can
choose one automatic provider under **Settings > Integrations**: **Off**,
**Jira**, or **Linear**. During migration, an eligible existing Linear
connection with a configured default team preserves its prior automatic
behavior.
When Jira is selected, Autosana creates at most one current ticket for each new
critical issue group. Jira must have a valid default project and issue type and
healthy webhook delivery. Issues marked **Don't show again** never create
automatic tickets.
## Webhook health and reconnecting
Jira webhooks normally update Autosana when a ticket changes status. The Jira
settings card shows webhook health and the latest verified delivery. Autosana
also reconciles stale ticket status and renews webhook registrations to repair
missed or expiring subscriptions.
Reconnect Jira when the settings card reports that authorization is required.
If a project or issue type was removed, choose a new default before enabling
automatic creation.
To remove the connection, select **Disconnect Jira** from the Jira settings
card. Existing Autosana ticket history remains available, but new Jira ticket
creation and status updates stop until Jira is connected again.
## Operator rollout and rollback
Use this order when enabling provider-backed ticketing in a hosted environment:
1. Apply the additive/rename migration and deploy backward-compatible backend reads.
2. Configure Atlassian OAuth callback and approve distribution/sharing for Jira Cloud tenants.
3. Configure Jira, Linear, and Slack webhook secrets and public callback URLs.
4. Reauthorize existing Linear installations so webhook permissions/secrets are present.
5. Enable the PostHog `ticketing-integrations` UI flag for internal organizations, then a small customer cohort.
6. Monitor unknown creates, webhook age, reconciliation failures, Slack interaction latency, and blocked grouping maintenance.
7. Expand rollout; for a normal cohort rollback, disable the PostHog flag and `automatic_provider` while leaving the deploy runtime enabled so ticket history and webhook ingestion continue.
`TICKETING_INTEGRATIONS_ENABLED` is the deploy-level runtime switch, separate
from the PostHog UI flag. Setting it to `false` removes the provider, ticket,
and webhook routes and stops ticketing background work. Reserve it for a full
runtime shutdown, not a normal UI/cohort rollback.
# Linear Tickets
Source: https://docs.autosana.ai/linear-integration
Connect Linear and choose where Autosana creates tickets
Connect Linear to create editable tickets from Autosana issue groups. A grouped
ticket contains the issue summary, evidence links, and affected run context
instead of creating a separate Linear issue for every failed run.
## Connect Linear
You must be an Autosana workspace admin or owner to install and configure
Linear.
1. Go to **[Settings > Integrations > Linear](https://autosana.ai/settings?tab=integrations\&integration=linear)**
2. Click **Connect Linear**
3. Authorize Autosana to access your Linear workspace
4. Select the default team for new tickets
Automatic Linear tickets remain unavailable until the default team is set and
webhook delivery is verified. Reconnect Linear if the settings card reports
that the installation needs new webhook permissions.
## Review and create tickets
Select **Create ticket** on an issue group or an individual flow-run issue. The
shared review dialog lets you choose Jira or Linear, select the destination
team, and edit the title and description before submitting.
Autosana displays the Linear key after creation and keeps completed and failed
attempts in ticket history. One issue group can have only one current external
ticket. When an issue returns after its Linear ticket was completed, the dialog
offers a linked regression and reuses the previous team while it remains
available.
## Automatic tickets
Automatic creation is **Off** for new connections. Workspace admins can choose
one provider under **Settings > Integrations**: **Off**, **Jira**, or
**Linear**. During migration, an eligible existing Linear connection with a
configured default team preserves its prior automatic behavior.
When Linear is selected, Autosana creates at most one current ticket for each
new critical issue group. Other issue groups remain available for manual review.
Issues marked **Don't show again** never create automatic tickets.
## Webhook health
Linear webhooks keep ticket status synchronized with Autosana. The integration
settings show webhook health, the latest delivery, and whether reconnection is
required. Autosana reconciles stale status in the background when a delivery is
missed.
If webhook health is degraded, manual creation may remain available, but
automatic Linear creation stays blocked until delivery is verified.
## Example issue
# Local Testing
Source: https://docs.autosana.ai/local-testing
Run Autosana flows on your local simulator, emulator, device, or browser
Have Autosana verify that your code actually works — on your local simulator, emulator, physical phone, or web browser, against your dev environment with hot reload, no build uploads needed.
## Prerequisites
* [Autosana MCP server](/mcp-setup) added to your coding agent
* One local target:
* **iOS Simulator**: macOS with the Simulator booted
* **Android Emulator**: macOS, Linux, or Windows with the emulator booted
* **Physical Android device**: a connected device with USB debugging allowed
* **Physical iPhone**: macOS with Xcode and a trusted, unlocked iPhone in Developer Mode — connected over **USB or Wi-Fi** (see [Wireless (Wi-Fi) iPhone](#wireless-wi-fi-iphone) for the one-time wireless setup)
* **Web**: a local dev server running on any OS (e.g. `localhost:3000`). HTTP and HTTPS dev servers are both supported — the scheme is auto-detected. Frameworks like `next dev --experimental-https`, `vite --https`, and mkcert-based setups work out of the box.
## Usage
Tell your coding agent (Cursor, Claude Code, etc.):
```
Test this new feature locally using Autosana
```
Everything else — CLI setup, dependencies, tunnel — is handled automatically.
## Run code-managed flows from the CLI
If you keep your tests as code ([code-managed flows](/code-managed-flows), YAML under `.autosana/`), you can run the flow you're editing on your local target without a coding agent: start a session with `autosana up`, then `autosana run --local`. The same command runs your working copy on Autosana's cloud devices with `--cloud`. Both are covered in [Running from the CLI](/code-managed-cli).
## Physical Device Local Testing
Use this when you need Autosana to control a real phone connected to your computer. Android devices use the same local testing path as Android Emulators. Physical iPhones need the stricter iPhone readiness checks below.
Tell your coding agent:
```text theme={null}
Test this on my physical device using Autosana local testing
```
Your agent should handle the setup, device detection, readiness checks, recovery, and flow execution. You should not need to copy internal connection details by hand.
Keep the device ready while the flow runs:
* Keep it awake, unlocked, and on the app being tested (Android over USB stays plugged in; a Wi-Fi iPhone stays on the same network as your Mac)
* For Android, allow USB debugging when prompted
* For iPhone, keep Developer Mode enabled and leave the Mac trusted
* For iPhone, use a longer Auto-Lock interval, or temporarily set Auto-Lock to Never while testing
* Do not lock an iPhone from iPhone Mirroring during a run
Autosana only runs on real screenshots from the device. If an iPhone display turns off or screenshots go black, the run stops with an actionable error instead of letting the agent continue blind.
### Wireless (Wi-Fi) iPhone
A physical iPhone connected only over Wi-Fi works the same way — your agent auto-detects the wireless connection and handles WebDriverAgent setup for you. There's a one-time setup, done in Apple's tools:
1. Plug the iPhone into your Mac once over USB and pair/trust it.
2. In Xcode → **Window → Devices and Simulators**, enable **"Connect via network"** for the device. After that you can unplug the cable.
3. Trust the developer certificate on the device when prompted: **Settings → General → VPN & Device Management** → tap your "Apple Development" entry → **Trust**.
4. Make sure **Developer Mode** is on (Settings → Privacy & Security → Developer Mode).
After that, just ask your agent to test on the device — no cable needed.
Wireless tips:
* Keep the iPhone on the **same Wi-Fi network** as your Mac, unlocked, with the screen on. Setting **Auto-Lock to Never** matters more here: the first WebDriverAgent build can take \~30-60 seconds, and a screen that locks mid-setup will interrupt it (your agent will tell you to unlock and retry).
* The first session builds and installs WebDriverAgent (slower); later sessions reuse the cached build and start quickly.
* Wireless is slower than USB and a bit more sensitive to network hiccups; for long unattended runs a cable is still the most reliable.
If something fails, ask your agent to check the local Autosana status and recover the session:
```text theme={null}
Check my physical device Autosana local status and recover it if needed
```
The agent uses Autosana's local status checks before running. A physical iPhone session is ready only when Autosana has proven both device control and non-black screenshots.
Physical iPhones are not part of multi-device auto-detection yet. Android devices are detected with the other Android targets. If more than one device is connected, your agent may ask you which one to use.
## Multi-Device Parallel Testing
Run the same flow across multiple devices at the same time — different screen sizes, OS versions, or just faster coverage.
### How it works
1. **Boot your devices** — open multiple iOS Simulators, Android Emulators, or both
2. **Tell your agent** what to test:
```
Test my app on all my simulators in parallel
```
3. The Autosana CLI **auto-detects all booted devices**, starts a separate session for each (with its own tunnel and Appium server), and runs your flows simultaneously across all of them
4. **Results come back per-device** — you see pass/fail for each device independently, with links to the run details on the dashboard
### What gets auto-detected
* **iOS**: All booted Simulators (via `xcrun simctl`)
* **Android**: All connected emulators and devices (via `adb`)
Physical iPhones are not part of multi-device auto-detection yet. Connected Android devices are included with Android targets.
No manual device configuration needed — boot the simulators you want, and the agent handles the rest.
### Example use cases
* **Screen size coverage**: Run the same flow on iPhone SE, iPhone 16 Pro, and iPad to verify layouts across screen sizes
* **OS version testing**: Boot simulators on iOS 17 and iOS 18 to catch version-specific bugs
* **Cross-platform**: Test on both an iOS Simulator and Android Emulator in a single run
## Close the Loop
The real power of local testing: your coding agent can build, test, and fix in a single loop.
Give your coding agent a task — a feature, a bug fix, a ticket:
```
Build . Test it locally using Autosana and fix anything that fails.
```
It writes the code, Autosana verifies it on your local target, and the agent fixes whatever breaks, looping until everything passes. With hot reload, the whole cycle takes seconds.
```
Task
↓
Agent writes code
↓
Autosana verifies it on simulator/emulator/browser
↓
Pass → next task
Fail → agent reads results, fixes bug → ↑ retest
```
## iPhone Duo pose changes (preview)
On a configured Duo simulator, a flow can ask the agent to close, open, or fold
the device into Book pose while preserving app state. Each action captures the
active app window. The replay uses a pose-specific schematic frame for Closed, Open, and Book. Wide
captures get a wider replay column, and step thumbnails preserve their captured
proportions. These preview frames do not represent final device hardware. The
replay labels the captured pose and uses that action's
screen dimensions for tap markers; Open and Book remain distinct even when their
screenshots have the same shape.
If capture metadata is unavailable, the replay keeps the full screenshot visible and disables live tap overlays instead of guessing the pose or coordinates.
Single taps and double taps are supported. Tap counts greater than two are not yet supported.
Duo currently uses one screenshot replay timeline. Continuous video, pinch, momentum flick, and
multiple visible app windows are not supported in this preview. Duo requires a
separately configured simulator host and is not yet available in the standard
cloud device pool.
# MCP Server
Source: https://docs.autosana.ai/mcp-setup
Connect AI assistants to Autosana for docs and test management
Connect your AI coding assistant to Autosana for seamless documentation access and test flow management directly from your IDE.
## Configuration
Connect Autosana with your existing account using browser sign-in. Grok Bot and Devin currently use the API-key setup described below.
Run in your project directory:
```bash theme={null}
claude mcp add --transport http autosana https://mcp.autosana.ai/mcp
```
Open Claude Code, run `/mcp`, select Autosana, and follow the browser sign-in prompt.
In Claude, open **Customize → Connectors → Add custom connector**. Enter **Autosana** as the name and this server URL:
```text theme={null}
https://mcp.autosana.ai/mcp
```
Choose **Connect** and sign in through your browser. Your Claude account must support custom connectors; if the option is disabled, ask your workspace administrator or use another supported client.
Add Autosana to `~/.cursor/mcp.json`, preserving any existing servers:
```json theme={null}
{
"mcpServers": {
"autosana": {
"url": "https://mcp.autosana.ai/mcp"
}
}
}
```
Open Cursor's MCP settings and click **Authenticate** beside Autosana.
Run:
```bash theme={null}
gemini mcp add --transport http autosana https://mcp.autosana.ai/mcp
```
In Gemini CLI, run `/mcp auth autosana` and follow the browser sign-in prompt.
Run:
```bash theme={null}
codex mcp add autosana --url https://mcp.autosana.ai/mcp
codex mcp login autosana
```
If you configure Codex manually, add the following to `~/.codex/config.toml`, then run `codex mcp login autosana`:
```toml theme={null}
[mcp_servers.autosana]
url = "https://mcp.autosana.ai/mcp"
```
A dot uses plugins installed in ChatGPT. In ChatGPT on the web, open **Settings → Security and login** and turn on **Developer mode** (Plus, Pro, Business, Enterprise, or Edu). Then open **ChatGPT Plugins**, select **+**, and create an app named **Autosana** with this server URL:
```text theme={null}
https://mcp.autosana.ai/mcp
```
Choose **OAuth** and approve access in your browser. Do not use an API key. In a chat, open **+**, choose **Developer mode**, and select **Autosana**. This connects the app in ChatGPT. It does not add a separate control inside the dot.
Full walkthrough: [Dots](/dots).
Grok Bot has no Settings form for a custom MCP URL, and Autosana is not in the marketplace. Open **[Settings → Integrations → Grok Bot](https://autosana.ai/settings?tab=integrations\&integration=grok-bot)** to create a key, then send this in a Grok Bot chat:
```
Add a custom MCP called Autosana at https://mcp.autosana.ai/mcp with header x-api-key:
```
Approve the confirm card. Next message: `@` Autosana and ask it to list or run a flow. Adding MCP in Cursor does not set up Grok Bot.
Full walkthrough: [Grok Bot](/grok-bot).
In Devin, open **Customize → MCPs → Add a custom MCP**. Choose **HTTP**, not STDIO. Open **[Settings → Integrations → Devin](https://autosana.ai/settings?tab=integrations\&integration=devin)** to create a key, then fill in:
```
Server name: Autosana
Transport: HTTP
Server URL: https://mcp.autosana.ai/mcp
Authentication: Auth Header
Header key: x-api-key
Header value:
```
Save, then use **Test listing tools**. Adding MCP in Cursor does not set up Devin.
Full walkthrough: [Devin](/devin).
Add this under `mcp_servers` in `~/.hermes/config.yaml` (keep any other servers). Hermes opens a browser the first time it connects:
```yaml theme={null}
mcp_servers:
autosana:
url: "https://mcp.autosana.ai/mcp"
auth: oauth
```
Or run `hermes mcp add --url https://mcp.autosana.ai/mcp --auth oauth autosana`. Do not use `--auth header` — that writes `Authorization: Bearer`. Adding MCP in Cursor does not set up Hermes Agent.
Full walkthrough: [Hermes Agent](/hermes-agent).
Run:
```bash theme={null}
openclaw mcp set autosana '{"url":"https://mcp.autosana.ai/mcp","transport":"streamable-http","auth":"oauth"}'
openclaw mcp login autosana
```
Follow the browser sign-in prompt.
In the browser, sign in to Autosana, check the agent name and company, and click **Connect**. The agent can work in any workspace you can access in that company. Return to your agent and ask it to list your Autosana flows.
If the authorization session ends before you approve, start a fresh connection from your agent. Do not reuse the old browser tab.
Existing API-key connections continue to work. To switch one to browser sign-in, remove its `x-api-key` header and reconnect. Keep the key until the new connection works; revoke it only if no other integrations use it.
You may need to restart your agent for the changes to take effect.
***
## Verify the Connection
Once configured, test the server:
```
List all my Autosana flows
```
```
How do suites work in Autosana?
```
If the connection is successful, you'll receive responses from both questions.
***
## Available Tools
The Autosana MCP server provides comprehensive test management, debugging, and documentation access.
### Flow Management
| Tool | Description |
| - | - |
| `flows_list` | List all flows in your organization, optionally filtered by suite |
| `flows_read` | Get details of a specific flow by ID |
| `flows_create` | Create a new test flow with name, instructions, and optional suite |
| `flows_update` | Update an existing flow's name or instructions |
| `flows_delete` | Delete one or more flows (supports batch deletion) |
| `flows_run` | Execute flows, with optional browser and extension loadout overrides |
| `add_hook_to_flow` | Add a setup or teardown hook to a flow |
### Suite Management
| Tool | Description |
| - | - |
| `suites_list` | List all test suites in your organization |
| `suites_read` | Get details of a specific suite by ID |
| `suites_create` | Create a new suite with flows and optional authentication |
| `suites_update` | Update a suite's name, description, or authentication setup |
| `suites_delete` | Delete one or more suites (supports batch deletion) |
| `suites_run` | Execute a suite with optional browser and extension loadout overrides |
### Hook Management
| Tool | Description |
| - | - |
| `hooks_list` | List all hooks in your organization, optionally filtered by flow |
| `hooks_read` | Get details of a specific hook by ID |
| `hooks_create` | Create a new hook with a script (cURL, Python, JS, Shell) |
| `hooks_update` | Update an existing hook's name, script, type, or description |
| `hooks_delete` | Delete one or more hooks (supports batch deletion) |
| `hooks_run` | Execute a hook and wait for results |
To attach a [built-in hook](/guides-otp-testing) such as **Lease email address**, pass `{"type": "builtin", "builtin_key": "lease_email_inbox", "hook_type": "setup"}` in the `hooks` list of `flows_create`, `flows_update`, `suites_create`, or `suites_update`. It's added to your organization automatically if it isn't there yet.
### Apps & Environments
| Tool | Description |
| - | - |
| `apps_list` | List active apps, optionally filtered by platform |
| `apps_read` | Get app details, recent builds, and attached dependencies |
| `apps_register` | Register an iOS, Android, web, or Chrome extension app |
| `builds_upload` | Upload a build from your computer ([details](/upload-build)) |
| `devices_list` | List supported cloud device and OS permutations |
| `environments_list` | List all environments for your organization |
Filter `apps_list` with `platform: "chrome-extension"` to discover extension
apps and their UUIDs.
Use `devices_list` to choose a model and OS. Set `model` or `os_version` to
`"latest"` for explicit rolling resolution, or omit `device` to use the newest
supported default for both.
Both `flows_run` and `suites_run` accept `dependencies` as a JSON array string
for web runs:
```json theme={null}
["extension-app-uuid", {
"app_id": "another-extension-app-uuid",
"app_build_id": "exact-extension-build-uuid"
}]
```
Omit `dependencies` to use the web app's attached defaults. Pass `"[]"` to
explicitly run without extensions. An app UUID uses that extension's active
build; an object with `app_build_id` pins the exact build. Extension-enabled
runs use Chromium, and one suite run uses the same extension loadout for every
flow in its shared browser session.
Run responses from `flows_run`, `suites_run`, and `labels_run` include a **Batch** link for viewing or sharing all runs launched together.
To run [code-managed tests](/code-managed-flows) as they are on a branch, pass `repo_full_name` and `branch_name` to `labels_run` or `suites_run`. The run uses that branch's `.autosana/` files, including tests that exist only on the branch, and the newest build uploaded for the branch. A suite that exists only on the branch has no ID yet, so pass `suite_key` (its folder path under `.autosana/`) to `suites_run` instead of `suite_id`. Without a branch, runs use the default branch's tests.
### Workspaces
| Tool | Description |
| - | - |
| `workspaces_list` | List the workspaces in your company and show which one is active |
| `workspaces_switch` | Switch the active workspace to another one in your company |
Every other tool acts in your active workspace. These two tools need a browser sign-in connection. An API key always acts in the workspace it was created in.
Switching applies to your whole Autosana login, not just one agent. The dashboard and any other agent signed in as you switch too.
### Results & Debugging
| Tool | Description |
| - | - |
| `list_suite_runs` | List recent suite run results with optional date filters |
| `list_flow_runs` | List recent flow run results with optional date filters |
| `list_hook_runs` | List recent hook run results with optional date filters |
| `read_suite_run` | Read detailed suite run info showing which flows passed/failed |
| `read_flow_run` | Read run details, actions, review, hooks, and extension build provenance |
| `read_hook_run` | Read detailed hook run info including status and output |
`read_flow_run` reports the exact dependency build IDs and available extension
metadata such as app/build IDs, app name, platform, source, branch, commit, and
upload time.
### Documentation
| Tool | Description |
| - | - |
| `search_autosana_docs` | Search Autosana documentation for information |
***
## Example Usage
#### Generate E2E tests from your codebase
```
Crawl my repo, come up with an E2E testing strategy, and create flows in Autosana
```
#### Increase test coverage from a PR
```
Look at this PR and get us to 80% E2E coverage by writing flows in Autosana
```
#### Debug failing tests
```
My login suite is failing. Help me figure out what's wrong and fix it.
```
#### Create hooks for backend setup
```
Create a setup hook that resets the test database before each flow runs
```
#### Check test run history
```
Show me the last 10 runs of my checkout flow and their success rates
```
#### Identify coverage gaps
```
Look at my Autosana coverage and see where we are missing tests
```
***
## Resources
* [MCP Protocol Specification](https://modelcontextprotocol.io)
* [Claude Code](https://docs.anthropic.com/en/docs/claude-code/mcp)
* [Claude Desktop](https://modelcontextprotocol.io/quickstart/user)
* [Cursor](https://docs.cursor.com/context/model-context-protocol)
* [Gemini CLI](https://google-gemini.github.io/gemini-cli/docs/tools/mcp-server.html)
* [Codex](https://developers.openai.com/codex/mcp/)
* [Dots](/dots)
* [Grok Bot](/grok-bot)
* [Devin](/devin)
* [Hermes Agent](/hermes-agent)
* [OpenClaw](https://docs.openclaw.ai/gateway/configuration)
* [Report an Issue](https://github.com/autosana/autosana/issues)
# Multi-Device Testing
Source: https://docs.autosana.ai/multi-device-testing
Test workflows that span two mobile devices with one Autosana agent.
Multi-device testing lets one agent control two phones in the same flow. It is
useful for messaging, sharing, collaboration, and other workflows where an
action on one device affects another.
Multi-device testing is an advanced feature. It currently supports up to two
cloud mobile devices.
## How it works
* The agent sees both screens and takes one action at a time on **Device 1** or
**Device 2**.
* Both devices run the same app build and platform, but can use different models
and OS versions.
* Both devices must be virtual or both must be physical.
## Set up a flow
In the flow editor, expand **Advanced options** and set **Number of devices** to
**2**. Refer to **Device 1** and **Device 2** in the instructions:
```text theme={null}
On Device 1, switch the app to light mode. On Device 2, switch it to dark mode.
Verify that each device keeps its selected theme.
```
See [Flows](/flows) for dashboard setup and [Code-managed flows](/code-managed-files)
for the `.autosana` schema. When adding a flow to a suite, all flows and Auth
Instructions must use the same device count. See [Suites](/suites).
## Run the flow
Choose a model and OS for each device when starting the run. The same ordered
device selection is supported across Autosana:
* [Runs API](/api-runs)
* MCP run tools, using the same device order as the [Runs API](/api-runs)
* [Automations](/automations)
* [GitHub Actions](/ci-cd-integration)
See [Real device testing](/real-device-testing) for physical-device availability
and requirements.
## Review results
The run viewer shows both screens and identifies the device used for each
action. Logs, performance data, network data, recordings, and other artifacts
are separated by device where available.
API results return ordered device metadata and a `device_index` for each
device-specific action. See the [Runs API reference](/api-runs) for the response
schema.
## Current limitations
* A flow can use at most two devices.
* Both devices use one app build and platform.
* Virtual and physical devices cannot be mixed in one run.
* State caching is disabled for multi-device runs.
# Network Allowlist
Source: https://docs.autosana.ai/network-allowlist
Source IPs to allowlist for firewalls in front of your apps or APIs
If your test target or the APIs your [hooks](/hooks) call sit behind a firewall, allowlist Autosana's source IPs. The current list is at `GET /api/v1/network/allowlist`.
If your web app is not reachable from public Autosana runner IPs because it lives inside a Tailscale tailnet, use the [Tailscale integration](/private-network) instead of IP allowlisting.
```bash theme={null}
curl -fsSL https://backend.autosana.ai/api/v1/network/allowlist \
-H "X-API-Key: YOUR_API_KEY"
```
Grab `YOUR_API_KEY` from [Settings → Integrations](https://autosana.ai/settings?tab=integrations).
```json theme={null}
{
"hook_runners": [""],
"flow_runners": {
"ios": ["", "..."],
"android": ["", "..."],
"web": ["", "..."],
"virtual_runners": {
"ios_simulators": ["", "..."],
"android_emulators": ["", "..."],
"web_browsers": ["", "..."]
},
"real_device_runners": ["", "..."]
},
"documentation_url": "https://docs.autosana.ai/network-allowlist"
}
```
* **`hook_runners`** — source IPs for HTTP traffic from [hook scripts](/hooks). Stable; rarely changes. Allowlist on firewalls protecting APIs your hooks call.
* **`flow_runners.virtual_runners`** — source IPs for iOS simulators, Android emulators, and web browsers hosted by Autosana.
* **`flow_runners.real_device_runners`** — shared source CIDRs for iOS and Android real devices. Allowlist these ranges when running on real devices.
Runner ranges can change as Autosana scales or its infrastructure providers update their networks.
## Webhooks
Run-result [webhooks](/webhooks), for every platform, are sent from the `flow_runners.virtual_runners.web_browsers` IPs. Allowlist those on the endpoint that receives them. The `ios_simulators` and `android_emulators` IPs never send webhooks.
The **Test** button on a webhook in **Settings > Integrations > Webhooks** sends from Autosana's API, whose IPs are not in this list. If your endpoint allowlists only the runner IPs, test webhooks can be blocked while real deliveries arrive.
## Staying in sync
We don't notify when IPs change — the endpoint is the source of truth. Two patterns:
1. **Poll from firewall-as-code** (Terraform, Cloudflare Worker, CI cron). The endpoint sends `Cache-Control: max-age=300`, so daily polling is cheap.
2. **Manual refresh.** Pull once a month, paste into your firewall.
If hooks start returning 401/403 or flows hit connection errors against your test target, your allowlist is likely stale — pull the endpoint and diff.
If run-result webhooks stop arriving at a firewalled endpoint, do the same.
# Network Traffic
Source: https://docs.autosana.ai/network-traffic
Inspect the HTTP(S) requests your app or site made during a test run
Autosana captures the HTTP(S) requests made during a test run. After a run completes, a **Network** section appears on the run detail page listing each request with its method, URL, status, type, timing, and size — so you can confirm an API call fired, inspect a failing request, or check what a screen loaded.
## What's captured
Full request list for the page under test: `GET`/`POST`/etc., URL, status, resource type (document, xhr, script, image, …), duration, and response size. Failed requests (DNS, abort, CORS) appear with `status: 0` and an `error` string.
HTTP(S) requests made by the iOS app during the test, including method, URL, status, duration, and response size.
iOS capture covers apps that use the system's trusted certificates. Apps that **pin** their TLS certificate reject interception and won't appear in the list. Request **metadata** is captured (method, URL, status, timing, size) — not request or response bodies.
## Fields
Each captured request includes:
| Field | Description |
| - | - |
| `method` | HTTP method (`GET`, `POST`, …) |
| `url` | Full request URL |
| `status` | HTTP status code; `0` for a failed request |
| `resource_type` | Request category (document/xhr/script/…) — web only |
| `duration_ms` | Time from request start to response end |
| `response_size` | Response body size in bytes |
| `error` | Present only on failed requests |
If a run makes a very large number of requests, the oldest are dropped and the artifact is flagged as truncated.
## Accessing captured traffic
* **Run page** — open a completed run and scroll to the **Network** section.
* **API** — fetch a run and read `network_log_url` (JSONL, one request per line). See [Runs API](/api-runs).
## Reading network logs in a flow
To assert on traffic mid-flow, use the **Read Network Logs** action — e.g. *"Read the network logs and verify the `POST` to `/api/login` returned 200."* See [Flows](/flows).
# Notifications
Source: https://docs.autosana.ai/notifications
Configure notifications for flow results
Get notified when flows complete so you don't have to constantly check the dashboard.
## Email
1. Go to **[Settings](https://autosana.ai/settings)** → **Notifications**
2. Toggle **Email Notifications** on
3. Add email addresses to receive notifications
**Not receiving emails?** Check your spam/junk folder, verify the address is in the recipient list, and add `notifications@autosana.ai` to contacts.
## Slack
Get real-time notifications in a Slack channel. See [Slack Integration](/slack-integration) for setup.
# Performance Monitoring
Source: https://docs.autosana.ai/performance-monitoring
Track memory, CPU, and rendering metrics during test runs
Autosana automatically collects performance metrics during test runs across all platforms. After a run completes, a **Performance** section appears on the run detail page with charts and diagnostics.
## What's Collected
| Metric | Description |
| - | - |
| **Memory** | App memory usage, sampled every \~2 seconds |
| **CPU** | App CPU usage over time |
| **Frame rendering** | Total frames, slow frames, stutter rate, and render time percentiles |
| **Device RAM** | Total device RAM for context (e.g., "350 / 2048 MB") |
| Metric | Description |
| - | - |
| **Memory** | App memory usage, sampled every \~2 seconds |
| **CPU** | App CPU usage over time |
iOS metrics are measured from the simulator. Absolute values differ from a real device — use them for **comparing across builds** and detecting regressions.
iOS performance monitoring is currently **not available for local CLI runs**. It works for cloud-hosted runs.
| Metric | Description |
| - | - |
| **Memory** | JS heap usage over time |
| **CPU** | Browser CPU usage over time |
| **Core Web Vitals** | Largest Contentful Paint, Layout Shift, Interaction Delay |
| **Page timing** | Server Response, Page Ready, Fully Loaded |
| **Runtime health** | Blocking tasks, total blocked time, page complexity |
Web Vitals thresholds follow [Google's recommendations](https://web.dev/articles/vitals):
* **Good** (green): LCP ≤ 2.5s, CLS ≤ 0.1, INP ≤ 200ms
* **Needs work** (amber): LCP ≤ 4s, CLS ≤ 0.25, INP ≤ 500ms
* **Poor** (red): above those thresholds
# Tailscale
Source: https://docs.autosana.ai/private-network
Route Autosana test traffic through your Tailscale network
Connect a Tailscale tailnet so Autosana web runners can reach apps that aren't publicly accessible, including staging behind an internal load balancer, split-DNS hostnames, and services behind subnet routers.
The Tailscale integration is for web flows. If your firewall only needs static source IPs, use the [Network Allowlist](/network-allowlist) instead. It is simpler and also covers hooks and mobile runners.
## How it works
The private network is connected once per workspace and applies to every environment in it, including environments created later.
1. **Preview**: a workspace admin enters a temporary Tailscale API access token, the exact Tailnet ID, and structured targets. Autosana reads the current tailnet configuration and shows the exact policy additions it proposes. Preview does not change Tailscale.
2. **Apply**: after a separate confirmation, Autosana adds the reviewed grants and creates a generated runner identity. The temporary token is discarded and is never stored.
3. **Check routing**: Autosana joins with a short-lived key and checks Tailscale DNS and route selection. This does not fetch your application or return application response content.
4. **Run**: each web flow run joins the tailnet only for the browser session and leaves when the run ends.
The dashboard shows **Autosana-managed Tailscale access**: the additions Autosana owns, not your tailnet's total effective access. Other policy you manage can grant additional access.
## Choose a target type
Enter each target as a logical hostname or URL, optional route CIDRs, protocol, and port. The preview shows the canonical origin that will be stored.
Logical targets derive TCP and the destination port from the hostname or URL. Protocol and port are entered separately only for route-only bindings.
### App Connector hostname
For a hostname already covered by a Tailscale App Connector, Autosana dynamically reuses that connector. Preview shows its name, configured domains and routes, tags, TCP ports, and device availability. DNS address changes behind the hostname do not require rewriting the Autosana-managed grant.
### Explicit private route
For a private hostname that is not covered by an App Connector, provide the exact application CIDR or CIDRs routed through your tailnet. Autosana preserves the logical hostname for test traffic while authorizing only those explicit routed networks at the selected port.
Split-DNS targets also need their private resolvers to be reachable. Preview lists every exact resolver dependency separately as an IPv4 `/32` or IPv6 `/128`, with TCP and UDP port 53, the matching DNS suffix, and its route source. Autosana does not authorize an entire DNS subnet just because one resolver is inside it.
### Route-only binding
Leave the logical hostname empty only when you want a route-only binding. You must provide at least one route CIDR, a protocol, and a port.
## Set up an App Connector
Complete this setup in Tailscale before Preview when the application hostname should route through a stable connector, such as a public staging site that allowlists only the connector's public IP.
You need a Tailscale Owner, Admin, or Network admin and a Linux connector device that is already joined to the tailnet, has IP forwarding enabled, and has a public IP address.
1. Choose a dedicated connector tag, such as `tag:app-connector-autosana`. This tag identifies your connector device; it is separate from the generated `tag:autosana-…` runner tag that Autosana creates during Apply. Treat `tag:autosana` and the `tag:autosana-*` prefix as Autosana-managed names.
2. In Tailscale **Access controls**, merge the connector tag owner and route auto-approvals into your existing policy. Do not create duplicate top-level sections.
```json theme={null}
{
"tagOwners": {
"tag:app-connector-autosana": ["autogroup:admin"]
},
"autoApprovers": {
"routes": {
"0.0.0.0/0": ["tag:app-connector-autosana"],
"::/0": ["tag:app-connector-autosana"]
}
}
}
```
These default-route entries let any device carrying the connector tag automatically approve the routes it advertises. Keep `tagOwners` tightly restricted and assign the tag only to the intended connector devices. The approvals do not grant Autosana access by themselves; Autosana adds separate runner-specific grants during Apply.
3. On the Linux connector device, advertise it with the same tag:
```shell theme={null}
sudo tailscale up --advertise-connector --advertise-tags=tag:app-connector-autosana
```
4. In Tailscale, open **Apps**, select **Add an app**, and enter:
* **Name**: any unique admin-console name, such as `autosana-app`.
* **Target**: **Custom**, unless the application is one of Tailscale's preset apps.
* **Domains**: the exact hostname or hostnames Autosana will test, such as `staging.example.com`. Do not include `https://`, paths, query strings, or ports. Use a wildcard only when every matching subdomain should use this connector.
* **Connectors**: select `tag:app-connector-autosana`, or the dedicated connector tag you chose in step 1.
5. Save the app and confirm Tailscale shows the connector as active.
6. In Autosana, enter the same application hostname as the logical target. Preview must classify it as **App Connector hostname** and show the expected connector name, tag, domains, and an online connector device before you Apply.
Autosana adds only the reviewed runner-specific grants and its generated runner tag. It does not provision the Linux connector, configure its domains, or change the connector tag's route auto-approvals. See Tailscale's [App Connector setup guide](https://tailscale.com/docs/features/app-connectors/how-to/setup) for device and policy details.
## Preview and apply
1. Create a temporary Tailscale API access token.
2. In Autosana, go to **Settings → Integrations → Autosana-managed Tailscale access** and select **Connect**.
3. Enter the Tailnet ID shown on Tailscale's General page and add the structured targets.
4. Select **Preview changes**.
5. Review the exact managed grants, App Connector details, application CIDRs, private-DNS dependencies, and warnings.
6. If Autosana finds legacy policy candidates, choose each one you want removed. They are never selected or deleted automatically.
7. Select **Review apply**, then use the separate **Apply changes** confirmation.
8. Revoke the temporary API token in Tailscale after Apply completes.
If policy, routes, DNS, or the current installation changes after Preview, Apply fails safely. Create a fresh preview instead of widening access or silently using the legacy setup.
Use **Upgrade/Reconnect** on an older connection or **Reconnect** on a current connection to preview changed targets without disconnecting first. The existing generation stays active until the replacement is published.
## Stored credential authority
Autosana stores one encrypted Tailscale OAuth credential with the `auth_keys` scope and its generated runner tag. The tag limits which tag newly created auth keys can receive.
The `auth_keys` scope is not mint-only: it can also read and delete machine auth keys through Tailscale's key API. It cannot administer tailnet policy, DNS, OAuth clients, or devices. Runtime keys are short-lived, single-use, ephemeral, and tagged.
## Disconnect and cleanup
Disconnect is tokenless. Before sending the request, Autosana asks you to confirm that it will immediately disable the local binding while Tailscale policy and the OAuth client may remain. Without temporary administrative authority, those remote resources may not be removable at that moment, and completing cleanup later can require a new full admin token.
If remote cleanup is deferred, the dashboard reports **Remote cleanup required** and lists backend-approved non-secret identifiers, resource status, and cleanup counts for follow-up. A later token-bearing setup can reconcile known resources, or an administrator can use those identifiers to remove them in Tailscale. A successful local disconnect does not promise that every remote resource was deleted.
## Troubleshooting routing
* **No primary route covers an application CIDR** — advertise and approve the exact subnet route in Tailscale, then run **Check routing** again.
* **A private DNS resolver is unrouted** — make the exact resolver `/32` or `/128` reachable through an approved primary route. Do not broaden the application CIDRs to compensate.
* **An App Connector is unavailable** — confirm that a uniquely matching connector is configured, tagged, online, and advertising the expected domain or route.
* **Routing configured but a test still fails** — routing readiness does not connect to the destination. Run an Autosana test to exercise the application data plane, then troubleshoot the application, TLS, or authentication separately.
* **Preview or Apply reports a conflict** — the plan became stale or another private-network operation is active. Wait for the current operation, then create a new preview.
# Quickstart
Source: https://docs.autosana.ai/quickstart
## Prerequisites
* An Autosana account ([sign up](https://autosana.ai/signup))
* A mobile app or website that you want to test (duh)
## When You Sign Up
A short setup walks you through it: name your workspace, then let your coding agent set things up or do it yourself and run your first flow.
Afterwards, the **Get started** card on your [home page](https://autosana.ai/overview) shows what's left. Click **Restart setup** on that card to go through it again.
## Getting Started
### 1. Add the MCP Server
Connect Autosana with your existing account using browser sign-in. Grok Bot and Devin currently use the API-key setup described below.
Run in your project directory:
```bash theme={null} theme={null}
claude mcp add --transport http autosana https://mcp.autosana.ai/mcp
```
Open Claude Code, run `/mcp`, select Autosana, and follow the browser sign-in prompt.
In Claude, open **Customize → Connectors → Add custom connector**. Enter **Autosana** as the name and this server URL:
```text theme={null} theme={null}
https://mcp.autosana.ai/mcp
```
Choose **Connect** and sign in through your browser. Your Claude account must support custom connectors; if the option is disabled, ask your workspace administrator or use another supported client.
Add Autosana to `~/.cursor/mcp.json`, preserving any existing servers:
```json theme={null} theme={null}
{
"mcpServers": {
"autosana": {
"url": "https://mcp.autosana.ai/mcp"
}
}
}
```
Open Cursor's MCP settings and click **Authenticate** beside Autosana.
Run:
```bash theme={null} theme={null}
gemini mcp add --transport http autosana https://mcp.autosana.ai/mcp
```
In Gemini CLI, run `/mcp auth autosana` and follow the browser sign-in prompt.
Run:
```bash theme={null} theme={null}
codex mcp add autosana --url https://mcp.autosana.ai/mcp
codex mcp login autosana
```
If you configure Codex manually, add the following to `~/.codex/config.toml`, then run `codex mcp login autosana`:
```toml theme={null} theme={null}
[mcp_servers.autosana]
url = "https://mcp.autosana.ai/mcp"
```
A dot uses plugins installed in ChatGPT. In ChatGPT on the web, open **Settings → Security and login** and turn on **Developer mode** (Plus, Pro, Business, Enterprise, or Edu). Then open **ChatGPT Plugins**, select **+**, and create an app named **Autosana** with this server URL:
```text theme={null} theme={null}
https://mcp.autosana.ai/mcp
```
Choose **OAuth** and approve access in your browser. Do not use an API key. In a chat, open **+**, choose **Developer mode**, and select **Autosana**. This connects the app in ChatGPT. It does not add a separate control inside the dot.
Full walkthrough: [Dots](/dots).
Grok Bot has no Settings form for a custom MCP URL, and Autosana is not in the marketplace. Open **[Settings → Integrations → Grok Bot](https://autosana.ai/settings?tab=integrations\&integration=grok-bot)** to create a key, then send this in a Grok Bot chat:
```
Add a custom MCP called Autosana at https://mcp.autosana.ai/mcp with header x-api-key:
```
Approve the confirm card. Next message: `@` Autosana and ask it to list or run a flow. Adding MCP in Cursor does not set up Grok Bot.
Full walkthrough: [Grok Bot](/grok-bot).
In Devin, open **Customize → MCPs → Add a custom MCP**. Choose **HTTP**, not STDIO. Open **[Settings → Integrations → Devin](https://autosana.ai/settings?tab=integrations\&integration=devin)** to create a key, then fill in:
```
Server name: Autosana
Transport: HTTP
Server URL: https://mcp.autosana.ai/mcp
Authentication: Auth Header
Header key: x-api-key
Header value:
```
Save, then use **Test listing tools**. Adding MCP in Cursor does not set up Devin.
Full walkthrough: [Devin](/devin).
Add this under `mcp_servers` in `~/.hermes/config.yaml` (keep any other servers). Hermes opens a browser the first time it connects:
```yaml theme={null} theme={null}
mcp_servers:
autosana:
url: "https://mcp.autosana.ai/mcp"
auth: oauth
```
Or run `hermes mcp add --url https://mcp.autosana.ai/mcp --auth oauth autosana`. Do not use `--auth header` — that writes `Authorization: Bearer`. Adding MCP in Cursor does not set up Hermes Agent.
Full walkthrough: [Hermes Agent](/hermes-agent).
Run:
```bash theme={null} theme={null}
openclaw mcp set autosana '{"url":"https://mcp.autosana.ai/mcp","transport":"streamable-http","auth":"oauth"}'
openclaw mcp login autosana
```
Follow the browser sign-in prompt.
In the browser, sign in to Autosana, check the agent name and company, and click **Connect**. The agent can work in any workspace you can access in that company. Return to your agent and ask it to list your Autosana flows.
If the authorization session ends before you approve, start a fresh connection from your agent. Do not reuse the old browser tab.
Existing API-key connections continue to work. To switch one to browser sign-in, remove its `x-api-key` header and reconnect. Keep the key until the new connection works; revoke it only if no other integrations use it.
Note: We recommend adding the MCP server at your project root so your agent has context across your frontend and backend.
You may need to restart your agent for the changes to take effect.
### 2. Onboard
Tell your agent:
```
Onboard me to Autosana
```
It'll handle CI/CD setup, test planning and creation, and more.
### Step 1: Add Your App or Website
**CI/CD Pipeline Integration (Recommended)**
1. Create an API key for your CI pipeline with [`autosana keys create`](/install-cli) or in **[Settings → API Keys](https://autosana.ai/settings?tab=api-keys)**
2. Follow the setup guide to integrate with your pipeline (GitHub Actions, Fastlane, Expo EAS, etc.)
3. Your first build will automatically create the app in Autosana
[Learn more about our CI/CD Integration →](/ci-cd-integration)
**Manual Upload**
Get your build file ready (iOS: `.zip` containing your .app bundle, Android: `.apk`). [Learn how to build your app →](/app-build-guide)
1. Navigate to the **[Apps](https://autosana.ai/apps)** page from the sidebar
2. Click **Create New App**
3. Fill in app name, bundle ID, platform, and optionally an environment
4. Drag and drop your build file, then click **Upload**
**CI/CD Pipeline Integration (Recommended)**
1. Create an API key for your CI pipeline with [`autosana keys create`](/install-cli) or in **[Settings → API Keys](https://autosana.ai/settings?tab=api-keys)**
2. Follow the setup guide to integrate with your pipeline (Vercel, Netlify, etc.)
3. Your preview URL will automatically be registered with Autosana on every deploy
[Learn more about our CI/CD Integration →](/ci-cd-integration#web-setup)
**Manual Setup**
1. Navigate to the **[Apps](https://autosana.ai/apps)** page from the sidebar
2. Click **Create New App**
3. Enter your app name, URL, select **Web** as the platform, and click **Create**
### Step 2: Create Your First Flow
A **Flow** is a test written in natural language that describes what you want to test.
1. Navigate to the **[Flows](https://autosana.ai/flows)** page
2. Click **Create Flow**
3. Write your flow instructions in natural language:
```
Open the app
Tap on the login button
Enter "test@example.com" in the email field
Enter "password123" in the password field
Tap the submit button
Verify that the home screen appears
```
4. Click **Create Flow**
See [Writing Effective Flow Instructions](/writing-effective-flow-instructions) for our style guide on writing effective flows.
### Step 3: Run Your First Flow
1. Find your flow in the Flows table
2. Click the **Play** button (▶️) next to your flow
3. Select your app from the dropdown
4. Click **Run Flow**
Your flow will now execute on our cloud infrastructure. You'll see real-time status updates as the flow progresses.
### Step 4: View Results
Once your flow completes:
1. Click on the status badge (Passed/Failed) in the Flows table
2. View detailed results including:
* Screenshots at each step
* Actions performed by the agent
* Any errors encountered
On the **Runs** page, open a batch's **Preview** and select **View details** to see its combined results. The address bar updates to a shareable batch URL. Closing the dialog restores your previous URL and filters.
### Next Steps
Group related flows and run them together
Schedule flows to run on builds, daily, or weekly
Upload builds automatically from your pipeline
# Mobile Testing
Source: https://docs.autosana.ai/real-device-testing
Test mobile apps on virtual and physical iOS and Android devices.
Run natural language flows against iOS and Android apps in the cloud. Use iOS Simulators and Android Emulators for fast coverage, or run the same flows on physical devices when you need real hardware.
## Available devices and OS versions
Every mobile run can target a specific device model and OS version. In the run dialog, choose a **Target**, then select a **Device** and **OS version**. If you do not change the selection, Autosana uses the recommended latest combination.
| Target | Current coverage |
| - | - |
| Android Emulator | Pixel 6 through Pixel 10 families on Android 14–17 |
| iOS Simulator | iPhone and iPad models on iOS 17–26 |
| Real device | Available iOS and Android phones and tablets; the exact catalog changes with device availability |
The device picker shows the exact model and OS combinations available for your app. iOS Simulator versions marked **Fast launch** use an optimized startup path; every listed combination is supported.
For API runs, use [List Devices](/api-runs#list-devices) to retrieve the current catalog, then pass one of the returned combinations in the run's `device` object.
## Physical Device Testing
Run any mobile flow on real hardware — actual iPhones, Pixels, and Galaxy devices — instead of an emulator or simulator. Same flows, same agent, real devices.
### Build requirements
What you can run depends on the build you upload:
| Platform | Build | Runs on |
| - | - | - |
| iOS | `.app` / `.app.zip` | Simulator only |
| iOS | `.ipa` | Real devices only |
| Android | `.apk` / `.aab` | Emulators **and** real devices |
For iOS, upload an `.ipa` built for physical devices. Android builds work on both targets with no changes.
### Running on a real device
Click **Run** on a flow (or suite), set **Target** to **Real device**, and pick a device model and OS version. The picker lists the live device catalog — newest models first — and defaults to the newest available device.
If your app's active build is an `.ipa`, the dialog opens on **Real device** automatically.
**Real device** stays visible but disabled when your organization has no real-device access or your app has no compatible build. Hover over or keyboard-focus the option to see why. For access, contact [founders@autosana.ai](mailto:founders@autosana.ai); for an iOS simulator-only app, upload an `.ipa`.
Pass one `device` object in the body — see [API → Run Flows](/api-runs).
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/v1/flows/run \
-H "X-API-Key: $AUTOSANA_KEY" \
-H "Content-Type: application/json" \
-d '{
"bundle_id": "com.example.app",
"platform": "ios",
"flow_ids": ["660e8400-e29b-41d4-a716-446655440001"],
"device": {
"physical": true,
"model": "iPhone 16 Pro",
"os_version": "18"
}
}'
```
Set model and OS to `"latest"`, or omit them, to run on the best available
physical device for the platform.
### Picking a device
The `device` object narrows which device the run lands on:
* `model` — e.g. `"iPhone 16 Pro"`, `"Pixel 9"`, or `"latest"`. Matches the names shown in the dashboard picker.
* `physical` — use `true` for real hardware; it defaults to `false`.
* `os_version` — `"18"` matches any 18.x; `"18.3.1"` matches only that version; `"latest"` keeps it rolling.
If no device matches your criteria, the run fails immediately with a clear error — pick a different model or OS version. Device availability is checked at run time; a model that's momentarily busy is still attempted.
### What you get
Real-device runs flow through the same pipeline as every other run:
* **Full video recording** of the session, with per-action replay markers
* **Screenshots and annotations** for every agent action
* **Device identity on the run page** — the exact model and OS the run executed on (e.g. `Pixel 9 · Android 16 · Real device`)
* Suites share one device session across their flows, so back-to-back flows start fast
Real-device runs take \~1–2 minutes longer to start than emulator runs while a device is acquired, and iOS sessions take a few extra minutes to finalize their video after the run completes.
### Native in-app purchases
Testing native Apple or Google purchase flows on private devices requires an enterprise contract with Autosana and coordinated onboarding. [Contact us](mailto:founders@autosana.ai) before preparing a build so we can confirm feasibility, device availability, and the signing and test-account setup. Selecting **Real device** alone does not configure native purchase testing.
See [Testing In-App Purchases](/guides-in-app-purchases#native-in-app-purchases-on-private-devices) for onboarding details and the self-service RevenueCat Test Store workflow.
### Push notifications
End-to-end push-notification testing on physical iOS and Android devices uses private devices that preserve the app's original signing and notification entitlements. This requires an enterprise contract. [Contact us](mailto:founders@autosana.ai) for availability and setup.
For self-service APNs and FCM testing, see [Testing Push Notifications](/guides-push-notifications) for the iOS Simulator and Android Emulator workflow.
# Slack Notifications
Source: https://docs.autosana.ai/slack-integration
Receive run results and review Jira or Linear tickets in Slack
Connect Slack to receive flow results, grouped issue summaries, and links back
to Autosana.
## Connect Slack
1. Go to **[Settings > Integrations > Slack](https://autosana.ai/settings?tab=integrations\&integration=slack)**
2. Click **Connect to Slack** and authorize Autosana
3. Select the organization default notification channel
Autosana requests `chat:write` and `chat:write.public` to send and update
messages, plus `channels:read` and `groups:read` to list available notification
channels.
## Route each app to its own channel
Apps use the organization default unless an organization owner or admin
overrides them:
1. Go to **[Settings > Integrations > Slack](https://autosana.ai/settings?tab=integrations\&integration=slack)**
2. Find **App-specific routing** in the Slack Notifications integration
3. Click **Add app route**
4. Choose the app and its dedicated Slack channel, then save
Only that app's run notifications go to the selected channel. Choose
**Organization default** to remove the override.
For private channels, invite the Autosana bot first so the channel appears in
the dropdown.
## Issue notifications
After an unsuccessful run, the Slack message first shows that issue analysis is
in progress. Autosana updates the same message when grouping finishes, showing
eligible issue groups in severity order.
Messages omit dismissed groups and groups marked **Don't show again**. If the
message would exceed Slack's block limit, use the included dashboard link to
review the remaining issues in Autosana.
An issue with a current Jira or Linear ticket shows its external key and link.
An eligible issue without a current ticket shows **Create ticket**.
## Link your Autosana account
Slack must know which Autosana member is approving a ticket. The first time you
select **Create ticket**, follow **Link Autosana account**, sign in to Autosana,
and confirm the link. The link is single-use, expires after ten minutes, and
only works for an active member of the same Autosana workspace as the Slack
installation.
If your Autosana membership is removed or Slack is reinstalled, link your
account again.
## Review a ticket in Slack
For a linked member, **Create ticket** opens a Slack review modal with the same
provider, destination, issue type, title, and description choices available in
the Autosana dashboard.
Submitting the modal acknowledges the request immediately and creates the
ticket in the background. When creation succeeds, Autosana updates the original
Slack notification with the Jira or Linear key and link.
If provider options are not available quickly, the modal directs you to the
dashboard rather than waiting. Recoverable failures preserve a dashboard retry
path. If the provider result is unknown, check Jira or Linear before retrying so
you do not create a duplicate.
Ticket creation always requires a person to review and submit the Slack modal.
The workspace's automatic-provider setting is a separate issue-group workflow.
# Suites
Source: https://docs.autosana.ai/suites
Group and manage related flows with test suites
Suites are collections of related flows that help you organize your flows and run them together efficiently. Think of suites as folders that group flows by feature, user journey, or any other logical grouping.
## What is a Suite?
A **Suite** is a container that groups multiple flows together. Suites provide:
* **Organization**: Keep related flows together (e.g., "Login Flows", "Checkout Flows")
* **Batch Execution**: Run all flows in a suite with one click
* **Shared Authentication**: Set up authentication once for all flows in the suite
## Creating a Suite
### Step 1: Click "Create Suite"
Navigate to the Flows page and click the **folder+** icon or **Create Suite** button.
### Step 2: Enter Suite Details
**Required:**
* **Name**: A descriptive name for your suite (e.g., "Authentication Flows")
**Optional:**
* **Description**: Additional context about what this suite tests
* **Suite Context**: Extra context the AI agent should keep in mind for every flow in the suite
* **Auth Instructions**: Shared authentication that runs once before all flows
* **Run flows in parallel**: Give each flow its own session and run them concurrently
### Step 3: Save
Click **Create Suite** to save. Your new suite appears in the Flows page.
## Suite Context
Suite Context is plain-English background that's injected into the AI agent's prompt for **every** flow in the suite — alongside any app-level context. Use it for product knowledge, conventions, or constraints that apply to the entire suite (e.g., "this suite tests the membership signup journey only — ignore marketing banners and cookie prompts").
Unlike Auth Instructions, Suite Context isn't executed as a flow. It just gives the agent shared situational awareness so each flow doesn't have to repeat the same background.
**Example Suite Context:**
```
All flows in this suite operate on the staging environment.
The user is expected to be onboarded; if you see the welcome tour, dismiss it.
Treat any "Beta" badge as expected — don't fail because of it.
```
Use Suite Context for *background* the agent needs ("this app", "this area"), and Auth Instructions for *actions* the agent must take before every flow ("log in as X").
## Auth Instructions (Setup Flow)
Auth Instructions are special setup instructions that run **once** at the beginning of the suite, before any flows execute. This is perfect for:
* Logging in with test credentials
* Accepting permissions or completing onboarding steps
* Navigating to a specific section of the app
**Example Auth Instructions:**
```
Login with test@example.com and password TestPass123
Tap "Sign In"
Wait for the home screen to load
```
### Benefits of Auth Instructions
* **Write once, use many times**: All flows in the suite start from an authenticated state
* **Faster execution**: Authentication happens only once, not before every flow
* **Easier maintenance**: Update login credentials in one place
## Suite Variables
Suite variables let you override environment variables for all flows in the suite. They take precedence over environment-level values, so you can customize configuration per suite without changing your environment settings.
**How to set them:**
1. Edit your suite
2. Expand the **Advanced** dropdown
3. Add key-value pairs (e.g., `TEST_EMAIL` = `suite-user@example.com`)
Flow-level variables can further override suite variables for individual flows. See [Variables](/variables#variable-precedence) for the full precedence hierarchy.
## Running Flows in Parallel
Expand **Advanced** while creating or editing a dashboard-managed suite, then enable
**Run flows in parallel**. Each flow runs in its own device or browser session.
Auth Instructions and suite setup/teardown hooks run independently in each session.
Parallel mode is useful when flows do not depend on state or variables produced by
earlier flows. Leave it off when flows must share a session or execute in order.
As you scale parallel testing,
give each session its own account or isolated data to avoid conflicts. See
[Making Suites, Parallelizable, Isolated, and Repeatable](/guides-parallelizable-isolated-tests)
for guidance.
API clients can optionally send `parallelize_flows` when launching a suite to
override its saved setting for that request. Explicit `true` or `false` takes
precedence; when the field is omitted or `null`, Autosana uses the suite's saved
**Run flows in parallel** setting.
## Adding Flows to a Suite
### Method 1: Create Flow in Suite
1. Expand your suite in the Flows page
2. Click **Create Flow** inside the suite
3. Write your flow instructions
4. The flow is automatically added to the suite
### Method 2: Add Existing Flows
1. Expand your suite
2. Click **Add Existing Flows**
3. Check the flows you want to include
4. Click **Add Flows**
### Method 3: Attach from Flow
1. Click the three dots (**⋯**) next to any flow
2. Select **Attach to Suite**
3. Check the suites you want to add the flow to
4. Click **Save**
## Managing Suite Contents
### Reordering Flows
Change the order flows run in:
1. Expand your suite
2. Drag and drop flows using the handle icon (⋮⋮)
3. Release to set the new order
Flows execute in the order they appear in the suite.
### Removing Flows from Suite
1. Expand the suite
2. Click the three dots (**⋯**) next to the flow
3. Select **Remove from Suite**
Removing a flow from a suite doesn't delete the flow—it just removes the relationship. The flow remains in your Flows list.
## Deleting a Suite
To delete a suite:
1. Find the suite in the Flows page
2. Click the trash icon on the suite card
3. Confirm deletion in the dialog
Deleting a suite does **not** delete the flows inside it. All flows are preserved and remain available in your workspace. Flows that belong to other suites stay attached to those suites.
## Running a Suite
### Running All Flows
1. Find your suite in the Flows page
2. Click **Run Suite**
3. Select your app
4. For a mobile app, choose a simulator or emulator model and OS version, or
select a real device
5. Click **Run Suite**
Virtual-device runs default to the latest supported permutation. **Fast launch**
choices usually start sooner; every listed permutation is supported.
Multi-device suites require every flow and Auth Instructions to use the same
device count. See [Multi-device testing](/multi-device-testing).
The suite executes as follows:
1. Auth Instructions run first (if configured)
2. Each flow runs, starting from an authenticated state
With **Run flows in parallel** enabled, each flow gets an independent session and its
own Auth Instructions and hooks. On the Runs page, those sessions appear together in
the same batch while remaining separate runs.
## Sharing Data Between Flows
When flows run sequentially in a suite, runtime variables from one flow automatically carry forward to the next. This includes values exported by hooks, variables set by the agent, and any per-flow or per-suite variable overrides. Parallel flows use independent sessions and cannot share runtime state with sibling flows.
Use the `${env:KEY}` syntax in later flow instructions to reference values from earlier flows.
`${env:KEY}` can only reference values that existed before the agent began running — it cannot reference values saved by the agent or exported by mid-flow hooks during execution. For those, use **Get Variable** instead.
**Flow 1:**
```
1. Complete the checkout process
2. Save the order number as a variable called order_number
```
**Flow 2 (runs after Flow 1):**
```
1. Navigate to order history
2. Search for ${env:order_number}
3. Verify the order appears
```
Hooks in later flows also have access to these variables via standard environment variable access (e.g., `os.environ.get("order_number")` in Python).
See [Variables](/variables#cross-flow-propagation-in-suites) for more details and examples of cross-flow data sharing.
## Best Practices
**Set Up Auth Instructions**
If your flows require authentication, use Auth Instructions instead of repeating login steps in every flow.
On the flow run viewer: **Shift+arrow** switches flows in a suite; **Shift+Ctrl+arrow** (Mac: **Shift+⌘+arrow**) switches suite runs.
## Next Steps
* [Automate suite execution with Automations →](/automations)
* [Integrate suites with our CI/CD →](/ci-cd-integration)
* [Define suites in your repo →](/code-managed-files#suites)
# Team
Source: https://docs.autosana.ai/team
Manage your organization's team members and roles
Manage your organization's team members and their access levels in **[Settings → Team](https://autosana.ai/settings?tab=team)**.
Only organization owners and admins can access team settings.
## Roles
| Role | Permissions |
| - | - |
| **Owner** | Full control: manage all settings, members, and integrations |
| **Admin** | Invite/remove members, manage integrations and agent settings |
| **Member** | Standard access to apps, flows, runs, and environments |
## Inviting Users
1. Go to Settings → Team
2. Click **Invite User**
3. Enter their name, email, and select a role
4. Click **Send Invitation**
The new user will receive an email invitation to join your organization.
You can also invite and remove members with the [Members API](/api-members).
## Renaming a workspace
Workspace owners and admins can rename the active workspace in **Settings → Workspace → Details**. Edit **Workspace**, then click **Save workspace name**. The new name appears in the workspace picker; existing tests, runs, and memberships stay with the workspace.
# Upload a Build
Source: https://docs.autosana.ai/upload-build
Send a build from your computer to Autosana
Test a build you just made on Autosana's cloud devices, without leaving your editor.
## With your coding agent
Tell your coding agent (Cursor, Claude Code, etc.):
```
Upload my latest simulator build to Autosana
```
It finds the build and uploads it for you.
## With the CLI
After `autosana login`:
```bash theme={null}
autosana builds upload MyApp.app
```
Works with an iOS Simulator `.app`, an iOS `.ipa` for [real devices](/real-device-testing), and an Android `.apk` or `.aab`. Not sure which build you need? See [Building Your App](/app-build-guide).
Xcode Archive builds are for real devices and won't run on the Simulator. Build for a Simulator destination instead.
## What an upload does
Either way, the upload becomes your app's active build, so runs that don't pin a build use it. It also starts the app's [CI/CD automations](/automations), which can start flow runs.
# Variables
Source: https://docs.autosana.ai/variables
Manage static configuration and dynamic runtime data across flows, hooks, and suites
Autosana has two types of variables that work together to make your flows flexible and data-driven:
1. **Environment Variables** — Static configuration (credentials, URLs, feature flags) set before runs
2. **Runtime Variables** — Dynamic values that accumulate during execution from hooks, flow/suite overrides, and the agent itself
## Environment Variables
Environment variables are key-value pairs managed in **Settings > Environments**. They store static configuration like API keys, test credentials, and URLs.
* Reference them in flow instructions and cURL hooks using `${env:VARIABLE_NAME}`
* Scripts access them via standard language methods (`os.environ.get()`, `process.env`, `$VAR`)
* Can be stored as secrets (encrypted, never displayed in plain text)
* The agent never sees the raw variable — only the resolved value substituted into instructions
See [Environments](/environments) for full details on creating and managing environment variables.
## Runtime Variables
Runtime variables are dynamic values that build up during a run. They come from these sources, and all feed into the same pool:
### Build Variables
Attached to a specific app build, either from your CI pipeline or the dashboard. These override environment variables for all flows that run against that build.
**When to use:** You need to pass context from your CI pipeline into flow instructions — PR numbers, branch names, feature flags, deployment URLs, or any other build-specific data.
**How to set them from CI (GitHub Action):**
```yaml theme={null}
- uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
platform: ios
build-path: app.zip
suite-ids: "uuid-here"
variables: 'PR_NUMBER=${{ github.event.pull_request.number }},BRANCH=${{ github.head_ref }}'
```
**How to set them from the API:**
Include a `variables` field in the request body of `/api/ci/confirm-upload`, `/api/ci/upload-web-build`, or `/api/v1/flows/run`. The value can be a string (`"KEY1=VALUE1,KEY2=VALUE2"`) or a JSON object (`{"KEY1": "VALUE1"}`).
**How to set them from the dashboard:**
1. Go to **Apps**
2. Expand the build history for an app
3. Click the **variables** icon on any build
4. Add key-value pairs and click **Save Variables**
You can also set variables when uploading a new build via the **Advanced** section in the upload dialog.
**How to use them in instructions:**
```
Navigate to the channel switcher. Enter pr-${env:PR_NUMBER} and apply.
```
The agent sees the resolved value (e.g., `pr-42`) and types it into the UI.
### Suite Variables
For code-managed suites, define `variables` in `_suite.yaml`. See [Variables in YAML](/code-managed-files#referencing-variables).
Suite variables override environment variables for all flows in the suite.
**When to use:** You want a different `TEST_EMAIL` or `API_URL` for a specific suite without changing the environment.
**For dashboard-managed suites:**
1. Edit your suite
2. Expand the **Advanced** dropdown
3. Add key-value pairs
### Flow Variables
For code-managed flows, define `variables` in `*.flow.yaml`. See [Variables in YAML](/code-managed-files#referencing-variables).
Flow variables override both suite variables and environment variables for that specific flow.
**When to use:** One flow in a suite needs a different value than the rest — for example, a flow that tests a different API endpoint.
**For dashboard-managed flows:**
1. Edit your flow
2. Expand the **Advanced** dropdown
3. Add key-value pairs
### Hook Exports
Hooks can export values by writing `KEY=VALUE` pairs to `/tmp/autosana.env`. These values become available to all subsequent hooks and flows in the suite.
```python theme={null}
# In a setup hook (Python)
with open('/tmp/autosana.env', 'w') as f:
f.write(f"AUTH_TOKEN={token}\n")
f.write(f"USER_ID={user_id}\n")
```
See [Sharing Data Between Hooks](/hooks#sharing-data-between-hooks) for full details and examples in all supported languages.
### Run Metadata
Set by Autosana for every run: `AUTOSANA_FLOW_RUN_ID`, `AUTOSANA_FLOW_ID`, `AUTOSANA_RUN_GROUP_ID`, `AUTOSANA_RUN_URL`, `AUTOSANA_RUN_GROUP_URL`, `AUTOSANA_BATCH_ID`, and `AUTOSANA_BATCH_URL`. They can't be overwritten.
**When to use:** Linking your own logs or error reports to an Autosana run from a hook. Don't use them in flow instructions, except `AUTOSANA_FLOW_ID`. The others change every run, so the flow can't be cached. See [Run Metadata](/hooks#run-metadata).
### Agent Variables
The agent can dynamically save and retrieve values during flow execution. This is useful when the data you need doesn't exist until the agent interacts with the app.
**Set Variable** — Tell the agent to save a value it sees on screen:
```
1. Navigate to the order confirmation page
2. Save the displayed order number as a variable called order_id
3. Navigate to the orders list
4. Verify that order_id appears in the list
```
**Get Variable** — Tell the agent to retrieve a previously saved value:
```
1. Get the variable auth_token
2. Verify the token is displayed in the debug panel
```
The agent automatically knows which variables are currently set, so you can reference them by name in your instructions without having to list what exists.
**Use cases:**
* Saving a generated order ID, username, or confirmation number from the screen
* Capturing dynamic data in one step and verifying it in a later step
* Storing values that subsequent hooks or flows need
Very large values are stored and available to hooks and subsequent flows (via `${env:KEY}`), but are not displayed to the agent when using Get Variable. Keep stored values concise — save specific data points (IDs, names, tokens) rather than large payloads.
## Using Variables in Instructions
Use the `${env:VARIABLE_NAME}` syntax to reference variables in flow instructions:
```
Log in with ${env:TEST_EMAIL} and ${env:TEST_PASSWORD}
Navigate to ${env:API_URL}/settings
```
This syntax resolves all variable types — environment variables, suite variables, flow variables, run metadata, and any runtime values (hook exports, agent-set values) that were set before the current flow started.
**`${env:KEY}` is resolved once, before the agent starts running.** This means it cannot reference values that the agent sets or that hooks export during the current flow. For those, use **Get Variable** in your instructions instead.
For **subsequent** flows in a suite, `${env:KEY}` works — runtime variables from earlier flows are available to later flows.
## Variable Precedence
When multiple sources define the same variable name, later sources override earlier ones:
| Priority | Source | Set by |
| - | - | - |
| Fixed | Run metadata | Autosana, can't be overwritten |
| Highest | Hook exports and agent-set variables | Hook scripts, agent using Set Variable |
| High | Flow variables | Flow editor, before the run |
| Medium | Suite variables | Suite editor, before the run |
| Low | Build variables | CI pipeline or dashboard, attached to the build |
| Lowest | Environment variables | Settings > Environments |
For example, if your environment defines `TEST_EMAIL=default@example.com` and your suite defines `TEST_EMAIL=suite@example.com`, the suite value wins for all flows in that suite.
If both a hook and the agent set the same variable name, whichever runs last wins — there's no fixed priority between them.
## Cross-Flow Propagation in Suites
When flows run together in a suite, runtime variables from one flow automatically carry forward to the next. This lets you chain data across flows:
**Flow 1** — Agent captures data:
```
1. Navigate to the product catalog
2. Save the name of the first product as a variable called product_name
3. Add it to the cart
```
**Flow 2** — Uses the data from Flow 1 (via `${env:KEY}`):
```
1. Navigate to the search page
2. Search for ${env:product_name}
3. Verify search results show ${env:product_name}
```
**Hook on Flow 2** — Also has access:
```python theme={null}
import os
product_name = os.environ.get("product_name")
print(f"Verifying product: {product_name}")
```
All runtime variable types carry forward — suite variables, flow variables, hook exports, and values set by the agent.
## Examples
### Example 1: Dynamic Data Capture Within a Single Flow
Capture a value from the app and verify it later in the same flow:
```
1. Navigate to the registration page
2. Fill in the form with a random email and submit
3. Save the displayed confirmation code as a variable called confirmation_code
4. Navigate to the verification page
5. Get the variable confirmation_code and enter it in the verification field
6. Verify the account is successfully verified
```
### Example 2: Passing Data Between Flows in a Suite
**Flow 1 — Create an order:**
```
1. Add "Wireless Mouse" to the cart
2. Complete checkout
3. Save the order number as a variable called order_number
```
**Flow 2 — Verify the order (uses `${env:KEY}`):**
```
1. Navigate to the order history page
2. Search for ${env:order_number}
3. Verify the order status shows "Processing"
```
### Example 3: Agent Sets a Variable, Hook Uses It
**Flow instructions:**
```
1. Log in and navigate to the profile page
2. Save the displayed username as a variable called current_username
```
**Teardown hook (Python) — reads the saved variable:**
```python theme={null}
import os
import urllib.request
username = os.environ.get("current_username")
# Clean up test data for this user
req = urllib.request.Request(
f"{os.environ.get('API_URL')}/users/{username}",
method='DELETE',
headers={'Authorization': f"Bearer {os.environ.get('ADMIN_TOKEN')}"}
)
urllib.request.urlopen(req)
print(f"Cleaned up user: {username}")
```
## Next Steps
* [Manage environment variables →](/environments)
* [Share data between hooks →](/hooks#sharing-data-between-hooks)
* [Organize flows with suites →](/suites)
* [Learn about all agent actions →](/flows#supported-agent-actions)
# Web Testing
Source: https://docs.autosana.ai/web-testing
Test web apps across Chrome, Firefox, Edge, and Chromium.
Run natural language flows against any website — production, staging, local, or a preview URL.
## Supported browsers
Pick a browser per run. Default is **Chrome**.
| Browser | What it is |
| - | - |
| **Chrome** *(default)* | Real Google Chrome stable — proprietary codecs (H.264, AAC), Widevine DRM, official Chrome user-agent. Use this unless you have a reason not to. |
| **Firefox** | Mozilla Firefox stable — different rendering and JS engine. Catches bugs that don't reproduce in Chromium-family browsers. |
| **Edge** | Microsoft Edge stable — for Edge-specific UI and Microsoft account flows. |
| **Chromium** | Open-source Chromium — no proprietary codecs, no DRM, vendor-neutral. |
## Setting the browser
Click **Run** on a web flow, pick the browser, hit go. Schedules ([Automations](/automations)) have the same picker. Re-running a flow keeps its original browser.
Pass `web_browser` in the body — see [API → Run Flows](/api-runs).
```bash theme={null}
curl -X POST https://backend.autosana.ai/api/v1/flows/run \
-H "X-API-Key: $AUTOSANA_KEY" \
-d '{"app_id":"...", "flow_ids":["..."], "web_browser":"firefox"}'
```
`flows_run` and `suites_run` accept a `web_browser` parameter (`chrome` | `firefox` | `edge` | `chromium`).
```
Run flow X on Firefox
```
Add `web-browser` to the [autosana-ci](/ci-cd-integration) action:
```yaml theme={null}
- uses: autosana/autosana-ci@main
with:
api-key: ${{ secrets.AUTOSANA_KEY }}
platform: web
app-id: my-web-app
flow-ids: "uuid-1,uuid-2"
web-browser: firefox
```
## Testing with Chrome extensions
Attach one or more Chrome extensions to a web app for an app-level default, or
select a different set in the Run dialog for one run. Passing an explicit empty
selection runs without extensions.
Extension runs use **Chromium**. Google Chrome and Microsoft Edge no longer
allow automation tools to side-load extensions. Chromium uses the same Blink
and V8 engines and supports Manifest V3 extensions in headless workers.
Autosana runs the real extension: service workers, content scripts, storage,
network interception, injected providers, and approval UI are not mocked.
Chrome side-panel approval surfaces (for example MetaMask connection requests)
are exposed to the agent as interactable browser pages.
Add extensions from **Apps → Create New App → Chrome Extension** using either:
* A Chrome Web Store URL
* A `.zip` of an unpacked Manifest V3 extension
See [Apps → Chrome Extensions](/apps#step-4-select-platform-and-upload).
### Choosing extensions for a run
* **App default:** Attach extensions from the web app card on the Apps page.
Every new run inherits those extensions.
* **One-run override:** In the Run dialog, select a different set. This replaces
the app default for that run.
* **No extensions:** Clear every extension in the Run dialog, or pass
`dependencies: []` through the [Runs API](/api-runs#run-flows).
You can load multiple extensions in one browser session. If two extensions
modify the same page behavior, their normal browser interaction rules still
apply.
### Exact build history
Autosana resolves each extension to an exact build when the run starts and
stores those build IDs with the run group. Updating an extension later does not
change historical results.
The Runs page shows a puzzle badge with the extension count. Hover it to see
names and build details. Group and flow detail pages show a collapsible
**Extensions** row; expand it and hover an extension to see the exact build and
upload date used by that run.
### Troubleshooting extension runs
* Confirm the extension uses Manifest V3.
* Use a `.zip` of the unpacked extension directory, not a `.crx` file.
* Put `manifest.json` at the archive root or inside one top-level folder.
* If a flow unexpectedly uses an extension, check the web app's default
attachments and the run's extension override.
* Chrome extension runs always use Chromium; Chrome, Edge, and Firefox cannot
be selected for that session.
# Webhooks
Source: https://docs.autosana.ai/webhooks
Receive HTTP notifications when flow runs complete
Get notified via HTTP POST request when Autosana completes a batch of flow runs.
## Setup
1. Go to **[Settings > Integrations > Webhooks](https://autosana.ai/settings?tab=integrations\&integration=webhooks)**
2. Click **Add Webhook**
3. Enter your HTTPS endpoint URL
4. Optionally add a signing secret for payload verification
If your endpoint sits behind a firewall, allowlist the `flow_runners.virtual_runners.web_browsers` IPs from the [Network Allowlist](/network-allowlist). Run-result webhooks come from those IPs for every platform, mobile included; the **Test** button sends from Autosana's API, which is not on that list.
## Payload Example
```json theme={null}
{
"event": "batch.completed",
"timestamp": "2025-01-31T12:34:56.789Z",
"app": {
"id": "com.example.myapp",
"bundle_id": "com.example.myapp",
"name": "My App",
"platform": "android",
"url": "https://jwvmttqclexlhnhfxuyq.supabase.co/storage/v1/object/public/app_builds/a1b2c3d4-e5f6-7890-abcd-ef1234567890/my_app.apk"
},
"git": {
"commit_sha": "a1b2c3d4e5f6g7h8i9j0",
"branch": "main"
},
"summary": {
"total_groups": 3,
"passed_groups": 2,
"failed_groups": 1
},
"run_groups": [
{
"name": "Login Suite",
"status": "passed",
"url": "https://autosana.ai/runs/groups/f1e2d3c4-b5a6-7890-abcd-ef1234567890",
"runs": [
{
"name": "User can sign in with email",
"status": "passed",
"url": "https://autosana.ai/runs/flow/12345678-abcd-ef12-3456-7890abcdef12"
}
]
}
]
}
```
The `git` object is only included when the app build has git metadata (e.g., uploaded via CI/CD with commit info).
## Verifying Signatures
If you configure a signing secret, payloads are signed with HMAC-SHA256. The signature is in the `X-Autosana-Signature-256` header as `sha256=`.
```python theme={null}
import hmac
import hashlib
def verify_signature(payload: str, signature_header: str, secret: str) -> bool:
expected = signature_header.removeprefix("sha256=")
computed = hmac.new(secret.encode(), payload.encode(), hashlib.sha256).hexdigest()
return hmac.compare_digest(expected, computed)
```
```javascript theme={null}
const crypto = require('crypto');
function verifySignature(payload, signatureHeader, secret) {
const expected = signatureHeader.slice(7); // Remove "sha256="
const computed = crypto.createHmac('sha256', secret).update(payload).digest('hex');
return crypto.timingSafeEqual(Buffer.from(expected), Buffer.from(computed));
}
```
# Writing Effective Flow Instructions
Source: https://docs.autosana.ai/writing-effective-flow-instructions
Write resilient flows that test user-visible behavior instead of UI implementation details.
## Default to journey-style
Users don't care if your button says "Continue" or "Next." They care whether they can do what they came to do — and your tests should reflect that.
By default, treat a flow as a contract with the **user-visible behavior** rather than the **UI implementation**. Behavior-level tests survive UI refactors; tests bound to specific buttons, colors, and positions don't.
Describe what the *user is doing*, not what the *UI looks like*. For tests where the UI itself is the contract, see [When the UI is the test](#when-the-ui-is-the-test).
## Structure
* **End with verification** — close the flow by asserting on the behavior you actually care about.
* **Bullets and multi-line steps are fine** whenever they read more naturally, such as a bulleted list of checks at the end of a flow.
* **Avoid numbered lists** — numbering locks in a rigid ordering that's painful to update later. Use plain lines or bullets instead.
## Good vs Bad Examples
**BAD — vague, no assertion:**
```
login to the app
type email
click submit
check if it works
```
**BAD — over-specified and brittle:**
```
Open the app
Tap the "Sign In" button at the bottom of the screen
Enter "test@example.com" in the email field
Enter "SecurePass123" in the password field
Tap the blue "Continue" button
Wait for the page to load
Verify that the heading "Welcome back!" is displayed
Verify that a profile icon is visible in the top right corner
```
This style locks the test to today's exact UI — a button rename or layout tweak makes it fail. Save it for tests whose subject **is** a single screen's behavior.
**Quoted UI labels are treated as exact ground truth.** Writing `Tap the "Sign In" button` couples the test to that exact label — rename the button to "Continue" and the test fails, even if the test isn't about that button. Only quote UI text when you're explicitly testing that the text appears, or when you need it to disambiguate between similar elements on the screen.
**GOOD — specific about intent, flexible about the steps in between:**
```
Login with these credentials:
email: test@example.com
password: SecurePass123
Submit and verify you land on the welcome screen.
```
## When the UI is the test
When the UI itself is the contract — such as form validation, error messages, disabled states, or exact copy — spell out the specifics.
## Waiting for the app
Describe the state you need, such as "verify the saved recipe appears in the folder," instead of adding fixed pauses between steps. Keep the expected result explicit on every platform: a still screenshot does not prove that loading is complete, regardless of how the run times its captures. Use a timed wait when the delay itself is part of what you are testing.
## Use environment variables
Don't hardcode anything sensitive, such as passwords, API keys, or tokens, or environment-specific values, such as URLs, test IDs, or feature flags. Use `${env:VAR_NAME}` so the same flow runs across environments. Mark sensitive values as **secrets** to encrypt them at rest and hide them from logs.
```
Log in with email ${env:TEST_EMAIL} and password ${env:TEST_PASSWORD}.
Open the account settings from the user menu.
Verify your email address is displayed.
```
See [Variables](/variables) for the full environment, suite, flow, build, and agent variable model.
The same `${env:VAR_NAME}` references work in [code-managed flow files](/code-managed-files#referencing-variables). Define flow and suite defaults in YAML; keep secrets in [Environments](/environments).
**Goal-oriented instructions create more flexible tests.** Instead of "click the gear icon, click Profile, click Change Email, type the new address, click Save, verify the success toast appears", write "update the user's email to '[new@example.com](mailto:new@example.com)' and verify it saves".
**Always include verification.** Close every flow by asserting on the outcome you care about — "verify the welcome screen appears", "verify the order shows in the list", or "check that the error message is shown".
**Web tests auto-load the site.** Don't include URLs unless you're explicitly testing URL functionality.