Skip to main content
autosana run runs the tests in your .autosana/ folder from the terminal, without opening a pull request. It is shorthand for autosana flows run. There are three ways to run, and the flag you pass decides where the tests come from and where they execute.

​
Three ways to run

Use --local while you iterate on a flow. Use --cloud to try an edit on real devices before you commit it. Use a branch run for what CI would do: the committed tests, at a commit, on cloud devices.

​
Before you start

Install the CLI and log in. Run autosana login from inside the repository and the login is stored for that repository, so different repos can use different organizations.
The organization you are logged in as must own the app you run against. A run that reports App not found. Upload a build first. almost always means the login belongs to a different organization than the app. Check autosana whoami before checking anything else. autosana run looks in .autosana/ in the current directory: --local and --cloud take the tests from there, and every mode reads config.yaml from there (a branch run takes its tests from GitHub). Pass --path to point at another folder, for example in a monorepo. (autosana flows validate takes the folder as its argument instead.)

​
Choosing the tests

Name flows by key and suites by folder:
A flow’s key is its path under .autosana/ without the .flow.yaml ending, so .autosana/checkout/cart.flow.yaml is checkout/cart. A suite’s key is its folder, so the folder .autosana/checkout/ with a _suite.yaml inside is the suite checkout.
A bare name is always read as a flow key. autosana run checkout fails with flow key 'checkout' not found when checkout is a suite folder. Use --suite checkout, or name a flow inside it such as checkout/cart.
A label matches the labels: list in your flow and suite files. Labels set only in the dashboard are not seen. A flow that belongs to a selected suite runs inside that suite; a flow selected on its own runs on its own.

​
Choosing the app

A run needs to know which of your apps to run against. For a mobile app that is its bundle id and platform. For a web app, --app-id takes its slug: a lowercase id such as my-web-app, the app-id input of the CI action, which the List Apps API returns as the app’s bundle_id. A branch run accepts only the slug; local and cloud runs also accept the app’s UUID.
Rather than type these on every run, commit them once in .autosana/config.yaml, next to your flows (key reference in Files & Schema):
With that file, autosana run --cloud --suite checkout needs no app flags: it uses the iOS app because default_platform says so. Add --platform android and it uses the Android one. --platform web, or default_platform: web, uses the app_id. A file with a single entry needs no default_platform at all. A local run picks the entry for whichever platform your autosana up session is on. A value you type beats one from the environment, which beats the file: --bundle-id on the command line wins over the AUTOSANA_BUNDLE_ID variable, which wins over config.yaml. The keys, their rules, and what the file is not (a test, or anything the sync reads) are in Files & Schema. Two more things affect the target:
  • Environment. --environment <name> (or environment: in the file) selects the variable set your flows resolve ${env:…} references against. It is required when one app spans several environments.
  • Local runs on mobile can leave the app out entirely when your organization has a single app for that platform. Cloud and branch runs always need it named, by flag, variable, or file.

​
Local runs

Start a session on your device or dev server with autosana up, then run against it:
A local run executes your uncommitted YAML on the target that autosana up connected. Web is inferred from --app-id; the mobile platform is taken from your session. Use --platform when you have sessions on more than one platform, or --session-id when several sessions match. Set-up details for simulators, emulators, physical phones, and dev servers are on the Local Testing page.
Local runs execute your flow’s instructions only. Hooks are not run locally because they manipulate the test environment rather than the target. Use a suite’s setup_flow for setup steps that can run on the target.

​
Cloud runs

--cloud uploads your uncommitted .autosana/ and runs it on Autosana’s devices, so you can try an edit before committing it. Select any number of flows, suites, or labels:
The first line names the app; the others leave it to config.yaml (see Choosing the app). The tests always come from your working copy. What --branch and --app-build-id choose is only the build: When no build was uploaded from the branch, the run stops and, if the app has an active build, names it so you can pin it with --app-build-id. Unlike --local, a cloud run does execute hooks. Hook scripts in your working copy are uploaded with the flows and take precedence; a hook a flow names that has no file in .autosana/ is taken from the dashboard.

​
Branch runs

With neither --local nor --cloud, the CLI runs the tests committed on a branch. It reads that branch’s .autosana/ from GitHub, so nothing uncommitted takes part:
--ref defaults to your current branch. Name flows with --flow here; the bare autosana run login form is for --local and --cloud only. The repository comes from your origin remote, or pass --repo owner/name. By default the run uses the app build made from the same commit as the tests, so every result names a single commit. Two flags change which build runs: The last row is the one to reach for when you changed only test files: no build exists at your commit, and rebuilding an app that did not change is wasted work. Reach for --build-match branch-latest when you want the branch’s newest build but do not have its ID to hand, in CI for example. If the option you choose matches no build, the run is refused and says so. Autosana never quietly falls back to the app’s active build, because a result that does not name the build it came from is worse than no result.
--build-match needs a branch name. Pass a full commit sha as --ref and the flag is refused, because a sha names no branch to take a latest build from. Name the build with --app-build-id instead, which works with any ref.
The dashboard runs branches too: the run dialogs on the Flows page take a branch and a build, and the build you pick there need not come from that branch.

​
What a run prints

Each run that starts is printed as a link you can open, followed by the batch that groups them:
The link opens the run in the dashboard. The batch id is what the Run Status API polls, if you want to wait on the result from a script.

​
Troubleshooting

​
App not found. Upload a build first.

The organization the CLI is logged in as does not own an app with that bundle id and platform. Run autosana whoami; if it names the wrong organization, run autosana login from inside the repository. If the organization is right, check the bundle id against the Apps page. A web app is looked up by app id, a mobile app by bundle id and platform.

​
flow key ‘x’ not found in the working copy

x is not a flow. Either it is a suite folder (use --suite x), or the file is not named <key>.flow.yaml, or you are in a directory without .autosana/ (pass --path).

​
no build was uploaded with branch_name ‘x’ for this app

A --cloud run found no build uploaded from that branch. Upload one from CI, or pin any build with --app-build-id. The error names the app’s active build when there is one.

​
My suite won’t run against a branch

A branch run fails with an error naming one or more flow files, instead of starting:
Autosana refuses to run a suite when one of its tests is missing. A suite that runs four tests out of five and still reports a pass would hide the fact that a test never ran. This happens when the suite has no flows: list: its tests are then every .flow.yaml beside its _suite.yaml, and on that branch one of those files could not be read. Possible Causes:
  • The file has a YAML error
  • The file couldn’t be downloaded from GitHub
  • Two files claim the same key, so both were dropped
Solutions:
  • Fix or delete every file the error names. It lists them all at once, so you can repair them in a single commit
  • Run autosana flows validate to reproduce a YAML error or a duplicate key
  • If a file failed to download, start the run again. If the same error comes back, check that the file is committed on that branch
  • Give the suite an explicit flows: list, so its tests are the ones you name rather than whatever the folder contains
Only runs you start against a branch are refused this way. The automatic check on a pull request is unaffected, because suite changes there don’t apply until you merge (see Pull request workflow).

​
Next Steps