astronaut
PricingDocs
Sign in
Overview
IntroductionGetting started
Install
iOS SDKSource attributionCreator attributionWeb analytics
Using astronaut
The dashboardIdentifying usersVerified revenueEnvironmentsExperimentsPush notificationsChat

Experiments

An experiment shows some of your new installs one version of a screen and the rest another, then tells you which one more people got through. You make it in the dashboard - its variants, how much of the audience it takes, what it is judged by - and your app asks one question: which variant does this install have?

Make the experiment

Under Experiments, choose New experiment and give it:

  • A name. This is what your app asks for, so it is kept to lowercase letters, digits and underscores.
  • Two to four variants. The first is the baseline: what you ship today, and what the others are compared against.
  • An audience: the share of new installs to enrol, split evenly across the variants.
  • A primary metric: the event that means it worked - or several, any of which counts - and how long each person gets to do it. Verified store events can be the metric too, so an experiment can be judged on a trial the store confirmed.

It is saved as a draft. Nobody is enrolled until you start it.

Ask which variant an install has

In your app, ask where the versions differ. You get the variant’s name back, or nothing for an install that is not in the experiment, which should see the baseline.

switch Astronaut.shared.variant("signin_apple") {
case "apple": AppleSignInView()
default:      EmailSignInView()
}

Ask where the difference is shown, not at launch. An install is counted when it asks, so asking early counts people who never reached the screen and waters the result down. If only some people should be in the experiment - say, those who can still start a trial - ask only for them.

Who is enrolled

New installs only. An install is dealt a variant the first time it opens the app while the experiment is running, and keeps it from then on. Nobody who has already seen your app as it was is moved to something new. The SDK fetches its variants at launch and remembers them, so the answer is instant and works offline.

Start, pause, stop

  • Start begins enrolling new installs.
  • Pause stops enrolling anyone new. Everyone already in keeps their variant, and results keep counting.
  • Stop ends it for good. Every install goes back to the baseline the next time it opens, and the result stays.

Once an experiment has started, its name and variants are fixed. The audience and the primary metric can still be changed.

Reading the result

Each variant gets a bar for its conversion rate, with a line across it for where that rate probably really is. When two lines do not overlap, the difference is real; when they do, it could still be chance.

Above the bars is the conclusion in a sentence. It says a variant is better or worse only once there is a 95% chance that it really is, and until then it says how many more people it would take. Nothing is concluded from fewer than 30 people in a variant.

Testing a variant yourself

Your own phone is dealt one variant like anyone’s. To look at another, pin it in a debug build:

Astronaut.shared.setVariantOverride("apple", for: "signin_apple")

The variant call lives in the iOS SDK.

← PreviousEnvironmentsNext →Push notifications
astronaut
Product
FeaturesPricingDocsLive demo
Legal
PrivacyTerms
© 2026 astronaut.shhello@astronaut.sh