Running a test you built yourself

There are two ways to run an A/B test with Ordinary, and both are scored the same way.

1. Build it in Ordinary. Open the visual editor, click the element you want to change, edit it, and publish. Ordinary writes the variant into your theme and shows the right version to each visitor.
2. Build it yourself. Your developer writes the variants and the logic that decides who sees what, then tells Ordinary which visitor saw which version using the JavaScript API below.

Either way, the results come from the same place: your reconciled Shopify revenue, matched to the visitors who actually saw each version.

When to build it yourself

Most tests don’t need code. Reach for your own implementation when the change is behavioural rather than presentational — a different checkout flow, a pricing rule, a search ranking change, a test that spans several templates, or anything already handled by logic in your own codebase.

Registering an exposure

Call this once per page view, as soon as your code knows which version the visitor is getting:

window.Ordinary.experiment({
  id: "summer-hero-test",
  variant: "b",
  source: "native",
});

id — your identifier for the test. Any stable string. (experiment_id also works if that’s what your data layer already calls it.)

variant — your identifier for the version this visitor saw.

source — optional label for where the test came from. Use native for a test you built.

Both id and variant are required. The call returns true when the exposure is registered and false if the arguments are unusable, and it never throws — it’s safe to drop into theme code without wrapping it. Calling it repeatedly for the same visitor and version is harmless; only the first registration counts.

To read back what Ordinary currently has for this visitor:

window.Ordinary.experiments();

This requires the Ordinary app embed to be enabled in your theme, which is the same thing that powers the rest of your tracking.

Naming the versions

Whatever you pass as variant is what you’ll see in your results, so use names you’ll recognise months from now — control and long-form-copy read better than a and b. Keep them consistent for the life of the test; changing a name mid-test splits one version into two in the results.

Reporting

A test you built yourself appears alongside the rest of your experiments and is scored identically — revenue per visitor, conversion rate, and the same peek-safe statistics. Ordinary reconciles revenue against Shopify, so a test you delivered in your own code is measured against the same numbers as everything else.

What Ordinary does not do is score a test it was never told about. Exposures arrive through the API call above, so a test that doesn’t make the call has no visitors recorded against it.

Ending a test you built yourself

While a test is running, Ordinary hides the versions a visitor isn’t assigned to. Ending the test stops that, and from then on your markup renders exactly as it ships in your theme.

A variant that ships hidden, with style="display:none" as described in Building a test in your theme code, stays hidden after the test ends. Visitors see your original, and the leftover markup is only tidying. A variant that relies on Ordinary to hide it shows next to your original the moment the test ends, and stays there until you remove it.

Ordinary can’t remove that markup for you — it’s your code, in your theme, and Ordinary never rewrites what it didn’t write. When you end a test you built, Ordinary checks the page and tells you what it found: markup that is showing, or hidden markup you can remove when convenient.

The order that keeps every visitor seeing a complete page:

1. End the test in Ordinary.
2. Make the version you’re keeping the only one in your theme, and remove the other version’s markup. Removing a version while the test is still running leaves the visitors assigned to it without their version. If your variant isn’t hidden inline, have this edit ready to make straight after ending.

Ordinary shows a confirmation on this before you end a test it didn’t deliver.

Tests where Ordinary wrote the variant behave like a hidden variant: ending the test leaves your original content showing on its own. You remove Ordinary’s markup from the Theme changes tab on the test, whenever you’re ready.

Related

• A/B testing and experiments — building and running tests in Ordinary
• Building a test in your theme code — shipping both versions in your theme and letting Ordinary choose who sees which
• How attribution is measured — how visitors are matched to orders