Running a test you built yourself

There are two ways to run an A/B test with Ordinary, and both are scored the same way.

1. Build it in Ordinary. Open the visual editor, click the element you want to change, edit it, and publish. Ordinary writes the variant into your theme and shows the right version to each visitor.
2. Build it yourself. Your developer writes the variants and the logic that decides who sees what, then tells Ordinary which visitor saw which version using the JavaScript API below.

Either way, the results come from the same place: your reconciled Shopify revenue, matched to the visitors who actually saw each version.

When to build it yourself

Most tests don’t need code. Reach for your own implementation when the change is behavioural rather than presentational — a different checkout flow, a pricing rule, a search ranking change, a test that spans several templates, or anything already handled by logic in your own codebase.

Registering an exposure

Call this once per page view, as soon as your code knows which version the visitor is getting:

window.Ordinary.experiment({
  id: "summer-hero-test",
  variant: "b",
  source: "native",
});

id — your identifier for the test. Any stable string. (experiment_id also works if that’s what your data layer already calls it.)

variant — your identifier for the version this visitor saw.

source — optional label for where the test came from. Use native for a test you built.

Both id and variant are required. The call returns true when the exposure is registered and false if the arguments are unusable, and it never throws — it’s safe to drop into theme code without wrapping it. Calling it repeatedly for the same visitor and version is harmless; only the first registration counts.

To read back what Ordinary currently has for this visitor:

window.Ordinary.experiments();

This requires the Ordinary app embed to be enabled in your theme, which is the same thing that powers the rest of your tracking.

Naming the versions

Whatever you pass as variant is what you’ll see in your results, so use names you’ll recognise months from now — control and long-form-copy read better than a and b. Keep them consistent for the life of the test; changing a name mid-test splits one version into two in the results.

Reporting

A test you built yourself appears alongside the rest of your experiments and is scored identically — revenue per visitor, conversion rate, and the same peek-safe statistics. Ordinary reconciles revenue against Shopify, so a test you delivered in your own code is measured against the same numbers as everything else.

What Ordinary does not do is score a test it was never told about. Exposures arrive through the API call above, so a test that doesn’t make the call has no visitors recorded against it.

Ending a test you built yourself

Remove your variant markup from your theme before you end the test.

While a test is running, Ordinary hides the versions a visitor isn’t assigned to. Ending the test stops that. If your variant markup is still in your theme when the test ends, your original and your variant content will both render on the page at the same time.

Ordinary can’t remove that markup for you — it’s your code, in your theme, and Ordinary never rewrites what it didn’t write. So the order matters:

1. Take your variant markup out of your theme, leaving the version you want to keep.
2. End the test in Ordinary.

Ordinary shows a confirmation on this before you end a test it didn’t deliver.

Tests where Ordinary wrote the variant behave differently: those variants are hidden in the theme file itself, so ending the test leaves your original content showing on its own. You remove Ordinary’s markup from the Theme changes tab on the test, whenever you’re ready.

Related

• A/B testing and experiments — building and running tests in Ordinary
• Attribution — how visitors are matched to orders