GemX CRO & A/B Testing: Help Center

Getting Started

Template Testing

May 05, 2026

Use Case: How to Increase Add-to-Cart Rate on Product Pages with GemX

Mar 21, 2026

When You Should Continue, Stop, or Iterate an Experiment in GemX

Mar 15, 2026

What to Do After Verifying the Winning Template for Your Test

Mar 03, 2026

How Long Should You Run a Test on Low-Traffic Shopify Store

Jan 26, 2026

How to Test the Product Information Section in My Product Page

Jan 16, 2026

Use Case: A/B Test Hero Section with GemX Template Testing

Oct 27, 2025

How to View and Read Template Testing Experiment Analytics

Oct 25, 2025

How to Create a Template Testing Experiment in GemX

Multipage Testing

Analytics

Accounts & Billing

Frequently Asked Questions

Test setup & duration

How many link for an experiment?

Just one — the Control link. GemX uses the Control page URL as the single experiment link and splits traffic at that URL across your variants. You don't add a separate link per variant.

I ran ads for my pages, do I need to change link?

No — provided your ads already point at the Control page URL. GemX uses the Control URL as the experiment link and splits traffic at that URL, so your ad campaigns stay unchanged.

How long should I run the test?

Judge by data stability, not a fixed number of days. GemX lets you start and stop whenever — a result is trustworthy once it's stable, not just because time has passed.

A reliable run covers weekdays and weekends, includes at least one full buying cycle, and has enough traffic and completed conversions. Keep the store stable during the test — no new promos, price changes, or theme edits.

Duration ranges by test type:
- CTA, headline, copy changes: 7-14 days
- Product page layout changes: 14-21 days
- Funnel / multipage experiments: 21-30+ days
- Pricing, upsell, or offer tests: 30+ days

Safe to stop when differences are consistent across several days, confidence has stabilized, and traffic looks normal. Note: Low-traffic stores need longer. These are starting ranges, not hard rules.

How does traffic splitting work

When a visitor enters through the Control URL (the experiment link), GemX randomly and automatically assigns them to a variant according to the split you set (e.g. 50/50, 70/30). The assignment is sticky for that visitor.

You can scope which visitors are included by: device, visitor type (new vs. returning), traffic source, and market / language.

How do I know when a test is statistically significant

GemX uses a Bayesian method to evaluate results. Instead of a pass/fail p-value, it estimates the full probability distribution of each variant's performance and reports a probability to win per variant in the Performance detail box, calculated only on the selected winning metric (Conversion Rate or Revenue). A variant is generally considered a winner once its probability to win reaches ~95%.

Distinguish two things:
- Statistical confidence: the probability the difference is not random chance.
- Decision confidence: whether the lift is stable and large enough to act on.

A result is decision-ready when the difference is consistent across multiple days, confidence has stabilized, the lift is meaningful (not merely measurable), it aligns with your hypothesis, and the run covers weekdays + weekends and is long enough for the traffic level.

Note: The winning metric can be changed after launch, which recalculates the probability to win.

Experiment perfomance

Will GemX slow down my site?

No. GemX test scripts are delivered via CDN and are designed not to slow down your storefront.

Why do I need to enable GemX Theme Helper?

When the GemX Theme Helper is enabled in your theme, A/B experiment logic is activated for all of your live experiments, so they run seamlessly. It also enables preview functionality, including browser-based previews.

Enabling the GemX Theme Helper won't affect anything else in your store.

Data accuracy

Data delay

Tracking data can take time to process and appear. Data is delayed by a maximum of ~30 minutes — recently placed orders may not show immediately.

If the delay is over 30 minutes, contact support to troubleshoot.

The buyer declined the store's cookie consent (or blocked cookies)

If your store uses a cookie consent banner and the buyer declines — or uses a third-party extension to block cookies — Shopify won't track behavior or send data to third-party apps, so GemX can't track the data.

For A/B testing: if a buyer accepts consent, the data won't be included in the experiment results, but it will still appear in Page Analytics.

The buyer used a tracking-blocker extension

If your buyer's browser blocks tracking — through a built-in setting or an installed extension — the website may be prevented from collecting behavioral data.

The order came from outside the page or experiment flow

Orders from a source outside the experiment flow aren't counted in the page or experiment flow.

Examples:
- Email campaigns
- Abandoned-cart recovery flows
- Direct purchases via platforms like Instagram, Facebook, etc.

Payment used an in-app method that didn't redirect back

If the buyer paid via an in-app payment method that didn't redirect back to the Thank-you page, the data won't be tracked.

Checkout was reached via a third-party checkout app or COD app

Orders that go through a third-party checkout app or a Cash-on-Delivery app — a checkout that replaces Shopify's standard checkout — may bypass the tracking flow and not be recorded.

The order was placed on a post-purchase page

GemX doesn't track events on a post-purchase (upsell) page.

A/B Testing Doesn’t Have to Be Complicated.

GemX helps you move fast, stay sharp, and ship the experiments that grow your performance

Start Free Trial

Start $1 Shopify