Running a Real A/B Test on Tabeer.ai's Merit Calculator CTA With PostHog Experiments

With server-side feature flags covering operational rollouts on Tabeer.ai's backend, the other half of the ask was A/B testing — and the right home for that is client-side, through the PostHog JS SDK that's already gated behind cookie consent, since an experiment's entire point is measuring what consenting, tracked users actually do.

Picking a Real Test, Not a Placeholder

Rather than invent a hypothetical feature to test, I used an actual, currently-live element: the final call-to-action button on /merit-calculator, one of the site's highest-intent pages — someone who's just calculated their merit is exactly the person deciding whether to sign up. Control copy was Start the free trial →. The variant: Try it now — free →.

Creating the Experiment via API

PostHog's API can create an experiment directly, which also auto-creates the underlying feature flag:

curl -X POST "https://us.posthog.com/api/projects/{project_id}/experiments/" \
  -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
  -d '{
    "name": "Merit calculator CTA copy",
    "feature_flag_key": "merit-calculator-cta-copy",
    "parameters": {
      "feature_flag_variants": [
        {"key": "control", "rollout_percentage": 50},
        {"key": "direct-cta", "rollout_percentage": 50}
      ]
    }
  }'

The response confirmed both the experiment (id: 461213) and its auto-generated feature flag (id: 867429) — but with "active": false and "start_date": null. Creating an experiment object and launching it are separate steps in PostHog's model, which makes sense once you think about it: you often want to finish wiring the variant-rendering code and deploy it before traffic starts actually splitting.

Wiring the Variant Into the Page

const ctaVariant = ref<'control' | 'direct-cta'>('control')
const finalCtaText = computed(() =>
  ctaVariant.value === 'direct-cta' ? 'Try it now — free →' : 'Start the free trial →'
)

onMounted(() => {
  const flag = window.posthog?.getFeatureFlag?.('merit-calculator-cta-copy')
  if (flag === 'direct-cta' || flag === 'control') ctaVariant.value = flag
})

<NuxtLink to="/register" class="btn-primary !h-auto !px-7 !py-4 !text-[17px]">
  {{ finalCtaText }}
</NuxtLink>

Default state (control) renders immediately on the server, so there's no layout shift or flash of missing content — the variant only overrides it client-side, once PostHog's flag evaluation resolves, using the same getFeatureFlag call the JS SDK already exposes for reading any flag, experiment-linked or not.

Launching It

With the code deployed, starting the experiment is a PATCH setting start_date:

curl -X PATCH ".../experiments/461213/" -d '{"start_date": "2026-09-04T21:06:50Z"}'

{"start_date": "2026-09-04T21:06:50Z", "feature_flag": {"active": true}}

feature_flag.active flipping to true in the response is the actual confirmation — that's the flag real visitors to /merit-calculator now start getting split against, 50/50, the moment they load the page.

Why This Wasn't Gated the Same Way as the Backend Flag

The server-side flags post covers the reasoning in full, but the short version: this experiment runs through the same cookie-consent- gated PostHog JS SDK the site already uses for analytics, deliberately, because there's no meaningful version of an A/B test that doesn't require tracking which variant a visitor saw and what they did afterward. A visitor who's declined analytics consent sees the control experience — correctly, since PostHog's JS client never initializes for them at all, and the ref stays at its default 'control' value.

What Happens Next

PostHog now needs enough traffic through both variants to reach statistical significance before the results mean anything — checking early would just be noise. The experiment can be stopped and a winner picked once that threshold is reached, without touching the deployed code again: the finalCtaText computed property already handles whatever PostHog reports back.

If you've got a real conversion point on your site and want to test copy or layout changes with actual statistical rigor instead of guessing, get in touch.