Skip to main content

Example: Testing a PayPal Button

Enterprise beta

Experimentation is in beta on Enterprise plans. Get in touch to join.

Our app currently accepts credit card payments only. We suspect offering PayPal will increase completed checkouts, and we want to prove it on a small slice of traffic before rolling it out. This page walks through the whole experiment.

1. Create the flag​

Create a multivariate flag called paypal_button. The flag's current behaviour (no PayPal) is the control; add one variation named show-paypal.

Your code will branch on the variation name, not the flag value, so keep names short and stable.

2. Create the metric​

With a warehouse connected, create a metric:

  • Name: Checkout Completion
  • Measure: Occurrence
  • Direction: Higher is better
  • Event name: checkout_completed

3. Create the experiment​

Create an experiment on paypal_button:

  • Hypothesis: "Offering PayPal at checkout will increase checkout completion by at least 10% within 30 days."
  • Rollout: 10%, split evenly. 5% of identities see PayPal, 5% act as control, and the remaining 90% stay out of the experiment entirely.
  • Primary metric: Checkout Completion, expected to increase.

Launch it from the Review step.

4. Instrument the checkout​

On the checkout page, resolve the flag (recording the exposure) and branch on the variation name. When a checkout completes, send the conversion event. The SDK must be initialised with enableEvents: true and the user identified; see Run an Experiment.

import flagsmith from '@flagsmith/flagsmith';
import { useExperiment } from '@flagsmith/flagsmith/react';

const CheckoutPage: React.FC = () => {
 // Evaluates the flag and records the exposure
 const flag = useExperiment('paypal_button');
 const showPaypal = flag?.enabled && flag?.variant === 'show-paypal';

 const onOrderConfirmed = () => {
  // Record the conversion
  flagsmith.trackEvent('checkout_completed');
 };

 return (
  <>
   <CardPaymentForm onConfirmed={onOrderConfirmed} />
   {showPaypal ? <PayPalButton onConfirmed={onOrderConfirmed} /> : null}
  </>
 );
};

Outside React, flagsmith.getExperimentFlag('paypal_button') does the same job as the hook. The same flow works in Python; see Run an Experiment for both SDKs.

If checkout is available to visitors who aren't logged in, identify them with a persistent anonymous GUID so bucketing and events stay consistent; see Identities.

5. Read the results and roll out​

On the experiment page, check the Exposures panel: enrolment should track about 10% of checkout traffic, split 50/50. Then watch the results as conversions arrive; see Analyse an Experiment.

When the result is conclusive, end the experiment, roll the PayPal button out to everyone, and clean up the flag.