🡠 Back
Landing Page A/B Testing
Reading time
1
A/B Testing
August 31, 2026
afrus.ai
Create an A/B test between two or more versions of a landing page, how to split traffic between them, and how to interpret the results.
In this article
+
🧪
What are Landing Experiments?
Compare versions of your landing page and find out which one converts better
It's the feature that lets you compare two or more versions of the same landing page. You create a copy, change what you want to test, and AFRUS automatically splits your visitors between the versions. When there's enough data, the panel tells you whether the difference is real or just chance.
🔀 How traffic splitting works
Your experiment link
→
Weight-based split
50% / 50%
50% / 50%
→
Control landing
the original
the original
Variant A landing
your modified version
your modified version
→
Donation or registration
attributed to the assigned version
attributed to the assigned version
Each visitor always sees the same version for 30 days, and their donations or registrations are attributed to the version they were assigned.
✅ Repeat visits don't inflate the numbers
If someone comes back five times, they still count as one visitor. Only the first entry generates an exposure.
⚠️ Only link traffic enters the experiment
Anyone who arrives directly at a landing URL is outside the measurement. Always share the experiment link.
📋
Before you start
Three requirements you need to have ready
🔓
Module enabled
Experiments is in beta and is enabled organization by organization. If you don't see the option in the menu, contact the AFRUS team.
🌐
An active landing page
The experiment takes an existing landing page as the reference. Only active landing pages appear in the selector.
🎯
A widget with a defined goal on that landing
The widget's goal determines whether the panel measures donations or leads. Without a widget, the panel cannot classify conversions correctly.
📝
Step by step
From creating the experiment to declaring the winner
Part 1 — Create the experiment
1
Access the Experiments module
Go to Landings and click the Experiments button (top right). You can also get there from the navigation menu: Fundraising → Landing Experiments.
2
Create a new experiment
Click New experiment. You'll see a form divided into three blocks.
3
Choose the control landing
In the 1. Base landing (control) block, open the selector and choose the landing to test. This becomes your control: the reference all versions are compared against. Leave the Weight at 50% for now.
Only active landing pages appear. If you can't find yours, check its status in the Landings module.
4
Add your first variant
In the 2. Variants block, click Add variant. AFRUS creates an identical copy of your control landing and adds it as Variant A.
⚠️ Change only one thing per experiment. If you modify the headline, image and button at the same time and the variant wins, you won't know which of the three changes worked.
5
Edit the variant content
In the variant row, click Edit this landing. The editor opens in a new tab. Apply the change you want to test and save.
🔴 Important: open and save each variant in the editor before sharing the link. Since the experiment starts in Running status, if you split traffic before that step, some of your visitors may reach an unpublished page.
6
Adjust the weights
The weight is the percentage of traffic each version receives. All together they must add up to exactly 100. Click Distribute evenly for AFRUS to calculate it. A green indicator confirms "Sum: 100 / 100".
For a standard test, leave it at 50/50: it's the split that reaches a conclusive result fastest.
7
Name the experiment
In the 3. Experiment data block, fill in:
- Name (required) — e.g.: "Urgency headline vs. gratitude"
- Description (optional) — for your team; the donor never sees it
- Default UTMs (optional) — up to 6 parameters (
utm_source,utm_medium,utm_campaign,utm_term,utm_content,utm_id) automatically added to each visit
✓
Create the experiment
Click Create experiment. The button only activates when the form is complete. You'll see a confirmation screen with the experiment link and a QR code ready to use.
Part 2 — Share the link
🔴 Never share the URL of an individual variant. A visit that goes directly to a landing doesn't go through the split, isn't measured and its donation isn't attributed to any version.
The entire experiment is distributed with a single link (the splitter URL). That's the link that goes in your ads, newsletters, posts and printed pieces.
| Where to find the link | How |
|---|---|
| Confirmation screen | Copy button next to the link, with QR code included |
| Experiments list | Copy splitter link action in the row |
| Edit screen | Splitter URL field with copy button |
💡 If you add your own UTM parameters to the link you share, they take priority over the experiment's default UTMs. This way you can use the same experiment in multiple campaigns with different tagging.
Part 3 — Declare the winner and close
🏆 1. Review the automatic suggestion
The top card suggests the variant with the best conversion rate among those that reached statistical significance. If none reached it, no suggestion is made.
✋ 2. Declare the winner manually
Click Change winner and choose the version. The Manual label will appear. With Back to automatic you undo the choice.
⚠️ Declaring a winner doesn't change the traffic split or the experiment status. For everyone to see the winning version you need to increase its weight or publish its content to your main landing page.
🔒 3. Close the experiment
Go to Edit, change the Status selector to Completed and click Save changes. The experiment becomes read-only and all link traffic returns to the control landing. Links you already shared keep working.
Completed is definitive, it cannot be reopened. If you just want to stop the test for a while, use Paused.
📊
How leer los resultados
The panel indicators and how to interpret them
📈 The six general indicators
| Indicator | What it measures |
|---|---|
| Total visits | All visits to the link, including bots |
| Filtered bots | Visits identified as automated traffic |
| Bot rate | Percentage of the total that was bot |
| Real visits | Human visits, after discarding bots. It's the basis of all analysis |
| Unique IPs | Different IP addresses that reached the link (encrypted) |
| Unique visitors | Different people identified by cookie |
🗂️ The per-variant table
| Column | What it tells you |
|---|---|
| Variant | The label and its landing. The control row appears marked |
| Real visits | How many people saw that version |
| Confirmed Donations / Leads | How many converted. The title changes based on your widget's goal |
| Conversion rate | Percentage of visitors who converted |
| Lift vs control | How much better (green) or worse (red) it converts compared to control |
| Significance | Whether the difference is solid or could be chance |
| Funnel | El recorrido: Real visits → Conversión |
The significance label — is this difference real?
Insufficient data
Not enough volume to conclude anything yet.
Keep collecting. Ignore any differences you see for now.
Not yet significant
There's volume, but the difference may be due to chance.
Keep collecting, or accept that they perform similarly.
Significant difference
The difference is unlikely to be chance. Includes the confidence level, e.g.: "97.3% confidence".
You can already decide. In the winner card, See details shows the Z-score and P-value.
💡 Significance ≠ importancia. Con mucho tráfico, una mejora del 0,3% puede ser estadísticamente significativa y aun así irrelevante. Mira siempre las dos cosas: el lift dice cuánto cambia, la significancia dice si puedes confiar en ese cambio.
Experiment statuses
Draft — Sends everything to control, without measuring
Running — Splits by weights, measures and attributes conversions
Paused — Sends everything to control, without measuring. Collected data is preserved
Completed — Sends everything to control, without measuring. The experiment becomes read-only
✅ Your links never break when pausing or completing. They always go to the control landing. This matters if the link is already printed or in an active ad.
🔬
How se calculan las estadísticas
The technical reference: formulas, thresholds and bot filtering
📐 Conversion rate y lift
Conversion rate = Conversiones / Unique visitors reales
Lift vs control = (Tasa de la variante - Tasa del control) / Tasa del control
El lift no se calcula para el control (es su propia referencia) ni cuando el control tiene tasa cero.
📏 Significance estadística — Test Z de dos proporciones
p1 = conversiones variante / visitantes únicos reales variante
p2 = conversiones control / visitantes únicos reales control
p_agrupada = (conv. variante + conv. control) / (visit. variante + visit. control)
error_estándar = √( p_agrupada × (1 - p_agrupada) × (1/n_variante + 1/n_control) )
z = (p1 - p2) / error_estándar
- Confidence level: 95%, two-tailed test. Threshold:
|z| ≥ 1.96 - The P-value comes from the two-tailed normal distribution
- The confidence shown in the panel is
(1 - P-value) × 100
⚠️ Umbrales mínimos de muestra — se exigen antes de calcular nada: tanto la variante como el control deben tener al menos 100 visitantes únicos reales y al menos 30 conversiones. Si cualquiera no llega, el veredicto es Insufficient data. Esto protege contra el error más común: cortar el experimento el primer día al ver una versión "ganando" por puro ruido.
🤖 Bot filtering — how it works
AFRUS marks a visit as a bot when:
- The visit declares no browser
- The browser corresponds to a known crawler (search engines, social media link previewers, command-line clients…)
- The same IP exceeds a threshold of visits per minute on the same experiment
Los bots siempre se redirigen al control (para que las previsualizaciones de WhatsApp/Facebook sean consistentes) y nunca producen conversiones atribuidas. Quedan fuera de Real visits, tasa de conversión, funnel y significancia.
🍪 The assignment cookie
- Lasts 30 days and stores which version the visitor was assigned to
- Not accessible from JavaScript and limited to the experiment's domain
- If the visitor clears their cookies, changes browser or enters from another device, they're treated as a new visitor and may be assigned to another version
⏱️ Delay in conversion attribution
Conversions appear with a delay compared to visits because the attribution process runs every 15 minutes. If you make a test donation and check the panel right away, it's normal for the visit to already appear but not the conversion yet.
✅
Best practices
Para obtener resultados confiables
- Change only one variable per experiment. If you want to test headline and image, run two separate experiments.
- Define in advance how long you'll wait. Set the volume or date before starting and don't peek at the results to decide when to cut.
- Let it run for at least a full week. Donation behavior changes a lot between weekdays, weekends and times of day.
- Don't change the weights mid-way if you can avoid it: results are cumulative and you'd be mixing two different periods.
- Don't run two experiments on the same landing at the same time. They interfere with each other and neither is reliable.
- Always share the experiment link, never the URL of an individual variant.
- Check the bot rate before trusting the numbers.
- Open and save each variant in the editor before splitting traffic.
⚠️
Limitations conocidas
Lo que la herramienta no hace en esta versión
| Limitación | Impacto y qué hacer |
|---|---|
| Results are not filtered by date | Numbers are always cumulative from the start. To analyze a period, manually record the values at the end of each one. |
| No metric for amount raised | Conversions are counted, not amounts. A version with more donations but smaller amounts may appear as the winner. Cross-check with the Transactions module before deciding. |
| The count does not filter by gateway status | Every attributed donation counts, whether credited or not. Relevant if you use asynchronous methods like PIX, boleto or bank transfer. |
| The funnel has only two steps | There is no intermediate step for widget view or checkout start. Planned for later. |
| No hay desglose de Unique IPs | The indicator shows the total, without detail by region or device. |
| A variant's landing cannot be reassigned | You must remove the variant and create a new one with the desired landing. |
| The winner doesn't apply itself | Declaring it records your decision, but redirecting traffic requires adjusting weights or publishing the winning content to the main landing page. |
❓
Frequently asked questions
The most common questions about experiments
1. How long do I need to let an experiment run?
Hasta que el panel muestre Significant difference, y como mínimo una semana completa. Los umbrales son: 100 visitantes únicos reales y 30 conversiones en cada versión. Con una landing que convierte al 3%, necesitas ~1.000 visitantes por versión (2.000 en total con reparto 50/50).
2. Does a visitor always see the same version?
Yes, for 30 days as long as they keep their cookie. If they clear cookies, change browser or enter from another device, they're treated as a new visitor.
3. ¿Por qué "Total visits" y "Real visits" no coinciden?
La diferencia son los bots: rastreadores, previsualizadores de enlaces y tráfico automatizado. "Real visits" es lo que queda después de descartarlos — es la base de todas las decisiones.
4. I made a test donation and it doesn't appear. Is it broken?
Probably not. Attribution runs every 15 minutes. Wait that long and refresh. If it still doesn't appear, check that you entered through the experiment link and not directly to the landing.
5. Can I add a variant while the experiment is already running?
Yes, without pausing it. Keep in mind that the weights must add back up to 100 and the new variant starts from zero, so it will take longer to accumulate enough sample.
6. What happens to links I already shared if I pause or complete the experiment?
They keep working and go to the control landing. You'll never leave a broken link by changing the status.
7. The bot rate is very high. What do I do?
First check where you published the link: if it circulated on WhatsApp, Slack, Facebook or Twitter, previewers generate automated visits. This is expected and doesn't contaminate results because they're excluded from decision metrics. If it's high without explanation, notify AFRUS.
8. Why does the column say "Conversions" and not "Confirmed donations"?
Because AFRUS couldn't determine the goal of your control landing's widget, usually because it has no associated widget or form. In that case the number combines donations and registrations. Check the landing configuration.
9. Una versión tiene mejor tasa pero dice "Not yet significant". ¿Puedo darla por ganadora?
Puedes declararla manualmente, pero no es recomendable. "Not yet significant" significa que la diferencia entra dentro de lo que produciría el azar. Si decides ahora, hay riesgo real de cambiar tu landing por una que no es mejor.
10. Does the winner automatically apply to my entire audience?
No. Declaring it records your decision, but doesn't change the split or status. For all traffic to see the winning version you need to increase its weight, or complete the experiment and bring its content to your main landing page.
11. Can I see the results for a specific month?
Not in this version. Results are always cumulative from the start. If you need to compare periods, note the values at the end of each one.
12. Are donated amounts compared between versions?
No. The panel counts conversions, not amounts. A version can win in number of donations and raise less. Before deciding, cross-check the result with the Transactions module.
13. What happens if someone goes directly to a variant's URL?
They see the page normally, but they're outside the experiment: no measured visit is generated and their donation isn't attributed to any version. That's why you always share the experiment link.
14. Does this affect my analytics tracking?
No. The link responds with a standard redirect. The UTMs you define propagate to the destination landing, and the ones you add to the shared link take priority. Each landing's analytics configuration keeps working independently.
15. Can I test two changes at the same time?
You can create three versions (control, "new headline" and "new image") and it's a valid experiment. What you shouldn't do is put both changes in one single variant, because if it wins you won't know which one worked. Each additional version needs its own sample of 100 visitors and 30 conversions, so the experiment takes longer.
Next tutorial
See all tutorials
→
Need help? Talk to the afrus team
In this article
✕
Related Tutorials
No items found.
This tutorial doesn't have others in its category yet. You might like these:
See all tutorials →