Using the A/B Split function, you can send two different versions of a campaign to a small percentage of all recipients. While the percentage can be defined by the user, the recipients used for the A/B split are chosen completely at random by NH Campaign. Consequently, you can analyze the statistics of a small but representative sample of your recipients to find out which campaign flavor is more successful.
The different campaign flavors you want to test may be based on the look of the template, on a header image, the time of sending, or on other factors, such as the subject line and the sender address used.
Once you have found out, which flavor of the campaign is more successful, you send this flavor to the remaining group of recipients (either manually or automatically). In the following we will showcase this functionality.
Note:
This option is available for integrated email campaigns and is displayed if you have selected the A/B Split action for the campaign.
It is recommended to only change one setting per A/B split to avoid drawing wrong conclusions.
Using the A/B Split Functionality:
Example: We want to prepare a guest campaign advertising our various products to the recipients. There is an existing template that we have used before and a new one that has been suggested by our agency. We want to find out which one is more successful, in other words, which of the two campaigns will achieve:
- A higher click rate?
- A higher open rate?
- Lower opt-out rate?
This is how it works.
- Set up a new campaign making sure that the Campaign Name indicates the comparison.
- Select the Use A/B Split
- Click Save.
Note: Do not start entering Email Settings at this point. With the Split functionality, you use the newly displayed A/B Split tab option to create two different email settings for each campaign flavor.
Once you have saved, two things happen. The system automatically displays a new tab option A/B Split in the Campaign Menu on top and grays out the Email Address and Template detail fields in the Email Settings.
- Go to the A/B Split tab and set up the two different campaign flavors.
For a description of all available fields, please see the Campaign A/B Split details. - First determine the Template Folder, where the system can find/locate the templates (both template A and template B need to reside in this folder).
In the field % of Results for Split Test you determine the percentage and consequently the amount of recipients which will be used for the A/B split. This A/B sample will be split in two, one A sample and one B sample, each in turn chosen at random. Consequently, if you enter a 10 % split, the system will use 5 % for A and 5 % for B.
Next, enter the Split Decision Type, i.e. determine the decision type based on which you will decide on your preferred campaign flavor.
You can also choose Undefined, if you want to decide this at a later time.
Split Decision Runtime: The run time fields allow you not only to schedule the sending of your campaign flavors, i.e. Split A and Split B, but also to send the winner of the Split decision automatically, by determining the run time and date.
- Once you have defined the General Settings for the A/B Split go to Split A and enter the Email Settings and the Runtime for campaign flavor A.
Note: You can send the sample campaigns automatically by setting a certain runtime, or manually by clicking the respective icons in the icon menu on top
(Run Split A or Run Split B).
- Next go to Split B and enter the Email Settings and the Runtime for Campaign flavor B.
In our example, we want to test which of two different template layouts is more successful. Therefore, we will leave all other fields identical, so as to get a fair estimate as to the success. Please note though that you can use A/B Split for checking the effectiveness of other things, such as subject lines, runtimes or email names, yet you should always only change and check one at a time.
When you have entered all the details, click SAVE.
- IMPORTANT and next: access Link Tracking and click the Add All Links icon in order to activate all links for tracking.
Caution: Make extra sure not to forget this step. If you send out the campaign without having added the links, you will not be able to do any tracking of clicked links whatsoever. In this case, any scheduled run time decisions, such as higher click rate, would be based on zero values.
Note: If the system is unable to detect a winner, it will chose Split A.
Alternatively, if you only want to include certain links in the split decision, use the Add links icon.
Per default, the Use for Split Decision checkbox will be selected for all added links. You can deselect them individually, by clicking into the respective link.
- Click Save.
- Next, go to the Queries tab and select/create the query. Run the query to see the number of results.
When you are ready to send the campaign to the chosen percentage of the group, go back to the Campaign Details and set the Status to In Process.
- When you are done, go back to the A/B Split option. See how the two Run icons in the actions menu have been activated.
You are now ready to run your campaign samples.
If you have entered runtimes, they will be sent automatically.
- In our example, we will trigger the sending by clicking the Run Split A icon rather than using a split run time date.
The system displays a confirmation message, since you cannot undo this step.
- Select OK.
The system starts sending campaign sample A.
Note how the field settings are grayed out as soon as you have sent the sample.
- Repeat this step for the campaign sample by clicking the Run Split B icon.
You have sent all your samples now and need to wait for the results to identify which of the campaign samples is more successful. On the bottom of the A/B Split option, you can view the Results in pie charts.
You can then use this information to send the more successful campaign. Caution: Remember that the scheduled runtimes and sending the entire campaign will only work if the Campaign Status has been set to In Process.
Campaign A/B Split Table Description of Fields
|
Field Name |
Description |
|
GENERAL SETTINGS |
|
|
Template Folder |
The Template Folder drop-down displays all folders that have physically been created as subfolders of the template folder as a further categorization of the templates. Typical subfolders can be, for instance, newsletter, birthday, loyalty, account statement, training, etc. Select the required folder from the drop-down list. |
|
% of Results for Split Test |
How many random recipients of your result list are to receive one of the two campaign flavors? Enter a percentage. Depending on the number of results you will probably enter something in between 1% and 10%. Note: You need to pick a percentage that is high enough to provide you with a meaningful result but not too high, since you would like to send the preferred campaign to most of your results. Tip: You can also use the A/B Split option to send out two different campaign flavors to ALL of your recipients. In this case, you would enter 100 here. |
|
Split Decision Type |
Determine the decision type based on which you will decide on your preferred campaign flavor. If you select undefined, you can take this decision later manually, after having seen and checked the result of your control samples. Once you have taken your decision, select the Split Decision Type drop-down again and choose either Manual Split A or Manual Split B. The other split decision types are automated and will be taken at the indicated run time date. You can base the decision on: - a higher click rate - a higher open rate or - a lower opt-out rate |
|
Split Decision Runtime |
The run time fields allow you not only to schedule the sending of (1) your campaign flavors, i.e. Split A and Split B (2); you can also send the winner of the Split decision automatically, by determining the run time and date. Select or enter the date and specify the preferred time settings. This is really helpful for large mailings in e.g. different time zones, since you can choose a ideal delivery time for a specific time zone. |
|
SETTINGS for SPLIT A |
|
|
Template (A) |
Select the template you want to use for this campaign, in our example, we use the OLD Template. |
|
Runtime for Split A |
Select or enter the date and specify the preferred time setting for running this campaign sample. Alternatively you can leave these fields empty and trigger the send manually. |
|
Subject (A) |
Subject line used for the campaign. |
|
Email From (A) |
Email Address that the campaign will be coming from, in other words, the sender email address. |
|
Email From Name (A) |
From Name used for the campaign, in other words, the sender name. Once the recipient receives the email, these three decisions will be in the email header, for instance. Recommended setting: checked (for regular mail campaigns) |
|
SETTINGS for SPLIT B |
|
|
|
Once finished you repeat the same steps for campaign flavor B, i.e. select the template, decide on the runtime, and define the address settings and the subject you want to use. |
|
Template (B) |
Select the template you want to use for this campaign, in our example, we use the NEW Template. |
|
Runtime for Split B |
Select or enter the date and specify the preferred time setting for running this campaign sample. Alternatively you can leave these fields empty and trigger the send manually, later. |
|
Subject (B) |
Subject line used for campaign. |
|
Email From (B) |
Email Address that the campaign will be coming from, i.e. the sender email address. |
|
Email From Name (B) |
From Name used for the campaign, in other words, the sender name. |
|
WEB ITEM OVERVIEW |
|
|
|
The Web Item Overview displays all activated template links that will be used for tracking the success of the campaigns. Additionally, the statistics will later show the number of emails that were sent, the open rate, the opt-outs and the click rate for A and B. |
0 comments
Article is closed for comments.