Variation sets allow you to run very similar tests on different versions of a design, or multiple tests on the same design, and ensure that all participants across all tests in the set are unique. This ensures that participants have not seen the design before, so will not be biased by previous tests.
This is great for iterating on a design or testing variations to see which performs best.
💡 Variation sets are available on our Basic and Pro plans.
How to create a Variation set
To create a variation set, head to any existing test you'd like to include in the set or create a new test. Click Make variation set on the top-right corner of your test to get started:
Next, you'll need to give your new variation set a name:
Your test will be added to the set automatically. You can now click Copy variation to create a duplicate of this test for quick editing:
You can also search for and add any other existing tests to your set:
Sharing your variation set with your own audience
Similar to other tests, a variation set has a unique link you can share with your audience to direct them to take a test from your set. Toggle on and copy this link from the Recruit participants tab.
Recruits who visit this link will be taken to one random variation. We'll split them evenly across variations. Your recruited participants will only be allowed to participate in one of the tests in this set, whether they revisit the set's recruitment link or visit the recruitment link of any test in the set.
If you are on the Pro plan, you can request demographic information from your participants by turning on Request demographics from the Recruit participants tab of your set. Information collected will be available with results: age, country, gender, and education level.
Collecting unique panel responses with variation sets
You might be curious about whether you'll automatically see new participants in all your tests. It's important to note that by default, we don't automatically provide unique panelists for each test. However, variation sets come to the rescue, allowing you to make sure that all participants across all tests in the set are indeed unique. This means that each participant has not encountered the design before, removing the potential for biases from previous tests.
Placing panel orders using variation sets
Unlike recruiting from your own audience, when you order responses from the Lyssna panel you cannot place an order against the variation set to receive an equal split of unique participants across your tests. Instead, you will need to place individual response orders for each test within your variation set.
For example, if you want to survey a total of 100 unique participants across 4 tests equally in a variation set, you will need to place an order for 25 responses on each test. By placing your tests inside a variation set we ensure that each response comes from a unique panelist.
Please note that this restriction can affect order delivery times if you order many responses (or from niche demographics) for tests within a variation set. The estimated turnaround time for the order you place on your last test may be longer than your first test as those panelists have been removed from the pool of eligible participants.
💡 Helpful tip: You can add older existing tests to any set to prevent those same participants from taking newer tests within the set, no matter when those older tests were run.
Results
The results for your variation set will not be combined—each test will have its own results. If you’re looking to compare or aggregate them, you can download the CSV for each test and combine them in your software of choice (Excel, Numbers, Sheets, etc.).
Have another question about your variation set? Chat with us from the bottom right of your screen or send us an email to support@lyssna.com.