Least Filled concept testing

Learn more about Least Filled concept testing and how to apply it.

Least Filled (also known as Least Full, Least Fill, or Least Filled Quota) is a concept testing methodology that prioritizes response collection for concepts which have the lowest number of completed responses.

In a Least Filled concept test, the application evaluates how many completions each concept has received whenever a participant begins the concept test portion of the survey. The evaluation occurs at the moment the participant enters the concept test portion, and it does not take into account incomplete responses or completions that occur while the participant is still in the concept test portion. When the evaluation is complete, the application shows the participant the concept that has the lowest number of completed responses.

If there is a tie in Least Filled response counts across two or more concepts, the group display order acts as a tiebreaker:

• Authored order

  The application goes down the concept list in authored order and displays the first occurring Least Filled concept.

• Random order

  The application changes the order of concepts each time the concept test is displayed to participants, goes down the randomized list, and displays the first occurring Least Filled concept.

• Rotated order

  The application selects a random concept and rotates the concepts above it to the bottom of the list (for example, a,b,c,d,e becomes c,d,e,a,b). Then the application goes down the rotated list and displays the first occurring Least Filled concept.

Eventually, the number of completed responses collected across all concepts is even. Least Filled concept testing can achieve even distribution with smaller sample sizes, unlike random distribution which works better with large sample sizes.

Note:

• Least Filled concept testing is available in modern surveys only.
• The Least Filled toggle must be enabled at all times while the survey is live. Turning the toggle on and off while the survey is in-field disrupts the ability to tabulate completed responses and determine which concept has the lowest number, thereby compromising the ability to achieve even distribution.

Create a basic Least Filled concept test

Example

Your clothing company wants to test 1 of 3 dress designs with customers.

1. Author your survey as desired.
2. Add a Group to your survey.
3. Edit the Group properties:
   1. Edit the Group name and Description.
   2. Select a Display objects in option:
      • Authored order
      • Random order
      • Rotated order
   3. Turn on the Apply Least Filled Concept toggle.
      

      When this toggle is turned on, the display option you chose changes and says using Least Filled at the end.

   4. Set the Number of objects to display to something other than All.
      In the example above, you'd select 1.
4. Add all the concepts inside the group.
   Note: If you want to nest survey pages inside the group, the Group object must be the top-level object and cannot be on its own page.

   

Create a Least Filled concept test with a qualification question preceding the concept test portion

Example

Your clothing company wants to test 3 of 12 clothing designs with customers. Some of the designs are only available in certain cities, so you want customers to only see designs that they will be able to buy. You add a qualification question before the concept test portion that asks for their home city. In the Least Filled group, you nest each concept inside survey logic that references the Home City question.

1. Author your survey as desired.
2. Add a qualifying question.
   In the example above, you'd add a Home City question. Ensure the qualifying question occurs before the concept test and is on its own page.
3. Add a Group to your survey.
4. Edit the Group properties:
   1. Edit the Group name and Description.
   2. Select a Display objects in option:
      • Authored order
      • Random order
      • Rotated order
   3. Turn on the Apply Least Filled Concept toggle.
      

      When this toggle is turned on, the display option you chose changes and says using Least Filled at the end.

   4. Set the Number of objects to display to something other than All.
      In the example above, you'd select 3.
5. Add all the concepts inside the group.
   Note: If you want to nest survey pages inside the group, the Group object must be the top-level object and cannot be on its own page.

   

6. Nest the conditional concepts inside their own Survey Logic actions.
   In the example above, you'd create a Survey Logic action for each clothing design that was only available in limited cities. For each Survey Logic action, you'd create a condition statement that specifies which cities.

   

Create a Least Filled concept test with qualification criteria on a concept

For this type of survey setup, the application evaluates the qualification criteria first and then applies the Least Filled concept test.

Example

Your clothing company wants to test 1 of 4 dress designs with customers. One of the designs is an expensive silk dress, so you want feedback from customers with a high household income ($150,000 or more) who are more likely to buy the dress. You already have Household Income stored as a profile variable. You can nest the silk dress concept inside survey logic that references that profile variable.

1. Author your survey as desired.
2. Add a Group to your survey.
3. Edit the Group properties:
   1. Edit the Group name and Description.
   2. Select a Display objects in option:
      • Authored order
      • Random order
      • Rotated order
   3. Turn on the Apply Least Filled Concept toggle.
      

      When this toggle is turned on, the display option you chose changes and says using Least Filled at the end.

   4. Set the Number of objects to display to something other than All.
      In the example above, you'd select 1.
4. Add all the concepts inside the group.
   Note: If you want to nest survey pages inside the group, the Group object must be the top-level object and cannot be on its own page.

   

5. Nest the conditional concept inside its own Survey Logic action.
   In the example above, you'd create a Survey Logic action to Show If HHI > $150K and nest the silk dress concept inside it.

Create a nested Least Filled concept test

Example

Your clothing company wants to test 2 designs from 3 apparel categories with customers:

1. Dresses (show 1 of 4)
2. Shirts (show 1 of 3)
3. Pants (show 1 of 3)

You first create a top-level Least Filled group for all the apparel categories. Then for each apparel category, you create a Least Filled group that shows 1 design from that category.

1. Author your survey as desired.
2. Add a top-level Group to your survey.
3. Edit the top-level Group properties:
   1. Edit the Group name and Description.
   2. Select a Display objects in option:
      • Authored order
      • Random order
      • Rotated order
   3. Turn on the Apply Least Filled Concept toggle.
      

      When this toggle is turned on, the display option you chose changes and says using Least Filled at the end.

   4. Set the Number of objects to display to something other than All.
      In the example above, you'd select 2.
4. Inside the top-level group, add a Group for each apparel category.
5. Edit the properties for each apparel category Group:
   1. Edit the Group name and Description.
   2. Select a Display objects in option:
      • Authored order
      • Random order
      • Rotated order
   3. Turn on the Apply Least Filled Concept toggle.
      

      When this toggle is turned on, the display option you chose changes and says using Least Filled at the end.

   4. Set the Number of objects to display to something other than All.
      In the example above, you'd select 1.
6. Add all the concepts inside the apparel category groups.
   Note: If you want to nest survey pages inside the group, the Group object must be the top-level object and cannot be on its own page.