If you apply significance testing to a Cross Tab report, all the results are compared to see where the differences are large enough be considered statistically significant.
The test performed is a t-test on proportions that has a 95% level of significance.
The application assumes that the column percentage is the independent variable and that the rows are the dependent variables. Testing is done within each banner and compares the banner variables to each other and to the total. Each column is assigned a letter, and when a difference is found to be significant, the larger result displays the letter of the smaller result beside it.
- If you select this option for a Cross Tab report, the report may take longer to generate.
- A statistically significant difference does not necessarily mean that it is significant in the analysis of the results.
Significance test
This is the formula used to calculate significance of proportions in CrossTab reports for ColumnAnswerA to be significant against ColumnAnswerB:
//ColumnAnswer values are taken as percentages of column
question total
//ColumnAnswerCount and QuestionTotal values are counts
//Statistical Significance is 95%
(ABS(ColumnAnswerA - ColumnAnswerB) - 1.96 * SQRT
((ColumnAnswerA * (1 - ColumnAnswerA)) / QuestionATotal +
(ColumnAnswerB * (1 - ColumnAnswerB))/QuestionBTotal)) > 0
AND
ColumnAnswerCountA > ColumnAnswerCountB