Sign in
x
☰

Check for straight-liners with standard deviation

Updated at 2020-11-17 19:31:02 UTC

There are different methods to check for outliers or low quality data in survey responses. Protobi even has a QC tool that helps you flag respondents based on pre-set conditions like straight-lining. However, there are some quality checks that are more complex and are better written out in data process. One example is using standard deviation to check for straight-liners in a battery of responses. 

Normal distribution curve displaying standard deviation zones from -3σ to +3σ, with labeled percentages showing 34.1% in the central zones, 13.6% in the middle zones, and 2.1% and 0.1% in the tails.

In the example below, Q13 is a battery of 7-point scale questions. You could use the QC tool to perform the pre-set straight-line check. However you might consider respondents  who gave very similar responses for each question suspicious in addition to true straight-liners. 

Survey question Q13 displaying responses for rating Product B device features using a 7-point scale where -3 is significantly worse, 0 is the same, and +3 is significantly better than the current device. Five features are shown: Q13_1 (Ease of daily use) with 34% at 0 Same, Q13_2 (Ease of packet and priming) with 30% at 0 Same, Q13_3 (Size) with 35% at 0 Same, Q13_4 (Wastage) with 22% at 0 Same, and Q13_5 (Dose counter) with 19% at 0 Same.

In project Pre-calculate you can define function that calculates the standard deviation between the children of a group.

Protobi Pre-Calculate settings screen with navigation menu on left (Open, Overview, Wiki pages, Data, etc.) and code editor on right displaying JavaScript for standard deviation calculation. The code includes a function to calculate standard deviation using d3.deviation and a checks array containing ["Q13_1", "Q13_2", "Q13_3", "Q13_4", "Q13_5"] with comments explaining the function usage.

Code

//Function used to calculate standard deviation

function std(row, kidKeys) {

    var vals = kidKeys.map(function(kidKey) { return row[kidKey] }); 

    if (vals[0]) var res = d3.deviation(vals)

    return res;

} 

//Groups/children for which you want to calculate std

var checks = {

    "Q13": ["Q13_1", "Q13_2", "Q13_3", "Q13_4", "Q13_5"]

}

//Create new "std_" variable standard deviation calculation

rows.forEach(function(row) {

    for (var key in checks) {

    row["std_"+key] = std(row, checks[key])

    }

});

return rows;

 

The code above create a new element with std_ appended as a pre-fix.  For the distribution below the round by is [0,0.5,1,3]. 

Protobi element std_Q13 displaying a horizontal bar chart of standard deviation distributions with blue bars. The largest segment is 0.6 to 1 at 48%, followed by 1.1 to 3 at 29%, 0.1 to 0.5 at 17%, and ≤ 0 at 6%. Below the bars, Mean is shown as 0.835 and N as 100.

If you click into the zero values in the standard deviation element, you should see that all those respondents were flagged in the straight-line check from the QC tool. 

Protobi interface displaying two elements side by side: Sflags element showing "straightline Q13" at 100% and "Not flagged" at 0% with N=6, and std_Q13 element showing standard deviation ranges where the ≤ 0 category is highlighted with a yellow 100% bar, while other ranges (0.1 to 0.5, 0.6 to 1, 1.1 to 3) show 0%, with Mean 0 and N 6.

If you click to drill in on respondents with a low standard deviation (e.g. 0.1 - 0.5), you can choose individual respids to check. This respondent did not straight-line, but they use the same rating for all but one of the questions in the battery. 

Protobi interface showing three elements: respid list with Filter and Apply button, std_Q13 showing standard deviation of 0.447 with 100% in 0.1 to 0.5 range (N=1), and Q13 battery results where all five product features show 100% selecting -3 Significantly worse compared to Top 2 Box, with column headers showing Bottom 2 Box, Middle 3 Box, Top 2 Box, and N.

Advanced support

There are complex cases where you might want to flag respondents, not limited to the example in this tutorial. Contact us at support@protobi.com to discuss your specific needs.