Sign in
x
☰

Quality check responses and flag outliers

Updated at 2020-11-11 17:42:17 UTC

Protobi has a built-in quality check or "QC" tool that allows you to identify outliers and flag respondents with low quality responses. 

Quality checks dialog for Q13 showing six conditional check options including 'Any child equals', 'Any child is more than', 'Any child is less than', 'All children are equal (aka straightline)', 'Sum of children not equals', and 'If custom condition is true', each with corresponding input fields.

Open the QC dialog

Select an element and press Shift + Q + C on your keyboard to bring up the QC dialog.

Horizontal bar chart for question Q1 asking 'Thinking about your Condition X patients, how many of those are currently receiving a GA (gamma antagonist)?' showing six response ranges with '1 to 15' having the highest percentage at 74.0%.

You can also select the element, then go to the "Advanced" button. Scroll to find the "QC check (beta)" option.

Dropdown menu from the Advanced menu button showing various options including 'QC checks (beta)' highlighted at the bottom with keyboard shortcut '(Ctrl G)'.

QC a single element

Single elements have three pre-set QC options. These checks are usually applied to numeric questions where you want to flag and cap outlier responses. 

Quality checks dialog for Q1 showing four check options with 'Q1 is more than' checked and set to value '50', while other options remain unchecked.

QC checks are stored in a special element

After you add a QC check, it creates a new element called "$flags". Every time you add a new QC check, it's added to this element. 3% of respondents are flagged as Q1 > 50 and those that do not meet the criteria are shown as [NA]. 

Horizontal bar chart for the $flags variable showing [NA] responses at 97.0% and flagged responses where 'Q1 > 50' at 3.0%, with a base of N=100.

QC a group

There are more QC options for a group of questions, for instance there is a preset to check for straight-liners.  

Quality checks dialog for Q13 displaying six checkbox options for flagging respondents based on Q13 children values, all currently unchecked with empty input fields.

Straight-line check

Select "All children are equal (aka straightline)" to flag respondents that entered the same answer for all questions in a group. 

Question Q13 showing a 7-point scale rating comparison for medical device features, with response categories ranging from ease of use to dose counter, with percentages displayed for each feature.

Result:

Horizontal bar chart for $flags variable displaying three categories: [NA] at 92.0%, 'straightline Q13' at 6.0%, and 'Q1 > 50' at 3.0%, with base N=100.

Remove flagged respondents

Shift + press on flagged respondents to exclude them from the sample. Then set a permanent filter on the question to exclude the flagged responses.

Split view showing $flags variable on the left with highlighted '[NA]' category at 98% in yellow, and Q13 question data on the right displaying device rating features with percentages.

Create your own custom flags

You can define a condition to create custom flags. Q3 asks how many Condition Y patients are on a gamma antagonist, and S8 asks how many Condition Y patients the respondent manages. 

Dual-panel view showing two questions (Q3 about Condition Y patients with GA treatment and S8 about number of patients managed per month) with Filter text boxes and Apply buttons above each chart.

You can set a custom condition that flags respondents that entered a greater value for Q3 than S8. Custom conditions can reference any element, you can even create the QC in an element, and not include it in the condition. 

Quality checks dialog for Q3 showing four check options with 'If custom condition is true' checked and the condition 'Q3>S8v2' entered in the input field.

Javascript operators

The custom condition text box will accept conditional statements written with Javascript operators. Below is a list of to create operators you can use to create custom conditions. 

Arithmetic

+

Addition
-Subtraction
/Division
*Multiplication

Comparison

==Equal
!=Not equal
>Greater than
>=Greater than or equal to
<Less than
<=Less than or equal to
===Equal to, and the same data type (Identical)
!==Not identical

Logical 

&&AND
||OR

Advanced support

There are complex cases where you might want to flag respondents, not limited to what's shown in this tutorial. Contact us at support@protobi.com to discuss your specific needs.

apply the filter