Protobi

← All help posts

View post

Detailed documentation is avaialble in the following links with a more explanatory discussion below.
* [Protobi REST API](https://help.protobi.com/api-references/protobi-rest-api)
* [Protobi JSON API](https://protobi.com/tutorials/json)
* [Protobi Chart Types configuration](https://protobi.com/tutorials/charttypes)

## Parts of a Protobi project
A Protobi project is defined by three key concepts:
 * **Dataset** Meta information including the project title, permissions, API connections etc.
 * **Elements**  An array of JSON object describing  each element in the project and how it is to  be displayed.
 * **Data** Data sources, including raw data (typically in CSV .csv or SPSS .sav), data processes (typically Javascript or SQL), and general documents.

All of the above can be set up manually online via the Protobi web site.  This is usually the simplest way to go for 1, 10, or even 100 projects.

Protobi can also automatically connect to survey engines to get all of this information programmatically over the web using API connections.  Protobi current can read Forsta (Decipher and Confirmit), OpinionRoute (formerly Jibunu) and TypeForm.  We're also more than happy to write custom pipeline connectors to your survey engine or your preferred vendor, just ask.  This is usually the simplest way to go for  100+ surveys.

## Create and update a project via REST

### Create a new project
To create a new project specify the project name, title, logo, etc. to Protobi, which will return a dataset object which has a unique dataset identifier:

```js
let myCompany = 'abcresearch'
let API_KEY = '********' // see https://app.protobi.com/account
var uri_create = `https://${myCompany}.protobi.com/api/v3/dataset`;
var content = {
  name: "Consumer survey Wave 1",
  title: "First wave of Consumer Preference Tracker Q22025",
  description: "Billing code A123345",
  logo_url: "https://mycompany.com/images/mylogo.png",
  image_url: null,
  primary_table: 'main',
  url: "https://mycompany.com",
  properties: {
    "foo": "bar"
  }
}

let dataset = await fetch(uri_create,  { 
	method: "POST",
	headers: {
	  'Content-Type': 'application/json',
	  'x-api-key': API_KEY
	},
	body: JSON.stringify(content)
}) 
```
### Set/Update project elements

To initialize the project with your survey configuration, post an array of JSON elements to the following URL with the API key in the `x-api-key` header and specify the action is to "update" the given elements (as opposed to "reset")

```js
let myCompany = 'abcresearch'
let myDatasetId = 'p192382933'
let elements = [
	{"key": "$root", "children": [ "S", "A", "B", "C"], "colors":  { "default": [ "#39C", "#26A", "#9B6"}}, 
	{"key": "A", "displayKey": "Demographics" , "title": "A few questions", "children": ["A1", "A2", "A3"]},
	{"key": "A1", "title": "What is your age?" , "survey": { scale: "numeric", dimension: "scalar"}},
... ]

let uri = `https://${myCompany}.protobi.com/api/v3/dataset/${dataset._id}/element?action=update`
const response = await fetch(uri, {
  method: "POST",
	headers: {
	  'Content-Type': 'application/json',
	  'x-api-key': API_KEY
	},
  body: JSON.stringify(elements)
});
```

### Upload a CSV or SPSS data file
To upload a data POST the CSV or SAV data as a binary blob in a multiform attachment:

```
let table_key = 'main' // same value as primary_table when creating dataset above
let url = `https://${myCompany}.protobi.com/v3/datasets/${dataset._id}/data/${table_key}`
let fd = new FormData();
fd.append('file', new File([new Blob([csv])], filename));
$.ajax({
    url: url,
    headers: { 'x-api-key': API_KEY },
    data: fd,
    processData: false,
    contentType: 'text/csv', // or 'application/spss-sav'
    type: 'POST',
    success: console.log,
    error: console.error
});
```

## Push vs pull 
There are two basic modes to set up Protobi projects programmatically:
* **“pull“** where Protobi calls the survey engine API to request the information
* **“push”** where your servers call Protobi to send the information

For most enterprise clients, Protobi “pulls” data from the survey engine given the survey identifier, and we write a short program to create and configure the Protobi project using business and design rules we work out with your team.  In this approach the Protobi team writes all the code to setup the project, and that's generally simplest for all and the approach we recommend.

Alternatively In the “push” mode, your programmers create the Protobi information and send it to our surveys.  This is more advanced as it requires more specific work by your team to decide how to configure the Protobi project to represent your survey.

The document below describes the push method in detail.

## Elements
The key thing to create is the elements. This is simply an JSON array which lists each question,  subquestion, and section in the survey as an “element”.  Each element is a JSON object with a set of attribute:value pairs.  The principal attributes you'll need to specify are:

Attribute |  Values | Example | Description
---------|--------|------------|--------
key | string | "Q1" | Unique identifier
displayKey | string | "Question Q1" | Identifier to present for display
title | string | "Q1. How are you?" | Question text
format | object | { "1": "Yes" } | Value labels for each response option
children | array | ["Q1a","Q1b"] | Identifiers for child questions or subquestions
type | string | "empty" | Specify "empty"  for elements that do not correspond to a data column
compactto | value(s)  |  `[4,5]` | Summary value for concise display
sortLast | array | [98] | Values that are outside the  scale, e.g. `98` for "Refused to answer"
survey.scale | string |  "discrete" | Response type, "discrete", "numeric", "text", "date" 
survey.subscale | string | "ranking" | Subtype, "general", "nominal", "binary", "rating", "ranking", "interval"
survey.dimension | string | "grid1d"| Dimensionality, "scalar", "grid1d", "grid2d", ""

### Root element

The first element to specify is the  root element. This must have the key `"$root"` and it can be as simple as the example below.  
```json
{
	"key":  "$root",
	"children": [ "S", "A", "B", "C", "D", "system"]`
}
```
Here `"S", "A", "B", "C", "D", "system"` are names of sections within the survey.  These can be anything, but ideally should match the naming convention of your survey engine.  For instance they could just as easily be `"screener", "questionnaire", "sample", "admin"` instead.

You can optionally specify additional details, such as brand colors and other style preferences:
```json
{
	"key":  "$root",
	"children": [ "S", "A", "B", "C", "D", "system"],
	"title": "Consumer Preference Survey Wave 1",
	"id": "respid",
	"weight": "wt_proj",
	"precisionPercent": 0,
	"precisionFreq": 0,
	"nsymbol": "N",
	"nunwtdsymbol": "N (unweighted)",
	"colors":  {
		"default":  ["#39C", "#26A", "#9B6", "#FA0", "#B22", "#508"],
		"ascending":  ["#26A", "#EEE",  "#9B6"],
		"descending":  ["#9B6", "#EEE",  "#26A"]
	}		
}
```

## Sections
Survey sections are presented as collections in Protobi.  For instance, the screener section could be identified as a collection with a list of questions in the screener:
```json
{
	"key":  "S",
	"displayKey": "Screener", 
	"title": "Before we begin, we need to ask you a few brief questions to determine if you match the specific participant profile we're looking for in this study. These screening questions will take approximately 1-2 minutes to complete.",
	"children": [ "S1", "S2", "S3", "S4", "S5"],
	"type": "empty"
	}
```

* **key** is a unique identifier for the section.  
* **displayKey** is an optional alternate identifier suitable for presentation to the end client. 
* **title** is the survey text or other descriptive information.
* **children** is an array of element keys for questions or subsections.
* **type** "empty" indicates that the screener does not have its own column in the data file.

Similarly each section would also be represented as a collection. 
```json
{
	"key":  "A",
	"displayKey": "Purchase history", 
	"title": "Let's begin the survey!  I'd like to ask you a few questions about your most recent purchases of inline skates",
	"children": [ "A1", "A2", "A3", "A4", "A5", "A6", "A6"],	"type": "empty"
	}
```

## Questions
Each question is represented in Protobi as an element.  For compound questions  each subquestion is also represented as an element.  The attribute "survey" gives Protobi hints as to how best to display the question for analysis.

It's optional but very helpful to include an attribute **questionType** with hints that let Protobi know how best to display the data.  This is an associative array with three attributes:
* **dimension**  which can be "scalar", "grid1d", "grid2d" (or blank)
* **scale**  which can be "discrete", "numeric", "binary", "boolean", "text", or "date"
* **subscale**  which desribes the flavor of the scale:
  * discrete:  `"nominal", "rating", "ranking"
  *  numeric:  "integer", "real"
  *  text: "short" or "long"
  *  date: "date", "date+time", "interval"

### Single choice question (discrete nominal scalar)
Here `"Q1"` has an optional `format` attribute which specifies how raw values are to be formatted for display to the user, an an optional string title attribute.

```json
{
	"key": "Q1",
	"field": "Q1",
	"title": "Q1. What is your primary specialty?",
	"color":  "#9B6",
	"format": {
		"1": "Primary care",
		"2": "Allergy",
		"3": "Pulmonology",
		"4": "Otolaryngologist",
		"5": "Other (specify)"
	},
	"survey":  {
		"scale" : "discrete", 
		"subscale": "nominal",
		"dimension": "scalar"
	}
}
```

Here 
* the attribute **format** is an associative array that specifies the choice options, and directly corresponds to the concept of value formats in SPSS.
* the attribute **title** contains the question text as presented to the respondent.
* the attribute **field** indicates the name of the data column in the data file for this question.  This is optional if the field and the key are identical.
* the attribute **questionType** describes the type of question
   *  "scalar" indicates that this question has a single value from a single column
   *  "discrete" indicates this question has a limited set of responses
   *  "nominal" indicates that the responses have no inherent order

### Scalar questions
Scalar questions are those that correspond to a single data column in the data file.  Generally these will be single response questions where the respondent provides just one answer. (However there can arise exceptions as Protobi can read multi-response delimited values)

### Grid questions
Grid questions are collections of questions with a common foundation, with the question text often containing a phrase like "For each of the following" or "Check all that apply".  Most commonly the grid question itself does not correspond to a data column, rather each subquestion does. There will be one Protobi element for the question itself, and one for each subquestion.

## Question types

### Single rating (discrete ordinal scalar)
Here `"Q2"` is a rating scale on a five-point ordinal scale

```json
{
	"key": "Q2",
	"field": "Q2",
	"title": "Q2. How much do you like the Boston Red Sox?",
	"color":  "#9B6",
	"format": {
		"1": "Greatest team ever",
		"2": "Great team",
		"3": "Representative of US MLB",
		"4": "Terrible team",
		"5": "Worst team ever"
	},
	"compactto": [1,2],
	"colors": "ascending",
	"survey":  {
		"scale" : "discrete", 
		"subscale": "ordinal",
		"dimension": "scalar"
	}
}
```

Here 
* the attribute **questionType.ordinal** indicates that the responses have an inherent order
* the attribute **compactto: [1,2]** indicates the top box score relevant for reporting

#### Simple binary questions
Here `"Q4"` is a numeric yes/no question:

```json
{
	"key": "Q3",
	"field": "Q3",
	"title": "Q3. Have you attended any major league baseball games in the last year?",
	"color":  "#9B6",
	"format": {
	   "0": "No",
		 "1": "Yes"
	 },
	"compactto": 1,
	"survey": {
	  "scale": "discrete",
		"subscale" : "binary", 
		"dimension": "scalar"
	}
}
```
* the scale " **discrete"** indicates that the responses are fixed choices.
* the subscale " **binary"** indicates that the responses are yes/no or true/false.
* the attribute **compactto: 1"** indicates that we should show value 1 as the summary metric

#### Simple numeric questions
Here `"Q4"` is a numeric open end question:

```json
{
	"key": "Q4",
	"field": "Q4",
	"title": "Q4. How many baseball games have you attended in the past 12 months?",
	"color":  "#9B6",
	"roundby": [0,1,5,10,25],
	"compactto": "$mean",
 "survey":  {
		  "scale": "discrete",
	 	 "subscale" : "binary", 
		 "dimension": "scalar"
	}
}
``` 
* the attribute **questionType.ordinal"** indicates that the responses have an inherent order
* the attribute **compactto: "$mean"** indicates the summary metric relevant for reporting

#### Simple text open end questions
Here `"Q5"` is a numeric open end question:

```json
{
	"key": "Q5",
	"field": "Q5",
	"title": "Q5. What do you like best about attending major league baseball games?",
	"color":  "#9B6",
	"survey":  {
		"scale" : "text", 
		"subscale": "general",
		"dimension": "scalar"
	}
}
```
 
* the scale **text** indicates that the responses are text open ends,
* the subscale **general** indicates that the responses are not expected to be short.

#### Binary grid questions
Here `"Q6"` is a binary grid1d question:
```json
{
  "key": "Q6",
  "field": "Q6",
  "title": "Q6. Which of these sports did you play in high school?",
  "color":  "#9B6",
	"format": {
	  "0": "No",
		"1": "Yes"
	},
	"children": [ "Q6a", "Q6b", "Q6c", "Q6d", "Q6e", "Q6f"],
	"type": "empty", '
	"compactto": 1,
  "survey":  {
    "scale" : "binary", 
    "dimension": "grid1d"
  }
}
```

Here 
* the attribute **questionType.scale"** indicates that the responses are binary responses
* the attribute **questionType.dimension"** indicates that the element is a group of scalar questions.

This question would typically be accompanied by an array of additional elements for Q6a, Q6b, etc. 
```json
{
  "key": "Q6a",
  "field": "Q6a",
  "title": "Q6a. Baseball",
  "color":  "#9B6",
	"compactto": 1,
  "format": {
	  "0": "No",
		"1": "Yes"
	},
  "survey":  {
    "scale" : "binary", 
    "dimension": "scalar"
  }
}
```

### Binary Grid1D questions
Binary Grid1D questions are "check all that apply" questions, where each response is either positive (e.g. "Yes", "True", or something like that), negative ("No", "false" or something like that) or blank.

Here `"Q6"` is a binary grid1d question:
```json
{
  "key": "Q7",
  "field": "Q7",
  "title": "Q7. Which of the following sports did you play in high school",
  "color":  "#9B6",
	"children": [ "Q7a", "Q7b", "Q7c", "Q7d", "Q7e", "Q7f", "Q7oe"],
	"type": "empty", '
	"compactto": 1,
	"format": {
	  "0": "Not selected",
		"1" : "Selected"
	},
  "survey":  {
    "scale" : "discrete", 
		"subscale": "binary"
    "dimension": "grid1d"
  }
}
```

* the scale value **discrete** indicates that the responses are closed end choices
* the subscale value **binary** indicates that there are just two choices (ignoring blank or missing values)
* the dimension **grid1d** indicates this is a collection of binary questions with a common question text and format

This question would typically be accompanied by an array of additional elements 
```json
{ 
   "key": "Q7a", 
	 "title": "Baseball", 
	"compactto": 1,
	"format": {
	  "0": "Not selected",
		"1" : "Selected"
	},
  "survey":  {
    "scale" : "discrete", 
		"subscale": "binary"
    "dimension": "grid1d"
  }
}
```

Here 
* the attribute **questionType.scale"** indicates that the responses are binary responses
* the attribute **questionType.dimension"** indicates that the element is a group of scalar questions.

This question would typically be accompanied by an array of additional scalar elements corresponding to each subquestion:
```json
{ 
   "key": "Q7a", 
	 "title": "Baseball", 
	"compactto": 1,
	"format": {
	  "0": "Not selected",
		"1" : "Selected"
	},
  questionType:  {
    "scale" : "discrete", 
		"subscale": "binary"
    "dimension": "scalar"
  }
}
```

### Other/specify
Grid questions may contain "other/specify questions" such as "Q7oe" in this example:
```json
{ 
   "key": "Q7ie", 
	 "title": "Other (specify)", 
  "survey"::  {
    "scale" : "text", 
		"subscale": "other/specify"
    "dimension": "scalar"
  }
}
```
* the scale **text** indicates this expe
* the attribute **questionType.dimension"** indicates that the element is a scalar question.

### Numeric Grid1D questions
Numeric Grid1D questions are collections of numbers such as "For each of the following what percent are...", where each response is a number.  The question may optinoally have a constraint such as "Must sum to 100" or "Must sum to more than 100".

Here `"Q8"` is a binary grid1d question:
```json
{
  "key": "Q8",
  "field": "Q8",
  "title": "Q7.  For each of the following sports how much did you spend in the last year?" ,
  "color":  "#9B6",
	"children": [ "Q7a", "Q7b", "Q7c", "Q7d", "Q7e", "Q7f", "Q7oe"],
	"type": "empty", '
	"compactto": "$mean",
	"roundby": "log",
  "survey":  {
    "scale" : "numeric", 
    "dimension": "grid1d",
  }
}
```

* the scale value **numeric** indicates that the responses are closed end choices
* the dimension **grid1d** indicates this is a collection of binary questions with a common question text and format

This question would typically be accompanied by an array of additional elements 
```json
{ 
   "key": "Q8a", 
	 "title": "Baseball", 
	"compactto": "$mean",
  "survey":  {
    "scale" : "numeric", 
    "dimension": "grid1d"
  }
}
```

* the attribute **questionType.scale"** indicates that the responses are binary responses
* the attribute **questionType.dimension"** indicates that the element is a scalar question.

### Rating Grid1D questions
Rating Grid1D questions are collections of ratings such as "For each of the following how much do you agree...", where each response is a discrete rating with an ordinal scale.

Here `"Q10"` is a rating grid1d question:
```json
{
  "key": "Q10",
  "field": "Q10",
  "title": "Q10.  For each of the following statements how much do you agree?" ,
  "color":  "#9B6",
	"children": [ "Q10a", "Q10b", "Q10c", "Q10d", "Q10e", "Q10f", "Q10oe"],
	"type": "empty", '
	"compactto": [4,5],
	"colors": "ascending",
  "format": {
		"1": "Strongly disagree",
		"2" : "Somewhat disagree",
		"3": "Neutral",
		"4": "Somewhat agree",
		"5": "Strongly agree",
	  "9": "Don't know",
	},
	"sortLast": ["9"],
  "survey":  {
    "scale" : "discrete", 
		"subscale" : "rating",

"dimension": "grid1d",
  }
}
```

* the scale value **discrete** indicates that the responses are closed end choices
* the subscale **rating** indicates that the responses are ordinal
* the dimension **grid1d** indicates this is a collection of binary questions with a common question text and format

In this example additioanl options are specified:
* **compactto** specifies that we'd like to summarize this with top-box scores
* **sortLast** indicates that response 9
* **colors** indicates that the scale is ascending responses should be colored as such

This question would typically be accompanied by an array of additional elements 
```json
{ 
   "key": "Q10a", 
	 "title": "I am really into sports", 
	"compactto": [4,5],
	"format": {
		"1": "Strongly disagree",
		"2" : "Somewhat disagree",
		"3": "Neutral",
		"4": "Somewhat agree",
		"5": "Strongly agree",
	  "9": "Don't know",
	},
  "survey":  {
    "scale" : "discrete", 
		"subscale" :"ranking",
    "dimension": "scalar"
  }
}
```

* the scale **discrete** indicates that the responses are single choice
* the subscale **rating** indicates that the choices are ordinal
* the dimension **scalar** indicates that this is a single choice

### Grid2D questions
Grid2D questions are two dimensional grids, such as "For each of the following brands, please rate your experience on each of the following attributes".
There would be two levels of elements

```json
{
  "key": "Q11",
  "field": "Q11",
  "title": "Q11.  For each of the following brands, please rate your experience on each of the following attributes?" ,
  "color":  "#9B6",
	"children": [ "Q11a", "Q11b", "Q11c"],
	"type": "empty", '
	"compactto": [4,5],
	"colors": "ascending",
  "format": {
		"1": "Strongly negative",
		"2" : "Somewhat negative",
		"3": "Neutral",
		"4": "Somewhat positive",
		"5": "Strongly positive",
	  "9": "Don't know",
	},
	"sortLast": ["9"],
  "survey":  {
    "scale" : "discrete", 
		"subscale" : "rating", 
    "dimension": "grid2d",
  }
}
```

There would be a Grid1D element for each child, e,g,.

```json
{
  "key": "Q11a",
  "field": "Q11",
  "title": "Yankees ,
  "color":  "#9B6",
	"children": [ "Q11a1", "Q11a2", "Q11a3", "Q11a4"],
	"type": "empty", '
	"compactto": [4,5],
	"colors": "ascending",
  "format": {
		"1": "Strongly negative",
		"2" : "Somewhat negative",
		"3": "Neutral",
		"4": "Somewhat positive",
		"5": "Strongly positive",
	  "9": "Don't know",
	},
	"sortLast": ["9"],
  "survey":  {
    "scale" : "discrete", 
		"subscale" : "rating", 
    "dimension": "grid1d",
  }
}
```

and there would be a scalar element for each grand child
```json
{
  "key": "Q11a1",
  "field": "Q11",
  "title": "Competitive team",
  "color":  "#9B6",
	"compactto": [4,5],
	"colors": "ascending",
  "format": {
		"1": "Strongly negative",
		"2" : "Somewhat negative",
		"3": "Neutral",
		"4": "Somewhat positive",
		"5": "Strongly positive",
	  "9": "Don't know",
	},
	"sortLast": ["9"],
  "survey":  {
    "scale" : "discrete", 
		"subscale" : "rating", 
    "dimension": "scalar"
  }
}
```

Publishing

Date

Status

Slug edit

Content

Thumbnail

Categories Manage

Tools

AI Database

Publishing

Content

Tools

Danger zone