structure_demo.qmd

---
title: "Machine-Readable Structure Demo"
toc: false
---

Here we will give an example of a basic machine-readable study description. This example uses JSON format, which could be converted into any structured format.

### Study framework

The primary unit is a study, which contains categories where different study components will be described, such as the hypotheses, methods, data, and analyses. Here, we will focus on how we can specify the hypotheses and their links to the analyses.

```
{
    "name": "Kinship and Prosocial Behaviour",
    "authors": []
    "hypotheses": [],
    "methods": [],
    "data": [],
    "analyses": []
}
```

### Hypotheses

At the most basic level, each hypothesis needs a verbal description. A study could contain multiple hypotheses, but our example contains only one. The example below contains placeholders for describing the criteria that will support or falsify the prediction, which we will fill in later in relation to the planned analysis output.

```
    "hypotheses": [
        {
            "description": "Cues of kinship will increase prosocial behaviour. Cues of kinship will be manipulated by morphed  facial self-resemblance. Prosocial behaviour will be measured by responses in the trust game. The prediction is that the number of trusting AND/OR reciprocating moves will be greater to self-resembling faces than to non-self-resembling faces.",
            "criteria": [],
            "support": {},
            "falsify": {}
        }
    ],
```

### Analyses

Next, we can specify the analyses. Here we will list the information necessary to run a t-test in R (indicated by the "software" value), but it would be possible to create machine-readable specifications for analyses using any other software package. We give the name of the function to be used and list the parameters. The notation ".data[kin]\$trust_self" refers to the "trust_self" column of a dataset called "kin" to be specified later. Here, we have two analyses: a t-test comparing trusting moves to self-resembling faces ("trust_self") versus non-self-resembling faces ("trust_non"), and a t-test comparing reciprocating moves to self-resembling faces ("recip_self") versus non-self-resembling faces ("recip_non").

```
    "analyses": [
        {
            "id": "trust_analysis",
            "software": "`r R.Version()$version.string`",
            "func": "t.test",
            "params": {
                "x": ".data[kin]$trust_self",
                "y": ".data[kin]$trust_non",
                "paired": true,
                "conf.level": 0.99
            },
            "results": {
                "conf.int": []
            }
        },
        {
            "id": "recip_analysis",
            "software": "`r R.Version()$version.string`",
            "func": "t.test",
            "params": {
                "x": ".data[kin]$recip_self",
                "y": ".data[kin]$recip_non",
                "paired": true,
                "conf.level": 0.99
            },
            "results": {
                "conf.int": []
            }
        }
    ]
```

### Criteria for support or falsification

The criteria for support and falsification now can be specified in relation to the planned analyses. Each criterion is specified with an ID, used to reference the criterion in the evaluation rules for support and falsification. We have two criteria that we need to check to determine support or falsification for each analysis: (1) Is the lower-bound of the 99% CI  larger than 0? and (2) Is the upper bound of the 99% CI larger than 0.2 (our smallest effect size of interest)? After data collection and running analyses, each criterion's "conclusion" value is changed from NULL to TRUE or FALSE.

```
            "criteria": [
                {
                    "id": "trust_lowbound", 
                    "analysis_id": "trust_analysis",
                    "result": "conf.int[1]", 
                    "operator": ">", 
                    "comparator": 0,
                    "conclusion": NULL
                },
                {
                    "id": "trust_highbound", 
                    "analysis_id": "trust_analysis",
                    "result": "conf.int[2]", 
                    "operator": ">", 
                    "comparator": 0.2,
                    "conclusion": NULL
                },
                {
                    "id": "recip_lowbound", 
                    "analysis_id": "recip_analysis",
                    "result": "conf.int[1]", 
                    "operator": ">", 
                    "comparator": 0,
                    "conclusion": NULL
                },
                {
                    "id": "recip_highbound", 
                    "analysis_id": "recip_analysis",
                    "result": "conf.int[2]", 
                    "operator": ">", 
                    "comparator": 0.2,
                    "conclusion": NULL
                }
            ]
```

### Evaluation of support or falsification

The values of "support" and "falsify" contain a verbal description of the criteria and an evaluation rule for determining what patterns of criteria support the hypothesis. In our example, the hypothesis is supported if both of the criteria are true for either the trust or reciprocation moves ("(trust_lowbound & trust_highbound) | (recip_lowbound & recip_highbound)") and the hypothesis is falsified if the second criterion is false for both the trust and reciprocation moves ("!trust_highbound & !recip_highbound"). All other patterns of results are deemed inconclusive.

```
            "support": {
                "description": "The hypothesis is supported if the 99% CI lower bound is greater than 0 and the 99% CI upper bound is greater than 0.2 (the SESOI) for either the trust or reciprocation moves.",
                "evaluation": "(trust_lowbound & trust_highbound) | (recip_lowbound & recip_highbound)"
            },
            "falsify": {
                "description": "The hypothesis is falsified if the 99% CI upper bound is smaller than 0.2 (the SESOI) for both trust and reciprocation.",
                "evaluation": "!trust_highbound & !recip_highbound"
            }
```

### Data

We can describe datasets using an existing codebook format from the [PsychDS](https://psych-ds.github.io/) project by setting the "\@context", "\@type" and "schemaVersion" values appropriately. Each dataset is given an ID for referencing in analysis scripts. 

```
    "data": [
        {
            "id": "kin",
            "@context": "https://schema.org/",
            "@type": "Dataset",
            "schemaVersion": "Psych-DS 0.1.0",
            "variableMeasured": [
                {
                    "type": "PropertyValue",
                    "unitText": "id",
                    "name": "id",
                    "description": "Subject ID"
                },
                {
                    "type": "PropertyValue",
                    "unitText": "trust_self",
                    "name": "trust_self",
                    "minValue": 0,
                    "maxValue": 3,
                    "description": "Number of trusting (P1) moves toward self-resembling faces"
                },
                {
                    "type": "PropertyValue",
                    "unitText": "trust_non",
                    "name": "trust_non",
                    "minValue": 0,
                    "maxValue": 3,
                    "description": "Number of trusting (P1) moves toward non-self-resembling faces"
                },
                {
                    "type": "PropertyValue",
                    "unitText": "recip_self",
                    "name": "recip_self",
                    "minValue": 0,
                    "maxValue": 3,
                    "description": "Number of reciprocating (P2) moves toward self-resembling faces"
                },
                {
                    "type": "PropertyValue",
                    "unitText": "recip_non",
                    "name": "recip_non",
                    "minValue": 0,
                    "maxValue": 3,
                    "description": "Number of reciprocating (P2) moves toward self-resembling faces"
                }
            ]
        }
    ]
```

<!--
``` r
simdata <- sim_design(
  n = 24,
  within = list(move = c("trust", "recip"),
                kin = c("self", "non")),
  mu = c(2, 1.5, 1.5, 1.5),
  r = 0.5,
) %>%
  mutate_if(is.numeric, round) %>%
  mutate_if(is.numeric, pmin, 3) %>%
  mutate_if(is.numeric, pmax, 0)

study <- study() %>% add_data(simdata, id = "kin")

data <- study$data[[1]]

data$data <- NULL
data$variableMeasured[[1]]$values <- NULL
data$variableMeasured[[2]]$values <- NULL
data$variableMeasured[[3]]$values <- NULL
data$variableMeasured[[4]]$values <- NULL
data$variableMeasured[[5]]$values <- NULL

data %>%
    jsonlite::toJSON(auto_unbox = TRUE) %>%
    jsonlite::prettify(4)
```
-->


### Full JSON

```
{
    "name": "Kinship and Prosocial Behaviour",
    "authors": []
    "hypotheses": [
        {
            "description": "Cues of kinship will increase prosocial behaviour. Cues of kinship will be manipulated by morphed  facial self-resemblance. Prosocial behaviour will be measured by responses in the trust game. The prediction is that the number of trusting AND/OR reciprocating moves will be greater to self-resembling faces than to non-self-resembling faces.",
            "criteria": [
                {
                    "id": "trust_lowbound", 
                    "analysis_id": "trust_analysis",
                    "result": "conf.int[1]", 
                    "operator": ">", 
                    "comparator": 0,
                    "conclusion": NULL
                },
                {
                    "id": "trust_highbound", 
                    "analysis_id": "trust_analysis",
                    "result": "conf.int[2]", 
                    "operator": ">", 
                    "comparator": 0.2,
                    "conclusion": NULL
                },
                {
                    "id": "recip_lowbound", 
                    "analysis_id": "recip_analysis",
                    "result": "conf.int[1]", 
                    "operator": ">", 
                    "comparator": 0,
                    "conclusion": NULL
                },
                {
                    "id": "recip_highbound", 
                    "analysis_id": "recip_analysis",
                    "result": "conf.int[2]", 
                    "operator": ">", 
                    "comparator": 0.2,
                    "conclusion": NULL
                }
            ],
            "support": {
                "description": "The hypothesis is supported if the 99% CI lower bound is greater than 0 and the 99% CI upper bound is greater than 0.2 (the SESOI) for either the trust or reciprocation moves.",
                "evaluation": "(trust_lowbound & trust_highbound) | (recip_lowbound & recip_highbound)"
            },
            "falsify": {
                "description": "The hypothesis is falsified if the 99% CI upper bound is smaller than 0.2 (the SESOI) for both trust and reciprocation.",
                "evaluation": "!trust_highbound & !recip_highbound"
            }
        }
    ],
    "methods": [],
    "data": [],
    "analyses": [
        {
            "id": "trust_analysis",
            "software": "`r R.Version()$version.string`",
            "func": "t.test",
            "params": {
                "x": ".data[kin]$trust_self",
                "y": ".data[kin]$trust_non",
                "paired": true,
                "conf.level": 0.99
            },
            "results": {
                "conf.int": []
            }
        },
        {
            "id": "recip_analysis",
            "software": "`r R.Version()$version.string`",
            "func": "t.test",
            "params": {
                "x": ".data[kin]$recip_self",
                "y": ".data[kin]$recip_non",
                "paired": true,
                "conf.level": 0.99
            },
            "results": {
                "conf.int": []
            }
        }
    ]
}
```