Table 1 Reproducibility-related characteristics. Variable numbers (N) are dependent upon study design. Full detailed protocol pertaining to our measured variables is available online (https://osf.io/x24n3/)
From: Reproducible and transparent research practices in published neurology research
Indicators of reproducibility included in present study | Significance of measure variable for transparency and reproducibility | |
|---|---|---|
Publications | ||
All (N = 400) | Publication accessibility (Is the publication open access to the general public or accessible through a paywall?) | The general public’s ability to access scientific research may increase transparency of results and improve the ability for others to critically assess studies, potentially resulting in more replication studies |
Funding | ||
Included studies (N = 389) | Funding statement (Does the publication state their funding sources?) | Explicitly providing source of funding may help mitigate bias and potential conflicts of interest |
Conflict of interest | ||
Included studies (N = 271) | Conflict of interest statement (Does the publication state whether or not the authors had a conflict of interest?) | Explicitly providing conflicts of interest may allow for full disclosure of factors that may promote bias in the study design or outcomes |
Publication citations | ||
Empirical studiesa (N = 271) | Citations by a systematic review/meta-analysis (Has the publication been cited by any type of data synthesis publication, and if so, was it explicitly excluded?) | Systematic reviews and meta-analyses evaluate and compare existing literature to assess for patterns, strengths, and weaknesses of studies regarding a particular field or topic |
Analysis scripts | ||
Empirical studiesb (N = 271) | Availability statement (Does the publication state whether or not the analysis scripts are available?) | Providing access to the analysis script helps improve credibility by providing the replicators the opportunity to analyze raw data with the same analysis procedure |
Method of availability (Ex: Are the analysis scripts available upon request or in a supplement?) | ||
Accessibility (Can you view, download, or otherwise access the analysis scripts?) | ||
Materials | ||
Empirical studiesc (N = 255) | Availability statement (Does the publication state whether or not the materials are available?) | Providing the materials list allows replicators to reproduce study using the same materials, promoting |
Method of availability (Ex: Are the materials available upon request or in a supplement?) | ||
Accessibility (Can you view, download, or otherwise access the materials?) | ||
Pre-registration | ||
Empirical studiesb (N = 271) | Availability statement (Does the publication state whether or not it was pre-registered?) | Pre-registering studies may help mitigate potential bias and increase the overall validity and reliability of a study |
Method of availability (Where was the publication pre-registered?) | ||
Accessibility (Can you view or otherwise access the registration?) | ||
Components (What components of the publication were pre-registered?) | ||
Protocols | ||
Empirical studiesb (N = 271) | Availability statement (Does the publication state whether or not a protocol is available?) | Providing replicators access to protocols allows for a more accurate replication of the study, promoting credibility |
Components (What components are available in the protocol?) | ||
Raw data | ||
Empirical studiesb (N = 271) | Availability statement (Does the publication state whether or not the raw data are available?) | Providing replicators with access to raw data can help reduce potential bias and increase validity and reliability |
Method of availability (Ex: Are the raw data available upon request or in a supplement?) | ||
Accessibility (Can you view, download, or otherwise access the raw data?) | ||
Components (Are all the necessary raw data to reproduce the study available?) | ||
Clarity (Are the raw data documented clearly?) | ||