Researchers who move an experiment online can choose among JavaScript libraries, graphical builders, hosting services and platforms that combine these functions. A recent review by Karagüzel and Erdoğdu (2025) compares a dozen lab-based and web-based packages on timing precision, stimulus presentation, ease of use and platform compatibility. The authors emphasise "aligning software choice with research context, considering technical expertise, participant accessibility, and experiment complexity". For online data collection they recommend Gorilla or jsPsych, and they describe lab.js as combining "a drag-and-drop interface with advanced scripting capabilities".
This post applies that advice to the online tools a researcher is most likely to compare: jsPsych, PsychoPy with PsychoJS and Pavlovia, Gorilla, Labvanced, JATOS, lab.js and Open Lab. The review by Karagüzel et al. (2025) does not cover Labvanced, JATOS, Pavlovia or Open Lab, so the descriptions below are taken from each tool's own documentation and from the published papers that describe them. Open Lab publishes this blog and is one of the tools compared, and Open Lab's developer is a co-author of lab.js (Henninger et al., 2022).
An online study needs a task, a host and a recruitment channel
Sauter, Draschkow and Mack (2020) describe what an online experiment requires as "(1) a programmed experiment, (2) a server to host the study and (3) a recruiting platform which advertises to participants". Some products provide the first two components in one account. Others handle a single component and have to be combined with other tools. In their classification, Gorilla and Labvanced are integrated service providers; jsPsych, lab.js and PsychoPy Builder are experiment builders; and JATOS, Pavlovia and Open Lab are hosting and study-management services.
Sauter et al. (2020) reported that no documentation, privacy policy or operator information was publicly available for Open Lab. The documentation had been online at open-lab.online/docs since 2019; a privacy policy followed in April 2020 and a legal notice in July 2020. All three are linked from the current site (documentation, privacy policy, legal notice).
Recruitment is usually handled by a separate service. Prolific's documentation describes the service as helping researchers "recruit participants for studies that are hosted on external survey or experimental platforms", with participants returned to Prolific at the end by a redirect or a completion code (Prolific Researcher Help). University participant pools such as Sona Systems work in a similar way. For any host, the practical check is whether it can receive the participant's ID from the recruitment service and send the participant back when the study ends.
Sauter et al. (2020) also note that the components constrain one another: "No host supports all experimental builds and no experimental build is compatible with all hosts." They therefore recommend deciding on the combination of tools as a whole rather than on each component separately. Grootswagers (2020) names cost, flexibility and ease of use as the factors to consider when choosing a host. The sections below describe the building tools first, then the hosts, and then the timing evidence that applies to all of them.
jsPsych: experiments written in JavaScript
jsPsych is a JavaScript framework in which an experiment is assembled from plugins arranged on a timeline (jsPsych documentation; de Leeuw et al., 2023). It is released under the MIT licence and maintained by a core team of Josh de Leeuw, Becky Gilbert and Björn Luchterhandt. Experiments are written as JavaScript code. The project's packaging tool, jsPsych Builder, is a command-line utility that bundles an experiment's files and can produce archives for JATOS; it is not a graphical editor (jsPsych Builder).
jsPsych runs entirely in the participant's browser and does not include a server. Its documentation states that "jsPsych doesn't provide a single built-in solution for the web server component of your experiment" and lists several options, including Cognition.run, JATOS, Pavlovia and a self-managed web server (jsPsych: Running experiments). A jsPsych study therefore always involves a second decision about where to host it. The arrangement suits labs that are comfortable writing JavaScript and want to choose their own host.
PsychoPy Builder, PsychoJS and Pavlovia
PsychoPy is a Python package for lab-based experiments, with a graphical Builder and a Coder for writing scripts (Peirce et al., 2019). Since version 3.0, Builder can generate a JavaScript version of an experiment in PsychoJS and upload it to Pavlovia, a hosting service run by the PsychoPy team. Pavlovia's documentation states that "Many of the existing Builder experiments should 'just work', subject to the Components being currently supported by PsychoJS" (Pavlovia: Creating experiments). The PsychoPy documentation lists which components work online. Hardware components such as EEG and eye trackers do not, and several stimulus and response components are listed as not yet supported by PsychoJS (PsychoPy: Status of online options).
Pavlovia also hosts experiments built with jsPsych and lab.js (About Pavlovia). Piloting is free, with results limited to a local download, and data collection is paid for either with per-participant credits or through an institutional licence (PsychoPy pricing). The combination suits labs that already run PsychoPy experiments in the lab and want to run the same experiment online, once they have checked that every component they use is supported in the browser.
Gorilla and Labvanced: building and hosting in one account
Gorilla and Labvanced are commercial platforms that combine a graphical builder with hosting and data storage. Gorilla is operated by Cauldron Science and Labvanced by Scicovery GmbH.
Gorilla describes itself as a graphical builder with "no coding necessary", with separate tools for tasks, questionnaires, games and the overall experiment flow (Gorilla; Anwyl-Irvine et al., 2020). Researchers who want to extend the tools can add scripts; the current Task Builder uses TypeScript for custom components (Gorilla: Scripting). Gorilla is priced per participant. Building is free, a token is used for each participant whose data the researcher keeps, and lab, department and institutional licences are also offered (Gorilla pricing FAQ). Gorilla states that it is hosted on Microsoft Azure in the Republic of Ireland (Gorilla: Due diligence).
Labvanced is also a graphical builder ("no coding required") and supports custom JavaScript, CSS and HTML (Labvanced feature overview). Its features include studies in which two or more participants take part together and webcam-based eye tracking. Kaduk et al. (2024) compared Labvanced's webcam eye tracking with an EyeLink 1000 and reported an overall accuracy of 1.4° and a precision of 1.1° of visual angle; three of the four authors are employees of Scicovery. Labvanced sells annual licences in tiers, with a free tier for trying the platform, and lists eye tracking and multi-participant studies under the higher tiers (Labvanced licences). It states that data are stored on servers in the European Union operated by Hetzner (Labvanced data privacy).
These platforms suit researchers who want to build without code and do not want to combine separate tools for building and hosting. This is the case Karagüzel and Erdoğdu (2025) have in mind when they describe Gorilla as one of the "accessible solutions for researchers with limited coding skills". Gorilla also lists a multiplayer builder among its tools, so both platforms support studies with interacting participants.
Hosting a task built elsewhere: JATOS, Pavlovia and Cognition.run
When the task comes from a library or a builder that does not host studies, the host is a separate choice. JATOS is free, open-source software (Apache 2.0 licence) that a lab installs on its own server to run studies and manage the data (Lange et al., 2015). Its documentation lists compatibility with jsPsych, lab.js, PsychoPy, OpenSesame and PsyToolkit (What is JATOS). The original paper defines its scope: "JATOS does not aim to help researchers write their study scripts. Instead, it provides a helpful framework to run custom scripts." Researchers without their own server can use MindProbe, a free JATOS server for non-profit research sponsored by the European Society for Cognitive Psychology, which does not provide technical support by email. Sauter et al. (2020) advise against running studies on a self-maintained web server without a study-management system such as JATOS.
Pavlovia, described above, hosts PsychoJS, jsPsych and lab.js experiments. Cognition.run hosts jsPsych experiments, with a free tier and paid plans. Among these hosts, the differences concern who maintains the server, where the data are stored and how the cost changes with sample size. With a self-hosted JATOS installation, the data stay on the lab's own server, and the costs are the server and the staff time needed to maintain it.
lab.js: a graphical builder that exports to several hosts
lab.js is an open-source study builder that runs in the browser. Its authors describe it as follows: "Through its visual interface, stimuli can be designed and combined into a study without programming, though studies' appearance and behavior can be fully customized using html, css, and JavaScript code if required" (Henninger et al., 2022). The core library is released under the Apache 2.0 licence and the builder under the GNU Affero General Public License (lab.js licence).
lab.js does not host studies itself. The builder exports a study in several forms: a bundle for offline data collection that saves the data as a CSV file, a version for a generic web server with a PHP backend, code for embedding in survey tools such as Qualtrics, and direct exports to JATOS, Pavlovia and Open Lab (Henninger et al., 2022; lab.js: Deploying studies). A study built in the public lab.js builder can therefore be moved to a different host by exporting it again.
What the timing benchmarks show
Two benchmark studies have measured several of these tools with external hardware. Bridges et al. (2020) tested PsychoPy/PsychoJS, Gorilla, jsPsych, lab.js and Testable online, alongside lab-based packages. They found that online studies "did not deliver the same level of precision as lab-based systems". Among the online packages, "PsychoPy and Gorilla, broadly the best performers, were achieving very close to millisecond precision on several browser/operating system combinations", and for response times "most of the packages achieved precision at least under 10 ms in all browsers". Anwyl-Irvine et al. (2021) compared Gorilla, jsPsych, PsychoJS and lab.js. They concluded that "no single platform stands out as the best in all features and conditions" and that "all platforms are reasonably accurate and reliable for studies not needing < 100 ms reaction time accuracy or < 2 frames presentation accuracy."
Several qualifications apply to these results. Each study was written by developers of one of the platforms tested, PsychoPy in Bridges et al. and Gorilla in Anwyl-Irvine et al., and both papers declare this. The versions tested date from 2019 and 2020, and all of the tools have been updated since. Neither study included Labvanced, which Anwyl-Irvine et al. name as untested "due to practical restrictions on our time and resources". The timing figures in Karagüzel and Erdoğdu (2025) are drawn mainly from Bridges et al. (2020), so specific values are best checked in the original study. Bridges et al. also recommend that researchers make "their own timing validation measurements for their own stimuli and computer configuration". For experiments that require the highest timing precision, Karagüzel and Erdoğdu recommend lab-based software such as PsychToolbox or PsychoPy. An earlier post, RT precision in the browser, discusses what these measurements mean for the design of reaction-time studies.
Open Lab: hosting and study management for lab.js tasks
Open Lab hosts lab.js tasks and manages the studies built around them, which is the category Sauter et al. (2020) place it in. It does not replace the lab.js builder. A task is built in a lab.js builder and sent to Open Lab, either directly from the builder or by uploading the exported JSON file. Both the public lab.js builder and Open Lab's own builder at builder.open-lab.online can send tasks to Open Lab, and tasks run with lab.js 20.2.4. The post Running a lab.js experiment online, end to end walks through these steps.
Open Lab organises the work at three levels. A task is the lab.js experiment itself and can be reused across studies. A study combines one or more tasks with landing and consent pages, branching, assignment to conditions and an ending or redirect, and it has its own participant link and its own data. A project groups related studies and the collaborators who work on them, each with a role that limits what they can view and change (project roles).
Around the task, Open Lab provides the following:
- Assignment to conditions. Between-subjects conditions are assigned by minimisation (Pocock & Simon, 1975), which steers each new participant toward the condition that is currently under-filled, so group sizes stay close to the specified ratio in small samples (between-subjects randomization).
- Recruitment. Open Lab does not supply participants. A study link can carry the participant's ID from a recruitment service, and participants are sent back at the end with a redirect or a per-participant completion code. The documentation covers Prolific, Sona Systems, CloudResearch Connect, Amazon Mechanical Turk and lab recruitment systems such as ORSEE and hroot (invitation types and panels).
- Data storage and encryption. Data are stored on servers in Germany. A study can use end-to-end encryption tied to the researcher's account key (the recommended option), end-to-end encryption with a per-study key pair that the researcher keeps, or server-side encryption with a key managed by Open Lab. New studies use server-side encryption until the setting is changed, so the setting is worth checking before launch (three encryption options). With end-to-end encryption, the response-pattern checks described below are limited or unavailable, because the server cannot read the responses.
- Checks before launch. Preview shows the participant's view of the study. A test run plays a task from start to finish without human input and reports mechanical problems, such as a task that never finishes or records no data. Demo mode runs the full study without adding the session to the dataset (piloting before you launch).
- Data quality. When a researcher enables paradata collection, Open Lab records passive, aggregate counts of how each participant interacts with the task. It uses these to mark each session as Verified, Suspicious or AI-detected. The verdicts are prompts for review, not automatic exclusions, and paradata collection is off by default (what paradata can and can't tell you).
- Analysis. Each study has a data view with summary charts, and the Data Explorer runs queries in the browser through a visual builder or SQL, with CSV download (from sessions to analysis).
- Ready-made tasks. The task library contains lab.js versions of common paradigms such as Stroop, Go/No-Go and n-back.
- Cost. There is a free plan, and paid plans carry a 50% discount for academic users (pricing).
Open Lab runs lab.js tasks only. An experiment written in jsPsych or PsychoJS needs one of the hosts described in the previous sections, or has to be rebuilt in lab.js. The combination of lab.js and Open Lab suits researchers who prefer a graphical builder with the option of adding code, who want study management, collaboration roles and recruitment integrations without maintaining a server, or whose ethics approval requires data storage in the EU or end-to-end encryption.
Questions that narrow the choice
Who will write and maintain the experiment? Labs that write JavaScript can use jsPsych and choose any host. Labs that prefer a graphical interface can use lab.js, PsychoPy Builder, Gorilla or Labvanced, all of which allow custom code when the interface does not cover a design.
Does the experiment already exist in one of these tools? An existing experiment narrows the options. A PsychoPy Builder experiment leads to Pavlovia. A jsPsych experiment can go to any of the hosts listed in the jsPsych documentation. A lab.js study can go to any of the hosts its builder exports to.
How precise does the timing need to be? According to Anwyl-Irvine et al. (2021), the browser-based platforms they tested are reasonably accurate and reliable for studies that do not need reaction-time accuracy finer than 100 ms or presentation accuracy finer than two frames. Studies that need more than that may require lab-based software, or a timing validation on the setup that will be used (Bridges et al., 2020).
Where must the data be stored, and who can read them? Gorilla states that its servers are in Ireland, Labvanced that its servers are in the EU, and Open Lab that its servers are in Germany. With JATOS, the data are stored wherever the lab's server is. End-to-end encryption, where the host cannot read the data, is available on Open Lab.
How will participants be recruited and credited? The host needs to receive the participant's ID from the recruitment service and return the participant at the end. Prolific's help pages list compatible software, and each host documents its own integrations.
How is the cost structured? jsPsych, lab.js and JATOS are free to use, and self-hosting costs server and staff time. Gorilla and Pavlovia charge per participant or offer institutional licences, Labvanced sells annual licences, and Open Lab has a free plan and paid subscriptions. Which model costs less depends on sample sizes and on the number of studies a lab runs each year.
Does the design need a specialised feature? Webcam eye tracking with a published validation is available in Labvanced (Kaduk et al., 2024). Gorilla and Labvanced both support studies in which participants interact with one another. Experiments that use EEG, external eye trackers or button boxes need to run in the lab, for example with PsychoPy.
