Amazon Mechanical Turk

MTurk offers two ways to publish a HIT that lives on another site. One of them works with Open Lab and one of them cannot — worth knowing before you build the study rather than after.

Survey Link HITs work; ExternalQuestion HITs do not

A Survey Link HIT sends the worker to your link and gives them a box to paste a code into. That is the pattern to use.

An ExternalQuestion HIT embeds your study in a frame and expects it to submit a form back to MTurk. Open Lab cannot do that — see why not below.

Setup (Survey Link HIT)

  1. Turn on completion codes (Open Lab)

    In the Study Builder, under participant settings, switch on Generate completion code. It is off by default. Each participant who finishes then gets a unique code.

  2. Show the code on your Ending page (Open Lab)

    Open the Ending component and switch on Show completion code on this page. Tell people in the surrounding text to copy the code back into the HIT before closing the tab — that sentence does more work than it looks like it should.

  3. Copy your study link (Open Lab)

    Invitations → Public links → Study link:

    https://app.open-lab.online/studies/ABCDE
  4. Create the HIT (MTurk)

    Use the Survey Link project template, paste the study link, and let workers submit the code they were shown.

  5. Check the codes (Open Lab)

    Export the Participants table. It carries one row per participant with both the participant code and the completion code, which is the file to check submitted codes against in bulk before you approve.

No Worker ID arrives on its own

The Survey Link template does not substitute worker identifiers into your URL, so nothing about the worker reaches Open Lab automatically. If you need the Worker ID — to pay a bonus, to exclude repeat participants, to link a session to a submission — ask for it inside the study and treat it as data you collected.

Why ExternalQuestion HITs cannot work

An ExternalQuestion HIT loads your study inside a frame and appends assignmentId, hitId, workerId and turkSubmitTo to the URL. Those arrive in Open Lab perfectly well — they are ordinary link parameters. The problem is the other end.

To register the assignment as submitted, the study must send a form POST to turkSubmitTo/mturk/externalSubmit carrying assignmentId. Open Lab's Redirect component follows a link, which is a GET. There is no way to express a POST in it, so the assignment would never be submitted and the worker could not be paid through MTurk's normal flow.

This is a real limitation rather than a missing setting, so the Survey Link route above is the supported one. If you need automatic approval, CloudResearch Connect and Prolific both support redirects and are the easier fit.

Related