Reviewing submissions: How do I decide who to accept/reject?
On Prolific, you have the option to reject low-quality submissions. However, before you reject submissions, please think carefully whether the rejection is justified. Nobody likes to get rejected, so please try to keep rejections to a minimum.
General rule of thumb when reviewing submissions: If a participant completed the survey in time and submitted a completion code, then they should be compensated.
Valid reasons for rejections are, for example (see also the decision tree diagram below):
- The participant completed your study exceptionally fast. By "exceptionally fast" we mean participants who are statistical outliers.
- The participant skipped crucial questions. Crucial questions are those that are inevitable to answering your research question. Demographic questions, for example, do not necessarily fall under "crucial questions" because participants have the right to withhold such sensitive information (according to guidelines by the British Psychological Society).
- The participant failed one or multiple attention checks (for more on attention checks or instructional manipulation checks please click here).