Skip to main content

​
Overview

Effort scoring is a unique feature of Roundtable Alias that provides a granular assessment of the effort and engagement demonstrated in each survey response. By analyzing various linguistic and behavioral factors, Alias assigns an effort score between 1 and 10 to each response, where:
  • 1 indicates minimal effort or engagement
  • 10 indicates exceptional effort or engagement
Effort scores enable you to quickly identify and prioritize high-quality responses for analysis while flagging low-effort responses for review or removal.

​
Interpreting Effort Scores

Effort scores are returned in the effort_ratings object of the API response, with a score for each question: 
"effort_ratings": {
  "Q1": 8,
  "Q2": 3,
  "Q3": 6,
  "Q4": 9
}
In this example, the response to Q1 received a high effort score of 8, while Q2 received a low score of 3, indicating minimal effort. Q3 and Q4 received moderate and high scores, respectively. To interpret effort scores effectively:
  1. Set a threshold: Determine a minimum effort score for responses to be considered high-quality or acceptable. This threshold may vary depending on your specific research goals and data quality standards.
  2. Identify low-effort responses: Flag responses with effort scores below your defined threshold for review or removal. These may include single-word answers, irrelevant responses, or copy-pasted content.
  3. Prioritize high-effort responses: Focus your analysis on responses with high effort scores, as they are more likely to provide valuable insights and meaningful data.
  4. Analyze effort distribution: Examine the distribution of effort scores across your survey to identify questions that consistently receive low-effort responses. This can help inform survey design improvements or participant screening criteria.

​
Customizing Effort Scoring

By default, Alias calculates effort scores based on its trained machine learning models. However, you can customize the effort scoring behavior using the low_effort_threshold parameter in your API requests. The low_effort_threshold parameter accepts an integer value between 1 and 10, representing the minimum effort score required for a response to be considered acceptable. Responses below this threshold will be flagged with the “Low-effort” check. For example, setting low_effort_threshold=5 will flag all responses with an effort score of 5 or lower as low-effort. Customize the low-effort threshold based on your specific data quality requirements and tolerance for false positives.

​
Next Steps

  • Explore how effort scoring complements Alias’s other fraud detection features, such as basic checks and duplicate detection.
  • Learn how to customize effort scoring behavior using the low_effort_threshold parameter in the API reference.