A small change to the internal judge process has been made to help remedy conflict between song judgments involving more than one judge, specifically with judge notes containing extremely varied scores or 'Fixes Required' requirements (Before-hand known as CQ, or Conditional Queue).
I've appended the changes under 'REVIEWING PROCESS' in the Original Post for this thread, but also included a copy of what was added below for easy visibility.
When judges have independently finished their notes (for sets involving two judges):
Both judges compare their file ratings and 'Fixes Required' flags before posting.
-Discuss reasoning on any 'Fixes Required' flags. Try to come to a mutually agreeable conclusion on them. If both judges can’t agree on a conclusion, the file and Fixes Required reasonings are Direct Message'd to a third judge. (Ideally a Head/Senior Judge but can be any judge) The third judge will review Fixes Required reasonings and decide whether to apply the requirement.
-If the difference between both judge ratings on a file is 2 or more points, where one judge has given a reject rating, the file is Direct Message'd to a third judge (Ideally a Head/Senior Judge but can be any judge). The third judge will give a brief judgment, reviewing the differing opinions and deciding a course of action.
When these cases are resolved, notes are posted as usual. If applicable, the third judge's notes are included as well.
There will be a slight change to how FFR files are judged, starting with those files assigned to judges after the Official Tournament 16 Batch closed.
Judgements will be structured a lot more like those done for the Dump Batch last year. Which is to say, judges will no longer be expected to write an in-depth technical review for each file.
There will be more generalized notes written rather than giving timestamps and listing individual suggestions. Judges will instead write a minimum of 2-3 sentences for each chart that details their rating justification.
All other factors of the judge system will remain the same. (All ratings x/10, Two judges per file, Third judge policy etc)
This judgement system will remain in effect for all files sent to FFR until further notice.
Thank you all for your patience as we work to get judgements out as quickly as possible!
Comment