Queue/Batch Discussion Thread v2

[kommisar]

Re: Queue/Batch Discussion Thread v2

I don't think it's necessary imo


oh nuts i get to judge two of my songs lmao. hi wyde

[justin_ator]

Re: Queue/Batch Discussion Thread v2

[$] The file is better than the queue'd file that uses completely the same song, and it should be replaced with the old one (there's similarity in difficulty or stepping style).

Shouldn't this be 'replaced with the new one'? Should probably fix this. Just throwing that out there.

[jimerax]

Re: Queue/Batch Discussion Thread v2

fixed.

[bmah]

Re: Queue/Batch Discussion Thread v2

More importantly, I'd like to hear from the people who are complaining about their files being judged with highly variable results and possible contradictions in opinions. qqwref? rushy? ichlie? Etc. It's the only reason why I made such a proposal. Read the previous pages to hear their concerns.

[leonid]

Re: Queue/Batch Discussion Thread v2

[<] There is already a better file for this song in the queue or in the batch, and the file is not suitable as a V2.
[$] The file is better than the queue'd file that uses completely the same song, and the old one should be replaced with the new one (there's similarity in difficulty or stepping style).

Not an important thing or anything; just wondering why they are not just [<] and [>].

[Wyde]

Re: Queue/Batch Discussion Thread v2

oh nuts i get to judge two of my songs lmao. hi wyde

'sup

Yeah funny you got landed with that lol. hope u liek

[qqwref]

Re: Queue/Batch Discussion Thread v2

I'm not sure if I agree that previous judges should judge the resubmission files again. On one hand, it would ensure that any fixes that got pointed out actually get fixed; on the other hand, if one person thinks the song or chart style is total crud, they are not going to improve their opinion much when they see that a few minor problems have been changed. I guess you can put me down for a tentative yes, but I'd still like to see the worst grade thrown out in cases of extreme subjectivemania (+ + + -, ++ + + ?, that kind of stuff).

[TC_Halogen]

Re: Queue/Batch Discussion Thread v2

Throwing out grades is a bad idea in the fact that a file might be objectively incorrect, but will have passing grades for multiple judges anyway for various reasons (judging fatigue, extreme bias toward song choice when chart is passable but not worthy), etc.

tbh, Arch's ratings were a bit all over the place. Hell, even I thought +. was a bit ridiculous for my chart Cheeseburger last batch -- that file shouldn't have gotten anything higher than a +?, lol

If the song has a poor rating, it should only be considered thrown away if the judge can't give legitimate reasons as to why. Obviously if a judge gives a bad grade, they've most likely done it for a reason and if they can prove it, then their grade shouldn't be thrown out because it's rude to them for judging it.

Take me for example in my last batch, and Altair (which I believe I gave it a [-]). What if it received good marks from all of the other judges? My notes get thrown away for no reason?

It's just a thought; it's not a bad idea, but it comes with a lot of issues behind it.

As for resubmissions: pass them around to different judges, because there might be an inevitable bias towards/against songs.

[buizel8888]

Re: Queue/Batch Discussion Thread v2

I think we should all go play o2jam. Just sayin

[qqwref]

Re: Queue/Batch Discussion Thread v2

Throwing out grades is a bad idea in the fact that a file might be objectively incorrect, but will have passing grades for multiple judges anyway for various reasons (judging fatigue, extreme bias toward song choice when chart is passable but not worthy), etc.

It's possible, but I think it's a lot more likely that one person messes up than that three people mess up. Anyway I don't think the notes should always be thrown out, or the judge booted, just that the lowest rating shouldn't be counted when we determine how well songs did.

If the song has a poor rating, it should only be considered thrown away if the judge can't give legitimate reasons as to why. Obviously if a judge gives a bad grade, they've most likely done it for a reason and if they can prove it, then their grade shouldn't be thrown out because it's rude to them for judging it.

This is a good idea - to ask the lowest judge (if that rating is the lowest by a lot) to explain in more detail why they gave that ranking and what could be fixed. Then we could throw out their result only if they can't give a reasonable explanation.

The thing which makes me want to throw out the lowest rating is that the current system can potentially give a judge too much power. If one person gives a file [?] or worse, there is nothing the other judges can do - it's basically getting flat out rejected. The purpose of team judging is to give each file multiple perspectives, but it's important to have a system where a judge can't personally prevent a file from getting ingame.

[leonid]

Re: Queue/Batch Discussion Thread v2

Divide the judging team into two groups (instead of three), and for each file, omit the highest and the lowest score

[ichliebekase]

Re: Queue/Batch Discussion Thread v2

On one hand, it would ensure that any fixes that got pointed out actually get fixed; on the other hand, if one person thinks the song or chart style is total crud, they are not going to improve their opinion much when they see that a few minor problems have been changed. I guess you can put me down for a tentative yes, but I'd still like to see the worst grade thrown out in cases of extreme subjectivemania (+ + + -, ++ + + ?, that kind of stuff).

You pretty much said it for me in this, but

If the song has a poor rating, it should only be considered thrown away if the judge can't give legitimate reasons as to why. Obviously if a judge gives a bad grade, they've most likely done it for a reason and if they can prove it, then their grade shouldn't be thrown out because it's rude to them for judging it.

This should be considered too. If the judge cannot give good solid reasons as to why it received a lower rating, then it really shouldn't be considered. Pretty much means that they didn't do their job [in the sense that they should be pointing out mistakes or flaws].

Put me down for a yes like qqwref.

[Arch0wl]

Re: Queue/Batch Discussion Thread v2

Throwing out grades is a bad idea in the fact that a file might be objectively incorrect, but will have passing grades for multiple judges anyway for various reasons (judging fatigue, extreme bias toward song choice when chart is passable but not worthy), etc.

tbh, Arch's ratings were a bit all over the place. Hell, even I thought +. was a bit ridiculous for my chart Cheeseburger last batch -- that file shouldn't have gotten anything higher than a +?, lol

Does it surprise you that someone whose position is that simfiles cannot be objectively evaluated would not have an objective method of judging? It is objective with respect to my charting style, but that's as far as anyone can go. A method is not objective just because it is systematic.

[Patashu]

Re: Queue/Batch Discussion Thread v2

I would have +'d cheeseburger if I was judging it. idg why you hate it. I guess the <3 of exotic swing is lost when you transfer it into ffr though

[Niala]

Re: Queue/Batch Discussion Thread v2

Divide the judging team into two groups (instead of three), and for each file, omit the highest and the lowest score

+100 for this.