Queue/Batch Discussion Thread v2

[Razor]

Re: Queue/Batch Discussion Thread v2 (Submissions CLOSED)

Is there a list (or link) of what files are in the resubmission section yet, or what ones got accepted? Sorry still new to this. Doubt I got in but got [.+] on both? Or so I need to calculate all of them to see haha

chillax and take a tampax until jimerax posts the final list of accepted files.

[Jonathan123Ply84]

Re: Queue/Batch Discussion Thread v2 (Submissions CLOSED)

I must have sent it to the wrong e-mail then T_T. Now I'll have to wait another few months for the third time to submit a simfile.

[TC_Halogen]

Re: Queue/Batch Discussion Thread v2 (Submissions CLOSED)

I'm wondering what's taking so long for the list to come out, because JX has determined what the acceptance average in the judgment area (or it would seem).

[Xx{Midnight}xX]

Re: Queue/Batch Discussion Thread v2 (Submissions CLOSED)

I'd really like to get this out of the way now

But would someone mind giving a detailed explanation of why this math formula for acceptance is better than averaging the scores of the judges and then setting a bar of acceptance after it's all calculated out?

On top of that, why is this formula not posted anywhere or explained in detail anywhere in the first place? How is this formula not a simple manipulation of numbers? Why not round decimals? What numbers go in where?

In short, I think this formula thing is extremely stupid but before I can say anything about it I'd need to know these things

plus this information should be available so steppers know what the shit to look for and how to interpret their notes to the best extent possible

because with the understanding I have of it right now It seems to me like simple number manipulation for no reason and it's cutting charts that got scores that aren't extremely strong but are still really good such as 7/10s and such.

And anyone who says that this sort of information is irrelevant is a complete moron because it does matter, there's no point to using this system if it doesn't make any sense

[kommisar]

Re: Queue/Batch Discussion Thread v2 (Submissions CLOSED)

Alright then, sounds fair enough. I will be looking forward into that. Thanks for responding back.




and I thought you were overqualified...guess not ahahaha i kid i kid

U do know that was a compliment right? Not underqualified

[megamon88]

Re: Queue/Batch Discussion Thread v2 (Submissions CLOSED)

I'd really like to get this out of the way now

But would someone mind giving a detailed explanation of why this math formula for acceptance is better than averaging the scores of the judges and then setting a bar of acceptance after it's all calculated out?

On top of that, why is this formula not posted anywhere or explained in detail anywhere in the first place? How is this formula not a simple manipulation of numbers? Why not round decimals? What numbers go in where?

In short, I think this formula thing is extremely stupid but before I can say anything about it I'd need to know these things

plus this information should be available so steppers know what the shit to look for and how to interpret their notes to the best extent possible

because with the understanding I have of it right now It seems to me like simple number manipulation for no reason and it's cutting charts that got scores that aren't extremely strong but are still really good such as 7/10s and such.

And anyone who says that this sort of information is irrelevant is a complete moron because it does matter, there's no point to using this system if it doesn't make any sense

It's been posted before, but it seems like it's not really available unless you look for it. Here's a post from the hard songs batch a long time ago:

For hi19,

Adjustment Method

If a file gets rating A from Judge A, "actual" rating will be adjusted with following formula;

A = raw rating from Judge A
M(A) = Mean(Judge A) -- average rating of Judge A, indicator of strictness
SD(A) = SD(Judge A) -- SD of Judge A's rating, indicator of variation in Judge A's ratings
M(t) = Mean(total) -- average of Mean(Judge A), Mean(Judge B)...
SD(t) = SD(total) -- average of SD(Judge A), SD(Judge B)...
X(A) = actual rating from Judge A after the adjustment


X(A) = M(A) + {A - M(A)}*{SD(t) / SD(A)} + {M(t) - M(A)}

Maximum X(A) = 10, and Minimum X(A) = 1.
After we calculate X(A), .... X(F), we'll discard the highest rating and the lowest rating. (THIS ONLY APPLIES WITH 6+ JUDGES)

Then, the average of other ratings will be defined as the score of the file from this judgement system.

We calculated both Average(X(A), ... X(F)) and Average (without min and max) this time since we had only 6 primary judges. If one of these parameters is above 6.5, the file passes primary judgement. (CUTOFF VARIES WITH EACH BATCH)

[mean/SD of every primary judge]

bmah - 5.543 / 2.393
samurai7694 - 5.489 / 2.372
kommisar - 6.089 / 2.042
Gundam-Dude - 6.68 / 1.816
i love you - 6.302 / 2.029
qqwref - 6.458 / 2.010

For Zierg:
well, I'm suggesting fix for only 48ths part in the beginning now.
You still think they are the best representations of the song, I won't say anything anymore and moving it to the queue as it is.

From what I understand, this method is better because it adjusts the ratings to the relative strictness of each judge. Let's say my judging average is 5 and someone else's is 8. If I give an 8, it's relatively better than if the other judge gives an 8. This way, if you get stuck with a strict judge in your group, it doesn't really matter all too much since their ratings get adjusted.

Oh, and here's a simplified formula (cancelling M(A)):

[ichliebekase]

Re: Queue/Batch Discussion Thread v2 (Submissions CLOSED)

The part saying that the highest and lowest grades will be dropped only counts when there's 6 or more judges per song. It doesn't apply when there's only 4 judges like this time.

[megamon88]

Re: Queue/Batch Discussion Thread v2 (Submissions CLOSED)

The part saying that the highest and lowest grades will be dropped only counts when there's 6 or more judges per song. It doesn't apply when there's only 4 judges like this time.

Oh yeah, I forgot to mention that, thanks.

[MarioNintendo]

Re: Queue/Batch Discussion Thread v2 (Submissions CLOSED)

Also, Middie, Wilson is the one who came up with that equation last year. You know you can trust a formula when it comes from Wilson, just ask him to explain it for you!

[Xx{Midnight}xX]

Re: Queue/Batch Discussion Thread v2 (Submissions CLOSED)

It's been posted before, but it seems like it's not really available unless you look for it. Here's a post from the hard songs batch a long time ago:



From what I understand, this method is better because it adjusts the ratings to the relative strictness of each judge. Let's say my judging average is 5 and someone else's is 8. If I give an 8, it's relatively better than if the other judge gives an 8. This way, if you get stuck with a strict judge in your group, it doesn't really matter all too much since their ratings get adjusted.

Oh, and here's a simplified formula (cancelling M(A)):

It seems to me like it'd be very helpful if this was posted in a clearly visible place like the OP of this thread or something so others don't have to dig for it. Thank you for digging for it though, I appreciate it. I couldn't decode how it works via what Bmah posted in the TB chat due to emotes taking over.

Also, Middie, Wilson is the one who came up with that equation last year. You know you can trust a formula when it comes from Wilson, just ask him to explain it for you!

I'm sure the formula makes sense as to how it can be used, being Wilson devised it, but I'm questioning the why is it used part. And Megamon pretty much gave me that explanation.

But that still leaves issues in my head as to if it's "better" to use such a formula. Because a judge's "strictness" is as subjective as their notes are, so I don't understand so much how this is better. Megamon's explanation at least shed a light on it for me though.

On top of that, I guess I'll suggest that if this system stays in place, that maybe it'd be easier to sort the notes differently and add this calculation into it.

Maybe put all of the notes for a chart in one place in a spoiler and then add this calculated score to the bottom so you know if it actually was accepted immediately or not. This waiting game with this is easily the worst thing you can do with it.

Sorry if I come off as rambling but I'm just trying to throw this out there.

[i love you]

Re: Queue/Batch Discussion Thread v2 (Submissions CLOSED)

U do know that was a compliment right? Not underqualified

I did. I was just playing along lmao

[Nullifidian]

Re: Queue/Batch Discussion Thread v2 (Submissions CLOSED)

part of Ichliebekase's review for Dear Noriko:

- Right at the beginning you go back and forth stepping the bass hits as jumps or single notes. This actually happens as the song picks up too on occasion.

This comment ticked me off so badly, it makes me wonder why you're a judge lol. It's called layering, something you should know if you're a judge. You know? Stepping one note to one sound, and another note to another sound. The basics of stepping? That's how it results in 1 note for the bass kick on occassion..

[NeoMasterPie]

Re: Queue/Batch Discussion Thread v2 (Submissions CLOSED)

lol FFR

[ichliebekase]

Re: Queue/Batch Discussion Thread v2 (Submissions CLOSED)

part of Ichliebekase's review for Dear Noriko:


This comment ticked me off so badly, it makes me wonder why you're a judge lol. It's called layering, something you should know if you're a judge. You know? Stepping one note to one sound, and another note to another sound. The basics of stepping? That's how it results in 1 note for the bass kick on occassion..

I went and looked again just to be sure I remembered, and I guess it's more of an opinionated statement. I understand that it's layering, it's not how I personally would have chosen to layer it. If I'm stepping the bass as jumps, then I continue them as jumps. If I'm stepping them as single notes, I continue them as single notes. I guess it's a matter of consistency, and that's what I was taught a while ago.

I still gave you a 7 anyway, it's not like it's that big of a deal. Honestly, the nasty comment wasn't necessary, and a PM would have sufficed.

[Nullifidian]

Re: Queue/Batch Discussion Thread v2 (Submissions CLOSED)

Regardless of the 7 you gave me, the critique was unjust and showed a big lack of experience in my eyes. It was your main point of criticism in your review after all. Sorry if it offended you, but you'll live huehue