Queue/Batch Discussion Thread

[psychoangel691]

Re: Queue/Batch Discussion Thread

I've posted this like a zillion times but the framework would be something like
-How well stepped is it? / How much could the purely technical quality be improved upon by fixes?
-How much did I like it? / Is it interesting to play, irregardless of errors?

errors along the lines of 'you missed a jump at blah blah blah' would lower the former rating, errors along the lines of 'you took a pretty boring approach to the song' or 'this pattern is aids to hit' would lower the latter rating. one could have a perfect technical score and a bad entertainment score if it's e.g. a xandertrax file (sorry lol), or a bad technical score and a perfect entertainment score if its ideas are great but it has lots of little missing notes because bad speakers were used or w/e.

this will lead to less worries about whether all those errors you found should lower the score and potentially reject an otherwise good file, since you just lower the appropriate rating.

when considering whether a file should be accepted or not, imo accept on the basis of entertainment ratings and then the lower technical ratings are the more you should push to have fixes made (e.g. conditional queue-y ness goes up)

Sorry I haven't seen this before I don't know how I missed it because I like this idea since it incorporates the fun factor as well as the technical stuff in it.

Well, in my humble opinion, there are two dimensions to the choice of the song: 1. Does this song fill my tastes (aka personal)? and 2. Is this song steppable/should it be stepped (aka the judges job to analyze the pertinence of the song)?

I think that this is definitely not a win-win situation. Song choice does seem to depend both on subjective and objective terms. It then seems somewhat logical that some points are attributed to it... no?

Sorry I should have been a little more clear. What I mean is it shouldn't be rejected because it's a song that just out of taste the judge doesn't enjoy. I can understand if a song is repetitive and doesn't make for a good chart or if the sound quality is bad. Things of that nature are different.

And just saw you're edit about patashu - yea lol he's awesome so is bmah XD

[Patashu]

Re: Queue/Batch Discussion Thread

I'd have a rule on song choice: Don't alter ratings if you dislike the song. If the song is bad for stepping it'll be reflected in a lower entertainment rating. If you don't like the song but it still makes a fun chart then other people will like it too.

Maybe a joke third rating for song quality :P

[MarioNintendo]

Re: Queue/Batch Discussion Thread

I'd have a rule on song choice: Don't alter ratings if you dislike the song. If the song is bad for stepping it'll be reflected in a lower entertainment rating. If you don't like the song but it still makes a fun chart then other people will like it too.

Maybe a joke third rating for song quality :P

This sounds like somebody will have to give birth to a sticky on "How to be a flawless judge" not to get them too confused with that new judgement process...! :P

I'm starting to wonder, assuming that FFR's community is not entirely made of unreasonable people, wouldn't a judging process inspired from Newgrounds' way of working optimally reflect the community's opinion on a song? It might never happen, but on paper, it works!

After reflexion, I'm taking that statement back.

[buizel8888]

Re: Queue/Batch Discussion Thread

The FFR Judges are all art haters

There. My 2 cents have been inserted

[who_cares973]

Re: Queue/Batch Discussion Thread

so i was expecting another dull day with the queue thread




oh my god i dont even know where to begin to quote

[kommisar]

Re: Queue/Batch Discussion Thread

when I judge, most of my criteria for accepting is fun value, the rest is structure. I don't really look at the song itself since that's VERY subjective, but if the song itself generates very long repetitive sections then it's a given


who didin't give staring at my spaceship a +?

oh aj. u haterrrrrrrrrrrrrr




here's what I usually base a 1-5 scale on (25 point average)

5 points for creativity/originality
10 points for fun/replay value
5 points for structure/technicality
5 points for flow/patterns

although it's obvious that some are more important than others, this usually works out. the reason we have multiple judges is to avoid any kind of huge bias. if all the judges reject your file, there's a pretty big chance most of others will agree too. I can think a certain pattern should be done one way, next judge says otherwise. there's no way to eliminate different opinions aside from getting input from several people.

[who_cares973]

Re: Queue/Batch Discussion Thread

i try to be as much of a robot as i can be when it comes to judging

[Xandertrax]

Re: Queue/Batch Discussion Thread

Ouch, don't appreciate the jab, Patashu.

and wow. 6 pages of ffdramalution. I missed this stuff. Not really.

The main problem is that 4 judges is far too small of a sample size to tell how 'good' a file is. Howabout, in hotly contested files that get ratings across the board, you bring in another judge or two to get a few more ideas? Not sure that will accomplish much.

Also, was there a final conclusion on Feux d'Artifice? It was thrown in there but I never saw the answer come up.

[jimerax]

Re: Queue/Batch Discussion Thread

Accepted files of this batch aren't finalized yet, just files that got 17 or more points are safe for sure. A few other files could be accepted, and we need to determine acceptance criteria for them. going by ratings, or using secondary judgement system.

I'm not thinking of conditional queue for borderline files (this is for files that are accepted but offsync/bad parts or features).

And yeah being subjective isn't always bad but putting too much weight on how you hate/like the way of stepping, the song, difficulty causes judgement inconsistency, which isn't a good thing as judgement.
I don't think all judgement should be completely objective since basically everyone has different opinions, but we need some standards for judgement for reducing inconsistencies.

[psychoangel691]

Re: Queue/Batch Discussion Thread

here's what I usually base a 1-5 scale on (25 point average)

5 points for creativity/originality
10 points for fun/replay value
5 points for structure/technicality
5 points for flow/patterns

See now this is actually a really good idea I don't know why I didn't think about mentioning something like this sooner. This actually reminds me of how judging was for the skate competitions that we had at the skatepark. It was like a judge would rate on the difficulty of a trick, originality, how technical it was and like the wow factor then it was averaged.

Why don't we do something like this judges look at those different aspects on a 1-5 rating, rate each of those and then the final will be the average of all the aspects?

So based on Kommis aspects he mentioned say a file was really fun it gets a 5 for that, has maybe a few technical errors so lets say a 3, overall flow is pretty nice but maybe a few off patterns giving it a 4 and we'll just say 4 for originality here. Which ends up giving it an overall rating of 4. I think that's a really nice idea. It still gives a ton of room for opinionated things but it's balanced out.

[ichliebekase]

Re: Queue/Batch Discussion Thread

I really think there needs to be some sort of guidelines put in place to avoid extreme differences in ratings such as the + and - on the same file because that's just a huge difference.

This :/ on two of my files, I got ratings of 1,2,3,4, exactly that. Only one had an even rating of 2,2,3,3.

I also wanted to say before that what i love you said about the file he judged on mine, he was out of line for saying things like "LMFAO" and I think "Seriously?" was thrown in there too, and he completely rejected the file. It was pretty rude the way he said it all. Also, I think it was the same file that Carlos had judged, and he flat out accepted it with a +.

Something was seriously wrong there.

[krunkykai22]

Re: Queue/Batch Discussion Thread

yea except they have to be perfect to "them". It's pretty much entirely subjective after the point of the file being synced and not having notes go to nothing etc..

I like files being different then them so oh well I guess.

I was going to say pretty much the same thing. What actually defines "perfect" for a file? With such a vast difference in opinion on subjective things you can't even really call something a "perfect" file.

I was subjective, but I'm capable of admitting that I was, and if you look at my notes, you'll even see that files with SUBSTANTIAL AMOUNTS of notes/fixes requested still ended up getting no lower than a [+?] because I'm able to separate between what should be there in my opinion, and what should be there in accordance to how the rest of the simfile is. Stop acting like I'm reacting like this because of this file - there's other files from the previous batch that got knocked down a ridiculous amount for stupid reasons, like song choice. There's even some files that had small amounts of errors and still got a [-].

But hey, I don't know a thing about judging or whatnot. I'm arguing with someone who thinks their opinion is right and nothing else because they couldn't even bother to address the issues that I pointed out -respectfully- like I asked to.

I'm done judging FFR files - if people are going to sit here and cut each other's throats off because they can't tell the difference between judging off of their preferences and what's right, then what the hell is the point. I love how the argument of subjectivity is coming out entirely because I choose -now- to speak up, rather than trying to focus on why people's votes are too -overly- subjective.

The rest of the judges can go have fun doing what they want. I feel sorry for the really strong judges who are consistently good because they know what they're doing having to deal with ratings that deviate ridiculously without justified reasons whatsoever. Apparently, getting a legitimate discussion when I actually asked questions about how to FIX something is a complete waste of time because some judges won't even bother to give a straight answer. What's the point?

Seeking perfection in simfiles is going to get you ABSOLUTELY nowhere because the users who are playing your game will not notice a good amount of the details that people are putting in.

Find another judge for me, I'm not judging on this team until I see a change in people's attitudes. It's not everyone, but there's people on the team who I simply don't agree with being on it for reasons of not being active with the site, not caring about the game, or not legitimately wanting to help someone out because they feel like flaunting around their elitism when no one gives a shit.

1.) You guys lost a judge.... A damn good judge at that. Just because he wanted his gf to get her first FFR file in game doesn't mean he is just defending her for the hell of it. Aj has a lot of good stepcharts for both FFR and SM AND ITG.... I think he knows what the hell he's doing. For him to actually go over and shadow things on that file, I'm sure he damn well listened more in depth than anything.

2.) Personally, I LOVED The Final Hour. I'm not judge, don't get me wrong, and I don't have a file in FFR, rather 2 1/2 on the DF Engine, but I think what Kayla did was ****ing perfect. That's my opinion, she did what she did because SHE felt it to be the best file she wanted it to be. Not to mention, when that file came out in one of the DF Tournaments when FFR was down, did you guys see the damn compliments? EVERYONE was telling her how amazing the file was. I don't give two ****s if they aren't FFR Judges or anything, take that of the community in with your own reasons. If you just reject the file based on one/two/three peoples mentality what's that to say of the community?

Here an example:

AJ creates a Stepfile for a rock song "Disturbed = Believe." 99% of the community LOVE the file that he had shown to them offline. 3/4 judges come in and give it an 11. Don't you think the community would be like "really? WTF Elitist assholes."

If the file was completely retarded don't get me wrong. Overstepped, off sync, notes being filled in empty spots, BPM changes ridiculously ....whatever, than I understand it. But, if it IS a good file to us (the community) Than why not just suck it up and put it in? What makes The Final Hour different from .... For FFR? Or any other Synthlight songs? Yeah they are history for FFR, so why can't The Final Hour be?

3.) I agree the strictness needs to subside just a bit.

4.) Kommi's rating system is beast. Personally that's a HUGE thing I believe and apparently with others, would be fantastic in judging. But, still, there will always be subjectiveness.


I personally don't know what more to say about it. Somehow I knew that because Kayla is Aj's fiance, that would get thrown into the mix. I'm completely against that. 100%. Kayla knows what she is doing, and so does AJ, no need to get personal about it. nuff said.

[psychoangel691]

Re: Queue/Batch Discussion Thread

I also wanted to say before that what i love you said about the file he judged on mine, he was out of line for saying things like "LMFAO" and I think "Seriously?" was thrown in there too, and he completely rejected the file. It was pretty rude the way he said it all. Also, I think it was the same file that Carlos had judged, and he flat out accepted it with a +.

Something was seriously wrong there.

That's another thing that really shouldn't be happening. I don't think at any point a judge should say things like this sucks or laugh at people. It needs to be kept in mind that some people are newer and they don't deserve to be treated like that when they are trying to learn.

Actually something I'm noticing and I see you lurking Byron and don't take this the wrong way because I'm not stabbing but I noticed you mention things being offsync but I didn't see anyone else mention it especially Carlos and he's usually the one to nail that stuff with ddream. On DF he was always pointing out that people's gaps were wrong. It's just really weird to me that sync seems to be coming up as an issue for you but not others. Maybe there's something up with your computer? I know when AJ would do stuff on the one desktop we had the sync would be all screwed up and yet when it was done on the laptop it was fine.


Just throwing this back up quoted since it ended up back a page and sometimes people don't go back

here's what I usually base a 1-5 scale on (25 point average)

5 points for creativity/originality
10 points for fun/replay value
5 points for structure/technicality
5 points for flow/patterns

See now this is actually a really good idea I don't know why I didn't think about mentioning something like this sooner. This actually reminds me of how judging was for the skate competitions that we had at the skatepark. It was like a judge would rate on the difficulty of a trick, originality, how technical it was and like the wow factor then it was averaged.

Why don't we do something like this judges look at those different aspects on a 1-5 rating, rate each of those and then the final will be the average of all the aspects?

So based on Kommis aspects he mentioned say a file was really fun it gets a 5 for that, has maybe a few technical errors so lets say a 3, overall flow is pretty nice but maybe a few off patterns giving it a 4 and we'll just say 4 for originality here. Which ends up giving it an overall rating of 4. I think that's a really nice idea. It still gives a ton of room for opinionated things but it's balanced out.

[woker-X]

Re: Queue/Batch Discussion Thread

Let's see...

Can Can - Yay

Capriccio No. 24 - Only problem is the song quality, someone has a better audio for this song?... Or have anyone seen Scintill (because he did the audio file)?

Four Blocks Madness - Fixing

Sexplosion - Fixing

TtMH - Fixing

awww... borderline files should be conditional

[i love you]

Re: Queue/Batch Discussion Thread

Everything above the next quote are all gibberish BS due to my anger and disagreements.

*yawn*

Okay, now to actually answer something that you or anyone else from DF didn't know about:

Bold point #2: I don't even know why you're coordinating stuff within the game in the first place when you've been gone for who knows how long. That's my honest opinion, since you've been so kind enough to be abrasive when this entire thing could have been avoided had you actually discussed what I asked for and not made a snide comment. In my eyes, being a judge is a privilege, and not a job - if your attitude is going to be complete garbage to others, then you shouldn't serve in the community to begin with.

I haven't been gone from the game, i just don't play competitive FFR anymore which is why you haven't seen any scores from me in years. When i do tier points, I play-test each file offline plus look at the top 200 scores seeing what other people got in the file. i guess when you hear the word "retired" you will assume that i don't play the game at all which is true but that isn't going to stop me from playtesting files. I think you once told me that you were able to view replays from other players on your site so how about you go and look at my replays and tell me the last time I've played. I'm pretty sure it wasn't recent but it sure wasn't forever ago. Also, my attitude towards the community, and my judging are mostly positive.


Anyway, back on topic:

when I judge, most of my criteria for accepting is fun value, the rest is structure. I don't really look at the song itself since that's VERY subjective, but if the song itself generates very long repetitive sections then it's a given.

This is very similar what i do which is the main reason why i play-test each file first to see how fun and replayable the file is. If i notice something while playing it, i would mention it within my notes. Song choice, is usually minor for me but if the file is repetitive then i would say something about it and that IMO is what makes the person lose a lot of points from me. In the editor, i mainly look at sync, inconsistencies, and pattern choices. Of course, there would be times when I'd be subjective here and there but that doesn't mean that the person should agree with everything. If i think something could be better, i would state it but that doesn't mean it would affect my overall judge about it.