Queue/Batch Discussion Thread v2

[who_cares973]

Re: Queue/Batch Discussion Thread v2 (Submission Closed)

Lol out of those 60 wips only like 6 are for ffr

[leonid]

Re: Queue/Batch Discussion Thread v2 (Submission Closed)

The thing is, I think the judges would be overwhelmed by the number of files.

I'm sure MANY, MANY people have well over 10, heck, even 50 files ready for submission. I think the current system is fine, except that the batch needs to open more often. I mean, come on. who_cares973 has like 60 files in his folder. -_-;

Also, having a batch open all the time would drastically DROP the quality of files.
People realize that batches do not open frequently, and only get to submit X number of files.
Thus, they must put their utmost effort into stepping songs, otherwise their X submissions that they're given every X months is wasted.

If the batch was open all the time, people wouldn't feel as pressured to step well.

There are always pros and cons; of course some tweaking is needed
One way to prevent submission spam is to restrict the number of files from the same person in the judge queue that haven't been judged yet: if your SAP is 4 and there are already four of your files in the judge queue, you are not allowed to submit before one of them gets judged.
Speaking of the pressure, I think people would feel more pressed if they have a deadline.

[iironiic]

Re: Queue/Batch Discussion Thread v2 (Submission Closed)

I feel that the current judging system is extremely exclusive (break up to teams and judge a specific third of the batch). I know there is a new ten-point system proposed recently and I like that so far. I think to further make this system effective, each judge should make contact with each other about the files they are all judging (not simultaneously). That way, this method can avoid skewed ratings and contradictory commentaries on a particular file. It will also enlighten judges to view files differently and to consider that as they give a more accurate and deserved rating.

Downside: time commitment. Everyone, including myself, is busy these days haha.

EDIT: This might be a second silly idea, but to give judges more incentive to judge files seriously, maybe implement a judgment rating for each person? If someone is consistently giving incorrect ratings on a file, lower the rating to the point where JX can fire Judge A. Just a thought.

With this idea, I think anyone who is interested in judging can take advantage of this to see if judging is the right thing for them. After all, we are in a high demand for judges.

[Rebirth0]

Re: Queue/Batch Discussion Thread v2 (Submission Closed)

My suggestion:

At the start of every month, open submissions for the first half of the month. 15 days. After 15 days, close it, and judges take the next two weeks to judge and submit notes. There can be two sets of 3 judges. One set judges files one month, the other does the next month. Everybody only does work 2 out of 8 weeks. If someone wants to go through every batch as an additional judge, let them.

4 or 5 songs per week all released on the same day, including one new skill token or regular token every week. I will even come up with them if I have to. It isn't hard. The token can be released every Sunday while everything else can be released some other day.

Submission caps can stay the same or become smaller, depending on how these submissions would go. If the quality of files gets lower with more submission time, then set a hard limit to what is ACCEPTED, not SUBMITTED. 30%? 20%? It shouldn't be a number, it should be a percentage if there is a limit at all.

I am ready to start working with this site again as whatever needed. As a judge, absolutely. FFR has well over 1.5 million users so obviously, with a huge community comes a huge workload. Deal with it. What is happening now isn't making anyone happy in the long run. We want people who will stick around and contribute eventually.

[00Razor00]

Re: Queue/Batch Discussion Thread v2 (Submission Closed)

Rebirth0, the voice of reason.

[qqwref]

Re: Queue/Batch Discussion Thread v2 (Submission Closed)

I feel like having a batch open for only a few days encourages people to rush files and get them in as quickly as possible, rather than having people take their time to get a good chart they are happy with. (This is bad because people rarely fix up their charts.) Of course, as we've seen, it also prevents certain people from sending in files meant for FFR just because they are too late, thus leading to people being unhappy with the entire system. I'd be in favor of leonid's approach, or else, just leaving the batch up for a set amount of time (a week, say) and not worrying about how many files are in it. If one person can judge 25 or so files in one day, I'm sure they will be able to handle 50 or more files in their group, given that they have a few weeks to get it done.

But hey, do whatever you want. I'm not the one in charge.

[bmah]

Re: Queue/Batch Discussion Thread v2 (Submission Closed)

I can't review 25 files in a day for the work schedule I have.

[hi19hi19]

Re: Queue/Batch Discussion Thread v2 (Submission Closed)

Just saying that if leonid's system was put in place with no checks, I would be very happy.
And the judges would be very overloaded.

Honestly, as happy as I would be, I do think it needs -some- restrictions if just for the crazies like me.

[i love you]

Re: Queue/Batch Discussion Thread v2 (Submission Closed)

I feel like having a batch open for only a few days encourages people to rush files and get them in as quickly as possible, rather than having people take their time to get a good chart they are happy with. (This is bad because people rarely fix up their charts.) Of course, as we've seen, it also prevents certain people from sending in files meant for FFR just because they are too late, thus leading to people being unhappy with the entire system. I'd be in favor of leonid's approach, or else, just leaving the batch up for a set amount of time (a week, say) and not worrying about how many files are in it. If one person can judge 25 or so files in one day, I'm sure they will be able to handle 50 or more files in their group, given that they have a few weeks to get it done.

But hey, do whatever you want. I'm not the one in charge.

I have to agree with this. Not much to say really.

[Anaru]

Re: Queue/Batch Discussion Thread v2 (Submission Closed)

leonid's post makes good sense, but we only have so many judges..

Even with the current system of reviewing files, it takes absolutely forever for batches to get judged. :/ But reviewing files on a weekly basis (who knows how many files they'll get.. 50? 100?) might be overwhelming.


Or, nominate some more judges.

[hi19hi19]

Re: Queue/Batch Discussion Thread v2 (Submission Closed)

Hmm, batch pic?
I realize I'm being that guy, but hey someone's gotta be him

[who_cares973]

Re: Queue/Batch Discussion Thread v2 (Submission Closed)

leonid's post makes good sense, but we only have so many judges..

Even with the current system of reviewing files, it takes absolutely forever for batches to get judged. :/ But reviewing files on a weekly basis (who knows how many files they'll get.. 50? 100?) might be overwhelming.


Or, nominate some more judges.

i really doubt there would be 50+ files in the batch every month. maybe for the first month or two but after that it would slow down especially with a submission cap like leonid said

[Anaru]

Re: Queue/Batch Discussion Thread v2 (Submission Closed)

i really doubt there would be 50+ files in the batch every month. maybe for the first month or two but after that it would slow down especially with a submission cap like leonid said

Really? I'm pretty sure there'd be more than 50 people submitting files every month.. I mean, we have 1.7 million registered users right? (Even if there's only 100k active or whatever..)

[bmah]

Re: Queue/Batch Discussion Thread v2 (Submission Closed)

Most people on this site are players, not steppers.

[jimerax]

Re: Queue/Batch Discussion Thread v2 (Submission Closed)

FYI, judgement process should be at "sustainable" pace. A system that lasts for only a few months is useless.

By "batch open all the time" syetem, it will be complicated to manage and we don't have a method for controling submission rate.

Judges who have time and judgement speed can judge files in all groups (If we have 50 files/group and 3 groups, 150), and Judges who don't can only judge files in one group (50). This is what I wanted to say.

ps batch pic.
some files were removed due to submission cap, or timing. 106 files, still.