Difficulty Consultant Applications

[psychoangel691]

Re: Difficulty Consultant Applications

We can screen that easily in a few releases. If the CM doesnt seem to consider lower level difficulty structure, people will notice.

The lower area has been neglected for years, I've complained about it a ton myself. Others used to but then I think they all gave up over time because it continued to be ignored.

Higher-level players, especially those who've been there a while forget what it was like to be at that lower level and what was hard when they were starting out. I'm trying to get away from that mentality. This system has been terribly flawed and needs to be approached differently.

[One Winged Angel]

Re: Difficulty Consultant Applications

High-level players tend to be too elitist and don't even bother to think about the little guy. Sorry but it's true.

This isn't a fair assessment at all considering how the difficulty shifts transpired.

stavie33 and I were put in charge of the initial shift from 1-12 to 1-100. Stavie claimed the lower half as he wanted to replay those charts and AAA files he had left unplayed at the time. I had no qualms with this so I took to the upper half. After the initial shift I noticed a lot of charts in the lower half out of place compared to where I believed they belonged but only mentioned the largest offenders so as not to overstep any boundaries. I also continued to make +/-1/2 adjustments on my end on an ongoing basis. I'm not sure how many adjustments stavie made, if any honestly.

Eventually stavie dropped out and it was public opinion that the lower half scaling made little to no sense in comparison to the upper, and there were several charts rated close to ~10 points off where the community believed they should lie. I spoke to several players that weighed in on difficulty discrepancies in that subforum that appeared to have balanced skillsets, could articulate themselves well with valid opinions for shifts, and were willing to help restructure the lower half. Zapmeister in particular seemed perfect for the job but his activity on the site fluctuated a lot, and I'm not even sure if he responded to the last PM I'd sent him years ago. Eventually ended up adding Zenith and Haku with Zenith having played through the entire lower half already and completing a revision combover a few years back. This was a substantial improvement from what the lower half used to look like. But I don't believe many revisions were done on an ongoing basis past this, and there's still a considerable number of outlier charts in that range, to the point where it becomes evident that rating new files accurately in the lower half of the spectrum is next to impossible when attempting any sort of proper comparative analysis.

Almost any new chart added under 50 effectively has nearly a 5 or 6 point confidence interval that it can be added as and people likely won't complain. On the high end, anything off more than 2 points gets laughed at. I don't think that's elitism, I think that's a testament to how well-maintained and curated the high end has been. This is evidenced by players with 90+ skill ratings still weighing in when a 60-something seems off: something in that difficulty range ultimately doesn't matter much to them, but it's still obvious enough to point out. Higher skilled players don't often bring up misrates on the lower half, not because they don't care, but because that large confidence range exists when it shouldn't.

I also agree with op, and will try to avoid typecasting all lower level players because exceptions surely exist (again going back to Zapmeister, he was perfect for this), but lower level players are generally unable to acknowledge how unbalanced their skillsets might be to the degree of D6+, which will likely lead to a lot of misrates. There are D6+ players entirely capable of the analysis required to add increased linearity to the lower difficulties that shouldn't be dismissed simply because they're too good at the game. They just need to put the time in to re-familiarize themselves with those charts.

[TheSaxRunner05]

Re: Difficulty Consultant Applications

I'll be putting an application in tomorrow on my day off. One thing I do think gets lost a bit when higher level players judge lower level songs is stamina. While some songs like Makiba appear to have it factored into its difficulty, other songs do not. When it comes down to it, I think it comes down to having these discussions, laying the rules for how it should be, and then sticking to them consistently across the range. Overall, there has been a lack of time really examining the difficulty in the lower range and I'd like to help out with that. I think this is a step in the right direction whomever is chosen - it needs time and work put into it.

[Precarious]

Re: Difficulty Consultant Applications

Can you provide arguments to explain your claims ? It's not obvious to me that this is true or not.

I tend to think the opposite: high level players (with a decent amount of experience, and having played most songs or at least a wide variety of files) usually have a better understanding of what constitutes the partly subjective difficulty metric we use, like bpms, patterns, transitions, stamina, spikes, etc.

You have to remember that players with a level higher than a file's difficulty should also agree with the arguments used to pick a certain difficulty number, otherwise it's a bit pointless. That being said, we don't gain much by having a lower level player be a DC, other than they might play the easier files more often, which I think isn't a heavy weight in the balance.

This is exactly the problem I sort of hinted at in my application. In no way am I as fluent in the jargon of FFR as you are--and I mean that both in terms of the actual vocabulary, and in reading and interpreting a given pattern or set of patterns in a file. But that's precisely the problem with the current system. Difficulty is subjective, not objective. You literally have the current number 1 average rank on the site. You were a snipe away from winning D7. There are files you can AAA effortlessly that I can't even play--so of course you'd be infinitely better suited to judging such material than myself.

But by the same token, at some point you lose your ability to determine why something is hard when it no longer is hard. You can have two files that are superficially similar--comparable bpms, types of patterns, stamina requirements, and so on--yet feel completely different to a player at that level. If everything is easy for you, then how can you meaningfully distinguish between these files? You can't--other than to refer to those preexisting criteria. But nothing occurs in a vacuum, and incidentals that can't be so easily quantified exist. Evaluating difficulty is an art, not a science--otherwise we could just user an auto-rating system a la Etterna.

What you're suggesting is only true if you honestly believe that difficulty is completely, statistically quantifiable. And yet that clashes with our existing statistics--and mars the player experience anyway. Yes, it's obviously true that lower players will see greater variation, as they will invariably have holes in their skillset that lead otherwise comparable files to move apart. But I'll take that over dispassionately calculating from a spreadsheet. To argue otherwise is to suggest you're engaging with lower difficulty files in the same way as someone fighting against them, and that's simply not true.

High-level players tend to be too elitist and don't even bother to think about the little guy. Sorry but it's true.

This isn't a fair assessment at all considering how the difficulty shifts transpired.

It's completely true--and has nothing to do with difficulty shifts. Look at Opkiller's response; it considers the DLow-Mid experience only from that of DHigh. This attitude, consciously or unconsciously, permeates the site, and I feel it's a problem both for player attraction and retention.

But that's not really what this topic is about. You've done more than anyone to make ratings here accurate (and I thank you for that), but even so your response is bound up in the assumptions born of your own considerable abilities.

So consider the following scenario. A new file is released, and ten D6/7/8 players all rate it a 52. Ten D2/3/4 players rate it 51, 54, 56, 58, 54, 60, 50, 61, 57, and 57. One might surmise that by quantifiable standards, it must be a 52. All of the more skilled/experienced players determined it to be such, likely by calculating backward from bpm and patterns, and comparing it against their own expectations for such a file. The lower division estimates were all over the place, but averaged to 55.8. I'd argue that assessment is closer, because the file is meant to be representative and demonstrative of their skillsets. Yes, the lower you go, the more gaps there will be in skillsets, and thus the larger the potential for variation and outliers. But difficulty itself is, by everyone's contention, subjective, and thus matters most of all by comparison to other files. Not just in terms of speed and length and stamina, but in how comfortable and playable it is for the player it challenges. At some point, playability needs to become the key concern--and that's something lost as one gets further from a given file presenting any sort of challenge.

I did read your reply, and I do think it's valid. Certainly, there's going to be a certain amount of give unless we revamp lower difficulty entirely. But that's the thing. We should do that. If your proposal for higher level difficulty/AAA equivalency revamp goes through (and I like your discussion, and I think it should, whatever form that takes), that seems to me a good excuse to revisit everything.

[xXOpkillerXx]

Re: Difficulty Consultant Applications

I dont think you can correctly argue the difficulty of any file, easy or hard, in terms that me (or owa, or any other high lvl player) can't understand. Therefore, it's only a matter of assessing difficulty differently depending the on a file's difficulty range. It's really that simple.

[PhantomPuppy]

Re: Difficulty Consultant Applications

coming from someone that isn't naturally good at rhythm games, i'd say that i agree with the statement that a lot of high level players overlook some of the patterns that would be considered difficult to a D3/4. it took me years to go from barely being able to get under 50g FC on sound of my dream rmx (a 17 on the current scale) to where i am now. and now that i've made it, i can honestly say that D7's tend to rank files through a very biased lens that reflect their own skillsets.

like, i can't AAA for shit. i legit still have an issue trying to consistently AAA some D4 songs because the patterns are pretty wonky, while others files will be ez pz lemon squeezy but ranked higher just cuz they are faster.

now this is just completely my opinion and DOES NOT apply to all D7/8 players, but a lot of us tend to skip over the fact that these weird ass patterns in lower ranked songs took a long time to learn and didn't just come naturally to us either.

edit: i just want to make clear that my statement is not a personal attack on anyone. k thx

[devonin]

Re: Difficulty Consultant Applications

I dont think you can correctly argue the difficulty of any file, easy or hard, in terms that me (or owa, or any other high lvl player) can't understand. Therefore, it's only a matter of assessing difficulty differently depending the on a file's difficulty range. It's really that simple.

If it's that simple, why does it commonly appear to be understood that it hasn't happened?

[xXOpkillerXx]

Re: Difficulty Consultant Applications

coming from someone that isn't naturally good at rhythm games, i'd say that i agree with the statement that a lot of high level players overlook some of the patterns that would be considered difficult to a D3/4. it took me years to go from barely being able to get under 50g FC on sound of my dream rmx (a 17 on the current scale) to where i am now. and now that i've made it, i can honestly say that D7's tend to rank files through a very biased lens that reflect their own skillsets.

like, i can't AAA for shit. i legit still have an issue trying to consistently AAA some D4 songs because the patterns are pretty wonky, while others files will be ez pz lemon squeezy but ranked higher just cuz they are faster.

now this is just completely my opinion and DOES NOT apply to all D7/8 players, but a lot of us tend to skip over the fact that these weird ass patterns in lower ranked songs took a long time to learn and didn't just come naturally to us either.

Do you not consider at all the possibility that you are better at speed stuff than technical ? "Weird ass patterns" is very vague. Also D7 very likely do the opposite of judge diff 50 files based on their skillset, as they excel in everything such a file requires. A level 50 who's good at jacks though might very well judge that same file with a bias towards jacks being easier.

[xXOpkillerXx]

Re: Difficulty Consultant Applications

If it's that simple, why does it commonly appear to be understood that it hasn't happened?

Because very, very few people actually met the criteria of (1) having the time for this "job", (2) putting in the effort of somewhat defining boundaries/structure for their decisions on difficulties and (3) have the experience to do that.

I'm not saying we should absolutely have a high lvl player do it, I'm arguing that high lvl players are usually better suited for the job.

[xXOpkillerXx]

Re: Difficulty Consultant Applications

Many people use the typical "difficulty is subjective" line when they can't or don't want to argue further on difficulty. Yes difficulty is subjective, however that doesn't mean it shouldn't have structure. The most common points I see lately are about bpms + patterns (and occasionally stamina/length). This combination has its subjectivity (just like any other basis of difficulty) but it's debatable in terms of what's harder than what. Lower end files are no different: some combinations are harder than others. Fun fact, Etterna uses pretty much that kind of structure, split into categories.

Lets debate this example of yours:

So consider the following scenario. A new file is released, and ten D6/7/8 players all rate it a 52. Ten D2/3/4 players rate it 51, 54, 56, 58, 54, 60, 50, 61, 57, and 57. One might surmise that by quantifiable standards, it must be a 52. All of the more skilled/experienced players determined it to be such, likely by calculating backward from bpm and patterns, and comparing it against their own expectations for such a file. The lower division estimates were all over the place, but averaged to 55.8. I'd argue that assessment is closer, because the file is meant to be representative and demonstrative of their skillsets. Yes, the lower you go, the more gaps there will be in skillsets, and thus the larger the potential for variation and outliers. But difficulty itself is, by everyone's contention, subjective, and thus matters most of all by comparison to other files. Not just in terms of speed and length and stamina, but in how comfortable and playable it is for the player it challenges. At some point, playability needs to become the key concern--and that's something lost as one gets further from a given file presenting any sort of challenge.

First of all, gotta remind people that difficulty is AAA based currently. If all these 10 players rate the file so high, that's perfectly fine... if at least one of them can say why. You don't consider the fact that these players who "fight" the file probably don't have the AAA. Thinking this is elitist is just refusing to accept that more experienced/skilled players will likely have a better understanding of difficulty structure in general. A flagrant example of how it should go is the later rounds of D8 in the last OT. Many players in high end, including myself, tried to give the files proper ratings, but Silly and Etienne's opinions were a bit different in general and we all kinda followed them based on their scores. Nobody complained that these two were being elitists, that would've been nonsense. Their scores and feedback were very helpful in agreeing on where these insane files should go. The same principle applies to any range.

Also, difficulty is relative to other files in a specific range. That being said, I'm pretty confident that it's more likely for a lower level player to get lost in the ratings if they dont set a minimal structure because, as it's been pointed out, the lower end is messy af. One could easily fall in the trap of X feels harder than Y so it seems like it should be bumped up, but the fact is that Y is an outlier of its range and should Also be changed.

All in all, if any DC explains to me their decisions without any attempted objectivity in their comparisions, especially if on the basis that "difficulty is subjective", then I'll just lol and chances are people will complain.

[psychoangel691]

Re: Difficulty Consultant Applications

I promise I'm not trying to be rude, I'm just not in the mental state of reading the big posts at the moment but ultimately the fact is the way it's been approached hasn't been working out.

I want a diverse team, I've made this clear and I'm not the only one who thinks it should be so. Yes, they may not know every little detail but that's why us seasoned people are there, to help them learn and understand what they may not while being able to give us insight that we may no longer have being in the upper area.

This site has always had an elitist problem, for many many many years. You can say no it's not true but I see it regularly. It's a shame how some of the upper level people even talk to some of the lower end when they try to bring up some of this stuff.

Anyway, this is going to be a long project and not something that's gonna happen straight away so if for some reason it's not looking like it's working out we'll re-evaluate and go from there.

[xXOpkillerXx]

Re: Difficulty Consultant Applications

I promise I'm not trying to be rude, I'm just not in the mental state of reading the big posts at the moment but ultimately the fact is the way it's been approached hasn't been working out.

I want a diverse team, I've made this clear and I'm not the only one who thinks it should be so. Yes, they may not know every little detail but that's why us seasoned people are there, to help them learn and understand what they may not while being able to give us insight that we may no longer have being in the upper area.

This site has always had an elitist problem, for many many many years. You can say no it's not true but I see it regularly. It's a shame how some of the upper level people even talk to some of the lower end when they try to bring up some of this stuff.

Anyway, this is going to be a long project and not something that's gonna happen straight away so if for some reason it's not looking like it's working out we'll re-evaluate and go from there.

I'm debating how difficulty should be assessed. I'm very ok with having a lower lvl player tackling this challenging task; it's not an insult that they may have a tougher time to do it than a more skilled player, it's just how it is. I will argue with any player, D7 or D1, if their explanations are bad. I already do it with Zenith and Haku. My goal isn't to provoke DC's, it's to have coherent difficulties.

[psychoangel691]

Re: Difficulty Consultant Applications

I'm debating how difficulty should be assessed. I'm very ok with having a lower lvl player tackling this challenging task; it's not an insult that they may have a tougher time to do it than a more skilled player, it's just how it is. I will argue with any player, D7 or D1, if their explanations are bad. I already do it with Zenith and Haku. My goal isn't to provoke DC's, it's to have coherent difficulties.

It's impossible to have coherent difficulties right now because the system is too flawed. Too many people are basing it on what "they" find hard or easy. Or looking at a small set of players. You guys can debate if you want to, we already have a plan in mind.

I honestly wish you all would just hold up and wait till we get this settled more formally and present our idea rather than it keep being this huge thing.

[M0nkeyz]

Re: Difficulty Consultant Applications

If a file is too easy to judge for a higher-skilled player, isn't it a possibility to just up the rate? From my personal entirely subjective experience what makes a file hard for a beginner that's playing index usually isn't that different from what makes a file hard for a more advanced player. It's usually just the speed and frequency at which the patterns come up.

Even if you have a level 3 file where arrows come in lets say every 0.5 seconds a [1,2,2,1] pattern is still going to be harder to hit for most players than a [1,2,3,4] pattern.

[SK8R43]

Re: Difficulty Consultant Applications

Being a pretty low D6 player whose ass at aaaing shit and pretty bad at dense patterns i definitely feel like we need lower level difficulty people. Having mostly higher end doesnt really make much sense since you dont really have trouble with most patterns. Over the past 2 days ive played a bunch of newer songs and i feel like a bunch of them are a bit off still. I'm level 87 and still have trouble with songs in the 60s and 70s range. We might not necessarily need a D2 OR D3 player but i think having maybe a D4,5, and/or D6 would be good since they can see harder patterning better than someone who plays 100s and aaas them with ease.

You really do need to think like a newer player. Starting out jacks and rolls/bursts are pretty hard. I feel like compared to older lower level files the newer ones are getting much more difficult in terms of actual patterning and layering. Like, hands were pretty rare back in the day and now ALOT of lower level songs have them. For a new player thats just learning they wont be able to comprehend them all that fast so something with hands might be harder in terms of difficulty but a high level player wont see that.