Crowdsourcing Data for Rates

[Trumpet63]

I am starting a little side project here, and I'm looking to get as many people as possible to contribute to this. I want to eventually be able to use rates in user tournaments, and that is going to need a formula of some kind. I need a bunch of scores using rates, and just using my data would take a long time and could be inaccurate.

SO, here's the guidelines for submitting scores:

>Play the song at normal speed three times, and then enter that third score (with a rate of 1). Do this even if you have played the song before.
>Then play that song on any rate you like
>Play on each rate no more than once, and submit only complete, legitimate plays.
>You are allowed to submit more than one data point for each song, ex: play on 1.1x, 1.2x, 1.3x, 1.4x etc. until it gets too hard.
>The more data points you can submit in a row, the better. Ideally, play until you have to start mashing. If you mash a lot during a play, don't submit.
>A complete submission consists of: Song Name, Rate Used, Raw Score, and Username
>You must submit for at least two songs of different difficulties, but do not submit a series of plays for the same song more than once


Other than that, anyone can submit data. If you're unsure just post in the thread. Keep in mind that I set this up so I didn't have to do all the data entry, that's your job

READ THE WHOLE POST BEFORE SUBMITTING DATA
Enter Data Here

[Rapta]

Re: Crowdsourcing Data for Rates

I'm going to have fun submitting my results, and I will be glad they will be put to good use. Good idea~

[Trumpet63]

Re: Crowdsourcing Data for Rates

A general formula. See if I can make one that's 'good enough'. It will return some value based on player score, rate, and song difficulty.

Eventually this could be used to inform a procedural difficulty calculation tool.

[One Winged Angel]

Re: Crowdsourcing Data for Rates

Someone from the site added me on skype to discuss something incredibly similar to this so I'll just relay my thoughts on here as well: it's impossible to generally conclude that, for example, a 70 AAA on 1.2 is equivalent to an 8X AAA using some sort of multiplicative factor or formula to determine the new difficulty on the rate. Files rated equally on 1.0 will not scale equally in difficulty when rates are applied. This is largely due to specific patterns (jacks, trills, walls, etc.) or any files that are structurally inconsistent in difficulty (spiky) approaching the human threshold limit of 'do-ability' much quicker than other types of patterns such as pure streaming.

For example, there's a file in the mid-high 50s called Fei Longer that emphasizes long jacks in the latter half of the chart. Most mid+ D7 players can AAA a number of mid-high 50s on 1.7/1.8 with relative ease. Fei Longer becomes vibrajacking (and thus significantly much more of a bitch to AAA) on probably 1.4 or higher (not entirely sure, I'd need to check myself).

In order to accomplish what I think you're trying to do, you'd need some sort of ban list for what I would assume to be several hundreds of files. And even then, I'd imagine that the remaining files won't scale all that well together :/

If I've completely misinterpreted what you're trying to do though pls ignore me.

[s1rnight]

Re: Crowdsourcing Data for Rates

oh! that was me that started that conversation haha. i'm not sure if this thread is very fortuitous coincidence~

i also brought up that people being able to log scores for off-1.0 rates might be very useful in establishing a difficulty curve for individual song rates. i support this thread very much!

[TheSaxRunner05]

Re: Crowdsourcing Data for Rates

Someone from the site added me on skype to discuss something incredibly similar to this so I'll just relay my thoughts on here as well: it's impossible to generally conclude that, for example, a 70 AAA on 1.2 is equivalent to an 8X AAA using some sort of multiplicative factor or formula to determine the new difficulty on the rate. Files rated equally on 1.0 will not scale equally in difficulty when rates are applied. This is largely due to specific patterns (jacks, trills, walls, etc.) or any files that are structurally inconsistent in difficulty (spiky) approaching the human threshold limit of 'do-ability' much quicker than other types of patterns such as pure streaming.

For example, there's a file in the mid-high 50s called Fei Longer that emphasizes long jacks in the latter half of the chart. Most mid+ D7 players can AAA a number of mid-high 50s on 1.7/1.8 with relative ease. Fei Longer becomes vibrajacking (and thus significantly much more of a bitch to AAA) on probably 1.4 or higher (not entirely sure, I'd need to check myself).

In order to accomplish what I think you're trying to do, you'd need some sort of ban list for what I would assume to be several hundreds of files. And even then, I'd imagine that the remaining files won't scale all that well together :/

If I've completely misinterpreted what you're trying to do though pls ignore me.

Try ABCDeath and see how that jumptrill goes at higher rates lol

[Trumpet63]

Re: Crowdsourcing Data for Rates

@OWA

You are completely right, but you'd be surprised how convincing a "not entirely correct" formula can be, ie my current tournament. And of course tournaments lend themselves well to hiding formula inconsistencies. More importantly, I'll be able to quantify the goodness of fit using this data, and derive some kind of nearly right formula. I plan to then use the formula in a tournament, testing it further, and later use the general intuition gained from the formula to push development of a difficulty calculation tool, which hopefully will then be used to replace this rate formula.

[One Winged Angel]

Re: Crowdsourcing Data for Rates

I have a feeling this won't work as well as the formula used for your current tournament, since the ratings for files in game don't take into account rate difficulty at all and in your tourney players just need to play files on 1.0 where, for the most part, files scale in difficulty linearly, but I'm extremely interested in seeing the results regardless.

[Trumpet63]

Re: Crowdsourcing Data for Rates

I have a feeling this won't work as well as the formula used for your current tournament, since the ratings for files in game don't take into account rate difficulty at all, but I'm extremely interested in seeing the results regardless.

That's right, you have faith in me to spit out something cool, and by golly I'm gonna do that! Thank you OWA, lol.

[hi19hi19]

Re: Crowdsourcing Data for Rates

On one hand I agree with OWA that you are going to find any formula that assigns some kind of multiplier depending only on rate and listed difficulty on FFR as inputs will be generally pretty inaccurate at making meaningful comparisons. The nature of how files unequally scale in difficulty with rates is much different than the roughly normal distribution of FFR file difficulties. Perhaps a "good enough" formula will surprise me though, just because I'm skeptical doesn't mean I'm not interested.
To make a better formula, I think the relevant things to look at will be NPS over time per finger and per hand, because those are the difficulty-determining attributes that are most affected when you toy with rates.


At the same time, regardless of how well you can make a formula, this will be a cool dataset to play with and a good resource to have, and I'll probably have fun contributing to it at some point. Just thinking of manipulating the data is making my inner economics major start to leak STATISTICS MATLAB GRAPHS

[FF_rules]

Re: Crowdsourcing Data for Rates

idk if this will help; also that 1x score is shit but that was my 3rd play through of it (I need to warm up more).
(the fact that this song starts off as just really REALLY slow jump stream and by increasing the rate I'm slowly facing my hardest pattern; Idk what to say)

-Let's throw you a curve ball (use this however you please, or even not at all)

(replaying after that slow down practice)

[s1rnight]

Re: Crowdsourcing Data for Rates

nobody's gonna put it into the spreadsheet for you haha

[FF_rules]

Re: Crowdsourcing Data for Rates

nobody's gonna put it into the spreadsheet for you haha

Oh right; spread sheet. Someone can do that for me =p
Put Waltz in, working on G Air; Finished with G air (Felt like lowered rates might be interesting for exceptionally difficult songs for those in lower divisions (albeit I couldn't stand 0.6x rate since it literally took forever)

[hi19hi19]

Re: Crowdsourcing Data for Rates

Low rates is interesting and definitely worth including.

[EzExZeRo7497]

Re: Crowdsourcing Data for Rates

Unintentionally deleted the first post which is what I wanted to keep, oops. But yeah, as much as I'm skeptical of a "general formula", I would really want to see this happen. There are some questions that I have in mind, but I think I'll be able to answer it on my own after looking at what would constitute as a good general difficulty formula.

I'll also contribute some scores of my own in a bit.

EDIT: goddamn those #N/A errors are driving me nuts, i hate u