Signature Performance - An Explanation

HMXFedHMXFed Harmonix Developer
edited September 2012 in Peer To Peer Support Forum
Since there's been a lot of discussion, here's some info about how signature performance works and two recent changes we made to it.

1. Until recently we were choosing a performance that might not have been on your best or your favorite instrument. Now it will be on the instrument you have the highest career (plus DLC) score on. (Keys/Pro-keys might be underrepresented because there are fewer songs that can add to their career score.)

2. We also recently started giving extra performance points for songs played on higher difficulties and for songs at higher difficulty tiers. Before, easier songs were too heavily represented in signature performances. This is called the "Charlene adjustment" because I didn't want Charlene to be my signature performance anymore.

Your specific rank # on a leaderboard doesn't affect signature performance because it's not normalized across songs. A rank of #1,000 on a leaderboard with 1,000 scores means something very different than #1,000 on a leaderboard with 100,000 scores. So it's not useful for comparing performance across different songs.

Stars and accuracy also aren't directly considered. What we do look at is your score, in a way that's normalized across songs. Scoring highly is the most important thing. Gold stars, 100% accuracy, and a high leaderboard rank will help because they go along with high scores, but they won't guarantee a signature performance.

Comments

  • Bront20Bront20 The Writing's on the Wall
    edited January 2011
    TheHundredDollarHeadache;4269291 said:
    A suggestion, related to the "signature performances might not be on our best or favorite instruments" bit, and the keys underrepresentation:

    Supplement the "Scores Highest On" instrument with a "Most Skilled On" instrument based on relative rank for disc song total score, and have our signature performances be on the "Most Skilled On" instrument.

    Or better yet, just letting us pick a favorite instrument like the Rock Band 2 days.
    Or do your signature performance on each instrument?
  • TheHundredDollarHeadacheTheHundredDollarHeadache Pedantic Broken Record
    edited January 2011
    Bront20;4269639 said:
    Or do your signature performance on each instrument?
    That would take up too much space on the front page, but would be great on a detail page. The Favorite or Most Skilled On instrument would just choose the instrument that shows up on the front.
  • Tom ImpTom Imp Road Warrior
    edited January 2011
    Bront20;4269639 said:
    Or do your signature performance on each instrument?
    I like this idea a lot. :)
  • TarrottTarrott Unsigned
    edited January 2011
    Mine changed from Like A Fool to Casey Jones, but I haven't played Casey Jones for a couple weeks (I don't think.) But, I'm not complaining. You guys can just go ahead and lock that one in for me permanently. Really. I don't mind.
  • hmanjihmanji Opening Act
    edited February 2011
    HMXFed;4265453 said:
    Stars and accuracy also aren't directly considered. What we do look at is your score, in a way that's normalized across songs. Scoring highly is the most important thing. Gold stars, 100% accuracy, and a high leaderboard rank will help because they go along with high scores, but they won't guarantee a signature performance.

    I thank for the changes, because now at least the genre is correct. My new signature performance is now Mötley crue's "Without you". While I don't mind having Mötley crue as my sig performance, Im still bit confused as to why I would have a song that I have played only once as a 'signature performance'. What does signature even mean then? Oh well, I complain slightly less now.
  • bclewisbclewis Road Warrior
    edited February 2011
    I don't particularly care, but I suspect songs that are over-tiered will be much more likely to appear as signature performances. I'm probably now stuck indefinitely with "Interstate Love Song" on pro drums, which is rated at four stars even though (IMO) it's pretty easy in comparison with others in that tier.
  • nbauernfnbauernf Unsigned
    edited February 2011
    May I provide a suggestion? It would be completely awesome to take into account the number of times you've played those songs. (Like an extra performance point per star ever earned playing that song on that instrument?)
  • Bront20Bront20 The Writing's on the Wall
    edited February 2011
    nbauernf;4280734 said:
    May I provide a suggestion? It would be completely awesome to take into account the number of times you've played those songs. (Like an extra performance point per star ever earned playing that song on that instrument?)

    That could quickly become "Most played", which isn't what it's supposed to be.
  • nbauernfnbauernf Unsigned
    edited February 2011
    What about as a tie breaker? Then it's your most played eligible song =).
  • madcapmattmadcapmatt Road Warrior
    edited February 2011
    How does Pro/Standard difficulty figure into signature? For instance, pro Bass on Smoke on the Water is A LOT harder than legacy Bass. On the flip side, some Pro Drum songs simply move up to the cymbals from being all pad hits in Standard.
  • JPS2010JPS2010 Opening Act
    edited February 2011
    And thanks... My signature performance now is exactly as I want it to be :)
  • Edify1Edify1 Rising Star
    edited February 2011
    hmanji;4278708 said:
    I thank for the changes, because now at least the genre is correct. My new signature performance is now Mötley crue's "Without you". While I don't mind having Mötley crue as my sig performance, Im still bit confused as to why I would have a song that I have played only once as a 'signature performance'. What does signature even mean then? Oh well, I complain slightly less now.

    I'm with you there. Can I please get my signature performance changed? It's Blue Christmas by The Pretenders. I don't like their version, and I've only played it once or twice (I only got it because I wanted some Christmas music :D ). For some reason, it's been stuck as my signature performance for ages, even though I'm certain I do much better on other songs. I've full comboed higher-tiered songs than that.
  • SheSaidSheSaidSheSaidSheSaid Washed Up
    edited February 2011
    I wish there was a way to pick it...mine used to be "The Killing Moon" on X Guitar and now it's "Killing Loneliness" I don't mind "Killing Loneliness" but I greatly, greatly prefer "The Killing Moon" (and it's a much more reliable FC for me, so I'm not sure why it got taken down)
  • BohemianMattBohemianMatt Headliner
    edited February 2011
    Mine hasn't changed in forever. I'm glad mine is Could You Be Loved by Bob Marley, but I'd like to see what else has been deemed worthy since then.
  • XebarsisXebarsis Opening Act
    edited February 2011
  • KMO_2000KMO_2000 Opening Act
    edited March 2011
    HMXFed;4265453 said:

    Your specific rank # on a leaderboard doesn't affect signature performance because it's not normalized across songs. A rank of #1,000 on a leaderboard with 1,000 scores means something very different than #1,000 on a leaderboard with 100,000 scores. So it's not useful for comparing performance across different songs.
    (Statistician hat on)

    That's true, but you shouldn't be comparing rank #, you should be comparing rank percentile (rank/number of scores):

    #1,000/1,000 = Top 100% = rubbish
    #500/1,000 = Top 50% = average
    #1,000/100,000 = Top 1% = pretty good
    #1/1,000 = Top 0.1% = very good
    #10/100,000 = Top 0.01% = fantastic
    #1/100,000 = Top 0.001% = Rock Legend.


    The only issue remaining then is easy songs where you can hit the top fairly easily = eg a song where 100 people are "first equal". To deal with that, the calculation should treat all tied ranks as equivalent to the middle rank, not the highest, so treat all those people as being rank "50.5" in this case, not 1.

    [Edit 14-Apr-11: maths correction - to get this precisely right, you need to always take a notional midpoint, even for solo scores. The numerator needs to be (your_rank+next_rank)/2-1. The calculations above should be:

    #100,000/100,000 = 99999.5/100000 = 0.999995 = abysmal
    #1,000/1,000 = 999.5/1000 = 0.9995 = rubbish
    #500/1,000 = 499.5/1000 = 0.4995 = average
    #1,000/100,000 = 999.5/100000 = 9.995E-3 = pretty good
    #1= among 100/100,000 = 50.5/100000 = 5.05E-3 = pretty good
    #1/1,000 = 0.5/1000 = 5E-4 = very good
    #10/100,000 = 9.5/100000 = 9.5E-5 = fantastic
    #1/100,000 = 0.5/100000 = 5E-6 = Rock Legend.
    ]

    If you wish to select from the player's overall best instrument - sounds reasonable - that should be standardised on a percentile too, rather than absolute total score.

    I think this calculation will do a better job of what you're trying to attempt, given your stated intent in the original post.

    In my case, I'm clearly best at Pro Keys compared to other players, based on "total score ranking divided by number of people ranked". And I'm #1 on a number of Pro Keys leaderboards, which would then mean this algorithm would probably select whichever song I'm untied in the lead of has the most ranked players, giving the lowest (rank/players) number. And the result of that would make more sense to me as a Signature Performance than my fairly unremarkable gold star Guitar run of "Killing Loneliness".
  • KMO_2000KMO_2000 Opening Act
    edited March 2011
    Actually, my signature performance by that algorithm would probably be Pro Keys on Roundabout or one of the other on-disc songs. My #1's are on DLC, and there are going to be more than 10 times as many people playing the on-disc songs, so my #9 for Roundabout is a better percentile than my DLC #1s. Still fine by me.
  • hmanjihmanji Opening Act
    edited March 2011
    KMO_2000;4322656 said:

    I think this calculation will do a better job of what you're trying to attempt, given your stated intent in the original post.

    I still don't have the foggiest idea what HMX is trying to achieve or what "signature performance" means.
  • DaijobuDaijobu Unsigned
    edited March 2011
    So, funny story. Or not. A friend of my husband's was over and decided that he'd play a song on my tag while I was out of the room. As a result, my signature performance is something that I didn't play, and don't think I can bury any time soon.

    Is there any hope, or do I just get to swear under my breath every time I see his score as my signature performance? ...jerk
  • TheHundredDollarHeadacheTheHundredDollarHeadache Pedantic Broken Record
    edited March 2011
    KMO_2000;4322656 said:
    If you wish to select from the player's overall best instrument - sounds reasonable - that should be standardised on a percentile too, rather than absolute total score.
    I agree with this. Unless the player plays practically nothing but Pro Guitar/Bass/Keys, one of the Basic "Instruments" or Pro Drums will be the Scores Highest On instrument.

    Using this system, my Scores Highest On instrument will switch to Pro Guitar (17th out of 22,900) from Pro Drums, and my Signature Performance will probably be my 5-star of Rainbow in the Dark.


    The "Most Played Genre" also needs standardization. Most players have plain old Rock as their most played genre, because of the sheer number of songs dumped into this ghetto of genericness. It should be done by the following formula:

    Adjusted Genre Play Count = Raw Genre Play Count / Number of Songs Released In That Genre.
    Daijobu;4342069 said:
    So, funny story. Or not. A friend of my husband's was over and decided that he'd play a song on my tag while I was out of the room. As a result, my signature performance is something that I didn't play, and don't think I can bury any time soon.

    Is there any hope, or do I just get to swear under my breath every time I see his score as my signature performance? ...jerk
    Play a different instrument enough to beat your guitar total score. I played enough Pro Drums to get rid of my joint first place on Llama as my Vocals signature performance.
  • SheSaidSheSaidSheSaidSheSaid Washed Up
    edited March 2011
    I just don't understand why it changes so much. Mine used to go back and forth among "PDA," "The Killing Moon," and "Killing Loneliness" all the time. Now it's "Real Wild Child." I like all four songs (naturally), and, also naturally, all four are gold-star FC's, but it changes sometimes to a song I haven't even played in a few weeks (noticed that with "PDA" a little while ago). I'm #2 in the world on "PDA," so I thought that made a perfectly logical signature performance, but if I'm even ranked on the two RB3 on-disc songs it's in the hundreds.
  • Bront20Bront20 The Writing's on the Wall
    edited March 2011
    I think they keep tweaking it a bit. I still wish they did a top 5. They'd swap out a little more often, and they wouldn't have to limit all of them to your top instrument.
  • BohemianMattBohemianMatt Headliner
    edited March 2011
    SheSaidSheSaid;4344909 said:
    I just don't understand why it changes so much.

    I have the opposite problem. Mine's been stuck on Could You Be Loved by Bob Marley for a month or two (since they first tweaked it). I'm waiting to see if it will ever change again. :p
  • hmanjihmanji Opening Act
    edited March 2011
    Yeah same for me, my signature performance basically doesnt change at all. Only change happened when Harmonix changed the formula somehow.
  • hmanjihmanji Opening Act
    edited March 2011
    No! Now it has changed... teen age riot... !"#!"#!

    Does it now just count the total number of points, which is reliant on the amount of notes in the song... GAH! So theres propably no way of me replacing this performance with any other since teen age riot has stupidly large amount of notes(strum) and is easy to play..

    Ive only played couple times... The signature performance still doesnt reflect what I actually play! What on earth is "signature" about it?!!!
  • LLCoolRay3000LLCoolRay3000 Unsigned
    edited March 2011
    Daijobu;4342069 said:
    So, funny story. Or not. A friend of my husband's was over and decided that he'd play a song on my tag while I was out of the room. As a result, my signature performance is something that I didn't play, and don't think I can bury any time soon.

    Is there any hope, or do I just get to swear under my breath every time I see his score as my signature performance? ...jerk

    I had a similar situation. A friend of a friend gold star'd "Living On A Prayer" on guitar under my profile, and since I'm a lowly medium player, it instantly became my signature performance. I love the song, but it was his signature performance, not mine.

    Well, here's a ray of hope for you: Now my signature performance is "Gimme Shelter" (my favorite song from RB1). I don't know how, but it is. And it's mine. And I'm very happy.

    I just wish they'd let us pick our own as a profile option.
  • KMO_2000KMO_2000 Opening Act
    edited April 2011
    TheHundredDollarHeadache;4343669 said:

    Using this system, my Scores Highest On instrument will switch to Pro Guitar (17th out of 22,900) from Pro Drums, and my Signature Performance will probably be my 5-star of Rainbow in the Dark.
    Rockbandscores.com is now up, which allows you to check this sort of thing out yourself, per-instrument at least. Just sort by "Rank %-ile", and see which comes top. In your case, it looks like "Break on Through (To the Other Side)" at #10/7226.

    Regretfully, rockbandscores.com doesn't quite get the %-ile maths right either. It regards all 1st places as equivalent 100.0%, as it looks only at the "top" of your ranking position, rather than taking the midpoint. So it thinks my #1/53 on "Coming in from the Cold" is better than my #20/20,779 on "Bohemian Rhapsody". Doesn't make sense - I'm one-in-a-thousand on the latter. But it's roughly right as long as you're not comparing songs with wildly different numbers of players.
  • Bront20Bront20 The Writing's on the Wall
    edited April 2011
    KMO_2000;4367099 said:
    Rockbandscores.com is now up, which allows you to check this sort of thing out yourself, per-instrument at least. Just sort by "Rank %-ile", and see which comes top. In your case, it looks like "Break on Through (To the Other Side)" at #10/7226.

    Regretfully, rockbandscores.com doesn't quite get the %-ile maths right either. It regards all 1st places as equivalent 100.0%, as it looks only at the "top" of your ranking position, rather than taking the midpoint. So it thinks my #1/53 on "Coming in from the Cold" is better than my #20/20,779 on "Bohemian Rhapsody". Doesn't make sense - I'm one-in-a-thousand on the latter. But it's roughly right as long as you're not comparing songs with wildly different numbers of players.

    It does the simple math, not the complex valuations.
  • KMO_2000KMO_2000 Opening Act
    edited April 2011
    It doesn't need to do anything complex. For that case, just calculate (53-0.5)/53=99.1% and (20779-19.5)/20779=99.9%, rather than (53-0)/53=100.0% and (20779-19)/20779=99.9%. And then there'd be a little more work needed to get tied scores right.
  • SheSaidSheSaidSheSaidSheSaid Washed Up
    edited April 2011
    Mine has held steady with "Real Wild Child" for a few weeks now. Still not really sure why that's any more "signature" than "PDA" or "Killing Loneliness" or "The Killing Moon" But I like the song just fine.
Sign In or Register to comment.