Cheaper/Free Trials of XNA for Peer Reviewers

G_tarRoCK3RG_tarRoCK3R Road Warrior
edited April 2010 in The Rock Band Network
This topic keeps popping up in various threads, so lets try and keep this convo in one place!

What is everyones thoughts on this topic? I feel that if we are only doing peer reviewing and playtesting that we should get to do it for free. Why? What benefit do we REALLY get for giving out our hard work. I really hope HMX is trying hard to get us cheaper/free playtest only accounts.

I for one don't really think I will renew my subscription and help out if I gotta pay another $100 a year. Now its not really about the money for me. Its the fact that what do I get out of helping the community in this aspect? I thought it would be fun and cool to see the song early, but after having being doing this a month or two now, I feel that its not enough and I can just play the demo's or whatever. Its ALOT of work and other people are benefiting from my hard work (authors, HMX, M$, labels)

Now of course I think there should be VERY strict guidelines so that this won't turn into "I love this song I will give it all positive reviews without even playing it!"

There should be a weighted voting system (I've heard mentioned before that would work very well) HMX could assign people with ranks based on their play tests and reviews. Say you have a new guy. His would be 0 count for input, until they move up the ranks once proven to be useful/helpful.

Comments

  • overdriveguitarmanoverdriveguitarman The Walrus
    edited April 2010
    I don't like that people are saying they don't want just anybody to be able to do it because they won't do a good job, and then say they need more people. That isn't going to work out both ways!

    I have a lot of free time (obviously) and I would surely do a great job! But I am not going to pay so much to do this!
  • G_tarRoCK3RG_tarRoCK3R Road Warrior
    edited April 2010
    overdriveguitarman;3620120 said:
    I don't like that people are saying they don't want just anybody to be able to do it because they won't do a good job, and then say they need more people. That isn't going to work out both ways!

    I have a lot of free time (obviously) and I would surely do a great job! But I am not going to pay so much to do this!
    When did I say I don't want just anyone? If people are just going to free load the service (take all songs free for instance) and not helpout/particiapte and give adequate reviews, what good are they doing? Then will lose the "prilage' of a free account and have to then pay the $100 a year. I'm saying that yea we need more people to do this and playtesters shouldn't have to PAY to give QA.
  • overdriveguitarmanoverdriveguitarman The Walrus
    edited April 2010
    G_tarRoCK3R;3620138 said:
    When did I say I don't want just anyone? If people are just going to free load the service (take all songs free for instance) and not helpout/particiapte and give adequate reviews, what good are they doing? Then will lose the "prilage' of a free account and have to then pay the $100 a year. I'm saying that yea we need more people to do this and playtesters shouldn't have to PAY to give QA.
    You didn't say it but in another thread somebody said they didn't want it to be free so that people would have to be REALLY committed to this, but there aren't that many people that will do that!
  • kingtonyxkingtonyx Unofficial
    edited April 2010
    I do like that system, but a cost+reward system would work better.
  • G_tarRoCK3RG_tarRoCK3R Road Warrior
    edited April 2010
    overdriveguitarman;3620143 said:
    You didn't say it but in another thread somebody said they didn't want it to be free so that people would have to be REALLY committed to this, but there aren't that many people that will do that!
    Did you read my OP? lol I addressed how it could work with it still being free....a weighted ranking system for voting powers.
    kingtonyx;3620144 said:
    I do like that system, but a cost+reward system would work better.
    Honestly Tony, why are we paying to help them? They need our help.....they benefit. What do we get? Honestly, more authors should be peer reviewing/playtesting. Its not fair to the others that are doing this.
  • overdriveguitarmanoverdriveguitarman The Walrus
    edited April 2010
    kingtonyx;3620144 said:
    I do like that system, but a cost+reward system would work better.
    Oh yeah that's an idea!

    But then that would require Harmonix to do more work wouldn't it?

    How would they even know who to reward?


    Say they decided to do a "You review it you keep it", then a lot of people would just go around reviewing as much stuff as they can without putting any care into it at all!
  • kingtonyxkingtonyx Unofficial
    edited April 2010
    overdriveguitarman;3620143 said:
    You didn't say it but in another thread somebody said they didn't want it to be free so that people would have to be REALLY committed to this, but there aren't that many people that will do that!

    the only problem with it being free to do is this...

    say Ke$ha puts a song up for review

    as soon as anyone caught wind of it, you've got 10,000 Ke$ha fans rushing the review site and the song is approved within 10 seconds with no one actually looking at the charts or anything

    those people leave and never come back
  • overdriveguitarmanoverdriveguitarman The Walrus
    edited April 2010
    G_tarRoCK3R;3620149 said:
    Did you read my OP? lol I addressed how it could work with it still being free....a weighted ranking system for voting powers.
    You must have edited it because I did read your first post and it was different
  • G_tarRoCK3RG_tarRoCK3R Road Warrior
    edited April 2010
    overdriveguitarman;3620151 said:
    Oh yeah that's an idea!

    But then that would require Harmonix to do more work wouldn't it?

    How would they even know who to reward?


    Say they decided to do a "You review it you keep it", then a lot of people would just go around reviewing as much stuff as they can without putting any care into it at all!
    We shouldn't get the song for free.....idk what kind of rewards there can be. Rewards would be good, but free songs isn't really the way to go. They have to be QUALITY reviews you know? After reaching a certain (high) rank, maybe then you start to get rewards. But its not fair to the authors, artists etc. Which is why free would be best, no cost. Weighted voting system.

    This is how I see it:

    New guy, 0 reviews = 0 points
    5-10 QUALITY reviews = 1 point
    10-25 quality reviews = 2 points
    25-50 reviews = 3 points

    etc. etc.

    With this way, to get a passing reviewed song you would need a certain # of voting points (lets say 15) If you get 5 (3) ranked person, your song can pass review. There should also be like a minimum # of people also like 12. This prevents people for just pushing reviews and positives without even looking at the song. HMX would have to go through a persons history and see how they give the reviews/play tests to be based off this.

    Now that I think about it, this ranking should be for each instrument as well.
    overdriveguitarman;3620160 said:
    You must have edited it because I did read your first post and it was different
    Yea sorry I was editing as I go xD
  • kingtonyxkingtonyx Unofficial
    edited April 2010
    G_tarRoCK3R;3620149 said:
    Honestly Tony, why are we paying to help them? They need our help.....they benefit. What do we get? Honestly, more authors should be peer reviewing/playtesting. Its not fair to the others that are doing this.
    Well I do enjoy playing the songs early. There are some people there who are awesome and I don't mind helping out. It's just sorta hitting me that it's going to come to me having to choose between actually being able to buy some of these songs that I've helped with or paying to continue to test others...
  • overdriveguitarmanoverdriveguitarman The Walrus
    edited April 2010
    The main issue here is how do you know if it is a "quality" review you know what I mean?

    There is no way to know how the person reviewed the song. Maybe they really looked into it and analyzed it for long time, then decided it was ready for the game and didn't comment on it (I don't know if people can comment on things) because it was perfect and didn't need commenting

    Well how do you see the difference between that review and the review that was just given to get the song through because they like it?
  • kingtonyxkingtonyx Unofficial
    edited April 2010
    I think you'd have to look at the quality of the feedback.
  • G_tarRoCK3RG_tarRoCK3R Road Warrior
    edited April 2010
    overdriveguitarman;3620178 said:
    The main issue here is how do you know if it is a "quality" review you know what I mean?

    There is no way to know how the person reviewed the song. Maybe they really looked into it and analyzed it for long time, then decided it was ready for the game and didn't comment on it (I don't know if people can comment on things) because it was perfect and didn't need commenting

    Well how do you see the difference between that review and the review that was just given to get the song through because they like it?
    I've done that plenty of times. Of course those would count because they looked at the song. They should note that I didn't see anything wrong with the song, I looked at this this and this. As long as its not check yes on everything, no comments in the thread and they did this on 15 songs. You know? They can't look at every song on different things and NOT have a comment.

    Example here is one of my reviews from today:


    The mix

    Sounding pretty good, maybe bring the vocals up a TINY bit. And the overall master mix up a little bit as well.

    Expert Drums

    I don't know if this is my hearing or what, but the charted blue sections sound like they should be 8th notes instead of quarter notes (unless that is tambourine I hear?) I slowed it down to half speed and heard hits not being charted so thats why....

    I suggest moving the yellow crashes to their normal blue, but they are fine on yellow (at least its consistent!)

    I'll keep looking :) Looks good so far!

    I notice chords on bass, is that what the bassist plays is chords??

    Easy Drums


    WHOA, Major problems! No gems+kicks allowed. There are double bass kicks, only single bass.

    Interesting you charted 8th notes in the blue sections here but not on expert.

    Medium Drums


    This looks like a hard chart to me. I would remove those off beat kicks and any bass that there is two in a row to make it only one.


    Its just little things, nothing really major (except for the easy drum chart lol) but note things, even if they might not be changed or significant.

    Another review I had was:

    Everything is looking good! Except for a few minor issues on medium guitar, and I feel that medium drums could be reworked a lot.

    Medium guitar

    You have a triple red note, I'd remove the middle note reoccurs a few times:

    M33.4, M37.4, M51.4, M55.4

    Another reoccurring difficulty is a quick double red might be too difficult, remove the second red:

    M34.4, M38.4, M52.4, M56.4, M60.1, M64.1

    Medium Drums


    Double kicks are spread throughout the song, remove the offbeat kicks

    The chart is all over the kit and will be extremely confusing to the medium player. Try reducing the chart more. If you would like examples let me know and I'll try and suggest something. There is too much back & forth.


    See how on the second review I said everything was ok? But I also noted other things. They are suggestions, and the author can choose not to change them (the author didn't in this case) but you throw it out there.
  • SayburrSayburr The Always Informative Rock Band Forum Guru
    edited April 2010
    I wouldn't mind doing peer reviews, but I am not going to pay $100 a year or $50 per third of a year to do it. I would rather spend that cash on songs that I can play through RB.

    Also, your suggestion is too complicated and Microsoft would have to revise their system for it to work.... better suggestion... if you want in on peer reviews, pay the $50 for four months. If your reviews are good and helpful, HMX could comp you the cost for the next full year. When the renew comes up, HMX could review your record and pay the fee if they think your worth it to keep songs flowing through the system. It would benefit them greatly and "paying" a reviewer $100 per year would not break the bank.

    Heck, they could even 1099 you for taxes if they felt it necessary or they could take it off their own taxes as marketing...
  • kingtonyxkingtonyx Unofficial
    edited April 2010
    For instance I used to just post "Thumbs up/down" in the threads for peer reviewing with maybe a small comment or two. However I thought about it and decided to give a little better feedback which brought about my report card style.

    Now even if a song gets A+ across the board they get a comment about each element. If they get an "A" instead I'll tell them why.

    I think that's a bit better. There are people who go really in-depth but honestly I don't know about some of the stuff they are looking for so I don't bother.
  • G_tarRoCK3RG_tarRoCK3R Road Warrior
    edited April 2010
    Sayburr;3620197 said:
    I wouldn't mind doing peer reviews, but I am not going to pay $100 a year or $50 per third of a year to do it. I would rather spend that cash on songs that I can play through RB.

    Also, your suggestion is too complicated and Microsoft would have to revise their system for it to work.... better suggestion... if you want in on peer reviews, pay the $50 for four months. If your reviews are good and helpful, HMX could comp you the cost for the next full year. When the renew comes up, HMX could review your record and pay the fee if they think your worth it to keep songs flowing through the system. It would benefit them greatly and "paying" a reviewer $100 per year would not break the bank.

    Heck, they could even 1099 you for taxes if they felt it necessary or they could take it off their own taxes as marketing...
    My sediments as well, which is why I will be debating on renewing next year. But HMX should be made aware of the general public's thoughts on the matter. Its awesome they are looking into it.

    well if it was free I would think HMX would be the ones paying (or M$ being really nice!) OR they could have the authors pay like a little "fee" and the collected fees could go to the reviewers.
  • Die-Guitar-HeroDie-Guitar-Hero Rising Star
    edited April 2010
    I am very interested in helping out. All of Nuclear Blast Records songs have sat in Review status for two weeks unchanged and it is driving me crazy. I just don't have $100 extra that I wouldn't rather spend on more DLC! I would probably be more inclined to part with the money if I had any prospects for authoring, but as it stands, I just have to watch all of my most anticipated songs (Fleet Foxes, Band of Horses, Testament) sit at maybe 2 or 3 reviews for weeks! I wish there was an easier way of helping out.
  • trg007trg007 Your Ever Rocking RBN Forum Guru
    edited April 2010
    overdriveguitarman;3620120 said:
    I don't like that people are saying they don't want just anybody to be able to do it because they won't do a good job, and then say they need more people. That isn't going to work out both ways!
    Fewer, more dedicated people > anyone and everyone, when it comes to Peer Review. Even with the current community, we have seen complaints that songs from big-name artists get through review too quickly without being looked at thoroughly, simply because the songs are popular (see: Semi-Charmed Life '09). Imagine that effect many times worse, if Peer Review were free.
    I have a lot of free time (obviously) and I would surely do a great job! But I am not going to pay so much to do this!
    You'd still need to be 18 or older.
    G_tarRoCK3R;3620149 said:
    Honestly Tony, why are we paying to help them? They need our help.....they benefit. What do we get?
    It is true that you don't really "get" anything for being a playtester, aside from the satisfaction of knowing that you are helping out and quite possibly preventing songs with bad-quality authoring from getting through to the store. If you don't feel that that is worth the cost, you're free to not purchase XNA and wait/hope for a cheaper option down the road.
    Honestly, more authors should be peer reviewing/playtesting. Its not fair to the others that are doing this.
    This is something I can agree with 1000%! It's actually called "peer" review for a reason - peers (authors) are supposed to be looking at each other's work. The fact that we even have people who have paid to only playtest, is amazing and is very appreciated by those of us who are authors. More author participation in playtesting is needed to keep songs flowing through.
  • Die-Guitar-HeroDie-Guitar-Hero Rising Star
    edited April 2010
    trg007;3620249 said:
    Fewer, more dedicated people > anyone and everyone, when it comes to Peer Review. Even with the current community, we have seen complaints that songs from big-name artists get through review too quickly without being looked at thoroughly, simply because the songs are popular (see: Semi-Charmed Life '09). Imagine that effect many times worse, if Peer Review were free.
    Maybe free isn't the answer, but I know at least I am an exception to that rule. I feel that I would provide playtesting on part with those who can afford to pay $100, and I think the "bigger" the song, the more scrutinizing I would be, not less.
  • xRage08xRage08 Unsigned
    edited April 2010
    I've said it before and I'll say it again. I think HMX should give a set amount of Microsoft Points in return for peer reviewing. Not really giving back the $100 it costs, but I mean I'd prob pay the $100 dollars if I got like $35 of MP back.

    And honestly, in response to trg's stab at Semi-Charmed Life '09, I didn't think it was all that bad a song. The chart was pretty good, and the audio isn't as bad as everyone claims either. I think it was good enough to release. Could it have been improved? Of course, but I still think it's perfectly good. If you don't like the song, don't buy it. If you claim "I have to buy it to see if I like it", then download the demo.
  • kiggidykevkiggidykev Thinks about pandas
    edited April 2010
    Offering an incentive to playtesters/peer reviewers would be a nice option to encourage more people to partake and help justify the XNA fee, but I don't think it's going to happen for some time (if at all).

    I've been doing the RBN thing since the start of February, and it felt really cool and unique to be part of the process. That feeling unfortunately didn't last long because of the amount of work involved in providing a worthwhile playtest or review with detailed notes (and even then, there's no guarantee that your advice will be followed).

    I don't mean to be a downer, but I have to agree with what Sayburr said earlier on: if you're not planning to author songs, then it doesn't seem like a good investment for 100 bucks to pay for the ability to give up your time and work for someone else. It takes the very dedicated with a deep passion to want to be part of that, and isn't that the ideal when it comes down to having a final version of a song?

    On the other hand, I would really prefer to see the reviewing system the way it is for the reasons mentioned before (avoiding no-effort reviews and tests, rushed songs to store). A weighted voting system would require the current system to be rebuilt from zero, and it would be terribly inefficient to do so.
  • Die-Guitar-HeroDie-Guitar-Hero Rising Star
    edited April 2010
    Harmonix should hire volunteer playtesters and waive the xna fee for them! There could be an audition/application process or they could base it off forum reputation similar to the moderator system. Hell, I'd even write an essay!
  • G_tarRoCK3RG_tarRoCK3R Road Warrior
    edited April 2010
    Die-Guitar-Hero;3620269 said:
    Harmonix should hire volunteer playtesters and waive the xna fee for them! There could be an audition/application process or they could base it off forum reputation similar to the moderator system. Hell, I'd even write an essay!
    I think this would be a doable system. I would sign up for that! :D Then HMX can monitor those "paid for" XNA subscriptions to see if they are giving the quality produced reviews needed. If not, give it to somoene else. Perhaps they could even give out the 4 month ones, that way can kick off someone quicker whos not producing good reviews.
  • aperfectorestesaperfectorestes Opening Act
    edited April 2010
    Die-Guitar-Hero;3620269 said:
    Harmonix should hire volunteer playtesters and waive the xna fee for them! There could be an audition/application process or they could base it off forum reputation similar to the moderator system. Hell, I'd even write an essay!
    I would do this as well. I think this would be a sufficient gate to keep out the people that need kept out.
  • overdriveguitarmanoverdriveguitarman The Walrus
    edited April 2010
    Crap you have to be 18 or older?

    Aww man...
  • ThatAuthoringGroupThatAuthoringGroup Numero Uno Super **** Fanboy #1
    edited April 2010
    I agree that this is an issue that needs to be looked at by HMX.

    But something would need to be in place to prevent shenanigans.

    Last month we had someone come in and just start giving positive reviews on all the peer review songs(in such rapid fashion that you could tell someone was just going through there and clicking positive without actually looking at the songs).

    So yes someone paid money out of pocket to come in and try and raise hell with the PR system. If it is deregulated to where it's free and anyone could come in and test, all hell would break loose.

    I like the idea of maybe HMX 'hiring' testers. If that means paying for the trial, offering a discount on DLC, or free downloads or something of that nature I don't know, but making it free to everyone would be a disaster.
  • ThatAuthoringGroupThatAuthoringGroup Numero Uno Super **** Fanboy #1
    edited April 2010
    overdriveguitarman;3620370 said:
    Crap you have to be 18 or older?

    Aww man...


    Yes.

    18 or older, I THINK you have to have a US xbox account, a 360 linked to a PC, pretty speedy internet connection.

    Those are the requirements I can think of off the top of my head.
  • SayburrSayburr The Always Informative Rock Band Forum Guru
    edited April 2010
    noble;3620374 said:
    Yes.

    18 or older, I THINK you have to have a US xbox account, a 360 linked to a PC, pretty speedy internet connection.

    Those are the requirements I can think of off the top of my head.
    ...and you have to be one of the most awesome people on the planet! But, humble enough to accept undying gratitude from those of us who are lacking on the awesomeness detector.
  • G_tarRoCK3RG_tarRoCK3R Road Warrior
    edited April 2010
    trg007;3620249 said:
    Fewer, more dedicated people > anyone and everyone, when it comes to Peer Review. Even with the current community, we have seen complaints that songs from big-name artists get through review too quickly without being looked at thoroughly, simply because the songs are popular (see: Semi-Charmed Life '09). Imagine that effect many times worse, if Peer Review were free.
    Yea perhaps free peer review would be bad. But I'm one of the good ones so I could help out. Like someone said, they can come in and positive review everything and not have looked at the song. But thats where my weighted system would come in. HMX would see this person just positive everything and no comments in threads or feedback and just ban them from their free account.
    trg007;3620249 said:
    It is true that you don't really "get" anything for being a playtester, aside from the satisfaction of knowing that you are helping out and quite possibly preventing songs with bad-quality authoring from getting through to the store. If you don't feel that that is worth the cost, you're free to not purchase XNA and wait/hope for a cheaper option down the road.
    I guess you missed where I said I was DEBATING keeping my premium XNA account, unless you are talking in generalities than sorry.

    I do get satisfaction from helping out the authors that are listening to my suggestions and advice, if I get ignored or blown off (or they are against videos, aka FREE advertising) then say goodbye to me reviewing your songs again.....I have a list of peeps I won't be reviewing or helping out. It is good when the authors acknowledge the hard work us play testers who PAY to help you guys out. At least say, I disagree thanks for your input, or I'll take a look at it, etc. (Not saying you do that, just saying generalities sister Margret (haha kidding, Anchorman FTW!))
    trg007;3620249 said:
    This is something I can agree with 1000%! It's actually called "peer" review for a reason - peers (authors) are supposed to be looking at each other's work. The fact that we even have people who have paid to only playtest, is amazing and is very appreciated by those of us who are authors. More author participation in playtesting is needed to keep songs flowing through.
    You are welcome :)
  • vedisvedis Road Warrior
    edited April 2010
    G_tarRoCK3R;3620149 said:

    Honestly, more authors should be peer reviewing/playtesting. Its not fair to the others that are doing this.
    there is always the option that reviewers STOP reviewing songs of authors that dont help out in the process(and focus completely on those that do).....

    like thats gonna happen
Sign In or Register to comment.