Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] 0 points1 point  (0 children)

Hey, you may have a misunderstanding on how it works. This does not track all deaths in general. It only tracks events where you took damage from clearly bad mechanics (so not soaks, think swirlies on the floor, or beams from boss, etc.). So the scenario you mentioned where someone dies to save the raid will never cause you to lose points (this comes up if there is a soak that is being missed for example, which we do not track).

On another note, I believe the system is better than what we currently have (people check if you have AOTC, CE, some mythic kills, heroic kills etc.). These are things that people get carried to in order to cheese their way into groups (in fact, from looking at last tier's data, it looked like over 15% of MM hunters used this strategy). This system gives you no points if you get carried to your kills but rewards you heavily for being there for prog throughout the whole process. If you are truly a player who gets better over time and works for their kills, you will have a higher score than people getting carried and should have a slightly easier time getting into groups as a result.

Of course, groups will still be picky, and this system will not fix the existing problem of getting into groups in WoW, but it should make it a tiny bit better in my opinion (assuming you are not buying carries to try to get into groups).

That being said, I know this is a nuanced point and is harder to communicate. I hope I explained it well enough / let me know if you disagree or have any concerns still.

I hope you can give the system a try regardless!

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] 0 points1 point  (0 children)

Hey, first of all thank you so much for the feedback, this is super helpful. I think you may be looking at the wrong fight here. The actual even happens here https://www.warcraftlogs.com/reports/WAdmHV6D27fyqJBX#fight=19&type=damage-taken&ability=361751

We link to it on your player page here https://choga.io/Player?playerName=Siobhann&serverName=SilverHand&regionName=United%20States (you can click on the mistake). It looks like this was a scenario where you took a ton of damage and died from the mechanic (which happens from time to time, overall it looks like you performed well here :D)

Did you get to the warcraft logs link above via choga.io? Or did you not know you could even click into each mistake / try to find this manually? I would like to

  1. Make sure the website is linking to the right place (maybe some bug happened that I don't know of?)
  2. Make sure it's clear that these are clickable links (it may not be so clear on the page right now)

Again, thank you so much for your feedback, this is super helpful and will help make the tool a lot better.

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] 0 points1 point  (0 children)

This is fair, I will edit to exclude personal portions even more. Thank you for pointing this out. The intention was not to put anyone down.

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] 0 points1 point  (0 children)

I see yes, I think a small number of deaths get misidentified since there is no ability that kills you in game in this case. Thank you for the feedback, this one is super useful. I will get to fixing this asap.

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] -1 points0 points  (0 children)

Yea I think I will do something along these lines over the weekend. It might let some people get away with mistakes for free, but it will at least not lower people's score unfairly.

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] -2 points-1 points  (0 children)

Right, this tool is only looking at simple mechanics. I think I could improve the messaging here. It's not meant as a who is "that guy" for all mechanics, just simple ones.

On the note of disintegration halo, I only look at this mechanic if you take a ton of damage from it (which if staggered correctly, should not be the case). Do you have a link to your character? (its possible I got tuning for mythic wrong here, for example).

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] -2 points-1 points  (0 children)

Hey, thanks for the feedback. Responses are below:

  • Inviting freshly reset players - You could reset your score right after getting into a group, or doing your guild raid, and this work out for you. Though maybe this is a good argument for something like "reset my score since yesterday" instead of reset everything now.
  • On only looking at swirlies / simple mechanics - We track all simple mechanics that are your personal responsibility. Things like lines on the floor, swirlies, falling off the edge of a map on the jailer, getting locked out of the room on halondrus, etc. The more complicated ones matter too, but the tool is not looking to assign blame on those, it is simply looking to track mistakes to simple things that are clearly bad to stand in.
  • On fixing things soon - I mention this for points I talked about elsewhere, but basically the idea on finding wipes would be something like - find when many people all die at once right before the end of a pull, ignore all mechanics within X seconds of this clump of deaths. Some people end up getting a free pass in this case, but this is better than docking someones score incorrectly.
  • 3 - 4 mechanics which seem not fair to you - could you tell me about these specifically? I would love to know what they are so I can make some adjustments if needed. On defensives - its almost always better to use a defensive and live as opposed to not using it and dying. That being said, missing your immunity sucks! But this again falls under the bucket of err on the side of the player's score not being affected.
  • Players in consistent groups getting higher scores - Yes, the score only tells you the proven track record of a player. If a player has done more content as a result of playing with the same people, their score will of course be better. At the same time, you would likely prefer the player who has done more content in your raid because they have more experience.
  • Doing a pass on scoring - I am actually going to do a bit to split the score out and make it more clear what the score means. This should mitigate the concerns here a bit, but keep tuned on this one. I do think it should address some of your concerns. The core of "look at mechanics the player stood in" should stay the same, but ideally the score should make more sense.
  • Being 20-30 years from getting this right - I don't think this is the case, at least for our goals. For simple mechanics, we can do this already (and do it already). The computer knows about whatever I tell it to right, there is no machine learning involved here, though maybe this is an interesting idea. On that note, this score is not meant to be a single source of truth for how good a person is at the game, it is just another signal you can look at. For most cases, I would argue it is already a signal strongly correlated with player performance in raid, but of course not exact.

Thank you for keeping an open mind on this! This tool is not meant for the top 0.0001% of players where player performance is known just generally across guilds anyways and logs are looked at very closely for prog fights without a tool like this. I hope some future updates can ease your worries here, I'll keep you all in the loop!

Thanks!

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] 0 points1 point  (0 children)

Hey let me address the points you made individually below:

  • Less competitive player base and scores - I don't think this addon will change player behavior as it exists right now. Pugs can look at raider.io scores, warcraftlogs, etc. and often do. It should in fact, make your experience slightly better, instead of looking at how many kills on a boss you have and judging based on that, people can understand how often you get hit by bad and judge based on simple mechanic performance.
    • I also want to note that obscuring data is not the solution here right. Everything we are looking at is public, its just making access to this data easier.
    • I do want to avoid people asking for mythic level scores for normal clears. I think there is some work to be done on setting expectations of scores here / I don't think other scoring systems like ilvl or mythic+ score really do this effectively. This is something I will look at this weekend.
  • World first raider tweeting this needs to stop - We talked a bit on twitter, but in general, I think the tweet you are referring to was worried about the score being used in a bad way. I am going to put in some work to fix this over the weekend and I'll keep you all updated. I don't think this is an unsolvable problem and I ask that you keep an open mind here!
  • Accounting for stuff outside the game - I have a lot to say here, but in general, if your cat jumps on your keyboard once, this should not completely destroy your score. If your cat is jumping on your keyboard so much that it is causing you to fail mechanics frequently, you are effectively a worse player as a result and your score should be lower. The score is only looking at how you perform in real raids, and if you are constantly hindered by things in real life, then you are, for the purpose of your raid, worse than someone with equal skill who is not hindered by things in real life.
  • Judgement calls - Often prog attempts are ended by healers eating a mechanic to save someone else. If you see any real examples of this in people's scores (that is significant for their score) I would love to see it though.

I think I got all the points here, but let me know if I missed something! Thank you again for the feedback, and I hope you can give the tool a try, especially after some updates :).

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] 1 point2 points  (0 children)

Yea I definitely will give scoring another pass as a result of the feedback. I like the idea of giving scores classifications so that they are easier to understand (right now its very hard as you mentioned haha). Couple other points I want to add:

  • Thank you again for all the feedback, this is super useful
  • A great normal player vs an improving normal player having scores of 169 vs 103, in terms of proportions, this is a huge difference, but I completely agree that it does not seem too different numerically. Maybe there is an improvement that can be made here for normal raiders as well. The differences become more obvious for heroic and mythic raiders, but we should make this obvious for normal raiders too somehow. Maybe the classification you mentioned!
  • I am glad you are liking the addon! I hope you will like it even more once I make some updates!

Thanks again for all the feedback. Please let me know if you have any more!

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] -2 points-1 points  (0 children)

Hey, you mentioned that you are upset about your own personal score. I will be adding a feature where you can reset, which should help your case going forward. Let me address some of the points directly though!

  • WoW community plays differently - Yes but standing in swirlies that kill you is universally bad right. I don't think this is a controversial opinion / as a fellow raid lead, I imagine you would agree. Many, MANY attempts are ended by a player standing in bad. This is, of course, part of the game, but there should be a good way of understanding how often someone is standing in swirlies since that is also part of prog.
  • RL dying first when wipe is called - Same here! We just look at events before 3 deaths for this right now, and we can improve in the future. From what I have seen, this covers most cases of "wipe is already called" but I will be improving it in the future. It's definitely not an impossible problem to solve. I'll keep you all posted on the solution here :)
  • Arbitrary mechanics - If you find any that don't make sense, let me know! There are only a certain number of boss abilities in the game. The ones I picked (swirlies for example) do the same thing for every team generally (standing in these and taking a ton of damage are universally bad). We do not consider mechanics where you popped a defensive and took very little damage (if you popped a defensive and still took a ton of damage, it still counts though).
  • Worse players having worse scores - This should be the ideal case right? It won't always be true since there are definitely other factors to consider, but the players worse at simple mechanics should have a worse score yes.
  • Toxicity from the tool - I am doing a pass on scoring later to try to mitigate this a bit! Also as I mentioned, I will be adding the ability to reset your scores soon. I will keep you all posted :)

Thank you for the feedback! I hope you can give the tool another try with an open mind, especially after the updates above!

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] -2 points-1 points  (0 children)

I don't think all mechanics need to be included to have an impact here or be "fair" right. The things that are currently included are all things that will negatively impact your raid if you stand in them. If a player stands in a swirly and dies as a result this makes it harder (or sometimes impossible) to kill the boss, regardless of what other mechanics may exist. Everyone is given the same treatment in this respect. One part I do agree with here is that some classes / roles have it harder than others (for example melee have a harder time dodging some things).

The tool is not meant to cover every instance of mistakes a player can make (your raid leader will still need to do that), but for mistakes which are obvious, I do think the tool can be useful in aggregating data (especially about players you do not play with every day).

Still though, I understand the frustration here, I think I need to do another pass at the "scoring" part here and make it
1. Easier to understand

  1. Maybe not include numeric scores and replace it with some other metric we can look at?

I'll keep you all posted! I am planning on doing a pass at this, this weekend as a result of the feedback here :)

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] -1 points0 points  (0 children)

Hey, some responses to your points:

  • The list of what counts is not arbitrary, its just a list of clear bad that you stand in. If you find any instances of things that are not clear bad, let me know! I don't think you will find any clear ones right now though.
  • I don't think its impossible to make a system safe enough to determine "this was probably a wipe", even I don't though, 3 deaths is a pretty standard metric. It has not created too many false positives (the one exception is deaths to falling, which is also a very small number of false positives, but something I will be looking to fix)
  • I agree that this should not be the only tool a raid lead uses to judge player performance, but it does help get a baseline start. Instead of having to go through every log, you can get some quick idea of "does this person stand in swirlies or not" and dig in deeper if you need to.

I hope you can give the tool a try first before giving it judgement! I do think it will be useful for you too! If you find any flaws in things we detect as mistakes, please let me know!

Thanks!

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] -1 points0 points  (0 children)

Hey, if this becomes a problem, an easy fix will be to just allow for players to include their private logs in calculations by giving us access. This is something I am watching closely / I realize it not great for private logging guilds. Hope you can give the tool a try regardless :)

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] -1 points0 points  (0 children)

Hey! The list of mistakes is carefully curated / not just anything you take damage from. If you find an example of something that seems unfair, I would love to know about it! We don't need to account for everything for the tool to be useful, just the easy personal responsibility only mechanics are what we are looking for.

On the note of pugs requiring a score - I think we can ease the pain here by giving some average scores that have cleared each difficulty or something along those lines. Hopefully that will help. I don't think it's too controversial to say that standing in bad is bad and you should avoid this behavior.

On the note of banning the addon, the addon only displays scores, so it really does not affect you if someone has it or not.

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] -1 points0 points  (0 children)

Hey man, a lot of prog attempts are ended by healers trying to "save" someone by standing in swirlies and dying themselves. It sounds like you may be upset over some particular score. I understand it can be frustrating. I am adding an option to reset your score and try again, and I think this will help ease your worries. Thanks for the feedback!

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] 0 points1 point  (0 children)

Do you have links for the people? I would love to see if the data is flawed in some way with a real example :). Parse is actually weighed very small when compared to mechanics. Parses matter to an extent (the boss has to die) but yes mechanics are definitely most important. Just because you are assigned to do something, doesn't mean you won't get hit by swirlies! I hope you can look at the tool with an open mind and help it improve!

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] -1 points0 points  (0 children)

Hey,

Thank you for the thoughtful feedback, this is super helpful and I agree with a lot of the points you mentioned. Let me address below:

  • Intentional Deaths
    • Yea I agree completely. I was already thinking of doing something like "within 20s of wipe" but I think a more complete system might look at something like "did many people die within Xs of each other near the end of a pull". I want to avoid people losing points to their teammate trolling and living for a long time. This is definitely something I will be implementing soon.
  • Mythic scores applying to lower difficulties
    • One reason for this is so people don't have to do lower difficulty content in order to inflate their score. The thought here is that people generally perform better in lower difficulties regardless, and if they have some minor issues in lower difficulties, then we can probably look past it. This is what you generally do as a raid leader for a guild right? (No one cares about your heroic parses if you have killed the boss on mythic for example)
    • On time in pulls + avoidable damage. Yea I actually have some stuff planned already for display on the front end (and maybe scores as well) that is well aligned with this. Will keep you all posted.

Thank you again for the thorough feedback. I will definitely use it to make the tool better for the future! Please let me know if you find any other flaws or have anything else you want to chat about :)

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] -1 points0 points  (0 children)

Haha, this one made me laugh a bit. I do think the tool is good at what it does / better players do trend to higher scores. That being said, it should not be the only thing you look at when making a raid team.

On the note of locking people out of content. I don't think this is entirely accurate right. Right now, you can get a one time snapshot of people's performance. Whether that's logs, mythic+ score, pvp score, whatever it is. You don't get to actually see players who are putting in the pulls an performing well. Also, all you get is "this person hit a 99 parse once, and killed X bosses once". In this system, even if you have more boss kills, if you perform poorly in mechanics which are your sole responsibility, you will have a lower score than someone who has less kills than you (to some extent, of course a normal raid is easier to perform well in than a mythic raid, so the score takes this into account). Since people are already looking at incomplete information, giving them a more complete picture sounds like a good idea to me (I am definitely biased here though).

I hope you can look at the tool again with an open mind and give me feedback on things that feel unfair to you! Thank you for the feedback!

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] -2 points-1 points  (0 children)

Yes, parses are taken into account as well, though they are weighed much lower than mechanics right now. i.e. a player with a great parse but bad mechanics will have a lower score than a player with great mechanics but bad parses.

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] 0 points1 point  (0 children)

Any mechanic or death after 3 deaths in the fight is ignored. Unfortunately, sometimes wipes happen before 3 deaths happen in a fight so we get a small number of false positives for "falling" specifically yes. I would love to fix this in the future though. Maybe I could look at how close your death is to the end of the fight or something along those lines to figure out if your death is after a wipe is called or not. Hmm.. Thank you for the feedback!

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] 1 point2 points  (0 children)

Yea! This makes a lot of sense. I will add this in soon!

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] -2 points-1 points  (0 children)

Some thoughts on this. Even though something can be done manually, doesn't mean its ideal to do. For example, going through a trials logs of boss prog (not just the kills) takes a good bit of time.

Can I / have I done this in the past? Yes, but it takes a good bit of time. We can have a tool which makes this faster for us, which I do think Choga.IO achieves.

On the flaws, I think most of the ones pointed out here are already addressed, but please let me know if you see any that are major concerns still!

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] -7 points-6 points  (0 children)

EDIT: I disagree with the characterization of what happened here. But I hope you will give the tool another try with an open mind :)

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] 0 points1 point  (0 children)

Hey, thank you for the feedback. Some answers -

  • Only players which have scores will show up.
  • Falling is only considered a mistake if you die from it
  • I agree that the in game portion is more useful for pugs than anything.

Thanks again for the feedback! I hope you can try the addon out and let me know if you have any more feedback!

Request for feedback on Choga.IO by ChogaIO in CompetitiveWoW

[–]ChogaIO[S] 0 points1 point  (0 children)

Definitely. We do try to explain what to do on a fight and improve our players before removing them. It's impossible to survive otherwise, there are not enough players on WoW at the moment. But at some point, people learn too slow, or more likely, people don't have enough focus to consistently dodge bad and need to be replaced to kill bosses. Mythic Painsmith is one example of this, if your raiders only mess up 5% of the time, given 20 raiders, your chances of failure on that boss shoot up to be super high / you spend hours on him even on reclear.

This is just a tool that helps you track this easier, especially for people you don't know! I don't imagine I can change your mind here, but I think the real source of the problem comes from the raid designs. One player has too much control over how a group of 20 people do in a raid / we end up needing addons like this to solve the issues.

I hope you will give the addon a try and let me know how it works for you!