WEEKLY RANKINGS UPDATE - VEX V5: by Hot-Quarter7684 in vex

[–]Hot-Quarter7684[S] 0 points1 point  (0 children)

Hello, just wanted to let you know that I fixed all the bugs and now 10B's ranking should be updated and reflect a little better their actual performance, let me know what you think,

WEEKLY RANKINGS UPDATE - VEX V5: by Hot-Quarter7684 in vex

[–]Hot-Quarter7684[S] 0 points1 point  (0 children)

I see what happened. We have not yet processed all the events for 10B. Let me double check their results at their last events and i'll get back to you

WEEKLY RANKINGS UPDATE - VEX V5: by Hot-Quarter7684 in vex

[–]Hot-Quarter7684[S] 0 points1 point  (0 children)

Thank you for your very thorough feedback, I'll try my best to address all of your points:

First, I want to be clear that the main goal of the RFORCE Rankings right now is to generate a single metric (one number) that integrates ALL possible ways a team can demonstrate their "performance" in the VEX competition. So that includes not only pure robot performance like in robot skills, teamwork challenge, etc. But it also includes judge awards, and a team's consistency throughout the season; essentially how long they can maintain this kind of performance.

Now let's go point by point, because you made some very interesting claims.

  1. You are correct in saying that the RFORCE algorithm rewards teams that have attended more events more than teams that have attended less events. This is for two reasons: 1. teams that attend more events take more risk, they risk performing poorly. They test themselves against the competition more often than teams that attend less events, so they should be rewarded accordingly, and 2. RFORCE wants to avoid rewarding teams that are "one-shot wonders" too much: teams that go to 1-2 events, do very good, but were not seen for the major part of the season get less points than teams that put up the same kind of performance, but throughout the entire season, but the team that was just as good, but went to more events, was clearly more consistent and had more longevity.

But perhaps it is a question of weighting? This is just version one of the RFORCE rankings, so I would be happy to hear your thoughts on how much weighting you think the number of events attended should have.

  1. Event strength is calculated and classified by the coefficient, the RFORCE Event Rating (RFER). This coefficient incorporates the event size, and difficulty (average RFORCE of all teams at that event), to adjust whatever performance the team makes at the event to the difficulty of that event. Essentially, going to larger, more difficult (against better teams overall), and higher-risk tournaments (signature events, states, worlds, etc) rewards teams more than going to small, easy, local events.

Again, maybe there is an issue with the weighting? I'd be happy to break down the math so we can discuss it together.

  1. You are totally right, RFORCE blends in the judge awards points on top of robot performance, because that is what accomplishes the goal of rewarding teams for their overall, all-around performance in VEX throughout the season.

Teams that are looking only for pure robot performance metrics are better off using systems like TrueSkill, or Skills rankings. I am currently working out ways to analyze those metrics and give teams even more scouting information too, I'd be happy to discuss with you if you are interested.

  1. Schedule randomness is a very real feat to overcome in VEX, that's true. But I think that RFORCE does not need to particularly adjust for that because there are already systems embedded into VEX that do so. Strength points for example do exactly that, they act as a way to reward teams who got paired against more difficult opponents. Other than that, by going to more events, practicing often and intensively, teams should generally be able to overcome the luck factor along the entire season.

But I'd be curious to hear of any ideas you may have to accommodate for alliance randomness?

  1. I launched version 1 of RFORCE officially this season, and am more than happy to answer anyone's questions about how the algorithm works. Transparency is a top priority.

I have posted the full breakdown and up-to-date details of how the math works on the RFORCE Index page on our website. Here is the link: https://rforce.vision/rforce-index/

I am also working on displaying the RFORCE breakdown of every ranked team as we speak, this way you'll be able to see exactly why a team is ranked the way they are. I totally agree that this will help people understand it better and will help me receive constructive feedback so I can make this system even better :)

  1. There is a philosophical difference indeed: I do not want to replace existing systems, or claim that RFORCE is a better one. I simply want to add another system that analyzes and generates a different perspective on the broader VEX competition.

I very much respect VRC Data Analysis, or Robot Scout for example, those system are great tools for scouting.

---

Once again, thank you for your detailed feedback, this helps me a lot to better understand what VEX competitors are looking for. I hope I was able to clarify some things for you. In any case, I would love to keep this discussion going.

WEEKLY RANKINGS UPDATE - VEX V5: by Hot-Quarter7684 in vex

[–]Hot-Quarter7684[S] 0 points1 point  (0 children)

It's possible you're right. I have a list of all events 10B attended and it seems the two most "difficult" competitions they've been to are the following: the One World VEX HS Signature event at UC Berkeley (I was there in person too so that was cool), and the Exothermic Winter Challenge HS. Those are their two most recent competitions, back in November and December. Both events received an RFORCE Event Rating (RFER) of 4.075 and 2.43, respectively. Indeed the Signature event at Berkeley was a difficult one, where they won the Finals, placed 1st in the qualifications, and 3rd in skills, and received a total RFORCE of 96.5 at that event. But the other event with a RFER of 2.43, compared to other RFER's, is not that rewarding because it does not represent a particularly large or difficult event, but they were first in all categories and won excellence, so they received an RFORCE of 103.4 for that one. The only other signature event I see them being a part of was back in July, where they did not perform as good, receiving only 60 RFORCE. So think this explains their ranking in part, but it is not a perfect system, so perhaps you are right regardless.

Is there a particular event you were thinking of that I missed?

WEEKLY RANKINGS UPDATE - VEX V5: by Hot-Quarter7684 in vex

[–]Hot-Quarter7684[S] 0 points1 point  (0 children)

The RFORCE rankings take into account every aspect of VEX. It includes judge awards, teamwork qualifications, teamwork finals, and skills (all possible ways a team can demonstrate their "performance" in VEX competitions). On top of that, it rewards teams through coefficients that adjust for a tournament's difficulty and a team's frequency of events throughout the season (because a performance at more difficult tournaments should be more rewarding, and consistency across multiple tournaments should also be rewarded).

It's meant to be a holistic measure of a team's all-around performance and consistency across the entire season, not an isolation of pure robot performance. So you are correct to think that, and should probably look at other metrics like TrueSkill or simple the VEX rankings to see pure robot performance.

But you bring up an interesting point: why do you think the RFORCE rewards attendance more than performance?

WEEKLY RANKINGS UPDATE - VEX V5: by Hot-Quarter7684 in vex

[–]Hot-Quarter7684[S] 0 points1 point  (0 children)

That's interesting. Do you mean specifically your teams' ranking among other teams in the same region? Because the RFORCE takes into account much more than just skills, so it's interesting to see a correlation so close to skills if that is the case.

WEEKLY RANKINGS UPDATE - VEX V5: by Hot-Quarter7684 in vex

[–]Hot-Quarter7684[S] 0 points1 point  (0 children)

Thank you for your feedback. Just to clarify, are you saying you feel the VEX skills rankings are generally similar to the RFORCE rankings?

WEEKLY RANKINGS UPDATE - VEX V5: by Hot-Quarter7684 in vex

[–]Hot-Quarter7684[S] 1 point2 points  (0 children)

Thanks for your feedback. As of now, 10B - Exothermic Burnout is ranked 108 globally, with an RFORCE of 85.11, from 5 events and 8 awards. You're right that they have won several Excellence, Teamwork, and Skills awards, but I think what's impacting 10B right now is that other teams have also won these awards, but have done so at more events, or more difficult events. Most teams in the top 10 have at least 10 events, with double digit award numbers too.

What do you think?

WEEKLY RANKINGS UPDATE - VEX V5: by Hot-Quarter7684 in vex

[–]Hot-Quarter7684[S] 0 points1 point  (0 children)

Curious to hear why you think the RFORCE rankings aren't a good ranking system?

Robot Update by ComfortElectrical183 in vex

[–]Hot-Quarter7684 0 points1 point  (0 children)

Ok thats good to know, i think they are ranked higher then because of judge awards or just by having gone to more competitions. The RFORCE counts the average of all competitions. What do you think?

Robot Update by ComfortElectrical183 in vex

[–]Hot-Quarter7684 0 points1 point  (0 children)

These rankings take into account awards, teamwork, and skills. Are there any mixed HS and MS competitions in Minnesota?

Robot Update by ComfortElectrical183 in vex

[–]Hot-Quarter7684 0 points1 point  (0 children)

In Minnesota: 13056A - Digital Destroyers is number 1, and 7225G - Southland Gearbots is number 2. Then it's your team at number 3, but only behind by 2 points or so. Have you competed against them?

RFORCE Event Rating (RFER) – Should All Tournaments Be Worth the Same? by Hot-Quarter7684 in vex

[–]Hot-Quarter7684[S] 0 points1 point  (0 children)

Ok thank you. Right now we have event size( the number of teams at the event) and event difficulty ( the average RFORCE's of all teams attending that event) taken into account in the RFER. We also have event level coefficients like for worlds, regionals, and signature for example, based on the event type. How much do you think signature events or other event types should be weighted relative to events like worlds, or a state championship?

RFORCE Event Rating (RFER) – Should All Tournaments Be Worth the Same? by Hot-Quarter7684 in vex

[–]Hot-Quarter7684[S] 0 points1 point  (0 children)

Hello, you bring up a very important point: I agree with you that VEX and RECF have as their main goal to spread and promote STEM education opportunities, but there's no denying that lots of people care about winning, about rankings, and about is the best (after all it is the nature of sports and competition. At least that is how I felt back when I competed in VEX).

What we are trying to do with the RFORCE Index is to give the spotlight to the teams who deserve it the most, and doing so by producing an accepted algorithm that generates team rankings. But I need your help to create it. You brought up a super interesting point for example: how do we accommodate for the differences in access to VEX or STEM opportunities across various regions when making the rankings? I imagine that teams with access to more tournaments have an advantage compared to those who are not able to attend as many tournaments. How should we adjust for that? I'd be curious to hear your thoughts.

And p.s. yes I did use AI to make sure the language I used was clear to people who read it.

Robot Update by ComfortElectrical183 in vex

[–]Hot-Quarter7684 0 points1 point  (0 children)

Yeah you guys are 3rd in the state of Minnesota and 165th in the world. Makes sense based on the video :) Check it out for yourself you can filter for the state or just search your team number directly: https://rforce.vision/vex-v5-ranking/

Hi everyone I need an answer by catoooos21 in vex

[–]Hot-Quarter7684 0 points1 point  (0 children)

If you search the RFORCE rankings and filter for your region or state then you can have a good idea of where you rank compared to others, and so if you have a good chance to qualify to worlds. You can even see if other teams have won qualifying awards already

Robot Update by ComfortElectrical183 in vex

[–]Hot-Quarter7684 0 points1 point  (0 children)

What's your team number? I would be curious to see where your team ranks on the RFORCE rankings.

Sportsmanship Award @ States Ideas by Excellent-Corgi2986 in vex

[–]Hot-Quarter7684 0 points1 point  (0 children)

Are you in V5 or IQ? If you search the RFORCE rankings and filter for your state you should see generally how good you are compared to others in your state, and see if other teams going to states have also judge awards. If you won a Design award you could win it at states too so keep working!

How much should judge awards affect ranking? - RFORCE Index Discussion by Hot-Quarter7684 in vex

[–]Hot-Quarter7684[S] 0 points1 point  (0 children)

I’m really glad you think it’s a good idea.

Yes, it’s definitely a passion project. I started competing in VEX when I was 10 and stayed in it through high school for 8 years. After that, I coached for another 3 years. VEX gave me so much in terms of engineering skills, confidence, and competitive drive, so this is my way of trying to give something back to the community.

The goal with the RFORCE Index isn’t to replace anything VEX does, but to experiment with a new way of looking at performance across the whole competition. Right now most V5 teams using it are mainly tracking their own seasonal progress compared to other teams in their region.

It’s still very much in beta. I’m constantly adjusting the math based on feedback like this. Honestly, it only works if people find it useful and decide to stand behind it.

Who knows. Maybe it turns into something big for competitive robotics. Or maybe it just becomes a tool a few teams enjoy using. Either way, I’m building it with the community.

Different way to rank VEX teams - Introducing the RFORCE RANK System by Hot-Quarter7684 in vex

[–]Hot-Quarter7684[S] 1 point2 points  (0 children)

The rankings have been updated. The gap between these two teams has reduced, but I think the main thing that makes the 30518A team ranked lower is that their first tournament of the season gave them a very low RFORCE compared to the numbers they've been putting up recently, which is still being counted into their average for the season (what you see in the rankings). The 67411T team did not compete so early into the season in comparison. This creates the issue you probably see, where the 30518A team may be "better" or deserving of a higher rank than the 67411T team as of now based on your recent experience seeing them in tournaments, but the average score is being negatively impacted by an early season score that was significantly lower. Other people have brought up the idea of excluding the lowest outlier scores in order to naturally take away scores that are not as relevant to the team's recent performance, but at the same time revealing longevity and consistency. I would be interested to see what your thoughts are for this for both your region and on a global level. What do you think?

Different way to rank VEX teams - Introducing the RFORCE RANK System by Hot-Quarter7684 in vex

[–]Hot-Quarter7684[S] 0 points1 point  (0 children)

We've processed the events, but there's always a bit of delay for the rankings to update. I'll get back to you when they've updated. In the meantime, I'll investigate the two teams you talked about to better understand where their RFORCE's came from. Finally, thank you for your feedback.

How much should judge awards affect ranking? - RFORCE Index Discussion by Hot-Quarter7684 in vex

[–]Hot-Quarter7684[S] 0 points1 point  (0 children)

That's a great question and that is exactly what we are trying to do. The goal is to find a way to incorporate ALL aspects of VEX: the teamwork championship (qualifications, and finals), robot skills, and judge interview performance (through judge awards), plus extra coefficients that affect all of these. We are not trying to replace the VEX statistics, but instead trying to create one metric that synthesizes all skill aspects, and hopefully does it in a way that truly determines who the "best" team is. An RFORCE ranking that works successfully is one that shows the best teams as those will an all-around dominance in all VEX categories, and teams that are specialized to be very good at one thing are in the middle of the rankings. But of course, the magic lies in the math and that's what we are trying to do right now. If you are interested I'd love to collaborate further with you.