SKATEBOARD JUDGING – A POINT OF VIEW
This is an old e-mail sent to World Skate July 18 – 2018 after a news article concerning a workshop done by the international skateboarding judging commission.
Hello,
I just read the article about the last judging commission workshop.
I saw the e-mail address at the end for possible questions and below are some comments.
Introduction
I have been very interested personally over the years and especially the last couple of years, with the Olympics in mind, around the skateboard judging issue and what could be done to make it better. I’m thinking primarily about the problems with the old judging system and that still today is mostly used with 5 judges giving overall impression 1-100 and where lowest and highest scores are removed. This is also the system that World Skate (judging commission) has selected for the Olympics for the Park event.
This being said the formats used in Street League and X-Games park has shown progress. And also happy to see that the Street League Judging Format will be used for Street in the Olympics.
The problem with skateboard judging
The problem I see is concerning the handling of judgement for runs. And the process of how you achieve that single overall impression point.
It’s true that if you judge well you may have invented your own system of how to make notes that will help you in the end to decide the overall point. Still these notes can’t be too elaborated since you as a judge need to keep your eyes on the skating at all times (preferably).
Here is what I think is a big problem today. There are no info published anywhere of how these notes should or could be done. I can’t find any judge who has dared to share their own note system. And still, this is the very thing we do need to make better judges and help new ones to dare starting to get involved in this precarious task. It’s a sensitive place to put yourself into which makes it hard finding people to judge at competitions in general.
Here we need a lot of work to help judges evolve and also for skaters to understand how that overall point has been achieved.
The problem is also that you never see these notes or even the personal overall impression done for the unique judges. Not even at the high level events. You only see the result point of all judges counted together.
The problem, in the past when people have tried to address this issue, is to creating different point categories for different things. This has always ended up in catastrophe. Our brain is good to feel an overall impression but bad to try and split it up in parts. And the parts are too complicated to understand even if you had the time to do it. And that time you don’t have as a judge at an event anyway.
“The aim is good, the list may be correct, but still… more or less useless.”
So to explain how judgement is done ends up with this list of things that a judge may take into account. As you have done in this article. The same good old list exist in small variations. The aim is good, the list may be correct but still… more or less useless. I can’t imagine any judge thinking of such a list when judging. If a new judge would try this they would fail. The list is more an after construction from what a gut-feeling should include.
And even if you are making great notes (that you don’t really have time to do) you can’t get away from that gut feeling. Depending on your notes it may be more or less difficult to translate your notes into a point.
And then you have the mathematics to translate that point into the 1-100 scale.
And make sure you are in phase with your judge colleagues.
As you see there are many things that complicates and also makes logical and mathematical errors come in play.
The good news is that the overall impression from a gut-feeling is actually working quite good.
The bad news is we really can’t use it straight off, like an overall impression coming out of a black box, if wanting to bring some more serious glow over the skateboard judgement. We need some sort of judgement logging to help judges being more structured and consistent.
For me the question has been how we can use the “gut feeling” in more controlled, structured, consistent way and also having it logged.
– How to explain how judgement is done?
It’s hard to explain a gut-feeling and you end up with a useless list that does not help much for a person to judge or understand anyway.
And you can’t say it’s a gut feeling because it does not help to understand.
And I don’t even know if it is possible for a judge to understand it themselves.
– Gut-feeling is not a fixed thing.
Gut feeling is devious and evolves all the time. The larger the field the harder to keep it intact.
– Logging
There is no good way of tracking or logging how an overall impression is made. Yes you may have the judge notes but they are often personal and impossible to decipher. Even for the judge himself a week after the event.
– Logging important
Still, logging is important for many reasons. They may be needed for the judge himself, for any check if there are protests, for any internal admin checking, to learn, to add to general knowledge and to use as base for learning and practice.
“Often there are too much chatter between judges”
– Independent judges
We want independent judges. Often there are too much chatter between judges. This is ok on low level events but a judge on a high level event must be able to judge himself without being affected by the other judges.
– Point balance between judges
How to make sure all the judges produce points on the same level?
This is often decided by that unwanted “judge chatter” on the first run.
But it is needed on more occasions than that to keep a good point balance during the heats. Same for where to put a very low point of the day.
Still it’s very easy to get off the judgement chart by always for example giving too low or too high points compared with your fellow judges. And then you points never count with the current system where low and high judge points are discarded.
But just because your point level is constantly higher or lower then the other judges does not mean good judgement is off.
– Point balance in total
Also it’s hard to know where to put the level of points before racers have skated. This is often achieved by being very reluctant to give high points early even if they may end up us the best runs of the day.
– Point sensitivity problem
Some judges may have small point differences between skaters and other use big differences.
– Judgement crunch
If you judge many skaters that should be judged on the same base, for example during qualification with many skaters, you may easily come into situations where you have looked yourself into a small point area where a lot of skaters end up and you have few points to separate them all as you would have liked.
– Taking away lowest and highest judge scores
I don’t like this at all if it is done only because a judge has constantly higher or lower points.
You in that case want to do it when a judgement differs from the own judge average judgement compared to other judges own average judgments.
There is a mathematical equation to sort that out.
Also now, especially in big events where you have invited very good judges on place, and then we don’t take their points into account for the wrong reasons. Not good.Only if there are signs of corrupt judging the system should interact. Corrupt judging may exist also even if you don’t have highest or lowest scores. Corrupt judging is not having constantly higher or lower scores. It’s point balance problem.
– Another way of ensuring safer judgement
Bring along extra judges at bigger events. You need fresh minds to judge well. Judging all day will lower your reactions and possibility to think clear. Same as if you were driving all day. Mix judges around during the day. Select the five best judges for the finals according to the judgement logging.
If you have a very big event you could even have enough judges to use extra judges as control judges in the background to comparisons and analyzes. The more judges the more information to research.
– Computerized aid
Some things a computer does far more quicker and safer than a human brain.
Other things the brain is far more better as for the pure judgement (gut-feeling).
Computerized aid could help a lot to make judgement process quicker and safer and more presentable for audience.
On bigger events computerized aid should be mandatory to let judges judge and let computers compute to get maximize speed and quality results.
Judges has enough to handle with the judgement and should not do calculations or translating notes into points.
The goal is to have minimal or better off no waiting times for judgement points.
“…there must be a clue to how this could be done…”
Judgement notes guidelines
Here is something that could help judges. The current free and personal judgement notes system could be structured up. Of all good skateboard judges out there, there must be a clue to how this could be done and be most easily used by all.
My own early judgement notes
Early on I started off with 50 points and then added or subtracted points in my head until the heat was over and I had the point.
It worked ok, I thought, but I could not know how consistent I really was. And I had no logging proving the steps I did in my head.
My own current judgement notes
The judgement notes system that I use today is writing down points for all tricks done during a run. In the end I sum up all the points to have a “raw point” as base to then translate into the 1-100 scale.
The good thing it is very consistent and I have a log.
Problem is it takes time to sum up the points and also difficult with that final translation from “raw point” to “public 1-100 point”. And all the other problems mentioned above.
My point of view for a better judgement system
Much of the problems I have noted above is what I saw with my own judging and I could not just accept there were no solution to all these problems. And why I have finally tried to find a way around all the problems.
“Which makes it an excellent system for training new judges”
I think I have found a way to avoid all the problems. I have tested it for myself on real live competitions and compared results and it seems to work. It is quite a simple system to learn. And since it is a logged system you can compare yourself with other judges and learn and understand differences. And be able to reflect if the difference is correct or not. Which makes it an excellent system for training new judges.
The system have a default setup but where it is possible for judges to set up their own setup. It does support judges having their own personal judging. And still make sure to avoid unbalance between all the different judges and their philosophies. The logging is done in numbers and can be easily compared between judges.
The only problem is…
– To make it work in a live environment there is need for computerized support.Calculations are made to make sure there is balance between the different judgments done and their different philosophies.
– There are no official 1-100 points during a competition level. These points may be calculated but only at the end once all racers have finished a level.
– Instead of 1-100 points you only see diffs during a competition level.
I.e. during competition you only see the ranking of skaters. And with a raw diff point showing the difference between them.
What is great is that judges can judge the way they want, much the way they already do today. Just adept to the general logging system, that maybe is something similar they already do, and you got a structured and consistent way of handling your judgement.
The solution is: Smaller is better
Since I have come to the conclusion that it is impossible to exclude the gut-feeling from run judging at least it could get structured up by doing the judgement procedure in smaller pieces. Judge directly every trick in a run instead of the big picture.
– Judging smaller pieces is easier than bigger pieces.
– It’s more safe and consistent judging smaller pieces
– Proof of decision and judgement logging possible
– Faster judgement
– Opens up for live run judgment and quick result presentations
– Using points as notes is universal to understand
– Using points as notes is usable for computers to handle.
– Opens up for different judgement format rules to be implemented
There is more details on this and I will not go into it all here. But it could be interesting for anybody who is interested in skateboard judging to get this point of view presented for them. I would really have liked to present this for the ISJC team some time. It would take about one hour. It’s depending on how much interaction there is during the presentation.
Comments on the article
Ok, finally. Below are some things that I would like to comment on after reading the article. With the above text of my views in mind.
– “One of the most important (responsibilities of World Skateboarding) is standardizing the process and criteria of how skateboarding contests are judged“
As important this may be, talking of a standardization may talk against the very goal you seem to push for. I don’t see much information of the process that I think is the key. And the criteria, even if mostly relevant, may be misunderstood. The thing is there are no specific criteria that a judge must follow.
On the other hand the process is something that very well could be standardized to help out the judgement. But I don’t see anything on this topic.
– “Quality and consistent judging are the basis of fair competition“
I agree 100%.
And we need a way of helping judges with quality and consistency. But I don’t see anything of this.
– “ensure that skaters around the world know what’s expected of them“
This is a tricky one.
It’s true that it’s very important for the racers to know the event format and judgement logic being used.
It’s false, though, to think that you may know what the judges will expect of you. Because it is a change of exception for every occasion. Judges all have (should have) a very free judgement philosophy. The only way to increase this knowledge is to try them out.
1. Either you see how they are judging you in important competitions. The problem is for even the most prestigious events you seldom know how individual judges have judged. You often only get the “average” value.
2. You could ask a judge to make a live judgement of your runs. You yourself or together with your fellow skaters. Just to see how things compare. What runs seems to make more impact on the judge.
3. You could even send an e-mail with footage to a judge to make a judgement of your runs. You yourself or together with your fellow skaters.
This way a skater may prepare for how you will be judged but it demands that…
1. You know who are acknowledged judges.
2. You may want to know what judges may be judging at a specific event.
3. You need a system where acknowledged judges have time/motivation to do this work.
– Poor judging … can actually cripple the progression of a sport
It’s true that poor judging is bad. But what is poor judging? Since judges have a very free judgement it can’t be the judgment philosophy because that may change between judges. But what could be poor judging is bad consistency, corrupt judging or just bad administration management of your judging.
Here are things we can do for the judges to help out. But I don’t see anything about this in the article.
– What makes good skateboard from something “you know when you see it”, to something that can be written down and explained.”
Again, this is a tricky one. The short answer, you can’t.
“Short answer, you can’t”
There are a multiple things included in a judgement that you as judge do in a split second. You don’t have time and possibility to write all things down and reflect while following the runs at the same time. You have to rely on your gut instinct. An instinct that get’s better and better the more you practice.
But with a better way of creating a structure around the judgement process we can make things a lot better.
– The value and appreciation…
“the value and appreciation of progression, creativity, and originality rose to the forefront as we labored to ensure that the criteria we produced supported the continual positive growth of skateboarding on all levels.“
I agree with the goal here but… don’t agree that the criteria produced “ensures” this.
And I don’t understand how you can ensure this at all actually. The only thing we can do is having an open format that does not force certain trick lists for example. And have judges that are allowed to have different skate philosophies.
Then it is up to the skater to do what they want to do and for judges to make judgement of how they like what is done.
The scene may welcome new tricks and new variations but in the end it is up to the judges to make the quick decision what it’s worth. It’s not just because a trick is new, creative or progressive that a judge may like it.
– ISJC MISSION STATEMENT
“The International Skateboarding Judging Commission (ISJC) Skateboarding Judging Criteria was created to foster the continual progression of skateboarding while highlighting the importance of creativity and originality of skateboarding in competition.Agree 100%!“
“JUDGING PHILOSOPHY“
- “Skateboarding judging doesn’t exist to standardize or define what is considered good skateboarding, it is strictly a tool to rank the performance of skateboarders against each other in a given competition.“
- “Scoring is based on judging the current performance or moment in time.“
- “Judging is based on a skater’s performance against the current field, not against their past performances or ability.“
Agree 99%
1% off because in the first point I don’t agree with…
“…against each other in a given competition”. It’s not only in a competition but more general like…
“…against each other in a given moment in time”.
– OVERALL IMPRESSION
- “The judging panel will use the below criteria to formulate a single score based on the overall impression of a skater’s performance within the parameters of the competition.“
Simply I don’t think a judge can formulate his overall impression of a run with all the given criteria. It’s more like “will use some of the below criteria”
I agree depending on how that overall impression is done. It’s that process that is the key.
And also interested in what that “within the parameters of the competition means”.
If we want good judgement administration and consistent scoring from judges we need to help them out with a structured way of doing their job.
An overall impression judgement can easily generate bad judgement administration, corrupt judging, bad consistency and multiple other problems known using this way of judging as I described earlier above. At least if there is a demand to be able to defend/explain/understand an overall impression.
To do it by pointing at a criteria list will not work.
I don’t see any mention of the problems that do exist and how to try and deal with them. I can understand this is not something you want to go out with “in public” but I see on the text and the conclusions that nothing has been addressed. It’s the old system of judging and I had hoped with all the experienced persons gathered that things could have been moved forward. It’s after all a big effort in the making because of the Olympics.
– JUDGING CRITERIA (in order of priority)
- Difficulty
- Trick Performed
- Obstacle Selection
- Execution
- Quality of Execution
- Style
- Fluidity / Power / Aggression / Aesthetics
- Speed
- Height / Distance
- Quality of Landing
- Use of Course
- Variety of Tricks
- A higher number of tricks will not necessarily result in a higher score as the value of difficulty and execution are more important in formulating an overall impression score.
- Variety of Obstacles
- Flow (for runs only)
- Variety of Tricks
- Consistency (for runs only)
- While consistency and runs without any bails are a key factor in any high-scoring skateboarding performance, consistency alone will not be enough to win a competition, as the value of difficulty, execution, and use of course are more important in formulating an overall impression score.
This may be something that looks good on paper but how do you explain the complexity of a gut feeling?
Also it may be good to separate such a list between run and trick judgement. Even though you by fact do trick judgement also in a run it’s another perspective on it. Take for example your point “quality of landing” in your list. I don’t think that is an issue in runs. Either you make it or not. While in a strict “trick judgement” format then it does matter. I would actually prefer two different judgement criteria lists. One for “Trick judgement format” and one for “Run judgment format”. The pure “Trick judgement” format should by pure logic be a lot more detailed on what you judge since it is only one trick in play. And maybe for a strict “Trick judgement format” even that hated trick list could come in play.
The criteria Iist can be done in different ways. I prefer making the same criteria list from another angle coming from the judgement perspective. In low level events where the judges take care of everything it doesn’t matter how you present it. But on a high level event it may play in how you want to divide things up on different responsibilities. And then this is something like how I would have structured things up.
Run judging criteria
1. Facts
Tricks facts
– Difficulty?, Variety tricks/obstacles/sections
– Bail (landing – made it or not?)
Performance facts
– Height, Length, Speed
– Obstacle, Spot
2. Judgement
– Quality of Execution Facts.
– Style (Flow, Fluidity, Power, Aggression, Aesthetics)
– Creativity, Originality, Risk
3. Administrational
To make the judgement process as quick as possible.
Possible computerized facts not needing “judge judgement”.
– Trick management (trick manager)
– Obstacle/section management (run manager)
– Bail (run manager)
– Point format handling (computer setup)
– Variety trick/groups
– Consistency (bail vs tricks done)
– Max trick count (best n tricks counted)
– Run statistics
– Logging
– …
The end
Sorry for those who does not like reading long texts. But I hope the interest in skateboard judging is high enough for the people in the ISJC (International Skateboard Judging Commission) to find it interesting enough to read and reflect.
The subject is a sensitive never ending story, I know, but I hope this may give some new angles on the subject.
Regards
/Hans “Corky” Koraeus, Sweden (July 18 – 2018)
