Why the NSS was Created
How it Came About
The NSS was born out of a desire to make a better and fairer way to score competitions. The creator of the NSS, Ed Carnes, has played, judged and coordinated fiddle contests for many years. According to Ed: “Being an information technology professional, I always look at things from an analytical standpoint. I've noticed that some competitions had better judging results than others and I looked closer. I realized that many people had a bigger problem judging than one would think. I found that a really great fiddle player is not necessarily a great judge. On the other hand, some mediocre fiddle players, or even knowledgeable non-fiddle players, are sometimes great judges, and it made me wonder why that was. It became my passion to figure out a better way to score competitions.”
When Ed and Howard Harris re-established the Grand Master Fiddler Championship (GMFC) in 2005, Ed started working on scoring methods. “Although I developed paper score sheets and had calculators, at first we just use Microsoft Excel to add everything up. It was tough to manually go through and drop high and low scores, so I started looking at programmatic ways to make this easier” Ed said.
Point Spread Bias
One of the biggest issues Ed found is what he calls “point spread bias”. It is very difficult for anyone other than an experienced judge to determining how many points to allow between various player performances. This difficulty is exacerbated because a judge has to allow points for later players; even if the very first player is potentially the best in the competition. He found that a competent andexperienced judge is able to rank contestants fairly well; but again, the number of points to put between the contestants is the issue. A judge who uses a very small point spreads between contestants has less influence on the contest outcome than a judge who uses a very wide point spread - but in theory all judges should have the same weight. Ed found it was possible for two judges to have someone in first but the third judge control the winner because their point spreads were wider. And unfortunately, that was not always a mistake. He observed several deliberate attempts by persons using point spreads to influence and determine the outcome of a contest.
An Example of Point Spread Bias
Ed talked to a lot of competition fiddlers in developing the NSS. One related this story to him: “I was in the US National Fiddler Championship many years ago and I scored the most points in five of the six rounds in the Open. However, another player beat me by so many points in round one that the accumulated scores were high enough and they had the most points overall; so, they won and I got second.”
Ed then asked: “Did you play that badly in round one, how was your score?”. They said “No, I played well, and scored high, just not as high as the other fiddler.”
Ed then replied: “How do you know they actually played THAT MUCH BETTER than you in round one? Every round had five different judges - so couldn’t it be that the point spreads in the individual rounds were very different and not necessarily uniform? Because if they were, almost certainly you should have won overall.”
We leave that to you to decide.
Ranked Scoring was the Solution
The idea of ranking came from the TOTFA Scoring system. But in the TOTFA system, judges verbally determine the final placement from what they feel, and scores are just used by each judge only for their personal ranking. The scores are not used to determine the winners.
In looking around, Ed saw one of the early NASCAR point systems assigned a number of points based upon the order of finish in car races. With this potential programmatic method, he started testing various methods of point assignment based the results (rank). Fiddle champion Daniel Carwile was a person Ed contacted as a sounding board. “Daniel and I are each so detailed and get so enthralled in a subject that once we get started talking, people will just look at us and shake their heads. Daniel was the perfect sounding board and he has a keen interest in fair scoring, just like I do,” according to Ed, “and he has been a major help to me with theory behind the NSS.”
For flexibility reasons, the NSS will also allow scoring with traditional, non-ranked scores. This was designed for those contests that may or may not be comfortable with ranking and gives them a way to compare ranked scores versus the traditional scoring as recorded by each judge.
Judging Accountability was Another Factor
Contest coordinators always strive to get fair and honest judges; but unfortunately, sometimes a person comes along with an agenda. One of the design components of the NSS was to have capability to monitor judges scoring patterns in real time. That monitor can be used to make sure there are no data entry problems, or perhaps a judge recorded a number that was mistaken for another number, as a bias check, and to see how the contest is going. In a sense, it gives a way to determine the effectiveness of the judges and look for any “agendas” or inexperience in judging.
Putting it All Together
Systems need a LOT of test data. In this, Ed was fortunate and had over 30+ years of GMFC data. He was also given anonymous data for multiple years of contests by the Texas State Championship Fiddlers Frolics and the National OldTime Fiddle Contest in Weiser, ID. The data was meaningful and supported what the NSS was designed to do.
Over the years, several contests heard about the NSS and inquired about using it. Ed was determined not to release it until it was ready tried and tested. After nearly 20 years usage in the GMFC, 12 years in the Twin Lakes National Fiddler Championship, 5 years in the Kentucky State Fiddler Championship, and 3 contest pilots in the past 2 years, the NSS is ready and now released! Like any good technology person, Ed determined that user guides were needed and those were developed, along with some training exercises to teach people how to use the software.
Share
Print 