Each year during the NCRA Convention & Expo, a few dedicated members sit behind closed doors working on the Speed and Realtime Contests. Their tasks include feverishly reviewing hundreds of papers from the contestants, poring over them for grading, creating lists of qualifiers for each of the legs, and more. It’s a lot of work, but when we sat down with Contests Committee members Pat Miller, CRI, CPE, and Kelly Moranz, CRI, to learn everything we always wanted to know about the Speed and Realtime Contests, they said that they have fun, learn a lot, and that they enjoy connecting with each other and the contestants after months of planning.
How many individual legs do you grade for the Speed and Realtime Contests?
The Committee grades every leg that is submitted after the contests. Every passing leg is ranked as a qualifier. Only contestants who pass one leg in each category are eligible to place for a medal. A contestant could place first in the Realtime literary leg but not qualify in the Realtime testimony. Recognition will be given for the stellar literary skill but the person is not eligible for a medal.
How do you protect the papers so you don’t know whose work it is?
Contestants are randomly assigned an ID number that is known only to the Committee Chair(s) and the contestants themselves, of course. It’s not perfect, but the Committee does not try to link the ID with the contestant at registration. We want to be as surprised as the members attending the Awards Luncheon. It’s exciting to grade the papers and to see the amazing skill of our colleagues. An exception is made for Canadian members who opt to transcribe using British variant spellings as allowed by the Merriam-Webster dictionary (colour instead of color). Contestants who are eligible for this option do sign a form that makes it clear they will not remain anonymous to graders. We want to give back as many points as we possibly can within the What is an Error? Contests guidelines. We just grade the words and punctuation, not the people.
Why do contestants talk about qualifying instead of passing or failing?
Contests are definitely not a pass/fail situation. Any member who schleps their equipment to the NCRA Convention & Expo, days ahead of the Convention at additional cost, time away from work, and after whatever practice the member may have added to preparation for the Contests, does not ever pass or fail. Let’s get real. How especially extra awesome are these members? Contestants have and give a great showing of skill every year.
We use the term “qualifier” for Contests. Contestants either qualify or do not qualify. More contestants qualify than do not on both literary selections and both testimony selections. If there is a leg on which fewer people qualify than not, it is definitely legal opinion.
What happens with the papers that don’t qualify? And, conversely, what happens to the papers that do?
Every leg that is submitted for grading is graded and marked with a summation of results, final grade initialed by the Committee Co-chairs. Papers that do not qualify are marked “DNQ.” We attempt to mark all folders with polite-sized digits and letters so that contestants may keep their results private-ish when they review their grades after the Awards Luncheon.
Of course, this is a contest where participants expect scores to be made public. All of the qualifiers names and scores are posted after the Awards ceremony. While the Committee does not talk about individual member results – qualifier or not – the contestants may do as they wish.
What kind of checks do you do on the audio and what do you grade against?
We have checkers in the Contests rooms who have copies of the material that will be dictated. They mark any variations, stutters, kerfuffels, and any other such events on the script as best they can. When the Committee meets after the Contests, the checkers compare their findings against the master transcripts. Notations are made so that the Co-chairs can make final decisions on how those sections will be handled in grading. If we do record the audio, then the audio is also compared to the master transcript for any additional adjustments.
Realtime Coach will make changes in the master transcript that is loaded into their system before beginning the grading process.
Are there areas where there is more than one right answer? How do you deal with that? What if there’s a slip by the speaker?
Yes, there is more than one “right” answer. There are places where more than one punctuation choice is acceptable. This is why the computer grading is called “first grade.” It gives a great assessment of each transcript and saves a huge chunk of time for the graders. An instant accuracy percentage is given to determine qualification for second grading. A second grade is critical and so the graders review each transcript to determine whether or not it had areas that prevented the computer analysis from making an accurate assessment. During this time if there is a “trend” of some sort noted in a consistent error, it will be reviewed as to the possibility of it being correct. If that is the case, the number of errors would be reduced for that test.
Sometimes there are third and fourth grades of papers. Each round is in a different color pen and is initialed by the person who graded. When there are enough Committee members present, we do not regrade a paper with the grader who did the prior pass. The Contest Co-chairs have final say in any error/not an error determination and in how to grade areas where a speaker “slipped.”
A Canadian won the Speed Contest most recently. Did he use any Canadian spellings? How do you deal with that? Are those considered absolutely wrong?
They are only wrong if you are not Canadian. Yes, he did use British variants. He also was informed of the grading policy and formally agreed to it. We require consistency for those papers, too. If the contestant wants to use British variants, they cannot also have the companion American variant in the paper even if it would be an acceptable spelling in other contestants’ papers.
Why are the percentages figured out to so many decimal points?
I’m about to do math. Three contestants with three errors, the top three results in the Contest. Realtime contestant A has one error on the LRT (literary realtime) and two errors on the TRT (testimony realtime). Contestant B has two on LRT and one on TRT. Contestant C has zero on LRT and three on TRT.
LRT = 1000 words minus 1 error equals 99.90.
TRT = 1125 words minus 2 errors equals 99.82
99.20 + 99.82 / 2 = 99.860 for Contestant A
LRT = 1000 – 2 = 99.80
TRT = 1125 – 1 = 99.91
99.80 + 99.91 / 2 = 99.855 for Contestant B
LRT = 1000 – 0 = 100.00
TRT = 1125 – 3 = 99.73
100.00 + 99.73 / 2 = 99.865 for Contestant C
Stating the obvious, all three qualify in both categories and so are eligible for the medal round.
Standings for three contestants with an equal error count:
LRT: Blue ribbon to C; Red to A; White to B
TRT: Blue ribbon to B; Red to A; White to C
Champion Level: in third place, B; in second place, A; Champion, C, with a difference of no error but .005 in the score.
Although all three contestants have the same number of errors – a total of three in both legs – whether they are made in the literary or testimony leg will make a difference in the overall score.
What qualifications do you need to be a grader?
If you volunteer to be on the Committee, you will be expected to help grade. It helps if you graded NCRA certification exams, have an open mind, can leave your ego and personal judgment about what “should” be done at the door, and have a great sense of humor. You must absolutely be able to be completely confidential in all dealings from the moment you join the Committee to the announcement of the scores at the Awards Luncheon. You must be available to put the time into grading, which means arriving at the Convention early in order to participate and to begin grading as soon as Realtime Coach completes the first round of grading. You will want to know the What is an Error? Contests guidelines comfortably and be able to check them quickly for guidance. To recap: We grade words and punctuation, not people. We go with the style we’re given and enjoy doing something we may not ordinarily do in our own (saleable) transcripts.
Graders must be able to “defend” their choices to the contestants at the postmortem. Of course, not every grader will attend the postmortem, but we must answer to our decisions. We do make mistakes. We have nightmares and daymares and agita, tsuris, and plain old anxiety about mistakes. When a mistake is pointed out to us, we feel a professional disappointment in ourselves that is probably not as bad as the feeling the contestant had that they had one fewer error than was marked. Graders need to be self-healing, fairly confident people who understand that excellence is sought but not always achieved by graders as well as contestants.
But mostly – did I mention that we laugh a lot? Not at the contestants – ever. But sometimes a translation is quite entertaining. It could also have to do with the lack of sleep involved! We’re court reporters. Words are our playthings. Graders also get to glory in the beauty of skill that presents us with some amazing transcripts. It’s a hard, busy time, but it’s an awesome time to be among your colleagues.
Registration for the Speed and Realtime Contests is open until all the seats are filled (and the Contests sell out every year). Visit NCRA.org/Convention to register online or complete and return the downloadable Convention Registration form to NCRA.
