(Note: Most of this appeared in the 2012 North Carolina AP Calculus Teachers’ summer newsletter.)

           In a few days I will fly to Kansas City to be a Question Leader (QL) for the AP Calculus exam.  This will be my sixth year as a QL and I have loved every minute of it.  Prior to serving as a QL, I served as a Reader, a Table Leader (TL), and a Question Team Member (QTM).  A QL’s main job is to train readers how to grade student work based on the scoring standard (rubric).  What readers see is a polished training session (or “briefing”) with a mic’d QL, numerous slides to explicate the issues and how to award partial credit, and actual student solutions (all identifying information hidden).  What readers do not see is all of the “behind the scenes” work that goes into the briefing.

Two days after students take the AP Calculus exam, the Chief Reader (CR) lets the QLs know which of the nine problems will be theirs.  In addition, we see draft scoring standards, find out who is on our team, and what the “flow” of the reading will be (i.e. when our question goes in the line up).  A few weeks later, as many as 80 student samples arrive via UPS to the QL’s domain.  The QL scores these papers and begins compiling lists of issues for the first round-table discussion.  By the time the QL boards the plane for the reading site, many questions are answered, but many questions remain unanswered and even unasked.

A colleague compiles issues in parts (a), (b), (c), and (d) of her question at home.

Here is what my list of issues looks like for part (d) of my assigned question:

AP Calculus QLs arrive on site for the “pre-reading” along with the CR and Exam Leaders (ELs) six days before the reading begins.  The first day of meetings is spent in an intense round-table (ok so it is actually a bunch of rectangular tables put together in a big open square) discussion where we spend about 45 minutes on each question, examining issues and giving suggestions.  The second and third day of meetings the QL spends with the two team members, looking more closely at student work, preparing slides, and selecting samples for the TL briefing.  During these Question Team meetings the Team is visited by the CR and the ELs who challenge the scoring rubric and ask a lot of “what ifs”.  I haven’t ever defended a dissertation, but I think I know what it feels like based on these meetings!

What the QL has conceptualized as a road map for awarding partial credit gets really challenged at the TL briefing.  To be honest, this is tougher than the “big” briefing since most of these people have way more experience that I do, and are way better mathematicians than I am.  But, their input is invaluable.  After the TL briefing, the Question Team is able to further hone and revise the presentation.  About 36 hours before the “big” briefing, the final meeting takes place with just the QL, the ELs, and the CR (often the AssistantCR and CRDesignate too, depending on the year).  At this meeting the final scoring standard gets set and all of the issues, questions, and comments we can collectively think of are picked apart.  Then the QL needs to refine the slides, rehearse for the “big” briefing, and participate in the reading by working in the communal work space we affectionately call “The QL Pod”.

After the “big” briefing, the QL sits in the front of The Pod to answer questions from TLs whose rooms are grading the problem.  At first there are a lot of questions as readers are getting on the standard, but then the number of questions decrease as the readers see more student work and begin to internalize the philosophy of the point allocation. It takes about 1.752 days to finish scoring a problem and then the readers will get trained on another question.  Meanwhile, the QL finalizes reports for the CR and the College Board.  The QL will help with test booklet irregularities (these range from work done in the wrong place to exams out of order to $1 bills found in the booklets) and will score additional exams right up until the readers are dismissed.  Then, it is time for the QLs and ELs to stay behind and clean up any unscored exams on site.  When every exam on site is graded and accounted for, it is time to say good-bye and go home!

The reading itself last for seven consecutive days, but leadership works for as much as twelve consecutive days (not including travel days on either end).  It is a both energizing and enervating, but like I said in the beginning, I love every minute of it!

See you in Kansas City!