Testlet

This is long so grab a beverage and have a read…

This past Thursday the Mississippi Department of Education (MDE) released a “testlet” for each of its tested areas (grades 3-8 and algebra one).  [There are similar documents released for English/Reading, but I am not sure about their release date(s).]  The testlets are meant to be practice tests for classroom teachers and/or administrators to use with the students before they take the end of year (EOY) course exams.  As an algebra one teacher and assessment aficionado, I of course had my paws on the printed document before our administrators walked it down to our classrooms.

The first thing my administrator said, document in hand, was “We need to decide as a group how we are going to use this.”  Because I am a crusty, seasoned, classroom teacher with 26 years of experience I replied, “Before we meet as a group, I need to check the quality of this document.  I am not about to waste two days of instructional time giving my kids a junk pretest.”  To be fair, my administrators are AWESOME and totally respected my need to comb through the document.

Fast forward a few hours and I was in a state of extreme testlet nausea.  Many of the questions were so bad I just didn’t know what to do.  I emailed several of the questions with my annotated concerns to both MDE and to my math colleagues nationwide.  My math colleagues were saddened, appalled, and in one case, not surprised.  MDE did acknowledge that on the first question THE GRAPH DIDN’T EVEN MATCH THE EQUATION and so none of the statements were valid, and in fact amended that question immediately.

It still wasn’t that great and so I fixed it.

Question 1 as it first appeared:

Screen Shot 2016-03-05 at 6.55.52 PM

Question 1 after MDE fixed it, and as it still appears in the teacher resources:

Screen Shot 2016-03-05 at 6.51.34 PM.png

(NOTE:  The answer key says that answers A and E are valid.  Don’t get me started.  I already am.)

Question 1 after I fixed it:

Screen Shot 2016-03-05 at 6.47.50 PM

It is honestly still flawed — will probably have to throw it out completely.

(As a fun exercise you could put the answers to my revised questions in the comment thread.)

Before I show you some more questions as they currently appear and how I fixed them, let me back up a few years.  Last year Mississippi students and teachers were subjected to PARCC testing.  It was horrible.  Too much testing (Universal screener, benchmark, PARCC PBA, PARCC EOY) and the PBA and EOY were all on-line so our students were testing in small batches for weeks.  [Our district does not have a one-to-one student to computer ratio.] Instructional time was interrupted, compromised, and some days nonexistent.  We jumped for joy when the legislature (or was it governor?) threw out PARCC.  But the celebration was naive and short-lived.  Mississippi signed a 10-year, multi-million dollar contract with a testing company I had never heard of: Questar.   To the best of my knowledge, Questar had never written a math assessment before Mississippi hired them.

At some point this past fall, Questar released its “item sampler” for algebra one.  It was a small batch of on-line interactive questions with the print version available too.  I was thrilled to see these questions, but found several mistakes in the answer key.  I emailed MDE and they referred me to Questar.  I emailed Questar and nothing changed on the answer key.  Five months later and still, the answer key has not been changed.  Makes me really confident that my students’ work will be scored correctly this spring!  Ugh.  In addition, I emailed Questar asking when we could expect a full-length sample test.  The email I got in response was so rude and amounted to, “Your job is to teach the curriculum, not teach to the test.”  I am sure the person emailing me was a twenty-five year old who has spent maybe 18 months in a classroom.  He probably doesn’t even work there anymore.

I am sure many superintendents, curriculum specialists, and district test coordinators (DTCs) had the same question I did, “When are we going to see more practice items?”  I guess releasing practice items was not part of Questar’s multi-million dollar contract. So, the under-funded, under-staffed MDE took it upon themselves to release testlets to its teachers.  Unfortunately, the algebra one result was not good.

I spent the better part of Friday fixing the testlet.  One of my administrators asked if he should get me a sub.  I declined, saying that my students would be so busy on a cooperative activity that they would barely need me.  I asked MDE if they wanted my fixed version but the reply I got was, “That won’t be necessary.”  Instead, lots of teachers in the northern districts are using my fixed testlet anyway via their DTCs, who got the fixed version from our DTC with my permission.

What follows are several questions, as they appear in the MDE teacher resources, and then as they appear in my fixed version.

MDE Question 8:

Screen Shot 2016-03-05 at 8.03.16 PM

(NOTE:  Their answer key says F, T, T, F, T.)

How I fixed Question 8:

Screen Shot 2016-03-06 at 3.30.00 PM

 

MDE Question 10:

Screen Shot 2016-03-05 at 8.05.40 PM

(Note:  The answer key says A and B are valid.)

How I fixed Question 10:

Screen Shot 2016-03-06 at 12.46.54 PM

MDE Question 18:

Screen Shot 2016-03-05 at 8.08.39 PM

(Note:  Answer key says A, B, E)

How I fixed Question 18:

Screen Shot 2016-03-05 at 8.10.02 PM

 

These items are but a small sample of the items I fixed.  Admittedly, my fixes are imperfect, but the items are a whole lot better than what I first saw.

So now I need to turn my attention to the 8th grade testlet.  Here we go…

Screen Shot 2016-03-05 at 8.56.37 PM

MDE answer key says B and E.  Nope.  Not correct.  B is the only statement that correctly describes this function.