Last spring, state education officials decided not to score a 10th-grade MCAS question after some teachers raised concerns it was racist and hurt students’ chances to pass the exam, which is required to graduate.
The exam item asked students to read a passage from “The Underground Railroad,” a novel by Colson Whitehead about an escaped slave girl who a white couple takes in. The wife, Ethel, speaks to the teenaged black girl, Cora, in a denigrating way, calling her “stupid” and telling her she smells. The question asked students to write a passage from the perspective of the white woman.
The controversy focused on the content of the exam question — specifically that it required students to write from the point of view of a character some found racist. But a close look at the development of the test question raises questions about the vetting process and the writing exercise.
Test questions for the English Language Arts portion go through multiple layers of vetting before landing on an MCAS test. Two committees of teachers and educators from around the state reviewed the question. The first committee of 10th-grade English Language Arts teachers focused on whether the reading was appropriate for the age level of the student and addressed established learning standards. The second committee focused on whether there was any bias in the question that would disadvantage some students. The question was piloted on more than 1,000 10th-graders. Two outside experts also reviewed the question, according to the Department of Elementary and Secondary Education (DESE).
The diverse members of the Bias and Sensitivity Committee voted unanimously to approve the question in June 2017 and, after receiving results of field tests, again in August 2018. Of 14 members who attended at least one meeting, nine were African-American, Asian or Hispanic.
Read more: 'It's Not A Healthy Situation': Review Committee Member Says She Objected To MCAS Question In 2017
What’s surprising is that several members of the bias committee said in interviews with WGBH News or written communication with state education officials that they didn’t remember the question. That might have to do with the volume of questions they review.
“We can have some days that are light and some days that are heavy,” said Kahris White-McLaughlin, a former history teacher who served on the Bias and Sensitivity Committee then. “We could have had more time.”
On June 28, 2017, when the bias committee reviewed the "Underground Railroad" question, they reviewed 214 new items, according to the DESE. The committee had already seen the reading passages, so they were focusing just on the question. The questions they looked at included 45 eighth-grade English language arts questions, 73 fifth-grade science and technology questions, and 96 10th-grade English language arts questions.
A little over a year later, the bias committee spent a day looking at the results of field test data for 329 questions, including for the "Underground Railroad" question.
In a statement, DESE officials explained some bias committee members’ inability to remember the question this way: “The committee members frequently serve for more than one year, and during their time on the committee, they review a wide variety of questions in multiple subjects.”
A member of the committee in charge of deciding whether the questions are age-appropriate, the Assessment Development Committee, said she also didn’t remember the question.
“The process is intense,” said Patty Casey Ridolfi, head of the English department at Marshfield High School. But Ridolfi says the workload doesn’t affect the quality of the review.
Off Site
During White-McLaughlin’s time on the Bias and Sensitivity Committee, the group went from meeting in person at a hotel conference center to meeting remotely.
“Which I didn’t like as much,” she said. “It’s difficult, because you’re doing it from a distance and really for this work, I need to see how people are saying it. Not just their words, but what’s important to them. And it’s much easier to do it when we’re all around one table.”
White-McLaughlin speculated that meeting through conference calls was a cost-saving move.
As a result of the MCAS controversy, state education officials say they have gone back to holding all bias committee meetings in person.
Field Testing
Typically, when the state field tests MCAS questions it is developing, it adds them to real MCAS tests but doesn’t score them.
In the case of the "Underground Railroad" question, that didn’t happen. Instead, it was tested along with other questions as part of a 2018 practice test to pilot the state’s new computerized testing system. When students took the test, they knew it wouldn’t count, said Michol Stapel, associate commissioner for student assessment at the DESE.
Of the more than 1,000 students who took the practice test, only 64 scored high – three or for on a scale of zero to four. Five hundred and six students scored zeros. Another 195 students left it blank.
The state didn’t think the high number of zeros or blanks had anything to do with the question itself, since the question wasn’t an outlier.
“When we selected items for the operational test, we took into account the lack of motivation on the 2018 field test items, with the expectation that student performance would improve,” the DESE said in a statement.
Literary Writing
Teachers who complained to the state worried that the "Underground Railroad" question asked students to write from the point of view of a woman they thought was racist. Some testing experts also worry this type of question was too challenging for high school sophomores.
After reading the passage from "The Underground Railroad," students were asked to write a “narrative response” using characters, settings, events and other details from the passage following this prompt:
Based on "The Underground Railroad," imagine how the story might have been told differently if it were written from the point of view of Ethel. Create a journal entry written by Ethel reflecting on the events that happened in the passage. Your journal entry should provide insight into Ethel’s thoughts and feelings, as well as her relationship with Cora. Be sure to use what you know about the characters, setting, and events from the passage to develop your journal entry.
“This is a task that is way beyond any 10th-grader that I have ever met,” said Sarah Beck, a professor of English education at New York University.
Beck said to answer the question well, students need to have a sophisticated understanding of slavery and U.S. history. In addition, this question is asking students to “produce literary writing” under pressure, and Beck wonders if many students can do that.
“I’m skeptical about that,” Beck said. “The state frameworks were never intended to justify test design. They were intended to inform teaching.”
Another education professor worried that students weren’t provided enough information in the passage to write a journal response.
“The test is asking the writer to put him or herself into a character's perspective with very little time to think about what that means,” said Sharon James McGee, a professor of Literature and Language at East Tennessee State University.
“In this passage, while we have a few clues about the character of Ethel, she is still not fully realized as a character in the prompt that you get. So students may resort to stereotypes and clichéd writing, which is problematic,” James McGee said.
Ridolfi, from the state’s Assessment and Development Committee that reviewed the question, said it was appropriate for 10th-graders because students are being assessed on a narrow set of skills.
The question is meant to test how students “write narratives to develop experiences or events using effective literary techniques, well-chosen details, and well-structured sequences,” according to DESE officials.
They were scored on how well they use proper spelling, punctuation and grammar along with “idea development,” according to state officials.
“I think it can be done, as long as you’re looking at a narrow skill-set. Is it different than how you assess it at the classroom level? Of course,” Ridolfi said.
“Looking at how tenth graders can write from a different point of view is a very important skill,” Ridolfi said, “because we’re asked to do that all the time.”
Our coverage of K through 12 education is made possible with support from the Nellie Mae Education Foundation.
