The PARCC (Partnership for the Assessment of Readiness for College and Careers) released a draft of their assessment frameworks for ELA and Math, and asked for feedback: http://www.parcconline.org/parcc-content-frameworks.
Here are my three major objections.
The Industrial Age Paradigm
The assessment framework reinforces the current model of schools as factories and children as raw material to be forged into a uniform product. The assessment framework is still based on a cohort of students starting in Kindergarten and learning a prescribed dose of skills and knowledge each year through 12th grade. For example, while the ELA standards allow teachers or schools to select the specific reading passages, passage specifications are bounded by grade:
Use of grade band-level complex text: Leveled texts that are below grade band level in complexity are not a substitute; the standards indicate students should be reading grade band-level complex text. Flexibility is built in for educators to build progressions of more complex text within grade band levels (e.g., grades 4–5, 6–8, 9–12) that overlap to a limited degree with earlier bands, but reading text from the appropriate band level lies at the core of the Model Content Frameworks.
Students are children and children are individuals as are teachers. Why can't we bust out of the early 20th century mentality that everyone should be learning the same things in the same years?
The flawed concept of effective valid standardized tests of cognitive behaviors
There are basically four types or results one can assess: feelings, knowledge, behaviors, and results.
- Feeling assessments involve questions like, "how confident do you feel that you learned how to do X?" Feelings are the least reliable and least expensive way to assess.
- Knowledge assessments are more objective. Multiple choice questions "Which is the main idea in the following paragraph?" are one way of testing knowledge, as are short answer questions, and, in some cases essays can also test knowledge (or skills). Knowledge assessments test whether the learners can recall facts or procedures, they are fairly straightforward to develop and score, although more expensive than feelings assessments. One reason knowledge assessments are flawed is that they can only test a subset of what a student has learned over a period of time. For example if a teacher happens to have spent time on the specific areas tested (from, say, tailoring classes to what was tested in previous years), while neglecting areas that aren't being tested, the test will give a false result on total knowledge. Most current high stakes assessments test basic knowledge.
- Behavior assessment tests whether the learner applies knowledge or skills when placed in a situation that calls for them. These assessments are much more expensive to develop, and need to place students in novel situations, so they cannot merely regurgitate facts or directly apply algorithms they have memorized. When I was developing content and assessments, our rule of thumb was that, when teaching and then using behavior assessment, we needed to allocate 1/3 of the budget for the cost of assessment development and administration, and content development and instruction was 2/3.
- Results assessment tests whether the students are better able to achieve organization-wide goals. In the case of education, we'd want to evaluate whether they become better citizens, better workers, or better parents. This type of assessment is extremely difficult; can we even come to an agreement on what a better citizen, worker, or parent is? How could we possibly measure it? How could we determine how student activities in 1st grade make students better citizens, and what could we do with that information?
The goals of the PARCC Assessment Framework (assessing to know if students have the cognitive skills and knowledge to perform real tasks, and determining interventions when they do not) are noble. But my experience tells me that the cost trade-off of deep assessment versus saving money never works out. Following my old rule of thumb, where 1/3 of the cost is assessment, and 2/3 of the cost is teaching, you could reallocate that money and effort to essentially teach 40-50% more by using a simpler assessment. Would you rather teach more students more skills and knowledge (and not completely know the effectiveness) or teach less, but assess better? When the final dollar allocation decisions get made, we always seem to tip the scales toward reducing assessment costs in favor of teaching or savings.
Thus, to me, it seems more likely that PARCC will eventually fall down when it comes time for states to put money behind developing and scoring the assessments, and we will end up with tests not significantly better than the ones we currently use.
High stakes assessments always get gamed
If we are going to use assessment to determine teacher pay, teacher retention, and funding for schools and districts, the people being rewarded or penalized will search for ways to maximize their outcomes, and some of those will affect the validity of the students' test results. I know I am not the only person who writes about this, but here are three previous PilotEd articles that address this issue:
Mapping state proficiency standards to the NAEP
Many states cheat on standardized tests
What's my recommendation?
With the current direction of the Common Assessment, it's as if we've taken some of very smart, competent people, and we've told them, "Build a great road to that mountain over there." And they'll spend a lot of time, effort, and money building a great road to the mountain. But what we really need is a road to the sea, so what we'll get is a great road leading us to a place we don't really want to go to.
The whole debate about what students should know, when they should know it, and how we assess it is extremely important for our society. Rather than using that debate to frame high stakes assessments that merely tweak our current educational paradigm, we should
- Rethink what it means to offer education.
- Use the Common Core Standards and the PARCC frameworks as the bases for defining the skills and procedures needed by educators and parents.
- Build out corresponding community, pre-service, and in-service curriculum for all who will guide student learning.
- Develop resources for educators and parents to unleash to potential for each child.
