The Development of Standardized Test Questions

Summer School In Chicago
Tim Boyle/Getty Images News/Getty Images

Have you ever wondered how standardized test questions are created? Test question development is a lengthy overall process. Every test question goes through a rigorous evaluation before it can become an official test question. This process easily takes over a year. Each question is carefully reviewed by a variety of educational stakeholders throughout the process. Even as the question moves through each review, there is still a possibility that it may not find itself on an actual standardized test.

Standardized testing has become such an crucial part of education. Therefore, it is imperative that test questions accurately measure intended learning standards. The testing industry is a multi-billion dollar industry, and much of that money goes into the development and screening of potential test items.

How Often Are New Items Written?

There are several reasons why new items need to be written. New items are written every year as old items are cycled through, retired, or released. It is also necessary to write new items when standards are added or revised. Currently states have their own set of educational standards and contracts out the development and scoring of standardized tests. Of course, this will all change with the development of the Common Core State Standards and the associated assessments. The Common Core State Standards demand that a new set of questions appropriately assessing those standards be developed and screened.

Who Writes Standardized Test Questions?

Standardized test questions are typically written by one of two educational stake holders. Some questions are written by content specialists that work directly for the individual testing vendors. The majority of questions are written by educators who are hired as independent contractors.

These independent contractors typically do not have any direct stake with the state for which they are hired to write questions. The independent contractors are typically paid per question that they develop.

What Drives the Development of New Test Items?

The single most powerful document that drives the development of a test items is the specific state standards. Each test question must be aligned with a specific content standard that it is supposed to assess. In addition, many states have developed specific item specifications or guidelines that guide an item writer as they develop an assessment item. These item specifications are much more specific that than the standards themselves and typically list both test limits and points within the standards that should be emphasized.

What Happens After a Question Has Been Written?

Several things happen to a test question after it has been written and submitted to a testing vendor. An artist develops any art, graphics, graphs, etc. that the question may need. Then the question is sent to a copy editor who provides the first true screening for the question. A technical review is conducted by the testing vendor to ensure that the question aligns with the appropriate standard(s), that there are no errors, and then to decide whether or not to keep, reject, or have the item revised.

If the item is approved, it is typically sent to an assessment specialist within the state department of education who focuses on that specific content. The assessment specialist reviews each item closely for alignment and errors. They also have the ability to keep, reject, or to have the item revised.

Once the question has passed through this stage, it is sent to a committee of educators with content knowledge specific to the question. The committee breaks down and dissects the question thoroughly often having intense discussions concerning the question’s alignment, format, and possible bias. Once the committee has discussed all of these a vote is taken on whether to accept, reject, or revise the question.

What Happens to a Question After It Has Passed All Screenings?

If the question has passed all screening committees, it is sent to be field tested in the next assessment period.

Each assessment has a series of operational questions, or questions in which the student will be scored, and then a smaller series of field test questions, which are do not count for or against the student, but provide data to determine whether or not a particular question is too difficult or too easy.

The data on a field test item ultimately determines whether the question will be placed in the test bank, revised further, or rejected for good. Items where the majority of students answer correctly or incorrectly are typically rejected. All data from the field test is reviewed and evaluated by another committee. This committee makes the final decision based on that data as to whether or not the question is acceptable to use.

What Happens Next?

Once a test item has been accepted it is added to the test bank where it awaits to be selected to be an operational question. It can spend several years as an operational test item. The item will eventually be retired or may become a released test item which effective teachers use as practice test questions.