rsleediscuss

Just another WordPress.com site

Using and Misusing Value Added Testing in Tennessee

A recent position paper published by The American Statistical Association (the largest professional organization for professional statisticians) questions the use of value added assessments to make personnel and other high stakes decisions. This is important for Tennesseans because Tennessee teachers have been subjected to such questionable use of TVAAS (Tennessee Value Added Assessment System) data since 2010.

First, a little about how TVAAS works. TVAAS uses a group of individual students’ past test scores to project estimated future test scores for the same group of students. So, every student who is starting school this month has a projected score of where the statistical model projects the average score to be at the end of the school year, thus measuring their “growth” by the end of the school year. For teacher evaluation in Tennessee if a class of students score above this projection a teacher is judged to have done a better than average job because the students improved more than the estimated projection. Should the students score lower than projected the teacher is judged to have done a subpar job. One might think that using TVAAS is simple as using a baseball batting average.

In fact, Tennessee has been using the TVAAS system for over 20 years and the system provides useful data since it shows much more than what percentage of students are proficient on testing standards from year to year. So, when used correctly an academic team, school or school district can tell if reform initiatives or program improvements are working (and how well they might be working or not working). For example, say a middle school wants to implement a new reading program, but they want to see if it works before trying it school-wide. So the school implements it in the 7th grade and the school decides to use their year-end TCAP tests to determine if the program worked. If the teachers and principal only look at the percentage of students who score proficient at the end of this year compared to the percentage of students scoring proficient at the end of last year in the 7th grade, then they can’t really tell if the program worked because they compared two different groups of students at different times. But if they have a projected score and an actual score on the same group of students, then they could more easily determine if their new reading program worked.

The problem is that value added assessments, like TVAAS, are designed to make programming decisions like in the example above. Value added systems are not designed to make personnel or teacher effectiveness decisions. According to the American Statistical Association, value added testing systems, “typically measure correlation, not causation: Effects – positive or negative – attributed to a teacher may actually be caused by other factors not included in the model”. In other words a value added system is unable to separate out teacher performance from all the factors (such as student motivation, if parents have hired tutors, if there has been a high level of student transience during the year, or other factors) that affect student achievement.

This creates a problem for teachers and administrators in Tennessee. One-third of the annual teacher evaluation is based on a statistical system that is being used differently from how it was designed. Yet despite these problems there are lobbying and advocacy groups who are supporting using this system for hiring, firing and compensation decisions for teachers; despite the knowledge that using such systems can lack of fairness and possess questionable statistical appropriateness.

Who do our policy makers listen to? Experts disagree with a large part of our teacher evaluation system. Maybe more people should question their state representatives, school board members and other policymakers regarding statistically questionable use of TVAAS data.

The complete ASA statement can be found at http://www.amstat.org/policy/pdfs/ASA_VAM_Statement.pdf

No comments yet»

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: