rsleediscuss

Just another WordPress.com site

Archive for August, 2014

A column all Tennesseans should read

Dave Cook’s column in this morning’s Chattanooga Times-Free Press is a worthwhile read for anyone interested in education and education policy in Tennessee. Check it out at: http://www.timesfreepress.com/news/2014/aug/13/education-movement-will-not-be-tcaped/?opinioncolumns

Using and Misusing Value Added Testing in Tennessee

A recent position paper published by The American Statistical Association (the largest professional organization for professional statisticians) questions the use of value added assessments to make personnel and other high stakes decisions. This is important for Tennesseans because Tennessee teachers have been subjected to such questionable use of TVAAS (Tennessee Value Added Assessment System) data since 2010.

First, a little about how TVAAS works. TVAAS uses a group of individual students’ past test scores to project estimated future test scores for the same group of students. So, every student who is starting school this month has a projected score of where the statistical model projects the average score to be at the end of the school year, thus measuring their “growth” by the end of the school year. For teacher evaluation in Tennessee if a class of students score above this projection a teacher is judged to have done a better than average job because the students improved more than the estimated projection. Should the students score lower than projected the teacher is judged to have done a subpar job. One might think that using TVAAS is simple as using a baseball batting average.

In fact, Tennessee has been using the TVAAS system for over 20 years and the system provides useful data since it shows much more than what percentage of students are proficient on testing standards from year to year. So, when used correctly an academic team, school or school district can tell if reform initiatives or program improvements are working (and how well they might be working or not working). For example, say a middle school wants to implement a new reading program, but they want to see if it works before trying it school-wide. So the school implements it in the 7th grade and the school decides to use their year-end TCAP tests to determine if the program worked. If the teachers and principal only look at the percentage of students who score proficient at the end of this year compared to the percentage of students scoring proficient at the end of last year in the 7th grade, then they can’t really tell if the program worked because they compared two different groups of students at different times. But if they have a projected score and an actual score on the same group of students, then they could more easily determine if their new reading program worked.

The problem is that value added assessments, like TVAAS, are designed to make programming decisions like in the example above. Value added systems are not designed to make personnel or teacher effectiveness decisions. According to the American Statistical Association, value added testing systems, “typically measure correlation, not causation: Effects – positive or negative – attributed to a teacher may actually be caused by other factors not included in the model”. In other words a value added system is unable to separate out teacher performance from all the factors (such as student motivation, if parents have hired tutors, if there has been a high level of student transience during the year, or other factors) that affect student achievement.

This creates a problem for teachers and administrators in Tennessee. One-third of the annual teacher evaluation is based on a statistical system that is being used differently from how it was designed. Yet despite these problems there are lobbying and advocacy groups who are supporting using this system for hiring, firing and compensation decisions for teachers; despite the knowledge that using such systems can lack of fairness and possess questionable statistical appropriateness.

Who do our policy makers listen to? Experts disagree with a large part of our teacher evaluation system. Maybe more people should question their state representatives, school board members and other policymakers regarding statistically questionable use of TVAAS data.

The complete ASA statement can be found at http://www.amstat.org/policy/pdfs/ASA_VAM_Statement.pdf

Effective teaching means developing relationships

Last Sunday’s Parade (August 3, 2014) magazine had an interesting cover story, “What Makes a Great Teacher”. Usually pieces like this make me groan with worry because often they are just fluff pieces and sometimes even have mis-information for parents or the community. Happily this article was actually quite good, particularly focusing on how good teachers often use student mistakes and/or misunderstanding to help them learn.

I think this would be a good start for learning more about what professional teachers do and how students learn, but only a start. What is missing, although hinted at, is the important nature of how teachers develop relationships with their students. One of the most difficult tasks teachers really face is how to motivate students to persist in completing quality, thorough work and how to help students risk making mistakes in front of other students. Creating such a relationship oriented environment takes more than awareness-it takes work every day. Some questions to ask about how your schools work might be:

Does the administration provide teachers the level of support necessary to develop such a relationship oriented environment?

Do teachers focus on helping work through mistakes or just point them out?

Are there structural barriers that interfere with relationship development (such as teachers who see 100-200 different students every day)?