Value Added Model Policy Analysis and Review

Value Added

Model

Application and Ethical Principles Reflection

Public Policy Implications

The higher duty of the Florida SCIG actions was in theory to create a statewide evaluation system that accurately improves or maintains educators’ highly qualified professional performance. But, the implementation of the VAM scientific theoretic model was totally unsuccessful due to the high frequency in error rates.

If moral duties are in conflict, what is the higher duty?

Is this conduct consistent with the norms of justice?

Consider the possible public policy implications, should the recommended behavior be prescribed through legislation or specific regulations?

Critical Normative Evaluation:

Intuition

The participants in this group analysis totally support the universalism of using a professional performance appraisal system to measure teacher effectiveness (Mintrom, 2012). However, we are not totally convinced that the VAM measurement is the correct rating scale to ethically evaluate educators. When 58% of teachers’ performance scores are reduced tremendously as a result of using the VAM, then policy makers, state representatives, and educators must revisit this unfair process to establish the problems of this crisis. Then, consider the best formula, tool, or variables to use in order to accurately evaluate educators’ effectiveness and salvage quality education. Moreover, with so many teachers being misrepresented, then the probability of Type I and II error rated need to reexamined in order to prevent or reduce this occurrence. Florida should also consider some of the factors or variables that were excluded from the original covariate VAM assessment in order to increase accuracy in teacher effectiveness. Some of the additional factors or variables that should be considered are students in Response to Intervention programs, school grades, availability of resources, coursework complexity, and school discipline, and referral history Florida Department of Education, 2014).

What are your first impressions or thoughtful

reactions to the ethical issues triggered by the

case – in other words, what does your moral

intuition say about the action or policy under

consideration:

Is it right or wrong?

Consider the issues from the viewpoint of ethical

theory and review the following questions in order

to develop a coherent rational that defense your

normative conclusion about the case:

Does the action (or policy) optimize the consequences

and generate the greatest net expectable utility for all

parties involved?

Does this course of action violate any ethical duties or

infringe upon any basic human rights?

At present, this VAM measurement challenges the integrity of the professional rating system, and the competency level of the individuals who agreed to implement it in the public school system. It seems that members of the SCIG committee knew that this appraisal system had so many errors and opted to still implement it, in order to have a policy that qualified for to the Race to the Top funding.

American Institute for Research (2011). Florida value added model technical report. Retrieved from: www.fldoe.org

Betts, J. R., Zau, A. C. and Rice, L.R. (2003). Determinants of student achievement: New evidence from San Diego. Public Policy Institute of California. Retrieved from: http://www.ppic.org/content/pubs/rb/RB_803JBRB.pdf

Braun, H. (2005). Using student progress to evaluate teachers: A primer on value-added model, Policy Information Perspective. Educational Testing Service, Princeton, NJ. Retrieved from: http://search.ets.org/researcher/query.html?col=all&ql=&qt=Braun%2C+H.+%282005%29.+Using+student+progress+to+evaluate+teachers%3A+A+primer+on+value-added+model.+++Policy+Information+Perspective.+

First Coast News (2014). Grading Florida’s teachers and schools. Retrieved from: http://www.firstcoastnews.com/story/news/education/2014/02/24/florida-times-union-teacher-effectiveness-scores-vam/5782275/

Kourkounis, E. (2014). How does the VAM formula measure teachers? Retrieved from: http://tbo.com/news/education/how-does-the-vam-formula-measure-teachers-2014030

O’Connor, M. K. & Netting, F. E. (2011). Analyzing social policy: Multiple perspectives for critically understanding and evaluating policy. John Wiley and Sons, Hoboken, NJ.

Rose, R., Henry, G., Lauren, D., and Carolina Institute for Public Policy. (2012). Comparing

value added models for estimating teacher effectiveness. Retrieved from publicpolicy.unc.edu/files/2014/02/EE_VAM_Briefing_2-7-12.pdf

Sass, T. R., Semykina, A. & Harris, D. (2014). Value added model and the measurement of teacher productivity. Economics of Education Review, 38, 9-23. Doi:10.1016/j.econedurev.2013.10.003.

Strauss, V. (2014). The forgotten factor in student achievement: the student. Retrieved from: http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/02/25/the-forgotten-factor-in-student-achievement-the-student/

The Florida Senate (2011). CS/CS/SB 736- Educational Personnel. Retrieved from: http://www.flsenate.gov/Committees/BillSummaries/2011/html/0736ED

US Department of Education (2009). Race to the Top: Executive Summary. Retrieved from: www.ed.gov

Conclusion and Recommendations

Conclusion

Recommendations

1. policy should have placed equal emphasis on students’ learning abilities, external factors, and teachers’ ability to make learning gains on standardized assessments.

2. taken into consideration parental involvement

3. account for the lack of resources for some students to receive academic enrichment programs needed to increase students’ academic skills and to improve performance on standardized assessments.

4. provide clarity to the stakeholders as to what the VAM is truly assessing and delineate those goals so that teachers could have an opportunity to consciously work towards those goals.

5. Non-core content teachers should be scored based on core content teachers' scores.

There are many factors that can negatively affect students’ performance on standardized assessment, and policy makers and school district officials must consider all possible parameters needed to successfully improve students’ overall performance on standardized assessments.

Data Revealed

Discussions

Findings

Concerns of Policy:

factors that play a role in a student's poor performance on a standardized test:

family related issues that distract them from focusing

learning disabilities

language barriers.

Non-core content teachers receive the school's overall score by default and not student performance. As a result of this, several highly qualified teachers leave low performing schools so that their professional evaluation is not smeared.

An additional critique of the VAM was that it claimed to “level the playing field” for teachers (Florida Department of Education, n.d). Teacher would receive only 50% of their ratings from the VAM model, realizing that teachers are not in direct control of everything that happens to students (FLDOE, n.d.). In actuality the VAM model even though recognizes the fact that teachers cannot really be held solely responsible for a child’s learning, does hold teachers accountable for the success of each student based on the single growth from one year to another.

Interviews with teachers and administrators:

1. lacked clear understanding of what the VAM assessed and what components were being evaluated in the model.

2. were evaded by the State of Florida Education Board representatives when asked to clarify VAM by referring teachers and administrators to state documents that was either extremely complicated in presentation of the model, or repeated the process of determining the model to use.

Concerns also arose as to if these variables aided in describing an effective teacher. The original intent of the policy was to use the Value-Added Model to measure the effectiveness of teachers. Unfortunately, because various model defined effectiveness differently, it opened the room for criticism.

Risk Assessment

Analysis Strategy

Analysis

&

Risk Assessment

The rational approach analysis of Holcomb and Nightingale’s Implementation Analysis Model (HNIAM) (O’Connor & Netting, 2011). The analysis focuses on the policy action and performance in an effort to strengthen the policy:

• major goals and assumptions of the policy operationalization and service delivery
• key management functions
• client activities

contextual factors that influence the policy (O’Connor & Netting, 2011).

1) Inadequate review of the VAM models in comparison to the VAM model selected by the State of Florida.

2) Limited access to key people who were influential in creating the policy

3) Limited time constraints for the group members

4) Lack of communication in the research/data collection of the information

5) Not being able to create a formula that would address the goal of the policy

References

Background

Introduction

Policy

Introduction

&

Background

Senate Bill 736 Valued Added Model Policy : Places sole responsibility of students’ academic success and performance on standardized assessments on the classroom teachers’ abilities and skills.

Problem: Policy fails to equate external or other factors and variables that may negatively affect students’ academic skills and performance on standardized assessments.

Reason for creating and implementing policy:

Address the second goal of the educational reform in the American Recovery and Reinvestment Act (ARRA):

build data systems that measure student growth and success and inform teachers and principals of the ways they could improve instruction.

The VAM was created to evaluate teacher performance using statistical models that measure a student’s learning growth and thereby attribute this growth to the teachers involved (American Institute for Research, 2011).

American Institute for Research (2011). Florida value added model technical report. Retrieved from: www.fldoe.org

Betts, J. R., Zau, A. C. and Rice, L.R. (2003). Determinants of student achievement: New evidence from San Diego. Public Policy Institute of California. Retrieved from: http://www.ppic.org/content/pubs/rb/RB_803JBRB.pdf

Braun, H. (2005). Using student progress to evaluate teachers: A primer on value-added model, Policy Information Perspective. Educational Testing Service, Princeton, NJ. Retrieved from: http://search.ets.org/researcher/query.html?col=all&ql=&qt=Braun%2C+H.+%282005%29.+Using+student+progress+to+evaluate+teachers%3A+A+primer+on+value-added+model.+++Policy+Information+Perspective.+

First Coast News (2014). Grading Florida’s teachers and schools. Retrieved from: http://www.firstcoastnews.com/story/news/education/2014/02/24/florida-times-union-teacher-effectiveness-scores-vam/5782275/

Kourkounis, E. (2014). How does the VAM formula measure teachers? Retrieved from: http://tbo.com/news/education/how-does-the-vam-formula-measure-teachers-2014030

O’Connor, M. K. & Netting, F. E. (2011). Analyzing social policy: Multiple perspectives for critically understanding and evaluating policy. John Wiley and Sons, Hoboken, NJ.

Sass, T. R., Semykina, A. & Harris, D. (2014). Value added model and the measurement of teacher productivity. Economics of Education Review, 38, 9-23. Doi:10.1016/j.econedurev.2013.10.003.

Strauss, V. (2014). The forgotten factor in student achievement: the student. Retrieved from: http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/02/25/the-forgotten-factor-in-student-achievement-the-student/

The Florida Senate (2011). CS/CS/SB 736- Educational Personnel. Retrieved from: http://www.flsenate.gov/Committees/BillSummaries/2011/html/0736ED

US Department of Education (2009). Race to the Top: Executive Summary. Retrieved from: www.ed.gov