Does teacher preparation affect student achievement? - The Maxwell ...

0 downloads 213 Views 340KB Size Report
Feb 7, 2011 - content knowledge and/or instructional methods. Thus ..... for teachers holding masters degrees have been
Does teacher preparation affect student achievement?



Gary T. Henry, Department of Public Policy & Carolina Institute for Public Policy, University of North Carolina at Chapel Hill Charles L. Thompson, College of Education, East Carolina University & Carolina Institute for Public Policy, University of North Carolina at Chapel Hill Kevin C. Bastian, Department of Public Policy & Carolina Institute for Public Policy, University of North Carolina at Chapel Hill C. Kevin Fortner, Carolina Institute for Public Policy, University of North Carolina at Chapel Hill David C. Kershaw, Carolina Institute for Public Policy, University of North Carolina at Chapel Hill Kelly M. Purtell, Population Research Center & Department of Human Development and Family Sciences, University of Texas at Austin Rebecca A. Zulli, Carolina Institute for Public Policy, University of North Carolina at Chapel Hill Corresponding author: Gary T. Henry, MacRae Professor of Public Policy Carolina Institute for Public Policy and Department of Public Policy University of North Carolina at Chapel Hill 122 Abernethy Hall, CB# 3435 Chapel Hill, NC 27599‐3435 919.962.6694 (voice) 919.962.5824 (fax) [email protected]



Manuscript submitted to Education Finance and Policy, February 7, 2011.

The authors are grateful for comments and advice provided by Alisa Chapman, Alan Mabe, Ashu Handa, Doug Lauen and the deans of colleges and schools of education in North Carolina; and assistance from Jade Marcus, Adrienne Smith, Elizabeth D’Amico, and Rachel Ramsay. This research was funded in part by the University of North Carolina General Administration Teacher Quality Research Initiative.

Does teacher preparation affect student achievement? One interpretation of recent research on teacher effectiveness is that teachers matter but teacher preparation does not. However, some comparisons that have been interpreted to pertain to teacher preparation have compared traditional and alternatively certified teachers, which reflects a teacher’s status at a particular point in their career, rather than the preparation they received prior to beginning teaching. In this study, we estimate the differences in adjusted average test score gains of students taught by teachers who entered teaching from twelve distinct “portals,” which are combinations of formal education and other preparation to teach. The main findings are: teachers prepared in out of state undergraduate programs are less effective than teachers prepared in the public institutions within the state in five of 11 comparisons, including elementary mathematics and reading, where out of state teachers constitute the largest source of teachers; lateral entry teachers are less effective in three of 11 comparisons, including high school overall, where they are the largest source of teachers, high school social studies and high school science; Teach For America corps members are more effective in five of nine reliable comparisons, including high school overall, mathematics, science, and English 1 as well as middle school mathematics.

1

1. Introduction One stylized but common interpretation of current research on teacher effectiveness is that teachers matter but teacher preparation does not. For example, Gordon, Kane, and Staiger (2006) examined recent research on the teacher labor force and recommended that entry into teaching be opened to anyone with a college degree and requisite subject matter knowledge. They further recommended that tenure be granted on the basis of teachers’ effectiveness in producing test score gains in their first two years on the job. This could reduce barriers into teaching, but would risk opening the door to more individuals who are unprepared to teach. In fact, the barriers into public

school classrooms have been substantially reduced in recent years (Boyd et al. 2009). In North Carolina, for example, 14% of the public school teachers in 2007‐08 first entered the classroom as lateral entry or alternatively licensed teachers, without the coursework or practice teaching required for certification (authors’ analysis). Clearly, the Gordon, Kane and Staiger recommendation is based on a premise that on the job training, combined with actual experience teaching, is as effective as requiring specialized preparation to teach prior to entering the classroom. However, on this point, limited evidence exists on the relative effectiveness of different types of teacher preparation, and the question of how teachers’ preparation relates to their effectiveness in the classroom remains open (NRC 2010). The limited evidence that is available suggests that there is more variation in student test score gains within the categories typically used to classify teachers’ preparation than between these categories (NRC 2010). This has been interpreted to mean that variables other than teacher preparation have greater influence on teachers’ effectiveness. But an equally plausible alternative explanation is that the categories of teacher preparation have been too broad or otherwise mis‐ measured. The numerous pathways into teaching, including some 130 alternative pathways documented in the NRC review (2010), have been frequently lumped into two types – (1) regular (also known as traditional) certification and (2) alternative certification or lateral entry (NRC

2

2010). Unfortunately, these categories often represent teachers’ current status rather than their preparation prior to entering the classroom, and are more fluid than is apparent at first glance. For example, in most states lateral entry teachers are reclassified as fully certified teachers if they remain in teaching for three or four years, complete the required coursework, and pass exams in content knowledge and/or instructional methods. Thus, two teachers with very different preparation at entry into teaching can be classified into the same category based on their

experience and the amount of coursework completed after entering teaching. Therefore, studies that rely on certification status at some point in time after teachers begin teaching rather than initial preparation may mask differences in effectiveness related to pre‐service preparation. In this study, we examine the relative effectiveness of teachers who entered public school classrooms with six distinct types of preparation that could be classified as traditional and five types of preparation that could be classified as alternative. For example, we separate teachers who were prepared out‐of‐state from teachers who were prepared within the state. The reasoning behind this distinction is that teachers from out‐of‐state are (1) less likely to have been prepared to teach to the standards and objectives for specific grades and content domains of the state in which they are teaching, especially in elementary and middle grades; (2) less likely to be familiar with the types of schools and students they encounter in different states, especially if the states are in different regions of the country; and (3) given the evidence about the preferences of teachers to work close to home (Boyd et al. 2005), less likely to have been able to find a teaching position in the state where they were prepared, and therefore, on average, less able as teachers. In addition, we separately assess teachers at each of the three grade levels—elementary, middle, and high—on areas of tested content, including reading and mathematics, and where available, science and social studies, because there is good reason to hypothesize that teachers with different types of preparation may do better (or worse) in some grades and subjects than others. For example, out‐ of‐state teachers may be less effective in elementary and middle grades where content objectives

3

by grade differ from state to state, but equally effective in high school algebra or biology, where the standards and objectives may be more similar across states.

We next review selected research in light of four major methodological challenges that must

be surmounted in order to link teacher preparation and student achievement convincingly. Then we detail the classification scheme that we used to classify teachers by initial preparation (Section 3). In section 4, we lay out our findings, and in section 5 we present our conclusions. 2. Review of Teacher Preparation Effectiveness Research Literature

A. Brief Summary of Prior Studies. Decades of research on education policy converge to a

central point: teachers exert great influence on student learning. In fact, consensus exists that teacher quality is the most important school‐level factor explaining variation in student achievement (Coleman et al. 1966; Rockoff 2004; Clotfelter, Ladd, and Vigdor 2007). Findings concerning the efficacy of teacher preparation and certification, however, have sparked contentious debates between those who champion traditional teacher preparation and certification (Darling‐ Hammond, Berry, and Thoreson 2001) and those who stress the need for alternative policies to attract abler people into teaching, meet demand, and increase diversity (Gordon, Kane, and Staiger 2006).

With selected research both sides can find empirical support. For example, most studies

find advantages accruing to regular or standard certification, as compared to non‐certified, alternative, or other teachers (Goldhaber and Brewer 2000; Clotfelter, Ladd, and Vigdor 2007, 2010; Boyd et al. 2006; Kane, Rockoff, and Staiger 2008). The effects are small, however, and appear to be both concentrated early in teachers’ careers and differ across grade levels (Clotfelter, Ladd, and Vigdor 2010; Boyd et al. 2006). More importantly, research documents greater variability in teacher quality within certification types than between them, a finding which is often

4

interpreted to mean that factors outside the teachers’ certification status are responsible for most of the differences in teachers’ effectiveness. This combination of conflicting findings and great variability within the categories of certification commonly employed, suggests that some important methodological and conceptual issues have not been fully resolved in prior research. The most important of these may be expressed in terms of four major questions: 1) How should teachers be classified into pre‐service preparation categories? 2) Should the sample of teachers be restricted to those who more recently entered teaching or include all teachers without regard to their years of experience? 3) What is the appropriate methodology to accurately estimate teacher preparation effects? 4) From what population is the sample of teachers drawn, geographically and across grade levels and subjects?

An examination of prior studies’ tacit or explicit responses to these questions will help us understand their strengths and limitations, as well as to set the context for interpreting the contributions of the current study. See Table 1 for a summary of the most pertinent prior research [Insert Table 1 about here]

B. How should teachers be classified into preparation categories? This first question poses

three distinct conceptual and operational challenges for research on teacher effectiveness: a) the need to establish consistent meaning for the categories used to classify teachers by type of preparation; b) the difference between teacher preparation variables that change over time (status variables) and those that are fixed (basis variables), and c) the great variety of teacher preparation that currently exists. Partly because of data limitations, but also because the field is still in an early

5

phase of development, prior research, as detailed below, has left room for improving teacher preparation categorization. In a path‐breaking but controversial study, Goldhaber and Brewer (2000) relied on self‐ reports by a national sample of teachers to estimate the effects of certification for high school math and science instructors. Teachers reported their certification status in five categories—standard, probationary, emergency, private, and non‐certified—but several issues were problematic. First, the definitions for each certification type may have varied due to differences in certification requirements among the states. Second, the classifications were based on the teachers’ report of their status at a particular point, not the type of preparation that they had received before entering the classroom. Finally, even at a given time, a teacher might have different statuses with respect to different subjects or grade levels. For example, a teacher could possess standard certification in science, but be teaching with an emergency certification in mathematics (Goldhaber and Brewer 2000). The overlap of teacher preparation experiences among these categories was further illustrated by Darling‐Hammond and colleagues’ analysis showing that many of the emergency‐ certified teachers shared characteristics, such as comparable content course preparation, with standard certification instructors (Darling‐Hammond, Berry, and Thoreson 2001).

More recent work by Clotfelter and colleagues in North Carolina elementary and high

schools classified teachers according to their licensure status into regular, lateral entry, or “other” categories (Clotfelter, Ladd, and Vigdor 2007, 2010). This scheme improved on past work by including a separate category for teachers who had been classified as lateral entry at some point within the time period from which the authors’ data were drawn, but currently held regular licenses. These studies also included other measures relevant to teachers’ preparation, including the competitiveness of their undergraduate institutions and whether they were certified to teach the subject matter of the course. Despite these improvements, the studies did not distinguish

6

among the types of preparation that teachers received prior to beginning teaching. For example,

formerly emergency and lateral entry teachers were included in the regular certification category.

Finally, two studies in New York City by Boyd and colleagues and Kane and colleagues

advanced the measurement of teacher preparation further, using an improved classification scheme. The Boyd et al. classification scheme divided teachers into six categories—college recommended, individual evaluation, New York City Teaching Fellows, Teach For America, temporary license, and other (Boyd et al. 2006). Kane, Rockoff, and Staiger (2008) included five groups—standard, New York City Teaching Fellows, Teach For America, international, and uncertified. Each of these studies treated teacher preparation/certification as a fixed trait, anchored to the teacher’s initial status upon first being hired in New York State (Boyd et al. 2006) or New York City (Kane, Rockoff, and Staiger 2008). Importantly, this means teachers were consistently classified over time. Furthermore, these papers recognized the current diversity in teacher preparation by using more groups in their classification scheme. Nonetheless, a still finer teacher preparation classification system may lead to more accurate distinctions between teachers that will allow researchers to ascertain differences in the effects of preparation. For instance, teachers recommended by a university for licensure ‐‐ a category used by Boyd et al. (2006) ‐‐ may include those who simultaneously obtain an undergraduate degree and complete licensure requirements along with those who come back to a given institution to complete licensure requirements some time after graduating with a degree in fields such as business administration or in the arts and sciences from another institution. Further, the teacher’s classification upon hiring in one jurisdiction may potentially differ from her original preparation/certification classification if she held a prior teaching job elsewhere.

C. Should the sample of teachers be restricted to those who entered teaching recently or

include all teachers without regard to years of experience? In addition to classifying each teacher’s preparation for entering the classroom, researchers must choose the experience levels of teachers

7

to include in their studies. This decision needs to strike a balance between allowing researchers to accumulate enough data to reliably estimate teachers’ effectiveness and reducing the amount of statistical noise produced by on the job learning. The research by Goldhaber and Brewer and Clotfelter et al. included all teachers teaching during the period of analysis (Goldhaber and Brewer

2000; Clotfelter, Ladd, and Vigdor 2007, 2010). An advantage of this approach is that the analytic sample included teachers with a full range of experience, reflecting the actual teaching force at the time of the study. However, the influence of colleagues (Jackson and Brugemann 2009), formal and informal professional development, and trial and error experiences within the classroom may significantly reduce the influence of teacher preparation that occurred 10 or 20 years earlier.

Boyd and colleagues took an important step in their analysis by including all teachers,

regardless of experience, but reported separate analyses on teachers early in their careers. Multiple analyses presented results for teachers in their first, second, and third year of experience separately, when effects of preparation are hypothesized to be most salient (Boyd et al. 2006). Like Boyd et al., Kane and colleagues focused their analyses and conclusions primarily on novice teachers in their first three years in the classroom, although their sample was difficult to precisely ascertain (a dichotomous variable for teachers hired before the 1999‐2000 school year was included in their analytical models) (Kane, Rockoff, Staiger 2008). Finally, in another study, Xu, Hannaway, and Taylor (2007) compared the performance of Teach For America corps members to both new and experienced high school teachers in the districts where TFA corps members served, and found that TFA teachers outperformed the other teachers in both types of comparisons. Therefore, as we considered the sample for this research, the decision about the range of teachers’ experience to include needed to balance the potential for the influence of teacher preparation to decay over time with the need to have sufficient power to detect effects on the educator workforce, which we will describe for this study in Section 3.

8

D. What is the appropriate methodology to accurately estimate teacher preparation effects? The third issue is how to analyze the data to produce unbiased or consistent estimates of the effects of teacher preparation. Prior research shows that teachers sort across and within districts based on observed characteristics (Lankford, Loeb, and Wyckoff 2002), and that students are not randomly assigned to classrooms (Clotfelter, Ladd, and Vigdor 2005). Therefore, studies attempting to estimate the effects of teacher preparation on student achievement face the same problem: how to control for variables that influence student performance that may not be balanced among the types of students, classes, and schools in which individuals with different types of teacher preparation experiences teach. Any modeling choice will require assumptions to infer causality and consider the effect estimates unbiased (Reardon and Raudenbush 2009). The prior studies vary (1) in their use of value added models that estimate year‐to‐year gains in test scores or models that estimate the achievement level of students at a particular time, which are known as “levels models” and (2) whether they estimated the effects with ordinary least squares regression or student and school fixed effects, which use only within‐student or within‐ school variance. Goldhaber and Brewer relied on a value‐added regression model, which they point out did not fully control for selection, because (Goldhaber and Brewer 2000) the value added models only eliminate bias if all of the sorting of students and teachers was based on the students’ prior test scores. However, that is not the only type of sorting that goes on in U.S. schools (Clotfelter, Ladd, and Vigdor 2005; Rothstein 2010). To control for selection, all the remaining articles reviewed here employed either student or school fixed effects (Wooldridge 2009). Student fixed effects estimate the effects of teacher preparation on differences in student’s test score gains for students who experience teachers with different types of teacher preparation. Essentially, this uses each student as his or her own control, thereby eliminating the influence of unobserved differences between students from the estimates. An important implication of using student fixed effects, however, is that the coefficient on the binary variable indicating whether a teacher was

9

prepared through a particular preparation program or portal into teaching (for example, Teach For America) is identified only on the sub‐sample of students who have had teachers from that preparation portal and the reference group, thereby omitting students who have been taught by only one type of teacher from the estimates (Henry and Purtell 2010). School fixed effects estimate the effects of teachers with different types on preparation within schools that have teachers with different types of preparation, essentially forcing estimates of teachers’ effectiveness to be relative to teachers within the same schools, rather than relative to the entire sample of teachers included in the data. Clotfelter et al. used student fixed effects within a gains model at the elementary school level (Clotfelter, Ladd, and Vigdor 2007) and student fixed effects within a levels model at the high

school level (Clotfelter, Ladd, and Vigdor 2010). That is, Clotfelter et al. (2010) omitted students’ prior scores in the models estimating the effects of different types of teachers on students’ content specific End‐of‐Course tests, because student fixed effect models do not allow student variables that do not vary across time, including test scores and other characteristics such as ethnicity, gender, and poverty status. Finally, Boyd et al. employed school or school by grade by year fixed effects within a value‐added model, which corrected for teacher‐level clustering (Boyd et al. 2006), and Kane et al. relied on grade and year fixed effects with cluster adjustments at the school level. Recent methodological research has revealed issues with models that rely on covariate adjustments, fixed effects, and random effects (Rothstein 2010) but that using rich sets of covariates and multiple prior test scores mitigates many of the issues (Koedel and Betts 2009). Our use of extensive covariates and multiple test scores when available is described in Section 3.

E. From what population is the sample of teachers drawn, geographically and across grade

levels and subjects? The samples analyzed for a particular study should be important in their own right, but are also important in reasoning through the extent to which the findings can be reasonably extrapolated to other places, grades, or subjects. The recent popularity of administrative data sets in education policy studies presents numerous advantages in the quality of

10

the data, but restricts analysis to particular states or local school districts and tested subjects and

grade levels. While studies drawing their samples from New York City or North Carolina provide useful information about those locations, it is important to consider whether similar findings would be expected in other states or districts. National probability sample studies can create issues as well. The nationwide NELS dataset used by Goldhaber and Brewer (2000) was criticized for the lack of consistency in certification requirements that formed the basis of their classification scheme. Therefore, these issues of geographic scope cannot be resolved other than through replication in other states and districts, and eventually, larger scale, multi‐state studies that combine administrative and survey data may be required to provide findings that are more credibly extrapolated beyond the specific study population. The effect of teacher preparation across grades and subjects raises other issues. As is shown in Table 1, collectively, the prior studies of different types of teacher preparation have focused on all three levels of schooling and three content areas: mathematics, science, and reading/language arts. The differences that have been found between traditional and alternatively licensed teachers can reasonably be expected to be different by subject and grade level. As previously mentioned, out‐of‐state teachers might be expected to do better where content standards are more similar state‐to‐state and worse where the standards vary by state, such as in elementary reading and mathematics. Lateral entry teachers, for example, might perform comparably to college recommended teachers at one level, yet be outperformed at another. All the studies discussed within this review either analyzed one level of schooling (Goldhaber and Brewer 2000; Clotfelter, Ladd, and Vigdor 2007, 2010) or two levels (Boyd et al. 2006; Kane, Rockoff, and Staiger 2008). However, because of the previously discussed differences in identification and estimation strategies, it is impossible to discern if differences among levels and subjects is due to actual differences or differences in the methodological approaches. There are significant policy conclusions to be drawn if teachers prepared in traditional programs excel in certain subjects and

11

grades, for instance elementary reading, but not in others. This knowledge could permit refinement of licensure/certification systems and hiring decisions. In the next section, we present our research questions with specific hypotheses. Research questions and hypotheses. In this study, we examine the effects of different types of teacher preparation, labeled as “portals,” on the achievement of students in elementary, middle and high schools. Portals are meant to convey the level of formal education and specific teacher preparation an individual received prior to beginning teaching. By specific teacher preparation we refer to experiences, such as clinical or student teaching and induction, orientation, or other training programs that instruct pre‐service teachers on the knowledge, skills, or dispositions expected to be helpful in the classroom. The central guiding research question is: How do the test score gains of students taught by teachers who have entered teaching though different portals compare to the gains of those taught by teachers who were traditionally prepared by in‐state public institutions, controlling for a wide variety of student, classroom, and school characteristics?

Drawing on prior research and additional, sometimes anecdotal, information about the

teacher preparation portals, we expect the following: 1. Alternative entry teachers entering classrooms without completing the specific teaching preparation requirements will perform worse than those from traditional in‐state public institutions. These teachers have not invested in the specific human capital to prepare themselves to teach prior to beginning teaching, are taking coursework to meet specific teacher preparation requirements while teaching fulltime, and prior research indicates that these teachers are often less effective than certified teachers (Clotfelter, Ladd, and Vigdor 2010). 2. Teachers entering classrooms selected and prepared by the Teach For America program will perform better than those from in‐state public institutions. These teachers have been selected from a highly‐ motivated, high achieving group of recent college graduates, receive

12

substantial support from the TFA organization, and prior research indicates that they tend to outperform other teachers (Boyd et al. 2006; Xu, Hannaway, and Taylor 2007). 3. Teachers entering classrooms from out‐of‐state (both undergraduate and graduate portals, will perform worse than those from in‐state public institutions, especially in elementary and middle grades. As stated earlier, teachers from out‐of‐state are likely to be (1) less familiar with the learning objectives for specific grades and content domains of the state in which they are teaching, especially in elementary and middle grades; (2) less familiar with the types of schools and students they encounter in different states, especially if the states are in different regions of the country; and (3) less skilled than teachers who were able to find jobs in their states where they were prepared. 4. Teachers entering classrooms through the Visiting International Faculty program will perform worse than those from in‐state public institutions for reasons similar to those offered for teachers from the out‐of‐state portals. 5. Teachers entering classrooms from traditional programs at in‐state private institutions will perform similarly to those from traditional in‐state public institutions. Private institutions orient their programs to preparing teachers in the state in which they are located and are likely to provide similar preparation in terms of courses and student teaching as their in‐ state public counterparts. 6. Teachers entering classrooms from in‐state public and private institutions with graduate degrees will perform similarly to those from in‐state public institutions. Most of these teachers are in Master of Arts in Teaching (MAT) or similar programs, and few advantages for teachers holding masters degrees have been found in prior research. 7. Teachers entering classrooms completing specific teacher preparation licensure requirements after completing their undergraduate degrees but before entering teaching will perform similarly to those from in‐state public institutions. These “licensure only”

13

teachers have returned to college to take the courses they need to be certified as teachers prior to beginning teaching. These teachers have invested in specific human capital and have participated in student teaching and other in‐school activities that prior research suggests can positively affect performance (Boyd et al. 2009).

In the next section, we describe the classification system, data, and methods used for this study. 3. On Portals, Data and Methods

The main objective for this study is to estimate the average effect of different types of

teacher preparation on student achievement in mathematics and reading/language arts and where available, science and social studies separately for elementary, middle, and secondary grades. This required developing a classification system for teacher preparation, applying that system to all teachers in the data base, matching students to their teachers by grade and subject or course, merging student test scores and characteristics into the files, and estimating the effects.

In each of these steps, we have been able to make significant advances in some of the

methods employed in the prior research, but cannot claim that all bias has been removed from the estimates, or that the findings can be directly extrapolated to other states or school districts. Therefore, we concentrate on describing the advances which we have made over some of the previous research on the effects of teacher preparation.

Most methodologists agree that random assignment of students and teachers to classrooms

would be required to estimate the effects of teacher preparation without bias. In addition, it would also be required to randomize assignment into “treatments”, which are the preparation programs, which seems for obvious, practical reasons infeasible. However, random assignment is only appropriate for looking at interventions that have been “deliberately manipulated” (Cook et al. 2010). The goal for this study is not to “deliberately manipulate” the assignment of teachers with different types of preparation in order to estimate the effects. Rather, it is our purpose, as well as

14

the express purpose of the prior research on this topic, to provide quantitative estimates of the

differences that teacher preparation makes in student achievement as the labor market and assignment patterns for teachers currently operate. It is difficult to develop covariate‐adjusted estimates that can be considered “negligibly” different from unbiased estimates, but it is important to continue building theory and evidence that will allow us to remove as much bias as possible from studies with these objectives. This is the type of evidence that is useful for informing opinions and decisions about teacher certification policies, including the relative risk involved in changing current policies. It is also useful for generating hypotheses and formulating designs for pilot interventions that could be tested with random assignment studies to develop unbiased estimates of the effects of deliberately changing existing policies.

In addition, we use a unique individual student level data set that we constructed from

administrative sources in North Carolina. While the context and therefore the extrapolation of the findings to other states is limited, we were able to use student‐teacher class rosters that allow us to precisely link students to their teachers by grade, subject and course. Other researchers who have data that can be similarly linked have argued that it is likely to improve the reliability of the estimates of teacher effectiveness (Harris and Sass 2008). Certainly, it allows a more precise estimate in elementary and middle schools where different teachers teach students different subjects and more than one teacher may be responsible for teaching students the same subject during the year. Most often this occurs with reading/language arts in our data set. To describe the study methods, we first turn to the classification system.

A. Classifying Teacher Preparation: Portals. The classification of teacher preparation can be

based on a teacher’s status at a point in time or on the basis that describes the combination of formal education and specific preparation to teach when she began teaching. We chose to classify teachers based on the latter and have labeled the different types of preparation as portals, which is

15

meant to convey the level of formal education and specific teacher preparation they had received prior to beginning teaching.

Four fundamental considerations structured our portal classification scheme. First, was the

teacher fully qualified – that is, had she met all requirements for licensure when she entered the classroom for the first time? Second, if so, was her qualification based on a set of education‐related courses taken in the process of earning an undergraduate degree (traditional) or only after earning an undergraduate degree (“licensure only”)? Third, what level of degree − undergraduate or graduate − did the individual hold when first entering a classroom? And finally, if fully qualified and degree‐holding, from what type of institution: one of the fifteen teacher preparation institutions within the University of North Carolina system, a North Carolina private college or university, or an out of state university – did the person earn the degree? We hypothesized that the three sets of institutions might offer different levels of exposure to North Carolina public schools, students within the state, and the state’s curriculum ‐‐ differences that could affect their graduates’ academic effectiveness. With these questions as a guide, we created twelve mutually exclusive categories that pinpointed the “portal” through which a person entered the teaching profession. A full listing of our twelve portal categories, where the first six represent traditional teacher preparation and the next five alternative certification, were as follows: In‐state Public Undergraduate Prepared, In‐state Public Graduate Prepared, In‐state Private Undergraduate Prepared, In‐state Private Graduate Prepared, Out of State Undergraduate Prepared, Out of State Graduate Prepared, In‐state Public Licensure Only, Other Licensure Only, Teach For America, Visiting International Faculty, Alternative Entry, and Unclassifiable. (See Table 2 for definitions of each portal). [Insert Table 2 about here]

16

In order to categorize North Carolina public school teachers into one of the portals listed above,

we relied on administrative datasets from three sources (See Table 2). First, institutional data from the University of North Carolina General Administration (UNCGA) that covered each of the constituent institutions identified in‐state public prepared teachers at the undergraduate, graduate, and licensure only level. Second, Teach For America provided us with data identifying their corps members in North Carolina. And third, we utilized the teacher education, licensure audit, and certified salary files from the North Carolina Department of Public Instruction (NCDPI). From these datasets we employed several key pieces of information to classify teachers into portals. First, we calculated the year an individual began teaching from the NCDPI certified salary file. Second, using the NCDPI licensure audit file, we identified the basis for a teacher’s original teaching license, which established the basis for her licensure when she started teaching in North Carolina. And last, using either the UNCGA graduated student data files or the NCDPI teacher education file, we determined an individual’s graduation year, degree type (undergraduate or graduate), and degree origin (in‐ state public institution, in‐state private university, or an out of state institution). If an individual earned multiple degrees prior to entering the classroom, we categorized her according to the one most proximate to beginning teaching. Putting all these variables together, we categorized 93 percent of North Carolina public school teachers into one and only one of these 11 portals (for more detail on the classification process, see authors 2010).

When data were not sufficient to classify a number of North Carolina teachers definitively

into one of the 11 portals described above, we placed individuals into an unclassifiable category. This happened in three situations: 1) they did not have a college graduation year in the datasets, 2) their highest degree earned prior to entering teaching was less than a Bachelor’s degree, or 3) administrative data recorded the person teaching more than one year prior to her graduation year. Fortunately, few of these teachers were part of the analysis sample, because they had been in the system longer than 5 years or were assigned to non‐tested grades, courses or subjects. We retained

17

this group as a portal, however, so that the student sample would not be affected (biased) by missing, incomplete, or potentially inaccurate data on these teachers. ‐‐ Figure 1 about here ‐‐ In Figure 1, we display the number of North Carolina teachers employed in 2007‐08 that came through each of the 12 portals. In total, the in‐state public university system supplied approximately 36,494 teachers, including those prepared through undergraduate programs (32%), graduate programs (3%), and licensure only programs (0.6%). In‐state private colleges and

universities supplied over 13,000 teachers, or about 13 percent of the workforce. Perhaps surprisingly, nearly 30,000 North Carolina teachers were prepared in another state, with 23 percent of the workforce coming from out of state with undergraduate degrees and going into NC classrooms. Over 16,000 teachers − more than 16 percent of the workforce − had originally begun teaching before completely meeting the state’s requirements for teacher licensure. Too little information was available to classify 7,685 teachers, many of whom had extensive experience or entered to teach career preparation courses. Clearly, the state has a great diversity in how its teachers were prepared, with teachers that were not fully qualified by the state regulations or were from out of state growing to about half of the teacher workforce. B. Study data. In addition to the data used to classify each teacher into her portal, data on students, teachers, and schools for the four year period, 2004‐05 through 2007‐08, were assembled for this study. We linked students and teachers using actual class rosters, which allowed us to match students to approximately 93% of individual instructors over the four‐year period. Also, we matched students’ test scores to their prior test scores, which allows us to estimate the additional learning or “value added” during each of the academic years being studied. Finally, numerous other

18

student, teacher, and school characteristics were merged into these files and used in the analysis to

adjust for factors other than teacher preparation that may affect student achievement. C. Study Observations. After matching student roster entries to prior test scores and merging in teacher characteristics based on roster matches, we limited the dataset to teachers with five years of experience or less in each year. The decision to limit analysis to teachers with five years or less experience is based on our desire to achieve a balance between (a) limiting the number of years of experience, with the expectation that as teachers gain more experience, their initial training becomes less important to their classroom behaviors, and (b) including several years of experience on the premise that teachers from different portals may develop their teaching prowess at different rates when they begin teaching. Removing teachers with more than five years of experience from the dataset left over 1.6 million complete test records for analysis. This analysis included over 900,000 students and 20,000 unique teachers over the four year time period across all analyzed grades. For the precise breakdown by level of schooling for tests, students, and teachers, see Appendix Table 1. D. Outcome Variables. Students’ current and prior test score performance are based on North Carolina assessment grade 3 pre‐test scores, End‐of‐Grade test scores, and End‐of‐Course scores standardized within subject, grade, and year. The assessments were developed using the standards and curriculum set by the North Carolina State Board of Education and using psychometric techniques that are commonly employed by states to implement federal and state accountability requirements. Elementary grades (3 – 5) models include test scores in reading and mathematics for the years 2005‐06 to 2007‐08. In middle grades (6 – 8), test score measures in reading, mathematics, and Algebra I were included for the years 2004‐05 to 2007‐08. Science End‐ of‐Grade (EOG) testing began in 2007‐08, and models were implemented for that school year only. In comparison to many states that fail to assess reading and mathematics during high school and/or only offer minimum proficiency high school graduation tests, a distinct advantage of North Carolina

19

high school data is that End‐of‐Course (EOC) tests can be linked back to specific teachers and their students. High school grades (9 – 12) analyses included test observations across all ten EOC tested subjects: English I, Algebra I, Algebra II, Geometry, Biology, Chemistry, Physical Science, Physics, US History, and Civics & Economics. English I and mathematics analyses are included across all four years, 2004‐05 to 2007‐08. Science courses were excluded from the analysis for 2005‐06 due to test piloting during that year, and social studies tests were included in analyses for years 2005‐06 to 2007‐08 due to test piloting in 2004‐05. Following Clotfelter et al. (2007, 2010) all test scores were standardized by year and grade to remove secular trends or other anomalies that might occur in the testing process from year to year. Standardization involves subtracting the statewide mean score on a test from a student’s actual (scale) score and dividing the remainder by the standard deviation of scores for that test in that year. Thus, a standardized score of zero on a test is equivalent to the average score for that subject in that year, and a student with standardized scores of zero in two successive years has gained in achievement as much as the average student. E. Covariates. The rich data set available for this research allowed us to include a wealth of covariates to adjust for many of the potential disturbing variables described above. We are concerned about both observed differences and unobserved differences in attempting to “balance” the differences in students, classrooms, and schools in which teachers who received different types of preparation are employed. Unfortunately, no study that is not either a randomized experiment, or perhaps a regression discontinuity design, can make a compelling case that selection bias has been eliminated from the coefficients of interest due to unobserved or omitted variables. However, an extensive set of covariates (Table 3), allow us to statistically adjust for many, if not most, of the plausible threats to imbalance. [Insert Table 3 about here]

20

1. Student‐level Covariates. The prior test score for each student in the subject being tested

in the current year is included in each of the models. For high school EOC tests, we include the student’s 8th grade reading and mathematics scores. One major advantage that comes from the use of rosters to match teachers to their students for particular subjects or courses is that we have individualized information on the peers in each student’s class. Using the same prior test score information outlined earlier, we calculate an individual level value for peer ability that is the average performance on prior exams for all students in a classroom other than the individual for whom the value is calculated. This peer ability variable should help to balance any differences in class assignments for teachers with different types of preparation. Other student‐level covariates include dichotomous indicator variables for male, Asian, black, Hispanic, multiracial, and American Indian (females and white students are the reference groups). Student exceptionality codes are used to generate an indicator variable if the student is gifted and another indicator variable if the student has a disability. Students are coded as having a disability if their school district has recorded an exceptionality code which includes behavioral, physical, learning or cognitive disabilities. Two covariates related to each student’s English proficiency are included. One variable indicates whether a student currently receives services for Limited English Proficiency (LEP), and a second variable indicates whether a student formerly received LEP services, but does not currently. Structural mobility refers to student moves necessary due to the grade level structure of a school, i.e. moving to 6th grade from an elementary school in which the highest grade is 5th. Other between year mobility refers to student mobility where a student is enrolled in a grade other than the first offered in the school (that does not require structural move) and her prior year test score is from a different school. Finally, within year mobility is based on membership (enrollment) data and identifies within year movers as those that were enrolled for at least one week less than most other students within a school.

21

Each student’s absenteeism was recorded based on the number of days that the student was

recorded as absent during the school year. In order to control for student’s prior school progress, we created dichotomous variables for students who were overage for their grade, indicating they may well have been held back for a grade, or underage, indicating they may have skipped a grade. These variables were created based on the combination of a student’s date of birth and grade level. Based on the enrollment cutoff date of October 16th, we mark students as underage if their birth date occurs before the cutoff date for their grade level and overage if a student’s birth date occurs after the cutoff date one year later. Finally, dichotomous variables representing the student’s grade level are included in high school analysis models. Parental education is coded into five dichotomous variables representing a student’s level of parental education in the following categories: 1) those with less than a high school education; 2) those who graduated from high school only; 3) those who attended some college; 4) those who graduated from college with a four year degree or higher, and 5) those whose values for parental education were missing. As a proxy for a student’s family income level, we use three dichotomous variables indicating free lunch eligible students, those eligible for reduced price lunch, and those whose lunch payment status is missing. In addition to these characteristics, high school models also include indicator variables to control for the average score differences between different EOC tests. During the time period covered by this study, ten tests were required and included in the analysis: Algebra I, Algebra II, Geometry, Biology, Chemistry, Physical Science, Physics, English I, Civics and Economics, and U.S. History. 2. Classroom Covariates. Four control variables related to classroom characteristics are included in the models. The number of students in a classroom is included to control for differences in achievement that may be related to class size. In the high school and middle school analyses, two

22

indicator variables are used to indicate classrooms with an advanced or remedial curriculum. Classrooms are labeled as an advanced curriculum classroom when their titles include the keywords ‘advanced’, ‘honors’, or ‘AP’. Remedial classroom labels are based on course titles including ‘remedial’ or ‘resource’. Also, we calculate peer dispersion, a classroom level variable indicating the range of students’ prior test scores within a classroom. This variable is calculated based on the standard deviation of prior year test scores for each student within a classroom, and is included to control for any potential effect of the range of prior abilities of the students within a classroom on students’ achievement. Teacher controls include indicator variables for National Board Certification, supplemental advanced degree, and level of teacher experience. Because prior research shows that the effectiveness of teachers increases rapidly in the first three years and then stabilizes, to better account for the differences in the effectiveness of teachers with more or less experience, we include indicator variables for (1) teachers in their first year of teaching, (2) those in their second year of teaching, (3) those in their third year of teaching, and (4) those in their fourth year of teaching. We also compare the type, area, and grade level of the teacher’s license with the class taught to generate a dichotomous variable indicating those teachers who are teaching the grade or subject in‐ field. 3. School Level Covariates. School level covariates include measures of the concentration of students within a school by ethnicity and free or reduced lunch status. The equations include the percentage of students who are African American, Hispanic, Asian, Native American, and multiracial, as well as those eligible for free lunch and eligible for reduced price lunch. In addition, the models include variables indicating school size, measured by the average daily membership of a school; per pupil expenditures, measured in hundreds of dollars; the average local supplement paid to teachers in the school also measured in hundreds of dollars; and two indicators of orderliness in

23

the school (suspensions per 100 students and violent acts per 1,000 students). A complete list of the covariates is included in Table 3. F. Estimation Method. For this study, we chose models that we believe best allowed us to

isolate the effect of the teacher entry portals on student achievement by controlling for the influence of several student, classroom, teacher, and school variables. In the end, we elected to use year‐to‐year value added models with extensive controls, based on criteria including reducing bias in the estimates of teacher portal effects and consistency of models across levels of schooling. To isolate the effects of a particular teacher preparation portal, the analysis should eliminate, to the greatest extent possible, variables that influence student achievement and are imbalanced across portals. For example, estimates of the effectiveness of teachers entering through a portal should not be affected if they happen to teach in underperforming schools with high concentrations of poverty. This required that we develop and use an extensive set of controls for student, classroom, and school characteristics. These controls do not mean that adjustments for student or school differences are automatic, nor necessarily set different expectations for different students or schools. If differences between students or schools are systematically related to students’ test score gains, however, then year‐to‐year value added models with extensive controls will adjust for those differences when estimating the effects of teacher preparation programs. Value added models (VAM) with student fixed effects, which compare students to only their own test scores in the same subject (either reading or mathematics), can only be estimated in the third through eighth grade (Clotfelter, Ladd, and Vigdor 2010). We assessed impacts on high school End‐of‐Course test scores as well as 3rd through 8th grade scores in order to be able to compare effects across subjects and grade levels. Since prior tests in these high school subjects (e.g. biology or chemistry) are not available and 8th grade test scores are non‐time varying for an individual student, student fixed effects are not feasible for this purpose.

24

Estimates of portal effects are based on comparisons with the reference group, in‐state

public undergraduate prepared teachers in their fifth year of teaching. The model coefficients provide estimates of the average difference in student achievement between teachers trained in traditional in‐state public undergraduate programs and those in the specified portal, controlling for the rich array of other variables discussed above. Mixed models allowing random effects at student, classroom/teacher, and school levels were estimated, which appropriately adjusts standard errors for nesting. Where fewer than ten teachers from a portal were included in a model, the coefficients are not reported. The equation used to estimate the effect of the teacher portals is: ⋯





is the test score for student i in classroom j in school s at time t;

, where



...

estimate of the average effect of the 11 portals (excluding the reference group, in‐

state public undergraduate prepared teachers); …

are indicator variables that equal 1 if the teacher entered teaching

through that portal and 0 if not; represents a prior test score (or scores) for student i; represents a set of individual student covariates;

is the estimate of the average effect of each individual student covariate; represents a set of classroom controls;

is the estimate of the average effect of each classroom control;

25

represents a set of school controls; is the estimate of the average effect of each school control; and ,

, and

are disturbance terms representing unexplained variation at the

individual, classroom, and school levels, respectively. In high school models where all subjects are included, individual test indicator variables adjust for subject differences between student outcomes. Algebra I serves as the reference subject in models for all subjects and for mathematics specific estimates. For science and social studies specific estimates, other subjects must be chosen as the reference subject. Biology is the reference subject for science‐specific models, and US History serves as the reference subject in models of high school social studies achievement. The effects of teachers entering via each portal compared to in‐state public undergraduate prepared teachers were estimated using the SAS Proc Mixed software program. In total, each portal was compared to the in‐state public undergraduate portal on 11 different outcome measures. In cases when the portal had fewer than ten teachers teaching the course or grade and subject associated with the test, the coefficient is not reported. In all, we report 97 comparisons. 4. Research Findings

The main objective of this study was to assess the relative effectiveness of teachers who

entered North Carolina public school classrooms through different portals. Because multilevel models require that one portal be the reference group for the comparisons, we use the largest and in some ways the most public policy relevant portal, in‐state public institutions, which we label in‐ state public undergraduate prepared. In Table 4, we provide the descriptive statistics for the data base used in the analysis separately for each of the three data sets, elementary, middle, and secondary grades.

26



‐‐ Table 4 about here ‐‐ Overview of findings. Of the 97 comparisons made, students taught by in‐state public

undergraduate prepared teachers performed better in 14 comparisons, worse in 9, and not significantly different in 74 comparisons. (Complete tables of the coefficients for each grade level and subject are included in Appendix Tables 3.A‐C.) As shown in Table 5, in‐state public undergraduate prepared teachers consistently performed better than teachers from three other portals in high school overall, five other portals in mathematics, and three other portals in social studies. In high school overall as well as in English I and science, in‐state public undergraduate prepared teachers performed worse than teachers from Teach For America and NC private graduate prepared. Out of state undergraduate prepared teachers, lateral entry teachers, and VIF teachers do worse than in‐state public undergraduates in high school overall, mathematics, and in the case of the first two, in social studies as well. As shown in Table 5, for the middle school grades (6‐8) in‐state public undergraduate prepared teachers were significantly different from the other portals in two comparisons. In middle school mathematics, Teach For America corps members outperformed in‐state public undergraduate prepared teachers, but in‐state public undergraduate prepared teachers outperformed in‐state public licensure only teachers. Clearly, the different portals are more similar in terms of effectiveness in teaching middle school than demonstrated in high school, which may have to do with the fact that test score increases decline dramatically in grades 6‐8. In elementary grades (Table 5), in‐state public undergraduate prepared teachers outperformed out of state teachers in both mathematics and reading, but lagged Visiting International Faculty teachers in reading. To facilitate interpretation of the magnitude of the coefficients, we converted the coefficient on the teachers from the out‐of‐state undergraduate performance into equivalent days of schooling by changing the standardized coefficients to actual points and then dividing the point difference by the average gain per day of schooling for students

27

in the same grades on the subject. Students who are taught elementary mathematics by out of state teachers lose the equivalent of 6.1 days of schooling compared to those taught by in‐state public undergraduate prepared teachers. Except as noted, in‐state public undergraduate prepared teachers perform similarly to teachers who entered the state’s public schools through other portals. ‐‐ Table 5 about here ‐‐ Alternative entry teachers . In three of the 11 comparisons of alternative entry teachers, these teachers performed worse than in‐state public undergraduate prepared teachers. These include high school overall, high school math and high school social studies. The significance of this finding is raised when we consider that alternative entry is the largest source of teachers in high schools in North Carolina. Teachers who entered teaching via alternative entry do not perform reliably better or worse than in‐state public undergraduate prepared teachers in middle or elementary grades. Teach For America. The portal that most consistently outperformed in‐state public

undergraduate prepared teachers was Teach For America (TFA), a small, selective program that provides intense summer training and continued professional development to their corps members. Teach For America corps members outperformed in‐state public undergraduate prepared teachers in five of nine comparisons, perform no differently in the other four comparisons, and in two comparisons their numbers were insufficient to yield reliable estimates (see Appendix Table 2 for the counts of teachers by grade level). Their positive effects were concentrated in high school and middle school subjects. Their positive effects on middle school mathematics were particularly large. In fact, the TFA coefficient on middle school mathematics translates into an advantage equivalent to approximately half a year of learning more than the in‐ state public traditionally prepared teachers. Although TFA has strong and consistent positive effects on student test scores, it is important to note that corps members make up only 0.3% of the teacher workforce in NC schools.

28

Out of State Undergraduate Prepared Teachers. As shown in the preceding tables, out of

state undergraduate prepared teachers performed significantly worse than in‐state public undergraduate prepared teachers in five comparisons: high school overall, high school math, high school social studies, elementary mathematics, and elementary reading. They performed no differently in the other six comparisons. This pattern of lower performance among out of state undergraduate prepared teachers is especially noteworthy because they make up nearly 1/4 of the teacher workforce in NC schools (see Appendix Table 2 for the teacher counts by grade level). Visiting International Faculty. Another group of teachers that we hypothesized would underperform in comparison to in‐state public undergraduate prepared teachers, in the nine reliable comparisons VIF outperformed in‐state public undergraduate prepared teachers in elementary reading but performed worse in high school overall and high school math. Teachers prepared at in‐state private institutions. We hypothesized that in‐state undergraduate prepared teachers would perform similarly to in‐state public undergraduate prepared teachers. In only one of 11 comparisons, high school mathematics, did the teachers prepared as undergraduates in public institutions outperform those prepared in private institutions. However, teachers prepared with graduate degrees, primarily MAT degrees, by in‐ state private institutions outperform in‐state public undergraduate prepared teachers in four of seven reliable comparisons, including all high school subjects other than social studies. Conclusion In this study of the impact of teacher preparation on student achievement in North Carolina schools, we found that in‐state public undergraduate prepared teachers, who constitute nearly 1/3 of the North Carolina teacher workforce, perform near the middle of the pack, better in 14 comparisons, worse in 9, and similarly to teachers from other portals in 74. On balance, teachers from private colleges and universities in‐state perform similarly to in‐state public undergraduate prepared, with in‐state private undergraduate prepared lagging their in‐state public counterparts

29

in high school mathematics but with in‐state private graduate prepared outperforming them in four high school comparisons, in one case (high school science) by a wide margin. In‐state public

graduate prepared teachers perform neither better nor worse than their undergraduate counterparts. The major divide in teacher performance is not between teachers from public versus private higher education institutions within the state, but between in‐state traditionally prepared teachers on the one hand and teachers from out of state and alternative entry teachers on the other. Out of state teachers represent 32% of the teachers teaching tested subjects in elementary grades, the largest percentage of teachers from any portal, and a much higher proportion than the percentage of out of state teachers in the total NC teacher workforce (23%). Alternative entry teachers, who comprise 15 percent of the NC teacher workforce, perform worse in high school mathematics and social studies in particular, and on average across all high school subjects, where they are concentrated. This indicates that there is currently a mismatch between where these teachers are more effective and where they are placed. In contrast to these portals whose performance lags in‐state public undergraduate prepared teachers, Teach For America corps members outperformed in‐state public undergraduate prepared teachers in 5 of the 9 comparisons. Teach For America teachers are chosen competitively from applicants graduating from top colleges and universities, provided with intensive training during a summer institute before entering the classroom, and supported through ongoing professional development during their two years in the program. The strong performance of Teach for America teachers suggests that TFA might represent a good source of guidance for innovation and teacher preparation program development. In fact, we believe that others interested in improving teaching quality might best regard Teach For America as a small scale innovation (since they make up 0.3 percent of the North Carolina teacher workforce) from which they have much to learn rather than as a threat to their share of the teacher preparation market. Scaling up Teach For America while

30

maintaining its present quality standards would help improve public school performance especially in the high poverty schools in which they serve, but because of the limited size of the program, the overall average effects on the state’s performance would be minimal. However, importing some key features of TFA into traditional teacher preparation programs might well have a larger total impact. Some features of the Teach For America program – including techniques for selecting,

preparing, and providing continuing support to TFA corps members ‐‐ represent potentially transferable innovations. A first example would be the selection of TFA corps members on the basis of soft skills such as perseverance, leadership, and their ability to engage students, as well as their college academic performance and demonstrated commitment to service. Second, Teach For America corps members also practice teaching during the summer prior to entering the classroom in the grade and subject they will be assigned to teach the following fall. Third new corps members develop their plans to meet the objectives of the state’s curriculum with advice and support of experienced teachers just before the school year begins. Fourth, new corps members are assigned to their schools with other corps members to create a cohort within the school. Fifth, TFA program directors, who have experience teaching in the grades and subjects the corps members are currently teaching, observe the corps members teaching and provide them with immediate, constructive feedback on ways to improve their teaching. Finally, TFA corps members receive professional development and support from TFA during their two‐year term of service that is focused on improving their teaching skills, especially remedying their weaknesses. These types of practices should be considered for pilot testing and evaluation to determine whether they actually improve the performance of teachers early in their careers, a period of serious and substantial weakness for teachers from all portals. Taken together, these findings suggest two routes to improving the preparation of NC public school teachers (1) increase the productivity of portals where other portals that supply large numbers of teachers perform worse; and (2) develop, pilot, evaluate, and implement effective

31

innovations to existing teacher preparation programs, perhaps based on the Teach For America training and support program. Examples of the former are producing more traditionally prepared teachers in public institutions and teachers prepared with graduate degrees from private in‐state institutions for the high school level (where they outperform lateral entry teachers) and traditionally prepared in‐state teachers for the elementary school level (where they outperform out‐of‐state teachers) as long as increasing the supply does not reduce the quality of teachers prepared through these portals. As for innovations that might be developed, piloted, evaluated, and adopted, teachers from out of state and lateral entry teachers might benefit from training to teach the state’s curriculum and frequent observation and feedback. Such programs might include summer institutes in which beginning teachers plan their objectives week‐by‐week for teaching the objectives set forth in the state curriculum for the grade(s) and subject(s) they are being assigned to teach the following year, which are similar to the backward planning exercises used to prepare Teach For America corps members. In summary, taking all comparisons into account, in‐state public undergraduate prepared teachers perform near the middle of the pack but slightly better than teachers from several other sources. Undergraduate prepared teachers from out of state lag their in‐state public undergraduate prepared counterparts in elementary reading and mathematics, the very grade levels and subjects where the out of state teachers are most heavily concentrated. Similarly, lateral entry teachers underperform in‐state public undergraduate prepared teachers where they are most heavily concentrated – in high schools. In contrast, Teach For America teachers outperform in‐state public undergraduate prepared teachers in five out of nine comparisons, in some cases by wide margins. But Teach For America is a very small program, contributing only three tenths of one percent of all North Carolina public school teachers. Even if it were ten times as large as it is now, Teach For America would supply only 3% of the state’s teachers. Thus, increased productivity and innovation in more traditional teacher preparation programs, rather than substitution of alternative

32

certification programs seems a logical approach to improving teaching and student achievement in the public schools.

33 References    Boyd, Donald; Hamilton Lankford, Susanna Loeb, and James Wyckoff. 2005. The draw of home: How teachers’ preferences for proximity disadvantage urban schools. Journal of Policy Analysis and Management 24(1): 113-132. Boyd, Donald, Pamela Grossman, Hamilton Lankford, Susanna Loeb, and James Wyckoff. 2006. How changes in entry requirements alter the teacher workforce and affect student achievement. Education Finance and Policy 1(2): 176-216. Boyd, Donald, Pamela Grossman, Hamilton Lankford, Susanna Loeb, and James Wyckoff. 2009. Teacher preparation and student achievement. Educational Evaluation and Policy Analysis 31(4): 416-440. Clotfelter, Charles, Helen Ladd, and Jacob Vigdor. 2005. Who teaches whom? Race and the distribution of novice teachers. Economics of Education Review 24(4): 377-392. Clotfelter, Charles, Helen Ladd, and Jacob Vigdor. 2007. Teacher credentials and student achievement: Longitudinal analysis with student fixed effects. Economics of Education Review 26(6): 673-682. Clotfelter, Charles, Helen Ladd, and Jacob Vigdor. 2010. Teacher credentials and student achievement in high school: A cross-subject analysis with student fixed effects. The Journal of Human Resources 45(3): 655-681. Coleman, James, Ernest Campbell, Carol Hobson, James McPartland, Alexander Mood, Frederic Weinfeld, and Robert York. 1966. Equality of Educational Opportunity. Washington, D.C.: U.S. Government Printing Office. Cook, Thomas, Michael Scriven, Chris Coryn, and Stephanie Evergreen. 2010. Contemporary thinking about causation in evaluation: A dialogue with Tom Cook and Michael Scriven. American Journal of Evaluation 31(1): 105-117. Darling-Hammond, Linda, Barnett Berry, and Amy Thoreson. 2001. Does teacher certification matter? Evaluating the evidence. Educational Evaluation and Policy Analysis 23(1): 57-77. Goldhaber, Dan, and Dominic Brewer. 2000. Does teacher certification matter? High school teacher certification status and student achievement. Educational Evaluation and Policy Analysis 22(2): 129-145. Gordon, Robert, Thomas Kane, and Douglas Staiger. 2006. Identifying effective teachers using performance on the job. The Brookings Institution: The Hamilton Project White Paper #2006-01. Harris, Douglas, and Tim Sass. 2008. Teacher training, teacher quality and student achievement. Calder Institute Working Paper No.3.

34 Henry, Gary, and Kelly Purtell. 2010. The impacts of out-of-field teachers on high school performance. In review, Educational Evaluation and Policy Analysis. Jackson, C. Kirabo, and Elias Bruegmann. 2009. Teaching students and teaching each other: The importance of peer learning for teachers. American Economic Journal: Applied Economics 1(4): 1-27. Kane, Thomas, Jonah Rockoff, and Douglas Staiger. 2008. What does certification tell us about teacher effectiveness? Evidence from New York City. Economics of Education Review 27(6): 615-631. Koedel, Cory and Julian Betts. 2009. Does student sorting invalidate value-added models of teacher effectiveness? An extended analysis of the Rothstein critique. University of Missouri WP 09-02, July. Lankford, Hamilton, Susanna Loeb, and James Wyckoff. 2002. Teacher sorting and the plight of urban schools: A descriptive analysis. Educational Evaluation and Policy Analysis 24(1): 37-62. National Research Council. 2010. Preparing teachers: Building evidence for sound policy. Washington, DC: National Academies Press. Reardon, Sean and Stephen Raudenbush. 2009. Assumptions of value‐added models for estimating school effects. Educational Finance and Policy 4(4): 492‐519. Rockoff, Jonah. 2004. The impact of individual teachers on student achievement: Evidence from panel data. The American Economic Review 94(2): 247-252. Rothstein, Jesse. 2010. Teacher quality in education production: Tracking, decay, and student achievement. The Quarterly Journal of Economics 125(1): 175-214. Wooldridge, Jeffrey. 2009. Introductory Econometrics: A Modern Approach. Mason, OH: South-Western Cengage Learning. Xu, Zeyu, Jane Hannaway, and Colin Taylor. 2007. Making a difference? The effects of Teach For America in high school. Calder Institute Working Paper No.17.

1 Table (1): Summary Table of Research Themes and Findings Studies

Classification Groups

Sample

Geographic Scope

Grade Levels/Subjects

Methodology

Goldhaber & Brewer (2000)

1) 2) 3) 4) 5)

Standard Probationary Emergency Private Non-Certified

All experience levels of Nationwide teachers included

High school math and science

OLS regression; value-added model

Clotfelter et al. (2007)

1) 2) 3)

Regular Lateral Entry Other

All experience levels of teachers included

North Carolina

Elementary math and reading

Student fixed effects within a gains model

Clotfelter et al. (2010)

1) 2) 3)

Regular Lateral Entry Other

All experience levels of teachers included

North Carolina

High school math, science, and English

Student fixed effects without prior score controls

Boyd et al. (2006)

1) 2) 3) 4) 5) 6)

College Recommended Individual Evaluation (IE) NYC Teaching Fellows (TF) Teach For America (TFA) Temporary Other

All experience levels of teachers included; focus principally on novice teachers in their first three years in the classroom

New York City

Elementary and middle school math and reading

School fixed effects within a value-added model; standard errors clustered at the teacher level

Kane et al. (2008)

1) 2) 3) 4) 5)

Standard NYC Teaching Fellows (TF) Teach For America (TFA) International Uncertified

Recently hired teachers in NYC; focus principally on novice teachers in their first three years in the classroom

New York City

Elementary and middle school math and reading

Grade and year fixed effects within a valueadded model; standard errors clustered at the school level



Results Standard certification outperforms noncertified in HS math; emergency certification performs comparably Regular licensure outperforms other in ES math and reading; lateral entry only worse in levels models Regular licensure outperforms other and lateral entry; “prior” lateral entry do no worse than regular IE, TF, temporary and other worse in math; TF, TFA, temporary and other worse in reading Negative results driven more by first year teachers; effects sizes are small International worse in math; TF worse in reading Find little differences by certification type; more variation within types than between

2

Table 5. Comparisons of In‐state Public Undergraduate Prepared Teachers’ Effects to Teachers from Other Portals on Test Score Secondary Middle Elementary Teacher Portals

Overall

Math

Eng. I

Science

Social Studies

Math Reading Alg. I

Summary Science Math Reading th (8 )

In-state Public 0.022 0.057 0.006 0.008 0.005 0.013 ‐0.011 NR NR 0.017 ‐0.009 +0,‐0 Graduate Prepared In‐state Private ‐0.016 ‐0.049* 0.000 0.005 ‐0.010 ‐0.007 0.005 0.030 NR 0.001 ‐0.002 +0,‐1 Undergrad Prepared In‐state Private 0.088* 0.051* 0.041* 0.163* 0.018 NR NR NR NR ‐0.065 ‐0.058 +4,‐0 Graduate Prepared Out of State ‐0.026* ‐0.057* ‐0.015 ‐0.012 ‐0.040* ‐0.007 ‐0.008 ‐0.013 0.023 ‐0.023* ‐0.013* +0,‐5 Undergrad Prepared Out of State Graduate ‐0.014 ‐0.045 ‐0.019 0.014 ‐0.056* 0.012 0.008 ‐0.139 0.048 ‐0.008 ‐0.004 +0,‐1 Prepared In-state Public ‐0.034 NR 0.008 ‐0.103 ‐0.016 ‐0.009 ‐0.035* NR NR 0.020 0.026 +0,‐1 Licensure Only NR NR NR NR NR NR NR NR ‐0.050 ‐0.035 Other Licensure Only ‐0.001 +0,‐0 0.222* 0.079 0.148* 0.024 NR NR 0.042 0.040 +5,‐0 Teach For America 0.172* 0.139* 0.085* ‐0.031 0.011 NA 0.005 0.014 ‐0.131 NR 0.017 0.029* Visiting Intl Faculty ‐0.078* ‐0.082* +1,‐2 0.007 ‐0.019 ‐0.042* 0.003 ‐0.004 ‐0.025 ‐0.048 ‐0.014 ‐0.010 +0,‐3 Lateral Entry ‐0.023* ‐0.033* ‐0.027 ‐0.037 ‐0.001 ‐0.045 ‐0.011 NR NR ‐0.021 ‐0.006 Unclassifiable ‐0.044 ‐0.169* +0,‐1 NR = Not reported because fewer than 10 teachers from the portal found in this cell; Bold coefficients with * indicate that coefficient was statistically distinguished from 0.



3

Appendix Table 2. Teacher Counts by Portal Secondary

Middle

Elementary

Alg. I

Science (8th )

845

132

69

2,420

2,448

25

61

3

6

119

127

106

177

187

15

9

1016

1022

15

20

5

7

4

1

18

19

313

181

217

661

782

127

86

2,457

2,484

53

62

76

60

112

174

20

20

469

478

93

25

10

15

44

26

42

4

2

92

96

18 88

4 24

4 21

10 24

1 20

8 39

8 65

1 8

1 1

33 45

33 49

93

14

47

34

1

101

81

17

7

170

168

2,057 76

504 17

689 17

637 17

342 27

1,140 46

1,438 87

159 9

138 8

617 211

641 211

Math

Eng. I

Science

Social Studies

1,458

381

517

210

376

716

257

69

54

57

80

372

94

140

44

64

14

15

868

182

244

Teacher Portals Overall In‐state Public Undergraduate Prepared In‐state Public Graduate Prepared In‐state Private Undergraduate Prepared In‐state Private Graduate Prepared Out of State Undergraduate Prepared Out of State Graduate Prepared In‐state Public Licensure Only Other Licensure Only Teach For America Visiting International Faculty Lateral Entry Unclassifiable

Math Reading

Math Reading

Teacher Preparation and Student Achievement: Tables and Figures

Table 4: Descriptive Statistics by Grade Level Secondary Count of Weighted Observations Teacher Preparation Portals In‐state Public Undergraduate Prepared In‐state Public Graduate Prepared In‐state Private Undergraduate Prepared In‐state Private Graduate Prepared Out of State Undergraduate Prepared Out of State Graduate Prepared In‐state Licensure Only Other Licensure Only Teach For America Visiting International Faculty Lateral Entry Unclassifiable Other Teacher Characteristics Infield teaching First year teacher Second year teacher Third year teacher Fourth year teacher Student Characteristics Prior grade scores combined (Elem) (Std.) Prior grade math score (Std.) Prior grade reading score (Std.) Average peer test score (prior grade) Days absent Structural student mobility Moved since prior year Moved since prior year missing Within year move Within year move missing Underage student based on grade Overage student based on grade Academically or intellectually gifted Disabled student Free lunch Reduced lunch Lunch status missing Parent education less than high school Parent education some college Parent education college graduate Parent education missing White Black Hispanic Multiracial American Indian

Middle

Elementary

349,914 Mean SD

569,697 Mean SD

472,415 Mean SD

0.290 0.047 0.075 0.012 0.147 0.042 0.019 0.003 0.015 0.017 0.322 0.011 0.532 0.223 0.246 0.202 0.180

0.454 0.211 0.263 0.109 0.354 0.201 0.138 0.050 0.121 0.128 0.467 0.104 0.499 0.416 0.431 0.401 0.384

0.268 0.013 0.056 0.002 0.219 0.039 0.011 0.001 0.011 0.023 0.337 0.020 0.653 0.187 0.224 0.208 0.190

0.443 0.114 0.230 0.041 0.414 0.193 0.103 0.037 0.105 0.151 0.473 0.138 0.476 0.390 0.417 0.406 0.392

0.335 0.015 0.140 0.002 0.317 0.059 0.012 0.003 0.005 0.019 0.066 0.026 0.851 0.197 0.232 0.208 0.188

0.472 0.121 0.347 0.044 0.465 0.235 0.110 0.055 0.070 0.137 0.249 0.160 0.356 0.398 0.422 0.406 0.391

‐‐

‐‐

‐‐

‐‐

‐0.011

0.922

0.075 0.079 0.069 9.035 0.394 0.043 0.036 0.039 0.000 0.014 0.225 0.137 0.084 0.279 0.073 0.008 0.049 0.236 0.325 0.319 0.577 0.321 0.050 0.019 0.012

0.915 0.900 0.593 9.989 0.489 0.204 0.185 0.193 0.018 0.117 0.418 0.344 0.277 0.449 0.261 0.092 0.217 0.425 0.468 0.466 0.249 0.467 0.219 0.138 0.108

‐0.009 ‐0.013 ‐0.018 8.040 0.272 0.087 0.000 0.056 0.020 0.012 0.247 0.153 0.100 0.365 0.083 0.032 0.073 0.206 0.267 0.056 0.533 0.327 0.078 0.027 0.013

0.970 0.968 0.720 8.227 0.445 0.281 0.000 0.230 0.140 0.109 0.431 0.360 0.300 0.481 0.277 0.177 0.260 0.404 0.442 0.229 0.499 0.469 0.268 0.161 0.115

‐‐ ‐‐ ‐0.020 6.349 0.034 ‐‐ ‐‐ 0.043 0.000 0.009 0.207 0.111 0.119 0.343 0.074 0.129 0.059 0.140 0.167 0.389 0.538 0.279 0.110 0.039 0.010

‐‐ ‐‐ 0.459 5.764 0.181 ‐‐ ‐‐ 0.202 0.000 0.095 0.405 0.315 0.324 0.475 0.262 0.335 0.236 0.347 0.373 0.487 0.499 0.448 0.313 0.193 0.102

Teacher Preparation and Student Achievement: Tables and Figures Asian Male LEP services recipient Previous LEP services recipient Algebra 1 Algebra 2 English 1 Geometry Biology Chemistry Physical Science Physics Civics and Economics

0.020 0.488 0.020 0.007 0.145 0.082 0.217 0.109 0.121 0.041 0.059 0.005 0.126

0.141 0.499 0.140 0.085 0.352 0.275 0.412 0.311 0.326 0.197 0.235 0.074 0.332

0.022 0.503 0.040 0.015 ‐‐ ‐‐ ‐‐ ‐‐ ‐‐ ‐‐ ‐‐ ‐‐ ‐‐

0.145 0.499 0.195 0.121 ‐‐ ‐‐ ‐‐ ‐‐ ‐‐ ‐‐ ‐‐ ‐‐ ‐‐

0.024 0.505 0.069 0.022 ‐‐ ‐‐ ‐‐ ‐‐ ‐‐ ‐‐ ‐‐ ‐‐ ‐‐

0.154 0.499 0.253 0.147 ‐‐ ‐‐ ‐‐ ‐‐ ‐‐ ‐‐ ‐‐ ‐‐ ‐‐

0.095 0.293 ‐‐ ‐‐ ‐‐ ‐‐ US History Classroom Characteristics 22.195 5.859 23.140 5.478 21.391 3.657 Students per classroom 0.610 0.140 0.655 0.179 0.873 0.157 Classroom ability dispersion 0.257 0.437 0.198 0.399 ‐‐ ‐‐ Advanced curriculum 0.011 0.104 0.042 0.202 ‐‐ ‐‐ Remedial curriculum School Characteristics 13.163 5.466 7.852 2.810 6.128 2.234 School size (per 100) 203.14 159.99 69.555 48.975 42.545 31.109 School size squared 75.445 15.471 74.790 16.756 81.295 16.698 Total per‐pupil expenditures ($100s) 31.423 15.850 47.764 29.329 33.122 16.141 Average teacher supplement ($100s) Short‐term suspension rate (per 100 33.429 27.031 34.682 33.037 8.112 11.622 students) 15.466 10.655 10.616 10.370 2.037 3.591 Violent acts rate (per 1000 students) 37.462 17.539 50.433 20.711 54.023 24.382 Free and reduced lunch mean 31.304 23.276 32.570 23.022 28.462 23.132 Black mean 6.150 4.872 8.757 7.312 11.458 10.407 Hispanic mean 1.959 1.254 2.791 1.589 3.851 2.290 Multiracial mean 1.196 5.243 1.324 5.791 1.052 5.231 American Indian mean 2.269 2.689 2.278 2.933 2.475 3.651 Asian mean Note: Teachers with less than 5 years experience in 2004‐05, 2005‐06, 2006‐07 & 2007‐08 school years.

Teacher Preparation and Student Achievement: Tables and Figures Table 2. Portal Definitions Teacher Portal 1. In‐state Pubic Undergraduate Prepared

A North Carolina public school teacher who completed the requirements for initial licensure at a UNC institution by earning (a) a Bachelor’s degree in education or (b) a Bachelor’s degree in another major while simultaneously completing the necessary education‐related coursework, before beginning teaching, including Teaching Fellows.

2. In‐state Public Graduate Prepared

A North Carolina public school teacher who earned a graduate degree from a UNC system institution and qualified for an initial license before beginning teaching, including Teaching Fellows.

3. In‐state Private Undergraduate Prepared

A North Carolina public school teacher who completed the requirements for initial licensure at a private (independent) institution in North Carolina by earning (a) a Bachelor’s degree in education or (b) a Bachelor’s degree in another major while simultaneously completing the necessary education‐related coursework, before beginning teaching, including Teaching Fellows.

4. In‐state Private Graduate Prepared

A North Carolina public school teacher who earned a graduate degree from a private (independent) North Carolina institution and qualified for an initial license before beginning teaching.

5. Out of State Undergraduate Prepared 6. Out of State Graduate Prepared 7. In‐state Licensure Only

8. Other Licensure Only

9. Teach For America



Definition

A North Carolina public school teacher who completed the requirements for initial licensure at an out of state institution by earning a Bachelor’s degree before beginning teaching. A North Carolina public school teacher who earned a graduate degree from an out of state university and qualified for an initial license before beginning teaching. A North Carolina public school teacher who, after earning a Bachelor’s degree at any public or private institution in any state, then separately completed the education‐related requirements for teacher licensure at a UNC system institution, before beginning teaching. A North Carolina public school teacher who, after earning a Bachelor’s degree at any public or private institution in any state, then separately completed the education‐related requirements for initial teacher licensure at a non‐UNC system institution, before beginning teaching. A North Carolina public school teacher who began teaching in NC after earning a Bachelor’s degree but before completing the remaining requirements for initial licensure and did so through the Teach For America program.

10. Visiting International Faculty

A North Carolina public school teacher who entered teaching in NC through the Visiting International Faculty program.

11. Alternative Entry

A North Carolina public school teacher who entered the profession prior to completing requirements for initial licensure (Teach For America corps members excluded).

12. Unclassifiable

A North Carolina public school teacher who cannot be classified into one of the portals above on the basis of available evidence.

Teacher Preparation and Student Achievement: Tables and Figures Figure 1. Distribution of Teachers by Portal, 2008

Distribution of Teachers by Portal, 2008 In‐state Public Undergraduate Prepared Out of State Undergraduate Prepared Lateral Entry In‐state Private Undergraduate Prepared Unclassifiable Out of State Graduate Prepared In‐state Graduate Prepared Visiting International Faculty In‐state Licensure Only Other Licensure Only In‐state Private Graduate Prepared Teach For America 0



5,000

10,000

15,000

20,000

25,000

30,000

35,000



Teacher Preparation and Student Achievement: Tables and Figures Appendix Table 1. Test Score, Student, and Teacher Counts for Analysis Data Set Total Test Scores Total Students Total Teachers High School High School – All Subjects 514,870 339,873 5,688 Middle School Algebra 1 Test Scores 19,150 Science 22,570 Reading 276,606 Math 251,371 All Subjects 371,224 6,388 Elementary School Reading 234,264 Math 238,151 All Subjects 227,919 7,864 Total Counts 1,556,982 939,016 19,940 Notes: Middle and high school models include 4 years of data (2004‐2005 through 2007‐ 2008); elementary school models include three years (2005‐2006 through 2007‐2008). Student counts do not reflect unique students over time, but rather, unique students within each level of schooling.

Teacher Preparation and Student Achievement: Tables and Figures Table 3. Covariates by Level in the Impact Models



Student

Classroom & Teacher

School

1. Prior test scores (reading & math) 2. Classmates prior test scores (peer effects) 3. Days absent 4. Structural mobility 5. Other between year mobility 6. Within year mobility 7. Race/ethnicity 8. Poverty 9. Parental education 10. Gifted 11. Disability 12. Currently limited English proficient 13. Previously limited English proficient 14. Overage for grade (held back or retained at least once) 15. Underage for grade (promoted two grades) 16. Grade level

17. 18. 19. 20.

25. School size (ADM) 26. Suspension rate 27. Violent acts per 1,000 students 28. Total per pupil expenditures 29. District teacher supplements 30. Racial/ethnic composition 31. Concentration of poverty

21. 22.

23.

24.

Years of experience Teaching infield Number of students Advanced curriculum Remedial curriculum Heterogeneity of prior achievement within classroom Additional: Teaching Fellows & Other Teacher Scholarships Supplemental Graduate, NBC, Praxis II

Teacher Preparation and Student Achievement: Tables and Figures Appendix Table 3A: Elementary school math and reading results Elementary School Elementary School Math Reading Coefficient Coefficient Teacher Preparation Portals UNC Graduate Prepared NC Private Undergraduate Prepared NC Private Graduate Prepared Out of State Undergraduate Prepared Out of State Graduate Prepared UNC Licensure Only Other Licensure Only Teach For America Visiting International Faculty Lateral Entry Unclassifiable

0.017 0.001 -0.065 -0.023* -0.008 0.020 -0.050 0.042 0.017 -0.014 -0.021

-0.009 -0.002 -0.058 -0.013* -0.004 0.026 -0.035 0.040 0.029* -0.010 -0.006

0.010 -0.081* -0.029* -0.015* -0.012

0.001 -0.048* -0.026* -0.023* -0.012*

0.693* 0.043* -0.008* -0.033 -0.061* 0.078* -0.114* 0.283* -0.054* -0.036* -0.022* 0.002 -0.019* 0.027* 0.092* -0.190* 0.018* -0.068* -0.063* 0.162* 0.092* -0.030* 0.047*

0.703* 0.036* -0.001* -0.062* -0.032* 0.060* -0.114* 0.179* -0.166* -0.068* -0.044* -0.008 -0.055* 0.035* 0.092* -0.140* -0.025* -0.042* -0.090* -0.041* -0.073* -0.193* -0.011

-0.002* 0.079*

0.000 0.062*

-0.014* 0.001* 0.000 0.001* -0.001* -0.001 -0.000 -0.000 0.001* -0.005*

-0.016* 0.001* 0.000 -0.0004* -0.001* -0.001 -0.001* 0.0003* 0.000 -0.000

Teacher Characteristics Infield teaching First year teacher Second year teacher Third year teacher Fourth year teacher

Student Characteristics Average prior grade EOG scores (Std.) Average peer test score (prior grade) Days absent Structural move Within year move Underage student based on grade Overage student based on grade Academically or intellectually gifted Disabled student Free lunch Reduced lunch Lunch status missing Parent education less than high school Parent education some college Parent education college graduate Black Hispanic Multiracial American Indian Asian Male LEP services recipient Previous LEP services recipient

Classroom Characteristics Students per classroom Classroom ability dispersion

School Characteristics School size (per 100) School size squared Total per-pupil expenditures ($100s) Average teacher supplement ($100s) Short-term suspension rate (per 100 students) Violent acts rate (per 1000 students) Free and reduced lunch mean Black mean Hispanic mean Multiracial mean

Teacher Preparation and Student Achievement: Tables and Figures American Indian mean 0.000 0.000 Asian mean 0.003* 0.002* Intercept 0.056 0.153* Note: Teachers with less than 5 years experience in 2005-06, 2006-07 & 2007-08 school years. *Indicates a given coefficient is significant at the .05 level.

Teacher Preparation and Student Achievement: Tables and Figures Appendix Table 3B: Middle school math, reading, algebra 1 and science results

Teacher Preparation Portals UNC Graduate Prepared NC Private Undergraduate Prepared NC Private Graduate Prepared Out of State Undergraduate Prepared Out of State Graduate Prepared UNC Licensure Only Other Licensure Only Teach For America Visiting International Faculty Lateral Entry Unclassifiable

Middle School Math Coefficient 0.013 -0.007 NA -0.007 0.012 -0.009 NA 0.148* 0.005 0.003 -0.045

Middle School Reading Coefficient -0.011 0.005 NA -0.008 0.008 -0.035* NA 0.024 0.014 -0.004 -0.011

Middle School Algebra 1 Coefficient NA 0.030 NA -0.013 -0.139 NA NA NA -0.131 -0.025 NA

Middle School Science Coefficient NA NA NA 0.023 0.048 NA NA NA NA -0.048 NA

0.023* -0.080* -0.018* -0.009 0.011

0.009* -0.030* -0.013* -0.005 0.003

0.024 -0.108* -0.060 -0.058 -0.014

-0.009 -0.061 -0.016 0.043 0.062*

0.591* 0.127* 0.114* -0.005* -0.024* -0.004 -0.055* 0.003 0.041* -0.066* 0.149* -0.057* -0.009* -0.005 0.001 -0.025* 0.025* 0.062* 0.043* -0.084* 0.003 -0.026* -0.038* 0.094* 0.018* 0.001 0.050*

0.188* 0.551* 0.071* -0.002* -0.013* 0.000 -0.031* 0.008 0.034* -0.059* 0.116* -0.116* -0.044* -0.033* -0.020* -0.047* 0.035* 0.066* 0.049* -0.085* 0.014* 0.001 -0.047* -0.006 -0.050* -0.124* 0.003

0.674* 0.151* 0.020 -0.010* 0.934* 0.012 -0.074* -0.037 0.124* -0.106* 0.139* 0.046 0.012 0.007 0.078* -0.073* 0.005 0.044* 0.094* -0.076* 0.013 -0.002 0.051 0.134* -0.012 0.117* 0.066

0.274* 0.400* 0.034* -0.004* 0.217* 0.060* -0.066* -0.201 0.089* -0.082* 0.175* -0.022 -0.048* -0.021 -0.095 -0.014 0.002 0.046* 0.139* -0.247* -0.079* -0.066* -0.161* -0.008 0.223* -0.067* -0.060*

-0.001 0.008 0.016* -0.011

0.000 0.023* 0.008 -0.028*

-0.003 0.068 NA NA

0.000 -0.030 0.081* 0.128

0.006 -0.001 0.000

0.000 0.000 0.000

0.028 -0.001 0.001

0.013 -0.001 0.001*

Teacher Characteristics Infield teaching First year teacher Second year teacher Third year teacher Fourth year teacher

Student Characteristics Prior grade math score (Std.) Prior grade reading score (Std.) Average peer test score (prior grade) Days absent Structural student mobility Moved since prior year Within year move Within year move missing Underage student based on grade Overage student based on grade Academically or intellectually gifted Disabled student Free lunch Reduced lunch Lunch status missing Parent education less than high school Parent education some college Parent education college graduate Parent education missing Black Hispanic Multiracial American Indian Asian Male LEP services recipient Previous LEP services recipient

Classroom Characteristics Students per classroom Classroom ability dispersion Advanced Curriculum Remedial Curriculum

School Characteristics School size (per 100) School size squared Total per-pupil expenditures ($100s)

Teacher Preparation and Student Achievement: Tables and Figures Average teacher supplement ($100s) 0.000 0.000 0.000 0.000* Short-term suspension rate (per 100 0.000 0.000 -0.0003* -0.001* students) Violent acts rate (per 1000 students) 0.000 -0.002 -0.001 -0.0004* Free and reduced lunch mean 0.000 0.000 -0.002 -0.003* Black mean 0.000 -0.001 0.0004* -0.003* Hispanic mean 0.000 -0.001 0.001 0.001* Multiracial mean -0.003 -0.020 0.004* 0.012* American Indian mean -0.001 -0.003 -0.001* -0.003* Asian mean 0.000 0.000 0.002 0.003* Intercept 0.052 0.064 -0.725 -0.004 Note: Teachers with less than 5 years experience in 2004-05, 2005-06, 2006-07 & 2007-08 school years. *Indicates a given coefficient is significant at the .05 level.

Teacher Preparation and Student Achievement: Tables and Figures Appendix Table 3C: High school overall, math, English 1, science and social studies results

Teacher Preparation Portals UNC Graduate Prepared NC Private Undergraduate Prepared NC Private Graduate Prepared Out of State Undergraduate Prepared Out of State Graduate Prepared UNC Licensure Only Other Licensure Only Teach For America Visiting International Faculty Lateral Entry Unclassifiable

High School Overall

High School Math

High School English 1

High School Science

Coefficient 0.022 -0.016 0.088* -0.026* -0.014 -0.034 -0.001 0.172* -0.078* -0.023* -0.044

Coefficient 0.057 -0.049* 0.051 -0.057* -0.045 NA NA 0.139* -0.082* -0.033* -0.169*

Coefficient 0.006 0.000 0.041* -0.015 -0.019 0.008 NA 0.085* -0.031 0.007 -0.027

Coefficient 0.008 0.005 0.163* -0.012 0.014 -0.103 NA 0.222* 0.011 -0.019 -0.037

High School Social Studies Coefficient 0.005 -0.010 0.018 -0.040* -0.056* -0.016 NA 0.079 NA -0.042* -0.001

0.027* -0.113* -0.044* -0.019 -0.011

0.034* -0.123* -0.048* -0.032 0.010

0.018* -0.032* -0.020 -0.005 -0.012

-0.003 -0.101* -0.027 -0.005 -0.009

0.060* -0.150* -0.038 -0.017 -0.008

0.370* 0.316* 0.131* -0.007* 0.020* 0.014 0.014* -0.067* -0.131 0.098* -0.080* 0.116* -0.030* 0.004 0.002 0.025* 0.008 0.030* 0.030* 0.013* -0.097* 0.003 -0.012* -0.068* 0.045* 0.052* -0.046* 0.048* -0.373* 0.059* -0.267* -0.020 -0.563* 0.235* -0.998*

0.561* 0.103* 0.131* -0.008* 0.036* 0.004 -0.022* -0.065* -0.093 0.079* -0.089* 0.129* -0.033* 0.023* 0.015* -0.001 -0.001 0.002 0.014* -0.007 -0.088* 0.004 -0.021* -0.042* 0.096* 0.000 0.040* 0.029 -0.373* --0.267* -----

0.218* 0.494* 0.080* -0.005* -0.066* -0.016 0.044 -0.042* -0.119 0.065* -0.096* 0.129* -0.181* -0.036* -0.033* -0.023 -0.021* 0.046* 0.035* -0.026* -0.067* 0.003 -0.008 -0.078* -0.025* -0.138* -0.154* 0.014 --

0.414* 0.311* 0.085* -0.008* -0.050* 0.041* 0.025* -0.087* -0.327 0.128* -0.075* 0.110* 0.014 0.011* 0.018* 0.017 0.009 0.034* 0.033* 0.012 -0.134* -0.015 -0.017 -0.085* 0.054* 0.119* -0.032 0.067* ---1.012* 0.478* 1.263* --

0.236* 0.445* 0.124* -0.007* -0.067* 0.019 0.045* -0.079* -0.325 0.143* -0.063* 0.119* 0.044* -0.001 -0.008 0.054* 0.030* 0.058* 0.053* 0.093* -0.099* 0.034* 0.000 -0.073* 0.027* 0.198* -0.043* 0.067* --------

Teacher Characteristics Infield teaching First year teacher Second year teacher Third year teacher Fourth year teacher

Student Characteristics Prior grade math score (Std.) Prior grade reading score (Std.) Average peer test score (prior grade) Days absent Structural student mobility Moved since prior year Moved since prior year missing Within year move Within year move missing Underage student based on grade Overage student based on grade Academically or intellectually gifted Disabled student Free lunch Reduced lunch Lunch status missing Parent education less than high school

Parent education some college Parent education college graduate Parent education missing Black Hispanic Multiracial American Indian Asian Male LEP services recipient Previous LEP services recipient Algebra 2 English 1 Geometry Biology Chemistry Physical Science Physics

-------

Teacher Preparation and Student Achievement: Tables and Figures Civics and Economics US History

-0.035* -0.092*

---

---

---

0.067* --

-0.002* 0.041* 0.135* 0.018

-0.002* 0.070* 0.234* 0.126*

0.001* 0.033* 0.094* -0.019

-0.004* 0.058* 0.127* -0.040

-0.004* 0.056* 0.118* 0.016

Classroom Characteristics Students per classroom Classroom ability dispersion Advanced curriculum Remedial curriculum

School Characteristics School size (per 100) 0.0002* 0.0002* 0.0001* 0.000* 0.0002* School size squared 0.000* 0.000* 0.000* 0.000* 0.000* Total per-pupil expenditures ($100s) 0.000 0.000 0.000 0.001* 0.001* Average teacher supplement ($100s) 0.000 0.000 0.000 0.000* 0.000* Short-term suspension rate (per 100 0.000 0.000 0.000 0.000 0.000* students) Violent acts rate (per 1000 students) 0.000 0.000 -0.001* -0.002* -0.002* Free and reduced lunch mean 0.000 0.000 0.001 -0.001 -0.001* Black mean 0.000 -0.001 0.000 -0.001* 0.001* Hispanic mean 0.001 0.002 -0.003 0.002* 0.003* Multiracial mean 0.002 0.001 -0.008 0.014* 0.014* American Indian mean 0.000 0.000 0.000 0.000 0.001 Asian mean -0.001 0.000 0.001 -0.004 -0.003 Intercept -0.036 -0.200* -0.095* -1.203* -0.416* Note: Teachers with less than 5 years experience in 2004-05, 2005-06, 2006-07 & 2007-08 school years. *Indicates a given coefficient is significant at the .05 level.