North Carolina

North Carolina

i

EXECUTIVE SUMMARY Hope Street Group (HSG) launched its first State Teacher Fellows program in Kentucky in 2013 and has since expanded to Arizona, Hawaii, North Carolina, Tennessee, and Utah. With the aim of amplifying teacher voices to inform state and district policies that impact educators, the program provides training, tools, resources, and support to cohorts of teacher fellows in each state. In North Carolina, the teacher fellows are called Teacher Voice Network Fellows. In fall 2017, Teacher Voice Network Fellows distributed surveys to North Carolina educators and held 79 in-person focus groups with 358 participants on professional development/learning. A total of 7,563 teachers across North Carolina completed the survey, and 2,917 answered the online focus group questions. This report presents findings from the 10 survey questions and four focus group questions. The following is an overview of survey and focus group findings and associated recommendations, organized by topic area.

i

SUMMARY AND RECOMMENDATIONS FINDINGS:

RECOMMENDATIONS:

Approximately half of teachers reported that their principal is in charge of coordinating professional development/learning, while the other half reported that another party is in charge. Regardless of who is primarily responsible for coordinating professional development/learning, most teachers (65 percent) feel that they have the autonomy to direct their experience. When selecting professional development/learning, teachers consider whether it will improve their instruction, whether they have time to participate, and how it will affect their teacher evaluation ratings.

➢

Teachers noted that the following would enhance their satisfaction with professional development/learning: time to attend, implement, and reflect; understanding the relevance of the content to their instruction; modeling regarding how to apply the content in the classroom; additional compensation based on evidence that these opportunities impacted their instruction; follow-up and administrative support; peer collaboration opportunities; financial support; differentiated instruction; and classroom resources aligned with these opportunities.

In informing and developing a statewide micro-credentialing framework, NCDPI should review what other states have done. For example, the American Institutes for Research and BloomBoard offer resources from state leaders in Tennessee, Delaware, Arkansas, and Florida who have recently adopted micro-credentials as options for professional learning and educator licensure. ➢

Because NCDPI does not currently offer a central location for accessing microcredentials or license renewal credits, NCDPI should consider hosting a dedicated website to help teachers access these options, which would support effective statewide communication and dissemination of these opportunities.

➢

NCDPI should consider how it might support districts in developing the conditions that teachers identified as making them more likely to complete an evidence portfolio, including providing time during work hours. NCDPI should also consider offering incentives to teachers who submit evidence of their learning and using portfolios as evidence in teacher evaluations. Should NCDPI decide to pursue an option for teachers to submit portfolios as part of the teacher evaluation process, it will be important to help teachers understand the relationship between the

Teachers noted that participation-based professional development/learning requires less time and offers networking and collaboration opportunities, but that it also may be less relevant to teacher needs, lack classroom applicability, and be less engaging than competency-based professional development/learning. Teachers also offered insight on two competencybased learning options—micro-credentials and a competency-based continuum. Few teachers have experience with micro-credentials. When pursuing a competency-based continuum, most teachers prefer to complete it independently but check in with a small group or to follow the continuum with a small group. Approximately 57 percent of teachers have not submitted evidence or artifacts that documented their application of professional development/ learning, but most (48 percent) would be willing to do so. When creating portfolios of professional development/learning experiences, teachers expressed concerns around the time required, the qualifications of evaluators, whether it would be a

These findings align with those from the latest North Carolina Teacher Working Conditions survey (2016), where 33 percent of teachers disagreed that professional development is differentiated and 24 percent disagreed that they receive followup support. Teachers’ preferences for professional learning suggest that the North Carolina Department of Public Instruction (NCDPI) should continue to explore competency-based options, such as microcredentials, for teacher professional development/learning.

ii

meaningful process, and what would be included. However, teachers also shared that portfolios could offer opportunities for growth and selfreflection, document and provide evidence of how learning is applied, and offer feedback to teachers. Teachers would be most likely to complete a portfolio that demonstrates their application of competency-based professional learning if they had time during work hours to complete it, if they understood the relationship between the portfolio and their effectiveness as a teacher, or if the portfolio could be used as evidence in their teacher evaluation.

portfolio and their effectiveness as a teacher. ➢

Effective implementation of a microcredentialing framework and clear communication about the framework to districts and schools will be particularly important, as most teachers are unfamiliar with micro-credentials and several expressed concerns around how a framework would be implemented. Digital Promise offers several resources to support districts in this area.

Finally, teachers discussed their feelings regarding a compensation system partially based on submitting evidence of completing competency-based professional development/learning. Most support a system like this, while others do not or are unsure. Teachers also expressed concerns about fairness, requirements, available time, monetization of professional learning, and penalties for nonparticipation. Several teachers suggested alternative compensation systems, such as higher compensation overall, pay incentives for master’s degrees, or covered costs for professional development.

iii

TABLE OF CONTENTS

1

Introduction

3

Professional Development/Learning

9

Competency-Based Learning

22

Summary and Recommendations

.

iv

INTRODUCTION Hope Street Group (HSG) launched its first State Teacher Fellows (STF) program in Kentucky in 2013. Since that time, the program has expanded to Arizona, Hawaii, North Carolina, Tennessee, and Utah. With the aim of amplifying teacher voice to inform state and district policies that impact educators, the program provides training, tools, resources, and support to cohorts of teacher fellows in each state. In North Carolina, the STF program is called the Teacher Voice Network program, and the teacher fellows are Teacher Voice Network Fellows. Policy Studies Associates (PSA) in Washington, DC, has conducted independent evaluations of the program and has shared the data with the North Carolina Department of Public Instruction (NCDPI) and other state partners. HSG has used the data to make improvements to the STF program. PSA is currently conducting an independent evaluation of the Teacher Voice Network program in North Carolina for 2017–2018. HSG staff will use PSA findings to make evidence-informed decisions and programmatic improvements (if necessary), which they will share with NCDPI in summer 2018.

Fall Data Collection In fall 2017, North Carolina Teacher Voice Network Fellows (Appendix A) distributed surveys to educators across the state and held 79 in-person focus groups with 358 participants on professional development/learning (Appendix B). EducationNC (EdNC) hosted and managed the survey, and NCDPI helped to advertise this opportunity. Network Fellows hosted focus groups at their schools and virtually across the state. A total of 7,563 teachers across the state completed the survey, and 2,917 answered the online focus group questions. All of the districts participated,

including charter schools and North Carolina Virtual Public Schools (Appendix C).

Analysis and Reporting HSG secured the services of Magnolia Consulting, located in Charlottesville, Virginia, to independently analyze survey and focus group data and to report findings. Magnolia Consulting presents descriptive data in figures within the body of the report. Magnolia Consulting also examined survey items for differences by demographic variables. Because there were no notable differences, these findings are not discussed. Magnolia Consulting also disaggregated survey data by education district, school type, and rural versus non-rural schools (Appendix D). A separate appendix, with complete “Other” text box responses, is available upon request to the North Carolina Teacher Voice Network Director. In addition to the survey figures, HSG and Magnolia Consulting developed figures— located in the body of the current report—to highlight the most commonly referenced themes for each focus group question (Appendix E), with darker shading reflecting themes noted by a higher number of respondents and lighter shading reflecting themes noted by a lower number of respondents. Additionally, HSG and Magnolia Consulting collaborated to develop recommendations, and Network Fellows offered their feedback on an initial draft of the recommendations (Appendix F).

Report Layout and Design The layout of this report differs from previous years’ reports. After consultation with Network Fellows, state-based partners, and NCDPI staff, HSG changed the report layout to provide additional graphics and a layout more 1

conducive to social media sharing; Network Fellows use social media heavily to convey report findings to their professional learning networks. Additionally, the new layout is intended to improve the readability and overall understanding of the data analysis and recommendations.

HSG is committed to making changes to the layout and organization of this report to help NCDPI better understand and act on the data and recommendations. As such, HSG welcomes feedback on the report from NCDPI. The North Carolina Teacher Voice Network Director, along with national office staff, will promptly attend to any feedback.

2

PROFESSIONAL DEVELOPMENT/LEARNING

1

Relevant Survey Findings Teachers were divided regarding who has primary responsibility for coordinating professional development/learning at their school. Approximately half of teachers reported that their principal is primarily responsible. The other half of teachers reported that other parties, including themselves, a team of teachers assigned to this task, a lead teacher, or another individual or group are primarily responsible (Survey Question 1). Principal

I am responsible for coordinating my professional development/learning.

Lead teacher

Other

Team of teachers assigned to coordinate professional development/learning

Approximately half of teachers indicated that their principal is primarily responsible for coordinating the professional development/learning at their school.

15%

8%

%

13%

53%

10%

While principals or other parties often have this responsibility, most teachers (65 percent) agreed that they have autonomy to direct their professional development/learning, while 35 percent disagreed (Survey Question 6). 3

Strongly disagree 7%

Strongly agree 12%

Disagree 28%

%

Nearly two thirds of teachers agreed that they have the autonomy to direct their professional development/ learning, while one third disagreed.

Agree 53%

When teachers choose their own professional development/learning opportunities, most consider the following: the likelihood that the course or session will improve their instruction, whether costs will be covered, or whether they have professional time available to participate. Most teachers (57 percent) agreed that their teacher evaluation ratings guide them in their decision-making around professional development/learning selection, but 43 percent disagreed (Survey Questions 3 and 7).

When choosing their own professional development/learning opportunities, teachers consider whether the course or session will improve their instruction. 37%

LIKELIHOOD OF COURSE/SESSION IMPROVING INSTRUCTION

21%

IF COSTS WILL BE COVERED BY SCHOOL/DISTRICT/OTHER

16%

PROFESSIONAL TIME AVAILABLE TO PARTICIPATE

12% 8% 5% 1%

LIKELIHOOD OF COURSE/SESSION IMPROVING MY LEADERSHIP AS A TEACHER PERSONAL TIME AVAILABLE TO PARTICIPATE I DO NOT SELF-SELECT PROFESSIONAL LEARNING/DEVELOPMENT IF PEERS ARE ATTENDING THE SAME COURSE/SESSION

4

Strongly disagree 8%

More than half of teachers agreed that their evaluation ratings help guide their decision-making in selecting professional development/ learning opportunities, but 43% disagreed.

Strongly agree 8%

Disagree 35%

% Agree 49%

When discussing how they apply professional development/learning to their instruction, many teachers noted the following challenges: a lack of time to reflect after learning, a lack of relevance to their instruction, and difficulty in transferring content to their instruction (Survey Question 2).

Many teachers indicated that a lack of time after the professional development/learning to reflect on the purpose is a challenge when applying it to their instruction.1 39%

LACK OF TIME AFTER THE PROFESSIONAL DEVELOPMENT/LEARNING TO REFLECT ON THE PURPOSE

32%

NOT RELEVANT TO MY INSTRUCTION

31% 22%

DIFFICULTY TRANSFERRING PROFESSIONAL DEVELOPMENT/LEARNING TO INSTRUCTIONAL ENVIRONMENT LACK OF CONCRETE MODELS OF APPLICATION LACK OF TIME BEFORE THE PROFESSIONAL DEVELOPMENT/LEARNING TO REFLECT ON THE PURPOSE LACK OF LEADERS’ SUPPORT IN IMPLEMENTING INSTRUCTIONAL CHANGES BASED ON PROFESSIONAL DEVELOPMENT/LEARNING EXPERIENCES

19% 15% 15% 14% 11%

1

OTHER POOR QUALITY OF THE PROFESSIONAL DEVELOPMENT/LEARNING INSTRUCTION POOR QUALITY OF THE PROFESSIONAL DEVELOPMENT/LEARNING CONTENT

Due to a survey issue, only 6,303 teachers completed this question.

5

Teachers responded that the following would increase their satisfaction with professional development/learning experiences: a better understanding of relevance to their instruction, increased compensation based on evidence that it impacted their instruction, and time during contract hours to reflect on it (Survey Question 5).

Better understanding of the relevance to instruction or evidence-based compensation related to learning would increase teacher satisfaction with professional learning experiences.

29%

BETTER UNDERSTANDING OF HOW THE PROFESSIONAL DEVELOPMENT/LEARNING IS RELEVANT TO MY INSTRUCTION

27%

INCREASED COMPENSATION BASED ON EVIDENCE THAT MY PROFESSIONAL DEVELOPMENT/LEARNING HAS IMPACTED MY INSTRUCTION

20%

RELEASE TIME FROM INSTRUCTION TO REFLECT ON PROFESSIONAL DEVELOPMENT/LEARNING

6% 6% 6% 5%

ABILITY TO DEMONSTRATE TO OTHERS HOW PROFESSIONAL DEVELOPMENT/LEARNING HAS IMPROVED MY INSTRUCTION OTHER

RECOGNITION FOR MY PROFESSIONAL DEVELOPMENT/LEARNING

NEW PATHWAYS FOR ADDITIONAL RESPONSIBILITIES BASED ON MY PROFESSIONAL DEVELOPMENT/LEARNING

Relevant Focus Group Findings In focus groups, teachers discussed what supports are required to enhance their professional development/learning experiences. Teachers often requested additional time to attend and implement their learning, follow-up support, opportunities to collaborate with peers, administrative support, and financial support to cover costs. Teachers also requested relevant and differentiated professional development that models how content could be applied in the classroom and more classroom resources aligned with professional learning opportunities (Focus Group Question 2).

6

Higher

Number of Respondents

Provide opportunities to collaborate with peers

Provide follow-up support Provide professional development that is relevant and differentiated

Lower

Provide classroom resources

Support for Professional Development

Offer administrative support

Offer financial support

Provide time in the schedule

Include real-world modeling

PROVIDE TIME IN THE SCHEDULE Teachers commented that they need time in their schedule to participate in professional development/learning experiences and to incorporate what they learned into their teaching. “Adequate time to participate, reflect, and put into practice.” “Having the time away from the classroom, not outside of my work hours.”

OFFER FINANCIAL SUPPORT Teachers expressed a need for financial support to cover expenses related to professional development. For example, many teachers mentioned enrollment fees, travel costs, and substitute teacher costs, or suggested stipends to attend. “Funding for attendance and class coverage.” “Being compensated, especially not having to pay for a sub.”

7

PROVIDE PROFESSIONAL DEVELOPMENT THAT IS RELEVANT AND DIFFERENTIATED Teachers requested choices and input in selecting professional development (PD)/ learning experiences. Additionally, teachers requested a wider variety of topics that are relevant to their specific learning needs. “Better choices by school systems for relevant and practical professional development would help enhance the experience.” “I would like to be able to have more choice on my PD.”

PROVIDE FOLLOW-UP SUPPORT Teachers requested that someone be available to provide follow-up support, particularly around implementation. Many teachers added that a designated personsuch as a teacher leader, coach, or mentorcould provide this support. “Always, always, always follow up! Provide support, feedback, follow through with PD.” “Support. Having that follow-up and coaching opportunities.”

PROVIDE OPPORTUNITIES TO COLLABORATE WITH PEERS Teachers stated that working with their peers, particularly teachers in the same content area, is helpful during and after professional development. “Opportunity to engage with my colleagues that are in the same role I serve at other schools.” “It would be helpful to have time to reflect with colleagues about what worked.”

PROVIDE RESOURCES TO USE IN THE CLASSROOM Teachers requested various classroom resources that align with their professional development experiences, such as handouts from the professional learning and technology to implement professional development content in the classroom. “Take-away materials that are easily applied to my classroom instruction.” “Materials and resources that are mentioned in the PD.”

OFFER ADMINISTRATIVE SUPPORT Teachers requested that school and district administrators support teachers in their professional development and learning experiences. “Administration that supports furthering professional goals.” “Administrative support for training and implementing.”

8

INCLUDE REAL-WORLD MODELING IN PROFESSIONAL DEVELOPMENT Teachers suggested that professional development opportunities should model how content could be used in the classroom environment. “Having it modeled in my classroom with all levels of students.” “Hands-on applications with visual modeling.”

2

COMPETENCY-BASED PROFESSIONAL LEARNING

Relevant Survey Findings NCDPI defines competency-based learning as learning activities that result in the participants demonstrating evidence of achievement of specified outcomes. As such, all questions in this section relate to demonstrating evidence of professional learning, through artifacts or microcredentials. Teachers provided their feedback on two competency-based learning options—microcredentials and a competency-based continuum. Teachers reported having little experience with micro-credentials, with 74 percent sharing that they are unfamiliar with them and have never explored them. Another 11 percent of teachers reported that they are familiar with microcredentials but have never explored them (Survey Question 8).

8%

7%

% 74%

11%

Yes, I have explored and obtained microcredentials. Yes, I have explored obtaining microcredentials, but not obtained them. No, I have not explored micro-credentials but I am familiar with what they are. No, I have not explored micro-credentials and I am unfamiliar with what they are.

Nearly three quarters of teachers have not explored microcredentials and are unfamiliar with what they are.

9

In pursuing a competency-based continuum (i.e., a series of related modules), most teachers said they would prefer to either complete the continuum independently but check in regularly with a small group or to follow the same continuum with a small group of peers (Survey Question 9). To follow the same continuum in a small group of peers

To complete the continuum independently and present to my peers at the conclusion

To complete the continuum independently but check in regularly with a small group

10%

%

44%

The majority of teachers prefer to either follow the same continuum in a small group of peers or to complete it independently but regularly check in with a small group.

46%

Approximately 57 percent of teachers have not submitted evidence or artifacts that document their application of professional development/learning in their classroom or school, but most would be willing to do so (Survey Question 4). Yes, I have submitted evidence/artifacts.

No, I have not and would not be willing to do so.

No, I have not but am willing to do so.

9%

%

43%

Over half of teachers have not submitted evidence or artifacts documenting that they applied their professional learning, but most are willing to do so.

48%

10

Teachers would be most likely to compile an evidence portfolio that demonstrates their application of competency-based learning if their administrator provided them time during work hours to complete their portfolio, if they understood the relationship between the portfolio and their effectiveness as an instructor, or if the portfolio could be used as evidence in their teacher evaluation (Survey Question 10).

Teachers would be most likely to compile an evidence portfolio if their administrator allocated time during the workday or they better understood the relationship of the portfolio to instructional effectiveness. 32%

MY ADMINISTRATOR ALLOCATED TIME DURING THE WORKDAY FOR ME TO WORK ON MY PORTFOLIO

23%

I BETTER UNDERSTOOD THE RELATIONSHIP OF THE PORTFOLIO TO MY INSTRUCTIONAL EFFECTIVENESS

18%

THE PORTFOLIO COULD BE USED AS EVIDENCE IN MY TEACHER EVALUATION

9% 7% 6% 4%

OTHERS IN MY SCHOOL WERE COMPILING A SIMILAR PORTFOLIO OTHER

I HAD A MENTOR WHO HELPED ME COMPILE MY PORTFOLIO. I HAD A MENTOR WHO FOLLOWED UP WITH ME TO APPLY THE EVIDENCE COLLECTED TO IMPROVE MY EFFECTIVENESS

Relevant Focus Group Findings Teachers discussed some of the challenges and opportunities related to different types of professional development, particularly participation-based versus competency-based. Several teachers highlighted challenges and opportunities related to the time required to attend, the relevance to their teaching, and the application to their instruction. Many teachers noted that attending and implementing professional development can be time consuming, and some commented on the benefits of shorter sessions. Teachers also commented on frustrations associated with attending irrelevant professional development and noted benefits when professional development is relevant to teacher needs. Additionally, teachers cited difficulties associated with a lack of application and commented that they appreciate when they can apply what they learn because it improves their teaching. Teachers also noted several additional challenges, such as uninteresting professional development, a lack of follow-up support after professional development, and a lack of funding to cover costs. One additional opportunity that teachers mentioned was being able to network and collaborate with their peers at professional development sessions. When discussing types of professional development, several teachers noted that participation-based opportunities require less time to complete and offer opportunities 11

for networking and peer collaboration, but they may be less relevant to teacher needs, lack classroom applicability, and be less engaging for participants than competency-based professional development (Focus Group Question 1). Higher


No follow-up support

Lack of funds to cover costs

Lack of engagement or interest

Irrelevant professional development Time required to attend and implement

Networking and peer collaboration

Relevant professional development

PD Challenges and Opportunities

Lack of application and forgetting of material

Lower

Able to improve teaching Less time required to attend

CHALLENGE: TIME REQUIRED TO ATTEND AND IMPLEMENT Teachers commented that the time required to attend professional development, reflect upon it, and implement it in their classroom is a challenge. “Time is needed to implement. Time is always an issue.” “Not enough time to reflect, plan and use.”

CHALLENGE: IRRELEVANT PROFESSIONAL DEVELOPMENT Teachers stated that it is challenging for teachers when professional development is not personalized, differentiated, or relevant to their instruction. Several teachers mentioned this topic when discussing participation-based professional development. “The challenges would be a lack of diversity and personalized learning.”

12

“Another challenge is everybody is on different levels. Some material is taught above and below people’s instructional levels.”

CHALLENGE: LACK OF APPLICATION AND FORGETTING OF MATERIAL Teachers cited challenges when professional development has no application to the classroom and is quickly forgotten afterward. Several teachers mentioned this topic when discussing participation-based professional development. “Participation-based is more likely to be forgotten.” “It’s easy to forget and not implement material covered once.”

CHALLENGE: LACK OF ENGAGEMENT OR INTEREST IN PROFESSIONAL DEVELOPMENT Teachers shared that some professional development is boring, and referenced a lack of engagement or interest. On a related point, several teachers commented that disinterested peers can be distracting. Several teachers referenced this topic when discussing participation-based professional development. “During sit and get, people who know they will get credit no matter what may …work on other things and mentally check out because they signed in.” “Some of the sessions are so long and boring, that you just can’t comprehend all that you need.”

CHALLENGE: NO FOLLOW-UP SUPPORT Teachers commented on a lack of follow-up support after some professional development courses, particularly participation-based professional development. “I haven’t seen real follow through.” “Sit and get does not always have follow through.”

CHALLENGE: LACK OF FUNDS TO SUPPORT PROFESSIONAL DEVELOPMENT Teachers shared that their school or district lacks funds to support professional development costs or attendance. “One of the challenges would have to be money.” “Lack of funds to attend PD.”

OPPORTUNITY: NETWORKING AND PEER COLLABORATION AT PROFESSIONAL DEVELOPMENT Teachers noted that some professional development provides teachers with the opportunity to network and collaborate with their peers. Several teachers mentioned this topic in reference to participation-based professional development. “One benefit is the opportunity to meet new people.”

13

“I love to network. Being in the same space allows me to gather a lot of information from others. I appreciate any information that I receive.”

OPPORTUNITY: RELEVANT PROFESSIONAL DEVELOPMENT Teachers commented on the opportunities associated with attending professional development that is personalized, differentiated, and relevant to their needs. Several teachers mentioned this topic in reference to competency-based professional development. “I think that competency-based professional learning can take into account where each professional is coming from.” “Opportunity = give people a chance to go for what they are interested in and will benefit their students the most.”

OPPORTUNITY: ABLE TO IMPROVE TEACHING Teachers stated that an opportunity of professional development is the ability to improve their teaching and apply content. “I learn best by doing and so I prefer to learn something, then apply it within a window of time to show that I have “mastered” what it is I was taught.” “It’s better to PLEARN—practice and learn and take something with you from the training that you can directly apply to your instruction.”

OPPORTUNITY: LESS TIME REQUIRED TO ATTEND Teachers commented on the opportunities associated with shorter professional development sessions, which require less time to attend. Several teachers mentioned this topic in reference to participation-based professional development. “Participation based has a concrete time to participate.” “Opportunities would include the ability to get information at one time cutting down on time away from the classroom.”

In focus groups, teachers spoke to the benefits and challenges of compiling an educational portfolio of annual professional development/learning experiences on which they can receive feedback. Teachers overwhelmingly mentioned that the time required to create a portfolio would be a challenge, with many noting that they already have too much to do. Teachers also questioned the qualifications of the individuals evaluating their portfolio, whether the portfolio process would be meaningful, and what would be included. When discussing benefits, teachers noted that portfolios could offer the opportunity for self-reflection and growth, document how they apply professional learning, provide physical evidence of learning—which would be useful for teacher evaluations and job interviews—and offer feedback to teachers on their practices. Some teachers stressed the importance of receiving timely, quality feedback on their portfolio (Focus Group Question 4).

14

Higher


What is included?

Would it be meaningful?

What is the quality of feedback?

Lower

Offers opportunity for self-reflection and growth Shows how professional learning is applied

Compile evidencebased portfolio

Provides a physical record

Who is the evaluator? Finding time to create a portfolio

Provides feedback on practices

CHALLENGE: FINDING TIME TO CREATE A PORTFOLIO Teachers commented that creating a portfolio of professional learning experiences would take considerable time, when many teachers already feel overwhelmed with their workload. “I just foresee a portfolio being ‘one more thing.’” “Lack of time, How can I possibly add one more thing to my plate?”

15

CHALLENGE: WHO IS THE EVALUATOR? Teachers questioned who would provide feedback on the portfolio, particularly in terms of their content knowledge and training. “It only benefits ME if the people providing feedback comprehend the subject I teach.” “Who can give someone feedback on their experience? Will their be training for the feedback?”

CHALLENGE: WHAT IS THE QUALITY OF FEEDBACK? Teachers emphasized the importance of providing quality feedback on portfolios. Some teachers stressed the importance of sharing timely feedback with teachers on a regular basis. “Actually receiving QUALITY feedback is the largest obstacle.” “There is a benefit if the feedback is genuine and timely.”

CHALLENGE: WOULD IT BE MEANINGFUL? Teachers questioned whether the educational portfolio would be meaningful or would serve as additional paperwork. “Doing something like this feels like a hoop.”

“The negative is that it will be just another item to add to the paperwork we already do and another one that we feel will not be actually used...”

CHALLENGE: WHAT IS INCLUDED? Teachers questioned what would be included in the portfolio and which pieces to highlight. “The challenge will be: what and/or how do you compile it or add to it.” “The challenge is determining what information to select that would best represent the work that you’ve accomplished.”

BENEFIT: OFFERS OPPORTUNITY FOR SELF-REFLECTION AND GROWTH Teachers suggested that portfolios could provide an opportunity for self-reflection and growth. For example, some teachers noted that a portfolio could provide insight into their instructional strengths and weaknesses. “One benefit of a portfolio is the ability to reflect…what worked and didn’t work.” “The benefits would be that I can monitor my growth. It would allow me to challenge myself.”

16

BENEFIT: SHOWS HOW PROFESSIONAL LEARNING IS APPLIED Teachers commented that it would be beneficial to provide evidence of how professional development is applied in the classroom. Some teachers believed this evidence would hold teachers accountable. “I would love this because it would be ongoing and would allow for me to show what I have actually ‘learned’ or worked on that year.” “It would be a benefit that it would hold people accountable to what you are learning.”

BENEFIT: PROVIDES A PHYSICAL RECORD Teachers noted that a portfolio could serve as a physical record of professional learning for multiple purposes. For example, teachers mentioned teacher evaluations, job interviews, or a general record of teacher professional learning. “I had to compile this information early in my career, and I have used this information in job interviews.” “One benefit would be that when you got to your evaluation, you would have these things that you did that show leadership or empowering students in the classroom. It would also be something that you can take with you if you go to another job or another school.”

BENEFIT: PROVIDES FEEDBACK ON PRACTICES Teachers stated that a portfolio review could provide feedback to teachers on their practices. “The benefit is getting critiqued on some of your work.” “Benefits—feedback that could let me know if I am on the right track with my teaching.”

17

Teachers also discussed their feelings regarding a compensation system partially based on evidence of completed competency-based professional learning experiences. Most teachers support a system like this, while others do not or are unsure. Some teachers expressed concerns about this type of system, including fairness, evidence requirements, limited time to participate, monetization of professional learning, and penalties for non-participation. Several teachers suggested alternative compensation systems, such as offering higher overall compensation, restoring pay incentives for master’s degrees, or covering costs associated with professional development. A final group of teachers emphasized that all professional development opportunities should be high quality and relevant to teacher needs (Focus Group Question 3). Higher


Limits of teacher time

Instead, offer higher compensation overall

Concerns about evidence requirements

Concerns about fairness of system

Lower

Concerns about monetizing professional learning Instead, restore master’s degree pay Instead, pay for fees associated with professional learning

EvidenceBased Compensation System

Do not create a penalty for nonparticipation

Unsure

Do not support Support

PD should be relevant and high quality

SUPPORT Many teachers indicated they would support a compensation system partially based on evidence of completed competency-based professional learning. Several teachers added that this type of system would offer a strong incentive for teachers to pursue professional learning and implement it in the classroom. “I feel that would be great incentive to complete PD.”

18

“I think it would be a good motivator. Often extra efforts teachers make are not compensated.”

DO NOT SUPPORT Other teachers indicated they are against a compensation system partially based on evidence of completed competency-based professional learning. Several of these teachers suggested that this type of system does not give credit to teacher professionalism. “This is a terrible idea and would mean more hoops to jump through to get paid.” “I don’t care for that idea. I am not going to chase irrelevant training in order to plump up my salary. That’s an insult.”

UNSURE Other teachers expressed uncertainty without knowing more about the proposed system. “I would need more information before responding.” “I am not sure at this time.”

CONCERNS ABOUT FAIRNESS OF SYSTEM Teachers expressed concerns about unequal access to professional learning opportunities. For example, many commented that educators who teach outside of “core” subjects have less access to relevant professional development or few opportunities to apply it in the classroom. “Would need to be fair to all teachers, even those that teach ‘special’ subject in an elementary school. Like art, music, PE, etc.” “I don’t think it would apply fairly to those with non-core classes. We can’t show as much evidence as the core subjects can.”

CONCERNS ABOUT EVIDENCE REQUIREMENTS Teachers shared concerns about the type of evidence required, who would be evaluating the evidence, and whether the evidence would be objectively evaluated. “I think I would be fine with it, if the criteria for pass/fail is set & reliable.” “It might get tricky because the evidence you complete might not represent what happens in the classroom every day...”

LIMITS OF TEACHER TIME Teachers expressed concerns over how much time would be required of teachers to participate.

19

“While extra money always sounds good, I’m not sure how I feel about having one more thing on my plate to do and keep up with.” “There is not enough time in the day to keep some sort of portfolio on this. It would be one more thing done at teachers’ homes and probably would get last priority since teaching is our first.”

INSTEAD, OFFER HIGHER COMPENSATION OVERALL Teachers suggested that educators should have a higher compensation structure overall rather than this type of system. “Would prefer overall higher compensation to increase the competitiveness of the field and raise the bar for all teachers instead of finding these other ways to compensate people.” “Teachers need to be compensated for what they do all day already. Asking us to do more for less is a slap in the face to teachers who already work very hard for the bare minimum.”

CONCERNS ABOUT MONETIZING PROFESSIONAL LEARNING Teachers expressed concern that attaching compensation to professional learning would lead some teachers to complete courses for additional pay rather than to benefit their students. “I feel that teachers would spend more time trying putting together a portfolio for compensation rather than using that time [to] access students and create lessons...” “Not a fan, because of repeated studies that demonstrate that people will game the system to increase their compensation. Employees have been repeatedly shown to focus on the goals that increase compensation and neglect those that don’t. I am insulted by being offered more money to participate in PD, because it sends the message that I am otherwise uninterested in professional learning.”

INSTEAD, RESTORE MASTER’S DEGREE PAY Teachers requested that NCDPI bring back compensation for master’s degrees rather than this type of system. “I’d rather have compensation for obtaining a master’s degree.” “The first step would be to restore the pay incentive for graduate education and degrees. The ‘learning atmosphere’ is stagnant in comparison to what it felt like to work with colleagues who were growing through their graduate work.”

INSTEAD, PAY FOR FEES ASSOCIATED WITH PROFESSIONAL LEARNING Teachers suggested a system that would reimburse teachers for all fees associated with professional learning, such as substitute teacher costs, enrollment fees, and travel.

20

“I don’t think I should have to be paid to attend PD. I WOULD like to have the fees for PD paid for. I would go to a lot more if I didn’t have to put out my own limited funds to do so.” “It would be nice. I have spent thousands in order to renew my license. I have had to pay for the conference, substitute teacher, gas, etc. every time.”

DO NOT CREATE A PENALTY FOR NON-PARTICIPATION Teachers requested that the system offer additional pay that is added to regular teacher salaries rather than a system that takes base pay away for non-participation. Some teachers added that no teachers should be penalized for non-participation. “I like the idea of being able to earn more money as long as it doesn’t take away from my base salary.” “I don’t feel like any money should ever be taken away from teachers. However, if teachers could choose to earn additional money for completing professional development, that would be appropriate.”

PROFESSIONAL LEARNING SHOULD BE RELEVANT AND HIGH QUALITY Teachers stated that any professional learning opportunities offered in this system should be high quality and relevant to teacher interests. “If the PD is relevant and proven to make me a better teacher, I would agree to the structure. Don’t just make me do training of something we are already doing just because it has been renamed with a fancy new name!” “This only makes sense if the professional development is applicable and relevant.”

21

3

SUMMARY AND RECOMMENDATIONS

Summary

In summary, regardless of who is responsible for coordinating professional development/learning at their school, most teachers (65 percent) feel that they have the autonomy to direct their experience. When selecting a professional development/learning opportunity, teachers consider the likelihood that it will improve their instruction, whether costs will be covered, whether they have time available to participate, and how it will affect their teacher evaluation ratings. Teachers noted that the following would enhance their satisfaction with professional development/learning experiences: • time during contract hours to attend professional development, implement what they learn, and reflect on their learning; • a better understanding of how professional development is relevant to their instruction; • modeling by instructors regarding how content could be applied in the classroom; • additional compensation based on evidence that professional development impacted their instruction; • follow-up support; • opportunities for peer collaboration; • administrative support; • financial support to cover costs; • differentiated instruction; and • additional classroom resources aligned with professional learning opportunities. When discussing types of professional development, teachers noted that participation-based opportunities require less time to complete and offer opportunities for networking and peer collaboration, but they may also be less relevant to teacher needs, lack classroom applicability, and be less engaging for participants than competency-based professional development. Teachers also offered their perspectives on two competency-based learning options—micro-credentials and a competency-based continuum. Few 22

teachers have experience with micro-credentials, with 84 percent of teachers reporting that they have never explored them and 74 percent reporting that they are unfamiliar with them. When pursuing a competency-based continuum (i.e., a series of related modules), most teachers prefer either to complete the continuum independently but check in regularly with a small group (46 percent) or to follow the continuum with a small group of peers (44 percent). Approximately 55 percent of teachers have not submitted evidence or artifacts that document their application of professional development/learning in their classroom or school, but most of these teachers would be willing to do so. When discussing educational portfolios of annual professional development/learning experiences, teachers expressed concerns about the time required to create a portfolio, the qualifications of individuals evaluating the portfolio, whether the process would be meaningful, and what would be included. However, teachers also noted that portfolios could offer selfreflection and growth opportunities, document how professional learning is applied, provide physical evidence of learning, and offer feedback to teachers on their practices. Teachers would be most likely to compile an evidence portfolio that demonstrates their application of competency-based learning if: • their administrator provided them time during work hours to complete their portfolio, • they understood the relationship between the portfolio and their effectiveness as an instructor, or • the portfolio could be used as evidence in their teacher evaluation. In focus groups, teachers discussed their feelings regarding a compensation system partially based on evidence of completed competency-based professional learning experiences. Most teachers support a system like this, while others do not or are unsure. Teachers also expressed concerns about fairness, evidence requirements, limited time to participate, monetization of professional learning, and penalties for non-participation. Several teachers suggested alternative compensation systems, such as offering higher overall compensation, restoring pay incentives for master’s degrees, or covering costs associated with professional development.

23

RECOMMENDATIONS ➢ Teachers highlighted the benefits of professional learning that is relevant, applicable to their classroom, differentiated, and low cost, and that includes follow-up support. Teachers also supported submitting evidence of their learning and receiving additional compensation based on that evidence. These findings align with those from the latest North Carolina Teacher Working Conditions survey (2016), where 33 percent of teachers disagreed that professional development is differentiated and 24 percent disagreed that they receive follow-up support. Teachers’ preferences for professional learning suggest that NCDPI should continue to explore competency-based options, such as micro-credentials, for teacher professional development. Recent microcredentials research suggests that teachers who earn micro-credentials find value in earning more and that micro-credentials allow teachers to apply and reflect on professional learning in their own classroom (Friday Institute, 2016). To inform and define the development of a statewide micro-credentialing framework, NCDPI should review work in other states. For example, American Institutes for Research (2017) offers a report on three states— Tennessee, Delaware, and Arkansas—that recently adopted micro-credentials as options for professional learning and educator licensure. The report notes the importance of identifying a purpose, starting with a pilot, offering teachers some choices, and communicating effectively. Additionally, BloomBoard (2017) offers an ondemand webinar in which state education leaders in Florida, Tennessee, and Delaware discuss their experiences with micro-credentials and share perceived impacts on teachers. ➢ Because NCDPI does not currently provide a central location for accessing micro-credentials or license renewal credits, NCDPI should consider hosting a dedicated site to assist teachers in locating these options, which would support effective statewide communication and dissemination about these opportunities.* ➢ NCDPI should consider how it might support districts in developing the conditions that teachers identified as making them more likely to compile an evidence portfolio, such as time during work hours to create a portfolio. NCDPI should also consider offering incentives to teachers who submit portfolios and using portfolios as evidence in teacher evaluations. Should NCDPI decide to pursue an option for teachers to submit portfolios as part of the teacher evaluation process, it will be important to help teachers understand the relationship between the portfolio and their effectiveness as an instructor, as teachers indicated that this relationship is unclear. ➢ Once NCDPI finalizes a statewide framework, it should share the Micro-Credential Implementation Roadmap from Digital Promise (2017) with districts and schools. This resource emphasizes six overarching implementation phases, each with embedded strategies and resources. The phases are exploration; planning; preparation; implementation; supporting, engaging, and tracking; and reflection, calibration, and resetting. For instance, the planning phase includes resources to support development of micro-credential goal setting and focus areas, creation of a step-by-step plan for implementation, and planning of micro-credential stacks that are aligned to student needs. Effective implementation and clear communication of the framework to districts and schools will be particularly important, as most teachers are unfamiliar with micro-credentials, and several have expressed concerns around how these systems will be implemented (e.g., fairness, evidence requirements, qualifications of individuals evaluating the portfolio). Digital Promise (2017) also offers several resources to support clear communication, including videos and infographics for teachers on micro-credentials and a guide for districts to support the use of social media in promoting the framework.

24 *A Teacher Voice Network Fellow suggested this recommendation.

➢

NORTH CAROLINA APPENDIX

25

Appendix Description The following pages provide additional information on Teacher Voice Network Fellows, survey and focus group questions, survey respondent demographics, disaggregated survey findings, number of respondents referencing each theme by focus group question, Network Fellows’ feedback on the draft recommendations, sample representativeness, and technical issues related to survey distribution.

Ai

APPENDIX TABLE OF CONTENTS A1

Appendix A

A2

Appendix B

A7

Appendix C

A13

Appendix D

A38

Appendix E

A41

Appendix F

About Network Fellows

Survey and Focus Group Questions

Demographics

Survey Findings

Focus Group Findings

Recommendations Survey

A45

Appendix G

A47

Appendix H

Sample Representativeness

Technical Issues

Aii

Appendix A This appendix provides a list of the North Carolina Teacher Voice Network Fellows and their respective districts.

Teacher Voice Network Fellows Marlow Artis Chapel Hill-Carrboro City Schools

Erik Matticola Onslow County Schools

Douglas Price Charter School/Durham

Tracy Bell Chapel Hill-Carrboro City Schools

Kimberly Mawhiney Currituck County Schools

Joanna Schimizzi North Carolina Virtual Public Schools

Jacqueline Cheek Maureen Joy Charter School

Elaine Miles Chapel Hill-Carrboro City Schools

Jessica Stockham McDowell County Schools

Christine Fisher Guilford County Schools

Marquitta Mitchell Charlotte-Mecklenburg Schools

Isaac Wells Henderson County Public Schools

Lauren Genesky Wake County Public School System

Jerica Morgan Lee County Schools

Christy Wilson Craven County Public Schools

Marci Harvey Winston-Salem/Forsyth County Schools

Brandon Morrison Guilford County Schools

Jaclyn Holland Chapel Hill-Carrboro City Schools

Linda Myers Surry County Schools

Mika Hunter Durham Public Schools

Justin Parmenter Charlotte-Mecklenburg Schools

Marcia Alicia Lyons Chapel Hill-Carrboro City Schools

Ben Pendarvis Cherokee County Schools

Tricia Lyons Charlotte-Mecklenburg Schools

Meredith Pinckney Wake County Public School System

A1

Appendix B This appendix provides the North Carolina survey, focus group, and demographic questions.

A2

1) Who is primarily responsible for coordinating the professional development/learning you receive at your school? a. Principal b. Lead Teacher c. Team of teachers assigned to coordinate professional development/learning d. I am responsible for coordinating my professional development/learning. e. Other (Write in Answer) 2) Which of the following do you believe are challenges when applying professional development/learning to your instruction? (Check all that apply) a. Poor quality of the professional development/learning instruction b. Poor quality of the professional development/learning content c. Lack of time before the professional development/learning to reflect on the purpose d. Lack of time after the professional development/learning to reflect on the purpose e. Not relevant to my instruction f. Difficulty transferring professional development/learning to instructional environment g. Lack of concrete models of application h. Lack of leaders’ support in implementing instructional changes based on professional development/learning experiences i. Other (Write in Answer) 3) Which of the following do you consider most when you choose your own professional development/learning opportunities? a. Professional time available to participate b. Personal time available to participate c. If costs will be covered by school/district/other d. If peers are attending the same course/session e. Likelihood of course/session improving instruction f. Likelihood of course/session improving my leadership as a teacher g. I do not self-select professional development/learning 4) Have you submitted evidence or artifacts documenting that you have applied professional development/learning in your classroom/school? a. Yes, I have submitted evidence/artifacts. b. No, I have not but am willing to do so. c. No, I have not and would not be willing to do so. 4.a) If you selected “Yes, I have,” what have you submitted? (Write-in Answer) 4.b) If you selected “No I have not and would not be willing to do so” why would you not be willing to? (Write-in Answer) 5) Which of the following would likely increase your satisfaction with professional development/learning experiences? a. Recognition for my professional development/learning (e.g., official certificate from your district or acknowledgment from administrator in front of colleagues) b. Better understanding of how the professional development/learning is relevant to my instruction A3

c. New pathways for additional responsibilities based on my professional development/learning d. Ability to demonstrate to others how professional development/learning has improved my instruction e. Increased compensation based on evidence that my professional development/learning has impacted my instruction f. Release time from instruction to reflect on professional development/learning g. Other (Write-in Answer) 6) a. b. c. d.

I feel that I have autonomy to direct my professional development/learning. Strongly Agree Agree Disagree Strongly Disagree

7) My teacher evaluation ratings guide my decision-making in selecting professional development/learning opportunities. a. Strongly Agree b. Agree c. Disagree d. Strongly Disagree 8) a. b. c. d.

Have you explored and/or obtained micro-credentials for re-licensure? Yes, I have explored and obtained micro-credentials. Yes, I have explored obtaining micro-credentials, but not obtained them. No, I have not explored micro-credentials but I am familiar with what they are. No, I have not explored micro-credentials and I am unfamiliar with what they are.

9) Which would you prefer when pursuing a competency-based continuum (i.e., a series of related modules) of professional development/learning, as opposed to one-time participation learning? a. To follow the same continuum in a small group of peers b. To complete the continuum independently but check in regularly with a small group c. To complete the continuum independently and present to my peers at the conclusion 10) I would be most likely to compile an evidence portfolio demonstrating the application of my competency-based learning if: (Check only one) a. Others in my school were compiling a similar portfolio b. My administrator allocated time during the workday for me to work on my portfolio c. I better understood the relationship of the portfolio to my instructional effectiveness d. The portfolio could be used as evidence in my teacher evaluation e. I had a mentor who helped me compile the portfolio f. I had a mentor who followed up with me to apply the evidence collected to improve my effectiveness g. Other (Write-in Answer) 11) Which grade do you primarily teach? a) Pre-K through Grade 5 b) Grade 6 through Grade 8 c) Grade 9 through Grade 12 A4

12) How many years have you been teaching? a) Less than four years b) 5-9 years c) 10-14 years d) 15 years or more 13) What is your age? a) Younger than 30 b) 30-49 c) 50-54 d) 55 or older 14) What is the highest degree you have earned? a) Bachelor’s degree b) Master’s degree c) Higher than a Master’s degree 15) What district are you in? • Alamance-Burlington School System • Alexander County Schools • Alleghany County Schools • Anson County Schools • Ashe County Schools • Asheboro City Schools • Asheville City Schools • Avery County Schools • Beaufort County Schools • Bertie County Schools • Bladen County Schools • Brunswick County Schools • Buncombe County Schools System • Burke County Public Schools • Cabarrus County Schools • Caldwell County Schools • Camden County Schools • Carteret County Public Schools • Caswell County Schools

• • • • • • • • • • • • • • • • • • •

Catawba County Schools Chapel Hill-Carrboro Schools Charlotte-Mecklenburg Schools Chatham County Schools Cherokee County School District Clay County Schools Cleveland County Schools Clinton City Schools Columbus County Schools Craven County Schools Cumberland County Schools Currituck County Schools Dare County Schools Davidson County Schools Davie County Schools Duplin County Schools Durham Public Schools Edenton-Chowan Schools Edgecombe County Public Schools

• • • • • • • • • • • • • • • • • • • • • •

Elizabeth CityPasquotank Public Schools Elkin City Schools Forsyth County Schools Franklin County Schools Gaston County Schools Gates County Schools Graham County Schools Granville County Schools Greene County Schools Guilford County Schools Halifax County Schools Harnett County Schools Haywood County Schools Henderson County Public Schools Hertford County Public Schools Hickory City Schools Hoke County Schools Hyde County Schools Iredell-Statesville Schools Jackson County Schools Johnston County Schools Jones County Schools A5

• • • • • • • • • • • • • • • • • • • •

Kannapolis City Schools Lee County Schools Lenoir County Schools Lexington City Schools Lincoln County Schools Macon County Schools Madison County Schools Martin County Schools McDowell County Schools Mitchell County Schools Montgomery County Schools Moore County Schools Mooresville Graded School District Mount Airy City Schools Nash-Rocky Mount Schools New Hanover County Schools Newton-Conover City Schools Northampton County Schools Onslow County Schools Orange County Schools

• • • • • • • • • • • • • • • • • •

Pamlico County Schools Pender County Schools Perquimans County Schools Person County Schools Pitt County Schools Polk County Schools Randolph County Schools Richmond County Schools Roanoke Rapids Graded School District Robeson County Schools Rockingham County Schools Rowan-Salisbury School System Rutherford County Schools Sampson County Schools Scotland County Schools Stanly County Schools Stokes County Schools Surry County Schools

• • • • • • • • • • • • • • • • • • •

Swain County Schools Thomasville City Schools Transylvania County Schools Tyrrell County Schools Union County Public Schools Vance County Schools Wake County Public School System Warren County Schools Washington County Schools Watauga County Schools Wayne County Public Schools Weldon City Schools Whiteville City Schools Wilkes County Schools Wilson County Schools Yadkin County Schools Yancey County Schools Charter Schools North Carolina Virtual Public Schools

Focus Group Questions: 1) What are some of the opportunities and challenges related to participation-based (“sit and get”) vs. competency based professional development/learning? 2) What supports would you require to enhance your professional development/learning experiences? 3) How do you feel about a compensation structure partially based on evidence of completed competency-based professional development/learning? 4) What are the benefits and challenges of compiling an educational portfolio of annual professional development/learning experiences on which you can receive feedback?

A6

Appendix C This appendix provides demographic data on the North Carolina respondents, including grade level, years teaching, age range, highest degree earned, district, and region.

A7

Which grade do you primarily teach? (n = 7,563)

Pre-K through Grade 5 42%

Grade 6 through Grade 8 21%

Grade 9 through Grade 12 37%

How many years have you been teaching? (n = 7,563)

Less than four years 17%

5–9 years 17%

10-14 years 19%

15 years or more 47%

What is your age range? (n = 7,563) Younger than 30 13%

30–49 55%

50–54 15%

55 or older 17%

What is the highest degree you have earned? (n = 7,563) Higher than a master's degree 6% Bachelor's degree 50%

Master's degree 44%

A8

What district and region are you in? (n = 7,563) 1,013 ▪ 13%

1,814 ▪ 24%

412 ▪ 5%

659 ▪ 9% 697 ▪ 9%

937 ▪ 12% NCVPS 113 ▪ 1%

785 ▪ 10%

925 ▪ 12%

Charter Schools 208 ▪ 3%

Source: North Carolina State Board of Education (2015).

Alamance-Burlington School System 74 ▪ 1.0% Alexander County Schools 124 ▪ 1.6% Alleghany County Schools 7 ▪ 0.1% Anson County Schools 14 ▪ 0.2% Ashe County Schools 10 ▪ 0.1% Asheboro City Schools 18 ▪ 0.2% Asheville City Schools 39 ▪ 0.5% Avery County Schools 11 ▪ 0.1% Beaufort County Schools 31 ▪ 0.4% Bertie County Schools 9 ▪ 0.1% Bladen County Schools 62 ▪ 0.8%

Brunswick County Schools 77 ▪ 1.0% Buncombe County Schools System 142 ▪ 1.9% Burke County Public Schools 58 ▪ 0.8% Cabarrus County Schools 58 ▪ 0.8% Caldwell County Schools 72 ▪ 1.0% Camden County Schools 12 ▪ 0.2% Carteret County Public Schools 43 ▪ 0.6% Caswell County Schools 11 ▪ 0.1% Catawba County Schools 65 ▪ 0.9% Chapel Hill-Carrboro Schools 166 ▪ 2.2% Charlotte-Mecklenburg Schools 330 ▪ 4.4%

A9

Chatham County Schools 58 ▪ 0.8%

Graham County Schools 7 ▪ 0.1%

Cherokee County School District 47 ▪ 0.6%

Granville County Schools 52 ▪ 0.7%

Clay County Schools 3 ▪ 0.0%

Greene County Schools 15 ▪ 0.2%

Cleveland County Schools 64 ▪ 0.8%

Guilford County Schools 160 ▪ 2.1%

Clinton City Schools 9 ▪ 0.1%

Halifax County Schools 9 ▪ 0.1%

Columbus County Schools 94 ▪ 1.2%

Harnett County Schools 116 ▪ 1.5%

Craven County Schools 195 ▪ 2.6%

Haywood County Schools 36 ▪ 0.5%

Cumberland County Schools 303 ▪ 4.0%

Henderson County Public Schools 244 ▪ 3.2%

Currituck County Schools 29 ▪ 0.4%

Hertford County Public Schools 9 ▪ 0.1%

Dare County Schools 36 ▪ 0.5%

Hickory City Schools 13 ▪ 0.2%

Davidson County Schools 86 ▪ 1.1%

Hoke County Schools 69 ▪ 0.9%

Davie County Schools 31 ▪ 0.4%

Hyde County Schools 23 ▪ 0.3%

Duplin County Schools 68 ▪ 0.9%

Iredell-Statesville Schools 64 ▪ 0.8%

Durham Public Schools 125 ▪ 1.7%

Jackson County Schools 7 ▪ 0.1%

Edenton-Chowan Schools 24 ▪ 0.3%

Johnston County Schools 202 ▪ 2.7%

Edgecombe County Public Schools 33 ▪ 0.4%

Jones County Schools 15 ▪ 0.2%

Elizabeth City-Pasquotank Public Schools 36 ▪ 0.5%

Kannapolis City Schools 21 ▪ 0.3%

Elkin City Schools 11 ▪ 0.1%

Lee County Schools 49 ▪ 0.6%

Forsyth County Schools 248 ▪ 3.3%

Lenoir County Schools 29 ▪ 0.4%

Franklin County Schools 106 ▪ 1.4%

Lexington City Schools 14 ▪ 0.2%

Gaston County Schools 150 ▪ 2.0%

Lincoln County Schools 43 ▪ 0.6%

Gates County Schools 6 ▪ 0.1%

Macon County Schools 56 ▪ 0.7%

A10

Madison County Schools 14 ▪ 0.2%

Roanoke Rapids Graded School District 12 ▪ 0.2%

Martin County Schools 17 ▪ 0.2%

Robeson County Schools 134 ▪ 1.8%

McDowell County Schools 122 ▪ 1.6%

Rockingham County Schools 47 ▪ 0.6%

Mitchell County Schools 12 ▪ 0.2%

Rowan-Salisbury School System 50 ▪ 0.7%

Montgomery County Schools 7 ▪ 0.1%

Rutherford County Schools 47 ▪ 0.6%

Moore County Schools 52 ▪ 0.7%

Sampson County Schools 17 ▪ 0.2%

Mooresville Graded School District 11 ▪ 0.1%

Scotland County Schools 19 ▪ 0.3%

Mount Airy City Schools 12 ▪ 0.2%

Stanly County Schools 60 ▪ 0.8%

Nash-Rocky Mount Schools 67 ▪ 0.9%

Stokes County Schools 112 ▪ 1.5%

New Hanover County Schools 193 ▪ 2.6%

Surry County Schools 63 ▪ 0.8%

Newton-Conover City Schools 15 ▪ 0.2%

Swain County Schools 18 ▪ 0.2%

Northampton County Schools 18 ▪ 0.2%

Thomasville City Schools 3 ▪ 0.0%

Onslow County Schools 111 ▪ 1.5%

Transylvania County Schools 20 ▪ 0.3%

Orange County Schools 38 ▪ 0.5%

Tyrrell County Schools 16 ▪ 0.2%

Pamlico County Schools 7 ▪ 0.1%

Union County Public Schools 72 ▪ 1.0%

Pender County Schools 72 ▪ 1.0%

Vance County Schools 21 ▪ 0.3%

Perquimans County Schools 5 ▪ 0.1%

Wake County Public School System 628 ▪ 8.3%

Person County Schools 20 ▪ 0.3%

Warren County Schools 10 ▪ 0.1%

Pitt County Schools 103 ▪ 1.4%

Washington County Schools 7 ▪ 0.1%

Polk County Schools 17 ▪ 0.2%

Watauga County Schools 39 ▪ 0.5%

Randolph County Schools 82 ▪ 1.1%

Wayne County Public Schools 100 ▪ 1.3%

Richmond County Schools 16 ▪ 0.2%

Weldon City Schools 10 ▪ 0.1%

A11

Whiteville City Schools 3 ▪ 0.0%

Yancey County Schools 40 ▪ 0.5%

Wilkes County Schools 71 ▪ 0.9%

Charter Schools 208 ▪ 2.8%

Wilson County Schools 123 ▪ 1.6%

North Carolina Virtual Public Schools 113 ▪ 1.5%

Yadkin County Schools 41 ▪ 0.5%

A12

Appendix D This appendix provides survey findings, disaggregated by education district, school type, and rural versus non-rural. It also includes aggregated survey findings in tables.

A13

Q1

Who is primarily responsible for coordinating the professional development/learning you receive at your school?

15%

Principal Lead teacher Team of teachers assigned to coordinate professional development/learning I am responsible for coordinating my professional development/learning Other

8% 13%

%

14%

14% 7% 53%

%

15%

7% 54%

9%

10% Aggregated (n = 7,242)

52%

10%

North Central (n = 1,814)

Southwest (n = 937)

14%

20%

%

16%

18%

5% 9% 8% 7%

%

55%

%

62%

10%

%

58%

18%

19% 6%

%

Northwest (n = 659)

49%

16% Southeast (n = 925)

8% 9% 6%

Sandhills (n = 785)

11% 8%

8%

53%

11%

Northeast (n = 412)

12%

%

16%

Piedmont Triad (n = 1,013)

9% 12%

%

48%

12% Western (n = 697)

A14

Q1

Who is primarily responsible for coordinating the professional development/learning you receive at your school?

15%

Principal

8%

Lead teacher

I am responsible for coordinating my professional development/learning Other

%

13%

Team of teachers assigned to coordinate professional development/learning

53%

10% Aggregated (n = 7,563)

8%

15%

11%

18%

15%

7% 13%

%

54%

15%

10%

14%

11% Non-rural (n = 5,903)

51%

20%

% 38%

Charter Schools (n = 113)

14%

%

%

11%

Traditional (n = 7,242)

7%

13%

NCVPS (n = 208)

18%

53%

9% 11%

%

52%

10% Rural (n = 1,660)

A15

Q2 Poor quality of the professional development/learning instruction Poor quality of the professional development/learning content Lack of time before the professional development/learning to reflect on the purpose Lack of time after the professional development/learning to reflect on the purpose Not relevant to my instruction Difficulty transferring professional development/learning to instructional environment Lack of concrete models of application Lack of leaders’ support in implementing instructional changes based on professional development/learning experiences Other

Which of the following do you believe are challenges when applying professional development/learning to your instruction? (Check all that apply)

11% 9% 16% 33% 27% 26% 18% 12% 12%

Aggregated (n = 6,048) 14% 23% 31% 32% 32% 20% 11% 13% 16%

Northeast (n = 332) 14% 25% 33% 30% 34% 18% 11% 13% 16%

Southeast (n = 726)

17% 25% 32% 33% 41% 22% 12% 15% 15%

North Central (n = 1,550) 11% 20% 30% 26% 33% 20% 8% 11% 17%

Sandhills (n = 598) 14% 21%

34% 33% 40% 18% 11% 14% 12%

Piedmont Triad (n = 880)

18% 21% 29% 34% 39% 20% 15% 17% 15%

Southwest (n = 798) 14% 17% 32% 37% 41% 15% 9% 11% 13%

Northwest (n = 563) 14% 20% 27% 31% 46% 16% 9% 10% 16%

Western (n = 601)

A16

Q2 Poor quality of the professional development/learning instruction Poor quality of the professional development/learning content Lack of time before the professional development/learning to reflect on the purpose Lack of time after the professional development/learning to reflect on the purpose Not relevant to my instruction Difficulty transferring professional development/learning to instructional environment Lack of concrete models of application Lack of leaders’ support in implementing instructional changes based on professional development/learning experiences Other

Which of the following do you believe are challenges when applying professional development/learning to your instruction? (Check all that apply)

14% 11% 19% 39% 32% 31% 22% 15% 15%

Aggregated (n = 6,303) 14% 11% 19% 39% 32% 31% 22% 15% 15%

Traditional (n = 6,048) 14% 11% 19% 40% 32% 32% 23% 15% 15%

Non-rural (n = 4,929)

13% 12% 15% 36% 31% 28% 20% 20% 15%


18% 16% 24% 36% 25% 27% 17% 16% 19%

NCVPS (n = 83)

12% 10% 19% 37% 31% 29% 20% 15% 15%

Rural (n = 1,374)

A17

Q3

Which of the following do you consider most when you choose your own professional development/learning opportunities?

16%

Professional time available to participate Personal time available to participate If costs will be covered by school/district/other

16%

8% 21%

1% 37%

12%

40%

11%

5%

12%

6%

Aggregated (n = 7,242)

4%


15%

Southwest (n = 937)

15%

9%

15%

10% 22%

7%

20%

1%

24%

1%

1%

36%

33%

12%

38%

15%

5%

I do not self-select professional development/learning

19%

1% 37%

Likelihood of course/session improving instruction Likelihood of course/session improving my leadership as a teacher

7% 21%

1%

If peers are attending the same course/session

16%

7%

10%

6%

Northeast (n = 412)

5%

Sandhills (n = 785)

17%

Northwest (n = 659)

18%

6%

12%

8% 24%

1%

5% 20%

24%

1%

1%

35%

13% 5%

Southeast (n = 925)

35%

13% 5%


43%

11% 4%

Western (n = 697)

A18

Q3

Which of the following do you consider most when you choose your own professional development/learning opportunities?

16%

Professional time available to participate Personal time available to participate If costs will be covered by school/district/other If peers are attending the same course/session

8% 21% 1% 37% 12% 5%

Aggregated (n = 7,563) 16%

18%

7% 22%

I do not self-select professional development/learning

15%

0.5%

1%

37%

39%

12%

27%

11%

5%

Likelihood of course/session improving my leadership as a teacher

21% 20%

1%

Likelihood of course/session improving instruction

17%

8%

15%

3%


4%


16%

NCVPS (n = 113)

15%

8%

7% 21%

25%

1%

1% 38% 12%

5%


35% 12% 6%

Rural (n = 1,660)

A19

Q4 No, I have not and would not be willing to do so

Have you submitted evidence or artifacts documenting that you have applied professional development/learning in your classroom/school?

43%

No, I have not but am willing to do so Yes, I have submitted evidence/artifacts

8%

9%

%

45% 48%

10%

%

44% 47%



6%

9%

46%

37%

%

48%

%

Southwest (n = 937)

10% 42%

%

%

54%

Northeast (n = 412)

Sandhills (n = 785)

10% 40%

% 50%

Southeast (n = 925)

47%

Northwest (n = 659)

9% 43%

46%

%

11% 41%

47%


% 48%

Western (n = 697)

A20

Q4

Have you submitted evidence or artifacts documenting that you have applied professional development/learning in your classroom/school?

9%

No, I have not and would not be willing to do so

43%

No, I have not but am willing to do so Yes, I have submitted evidence/artifacts

%

48%


8%

9% 43%

%

42% 48%


%


48% 50%


%

40%

NCVPS (n = 113)

8%

10% 42%

%

12%

44% 48%

%

47%

Rural (n = 1,660)

A21

Q5

Which of the following would likely increase your satisfaction with professional development/learning experiences?

6%

Recognition for my professional development /learning*

6%

5%

7%

Ability to demonstrate to others how professional development/learning has improved my instruction Increased compensation based on evidence that my professional development/learning has impacted my instruction

5% 26%

20% 6%

21% 7%

North Central (n = 1,814) 8% 33% 6%

5%

Southwest (n = 937) 5%

30% 6%

31%

20% 7%

8%

25% 5%

27%

Aggregated (n = 7,242) New pathways for additional responsibilities based on my professional development/learning

30% 5%

6%

Better understanding of how the professional development/learning is relevant to my instruction

7%

29%

8% 28%

32% 2% 5%

25%

16%

29%

14%

7%

20%

6%

Northeast (n = 412) 7%

8%

Sandhills (n = 785) 6%

28% 6%

3%

29% 6%

6%

Northwest (n = 659)

6% 29%

30% 5% 7%

26%

19%

24%

22%

24%

Release time from instruction to reflect on professional development/learning

Southeast (n = 925)

Other

*e.g., official certificate from the district or acknowledgment from an administrator in front of colleagues

5%

6%


7%

Western (n = 697)

A22

Q5

Which of the following would likely increase your satisfaction with professional development/learning experiences?

6%

Recognition for my professional development /learning*

29% 5% 6%

Better understanding of how the professional development/learning is relevant to my instruction

27% 20% 6%

Aggregated (n = 7,563) New pathways for additional responsibilities based on my professional development/learning Ability to demonstrate to others how professional development/learning has improved my instruction Increased compensation based on evidence that my professional development/learning has impacted my instruction

6%

6% 29%

5%

9% 26%

6%

6%

9%

9%

27%

30%

20% 6%

6%

36%

18%

12%

5%


23% 9%

3%


NCVPS (n = 113)

7% 28%

32%

5%

5%

6%

6% 28%

26%

20%

18%

Release time from instruction to reflect on professional development/learning


Other

*e.g., official certificate from the district or acknowledgment from an administrator in front of colleagues

6%

7%

Rural (n = 1,660)

A23

Q6

I feel that I have autonomy to direct my professional development/learning.

7% 12%

7% 9%

8% 14%

Strongly agree Agree

28%

Disagree

%

33%

26%

%

51%

53%

% 53%

Strongly disagree Aggregated (n = 7,242)


5% 14%

8% 11% 25%

22%

%

Sandhills (n = 785)

% 54%

Southeast (n = 925)

% 55%

59%

6% 12% 28%

7% 11%

27%

%

57%

Northeast (n = 412)

Southwest (n = 937)

Northwest (n = 659)

7% 13%

8% 10% 31%

28%

%

%

52%


52%

Western (n = 697)

A24

Q6

I feel that I have autonomy to direct my professional development/learning.

7% 12%


28%

%

Disagree Strongly disagree

53%


4%

7% 11%

22%

18% 28%

%

5% 27%

18%

%

%

56%

50%

53%



7% 12% 29%

7% 12% 24%

% 52%


NCVPS (n = 113)

% 57%

Rural (n = 1,660)

A25

Q7

My teacher evaluation ratings guide my decision-making in selecting professional development/learning opportunities.

9% 7%

8% 8%

9% 8%


35%

Disagree

%

38%

49%

%

46%

35%

%

48%

Strongly disagree Aggregated (n = 7,242)


5% 11%

8% 11% 27%

28%

%

Sandhills (n = 785)

%

53%

Northwest (n = 659)

9% 5%

7% 8%

%

38%

%

51%

Southeast (n = 40)

35%

55%

6% 9% 34%

6%6%

%

53%

Northeast (n = 412)

Southwest (n = 937)

Piedmont Triad (n = 40)

47%

37%

%

49%

Western (n = 104)

A26

Q7

My teacher evaluation ratings guide my decision-making in selecting professional development/learning opportunities.

8% 8%


%

35%

Disagree

49%

Strongly disagree


8% 8%

35%

%

8% 9%

50%

%


40%

%

54%


38%

NCVPS (n = 113)

7% 9%

8% 8%

36%

%

29%


9% 13%

32% 49%

% 52%

Rural (n = 1,660)

A27

Q8

Have you explored and/or obtained micro-credentials for re-licensure?

8% Yes, I have explored and obtained microcredentials. Yes, I have explored obtaining microcredentials, but not obtained them. No, I have not explored microcredentials but I am familiar with what they are. No, I have not explored microcredentials and I am unfamiliar with what they are.

%

9%

7% 11%

11%

% 74%

74%


8%

%

7% 8%

70%

%

% 75%

Southeast (n = 925)

10%

%

14%

78%

Northwest (n = 659)

9%

7%

7% 6% 9%

8%

Sandhills (n = 785)

8%

9%

Southwest (n = 937)

71%

Northeast (n = 412)

%

7%

76%


14%

7%

7%

%

8% 5% 9%

5% 12%

73%


% 78%

Western (n = 697)

A28

Q8

Have you explored and/or obtained micro-credentials for re-licensure?

8% Yes, I have explored and obtained microcredentials.

7% 11%

%

Yes, I have explored obtaining microcredentials, but not obtained them.

74%

Aggregated (n = 7,563) No, I have not explored microcredentials but I am familiar with what they are. No, I have not explored microcredentials and I am unfamiliar with what they are.

8%

%

10% 8%

7% 11%

%

11% 6%

10% 60%

%

23%

73%

74%


8%

% 75%



8%

6% 11%

%

NCVPS (n = 113)

8% 12%

73%

Rural (n = 1,660)

A29

Q9

Which would you prefer when pursuing a competency-based continuum (i.e., a series of related modules) of professional development/learning, as opposed to one-time participation learning?

To follow the same continuum in a small group of peers To complete the continuum independently but check in regularly with a small group To complete the continuum independently and present to my peers at the conclusion

10%

10%

%

46%

44%


42% 43%

Northeast (n = 412)

Southeast (n = 925)

%

46%

%

43%

10% 46% 46%

%

44%

Northwest (n = 659)

7%

10% 45%

%

Southwest (n = 937)

Sandhills (n = 785)

10%

45%

46%

10%

46%

%

46%


12%

%

%

44%

10%

43%


47%

%

46%

Western (n = 697)

A30

Q9

Which would you prefer when pursuing a competency-based continuum (i.e., a series of related modules) of professional development/learning, as opposed to one-time participation learning?

To follow the same continuum in a small group of peers

10%

To complete the continuum independently but check in regularly with a small group

%

46%

To complete the continuum independently and present to my peers at the conclusion

44%


10%

45%

%

13%

%

45%

19% 38%

20%

%

50% 60% Traditional (n = 7,242)


10%

46%

%


NCVPS (n = 113)

10% 44% 44%

%

46%

Rural (n = 1,660)

A31

Q10

I would be most likely to compile an evidence portfolio demonstrating the application of my competency-based learning if: (Check only one)

9%

Others in my school were compiling a similar portfolio My administrator allocated time during the workday for me to work on my portfolio I better understood the relation of the portfolio to my instructional effectiveness The portfolio could be used as evidence in my teacher evaluation I had a mentor who helped me compile the portfolio

9% 34%

23%

22%

19%

4% 7%

18%

5%

5%

4%

4%

7%


8%

North Central (n = 1,814) 10%

30%

Southwest (n = 937) 10%

22%

22%

33%

25%

22%

24%

20%

5%

13%

10%

4%

8%

7%

6%

3%

6%

8%

9%

Sandhills (n = 785) 8%

7% 5% 7%

Southeast (n = 925)

Northwest (n = 659) 10%

34%

33%

21%

34%

23%

17%

Other

33%

22%

18% 6%

Northeast (n = 412) I had a mentor who followed up with me to apply the evidence collected to improve my effectiveness

10%

32%

22%

20% 5% 4% 7%


15% 7% 3% 9%

Western (n = 697)

A32

Q10

I would be most likely to compile an evidence portfolio demonstrating the application of my competency-based learning if: (Check only one)

9%

Others in my school were compiling a similar portfolio My administrator allocated time during the workday for me to work on my portfolio I better understood the relation of the portfolio to my instructional effectiveness The portfolio could be used as evidence in my teacher evaluation I had a mentor who helped me compile the portfolio

32% 23% 18% 6% 4% 7%


7% 32%

32%

22%

26%

21% 3%

4% 7%

4%

5%

9%

27% 5%

2%

5%


NCVPS (n = 113)

11% 33%

29%

22%

24%

18%

Other

24%

29%

18% 6%

Traditional (n = 7,242) I had a mentor who followed up with me to apply the evidence collected to improve my effectiveness

10%

6% 4% 7%


18% 7% 4% 7%

Rural (n = 1,660)

A33

1) Who is primarily responsible for coordinating the professional development/learning you receive at your school? Table D1 Principal Lead teacher Team of teachers assigned to coordinate professional development/learning I am responsible for coordinating my professional development/learning Other Total

n 4,010 791

% 53% 10%

1,019

13%

590

8%

1,153 7,563

15% 100%

2) Which of the following do you believe are challenges when applying professional development/learning to your instruction? (Check all that apply) Table D2 Unchecked n % Poor quality of the professional development/learning instruction Poor quality of the professional development/learning content Lack of time before the professional development/learning to reflect on the purpose Lack of time after the professional development/learning to reflect on the purpose Not relevant to my instruction Difficulty transferring professional development/learning to instructional environment Lack of concrete models of application Lack of leaders’ support in implementing instructional changes based on professional development/learning experiences Other

Checked n %

5,447

86%

856

14%

5,594

89%

709

11%

5,102

81%

1,201

19%

3,842

61%

2,461

39%

4,289

68%

2,014

32%

4,331

69%

1,972

31%

4,919

78%

1,384

22%

5,361

85%

942

15%

5,374

85%

929

15%

3) Which of the following do you consider most when you choose your own professional development/learning opportunities? Table D3 Professional time available to participate Personal time available to participate

n 1,198 576

% 16% 8% A34

If costs will be covered by school/district/other If peers are attending the same course/session Likelihood of course/session improving instruction Likelihood of course/session improving my leadership as a teacher I do not self-select professional learning/development Total

1,626 72 2,798

21% 1% 37%

914

12%

379 7563

5% 100%

4) Have you submitted evidence or artifacts documenting that you have applied professional development/learning in your classroom/school? Table D4 No, I have not and would not be willing to do so No, I have not but am willing to do so Yes, I have submitted evidence/artifacts Total

n 707 3,633 3,223 7563

% 9% 48% 43% 100%

5) Which of the following would likely increase your satisfaction with professional development/learning experiences? Table D5 Recognition for my professional development/learning (e.g., official certificate from your district or acknowledgment from administrator in front of colleagues) Better understanding of how the professional development/learning is relevant to my instruction New pathways for additional responsibilities based on my professional development/learning Ability to demonstrate to others how professional development/learning has improved my instruction Increased compensation based on evidence that my professional development/learning has impacted my instruction Release time from instruction to reflect on professional development/learning Other Total

n

%

466

6%

2,205

29%

398

5%

486

6%

2,056

27%

1,485

20%

467 7563

6% 100%

6) I feel that I have autonomy to direct my professional development/learning. Table D6 Strongly disagree Disagree Agree

n 529 2,115 4,028

% 7% 28% 53% A35

Strongly agree Total

891 7563

12% 100%

7) My teacher evaluation ratings guide my decision-making in selecting professional development/learning opportunities. Table D7 Strongly disagree Disagree Agree Strongly agree Total

n 579 2,638 3,743 603 7563

% 8% 35% 49% 8% 100%

8) Have you explored and/or obtained micro-credentials for re-licensure? Table D8 No, I have not explored micro-credentials and I am unfamiliar with what they are. No, I have not explored micro-credentials but I am familiar with what they are. Yes, I have explored obtaining micro-credentials, but not obtained them. Yes, I have explored and obtained micro-credentials. Total

n

%

5,608

74%

837

11%

498

7%

620 7563

8% 100%

9) Which would you prefer when pursuing a competency-based continuum (i.e., a series of related modules) of professional development/learning, as opposed to one-time participation learning? Table D9 To follow the same continuum in a small group of my peers To complete the continuum independently but check in regularly with a small group To complete the continuum independently and present to my peers at the conclusion Total

n

%

3,352

44%

3,448

46%

763

10%

7563

100%

10) I would be most likely to compile an evidence portfolio demonstrating the application of my competency-based learning if: (Check only one) Table D10 Others in my school were compiling a similar portfolio

n 710

% 9%

A36

My administrator allocated time during the workday for me to work on my portfolio I better understood the relationship of the portfolio to my instructional effectiveness The portfolio could be used as evidence in my teacher evaluation I had a mentor who helped me compile my portfolio I had a mentor who followed up with me to apply the evidence collected to improve my effectiveness Other Total

2,407

32%

1,715

23%

1,381

18%

468

6%

330

4%

552 7563

7% 100%

A37

Appendix E This appendix provides information on the focus group analysis and the total number of respondents by most common themes. HSG and NCDPI collected responses from EdNC-administered surveys2 and in-person focus groups. HSG then sent the results to Magnolia Consulting for analysis. After receiving the focus group data, Magnolia Consulting cleaned and prepared them for coding in ATLAS.ti, a qualitative data analysis software. ATLAS.ti allows users to divide data into segments, attach codes to the segments, and find and display all instances of similarly coded segments for analysis (Miles & Huberman, 1994).3 This enables efficient data organization and analysis. Next, Magnolia Consulting conducted a content analysis of the data, which involved identifying, organizing, and categorizing recurring themes in the survey answers (Patton, 2015).4 Magnolia Consulting staff regularly met to review codes and establish raters’ agreement on recurring themes. The total number of respondents who referenced each theme is located in Tables E1–E4.

2 3 4

Approximately 16% of survey respondents elected to complete the qualitative questions. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, CA: Sage Publications. Patton, M. C. (2015). Qualitative research and evaluation methods (4th ed.). Thousand Oaks, CA: Sage Publications.

A38

FG

Focus Group Questions

FG1. What are some of the opportunities and challenges related to participationbased (“sit and get”) vs. competency-based professional development/learning? Table E1 Count Challenge: Time required to attend and implement Challenge: Irrelevant professional development Challenge: Lack of application and forgetting of material Challenge: Lack of engagement or interest in professional development Challenge: No follow-up support Challenge: Lack of funds to support professional development Opportunity: Networking and peer collaboration at professional development Opportunity: Relevant professional development Opportunity: Able to improve teaching Opportunity: Less time required to attend some types of professional development

768 537 229 210 124 93 203 176 170 95

FG2. What supports would you require to enhance your professional development/learning experiences? Table E 2 Theme

Count

Provide time in the teaching schedule Offer financial support Provide professional development that is relevant and differentiated Provide follow-up support Provide opportunities to collaborate with peers Provide resources to use in the classroom Offer administrative support Include real-world modeling in professional development

1,278 643 584 298 186 136 131 102

FG3. How do you feel about a compensation structure partially based on evidence of completed competency-based professional development/learning? Table E 3 Theme

Count

Support Do not support Unsure

1,026 258 208 A39

Theme Concerns about fairness of system Concerns about evidence requirements Limits of teacher time Instead, offer higher compensation overall Concerns about monetizing professional learning Instead, restore master’s degree pay Instead, pay for fees associated with professional learning Do not create a penalty for non-participation Professional learning should be relevant and high quality

Count 173 151 133 121 87 69 67 66 61

FG4. What are the benefits and challenges of compiling an educational portfolio of annual professional development/learning experiences on which you can receive feedback? Table E 4 Theme

Count

Challenge: Finding time to create a portfolio Challenge: Who is the evaluator? Challenge: What is the quality of feedback? Challenge: Would it be meaningful? Challenge: What is included? Benefit: Offers opportunity for self-reflection and growth Benefit: Shows how professional learning is applied Benefit: Provides a physical record Benefit: Provides feedback on practices

1,629 238 221 203 118 433 364 317 159

A40

Appendix F This appendix provides Teacher Voice Network Fellows’ responses to an initial draft of report recommendations.

A41

TEACHER VALUE AND EFFECTIVENESS In summary, while principals or other parties are primarily responsible for coordinating professional development/learning, most teachers (65%) feel that they have the autonomy to direct their experience. When selecting professional development/learning opportunities, teachers consider: the likelihood that it will improve their instruction, whether costs will be covered, whether they have time available to participate, and their teacher evaluation ratings. Teachers noted that the following would enhance their satisfaction with professional development/learning experiences: • • • • • • • • • •

time during contract hours to attend, implement what they learned, and reflect on their learning; a better understanding of how it is relevant to their instruction; modeling how content could be applied in the classroom; additional compensation based on evidence that professional development impacted their instruction; follow-up support; opportunities for peer collaboration; administrative support; financial support to cover costs; differentiated instruction; and additional classroom resources aligned with professional learning opportunities.

When discussing types of professional development, teachers noted that participation-based opportunities require less time to complete and offer opportunities for networking and peer collaboration, but they may be less relevant to teacher needs, lack classroom applicability, and be less engaging for participants than competency-based professional development. Teachers also offered their perspectives on two competency-based learning options—microcredentials and a competency-based continuum. Few teachers have experience with microcredentials, with 84% of teachers reporting that they have never explored them and 74% reporting that they are unfamiliar with them. When pursuing a competency-based continuum (i.e., a series of related modules), most teachers prefer to either complete the continuum independently but check-in regularly with a small group (46%) or to follow the continuum with a small group of peers (44%). Approximately 55% of teachers have not submitted evidence or artifacts that documented their application of professional development/learning in their classroom or school, but most of these teachers would be willing to do so. When discussing educational portfolios of annual professional development/learning experiences, teachers expressed concern about the time required to create a portfolio, the qualifications of individuals evaluating their portfolio, whether the process would be meaningful, and what would be included. However, teachers also noted that portfolios could offer self-reflection and growth opportunities, document how professional learning is applied, provide physical evidence of learning, and offer feedback to teachers on their practices. Teachers would be most likely to compile an evidence portfolio that demonstrates their application of competency-based learning if:

A42

• • •

their administrator provided them time during work hours to complete their portfolio, they understood the relationship between the portfolio and their effectiveness as an instructor, or the portfolio could be used as evidence in their teacher evaluation.

In focus groups, teachers also discussed their feelings regarding a compensation system partially based on evidence of completed competency-based professional learning experiences. Most teachers support a system like this, while others do not or are unsure. Teachers also expressed concerns about fairness, evidence requirements, limited time to participate, monetization of professional learning, and penalties for non-participation. Several teachers suggested alternative compensation systems, such as offering higher overall compensation, restoring pay incentives for master’s degrees, or paying fees associated with professional development. Recommendation: •

Teachers highlighted the benefits of professional learning that is relevant, applicable to their classrooms, differentiated, low cost, and that provides follow-up support. Teachers also supported submitting evidence of their learning and receiving additional compensation based on that evidence. These findings are similar to those from the latest North Carolina Teaching Conditions survey (2016), where 33% of teachers disagreed that professional development is differentiated, and 24% disagreed that they receive follow-up support. Teachers’ preferences for professional learning suggest that NCDPI should continue to explore competency-based options, such as micro-credentials, for teacher professional development. Recent micro-credentials research is encouraging and suggests that micro-credentials are beneficial to educators and supportive of higher-level thinking and application (The Friday Institute, 2016). In developing a micro-credentialing statewide initiative, NCDPI should review what other states have done. For example, American Institutes for Research (AIR; 2017) offers a report on three states (i.e., Tennessee, Delaware, Arkansas) that are early adopters of micro-credentials. The report notes the importance of identifying a purpose, starting with a pilot, offering teachers some choices, paying attention to scoring or evaluation of microcredentials, and effective communication. Additionally, BloomBoard (2017) offers an ondemand webinar in which state education leaders in Florida, Tennessee, and Delaware discuss their experiences with micro-credentials and share perceived impacts on teachers. NCDPI should consider how these recommendations might support the development of a micro-credential system for educator re-licensure. NCDPI should also consider how it might provide districts with resources to support them in addressing the conditions teachers identified that would make them more likely to compile an evidence portfolio, including time during work hours to create a portfolio. NCDPI could also consider offering incentives and using portfolios as evidence in teacher evaluations. Should NCDPI decide to pursue portfolios, it will be important to help teachers understand the relationship between the portfolio and their effectiveness as an instructor, as teachers indicated that this relationship is unclear. Once NCDPI finalizes a statewide initiative, it could share a Micro-Credential Implementation Roadmap from Digital Promise (2017) with districts and schools. This resource emphasizes six overarching implementation strategies, each with embedded A43

strategies and resources. The stages include: exploration; planning; preparation; implementation; supporting, engaging, and tracking; and reflection, calibration, and resetting. For instance, the Planning stage includes resources to support development of micro-credential goal setting and focus areas, creation of a step-by-step plan for implementation, and planning of micro-credential stacks that are aligned to student needs. Effective implementation and clear communication of the initiative to districts and schools will be particularly important, as most teachers are unfamiliar with micro-credentials, and several have expressed concerns around how these systems will be implemented (e.g., fairness, evidence requirements, qualifications of individuals evaluating the portfolio). Digital Promise (2017) also offers several resources to support clear communication, including videos and infographics for teachers on micro-credentials and a guide for districts to support the use of social media in promoting the initiative. Do you agree with this proposed recommendation?

% Yes 100% What specific changes would you suggest? • • •

Since leading these focus groups, I have completed an online course and some microcredentials. I had to find my own courses, however, so I hope NCDPI will have a dedicated site to help teachers locate course options for micro-credentialing and renewal credits. I think it’s great as is. I was very happy to see the reference to the NC Teaching Conditions Survey to add to the validity of the findings. The links are very helpful. I didn’t watch the Bloomboard webinar, but read the articles and they provided context for the points being made. I don’t think we have this ability, but are we able to break down the PD data by rural and urban? Recently the massive differences in opportunities have been brought up, but I don’t know if there is a trend in our data or if that can be filtered out.

A44

Appendix G This appendix provides information on the representativeness of survey respondents, based on National Center for Education Statistics data.

A45

SAMPLE REPRESENTATIVENESS In an effort to determine how representative the respondents to this survey are, demographic information was collected. This demographic information was collected to compare it to demographic data collected by the National Center for Education Statistics (NCES). The comparisons appear below and are based on NCES data from 2011–2012. Years of Experience Less than 4 years 4–9 years 10–14 years 15 or more years

Survey Respondents 17 --* 19 47

NCES 11.7 31.4 20.4 36.4

NCES data can be found at: http://nces.ed.gov/surveys/sass/tables/sass1112_2013314_t1s_003.asp

*The survey asked respondents to select whether they had 5–9 years of experience; therefore, we could not calculate a percentage for survey respondents in the NCES-utilized range of 4–9 years. Highest Degree Completed Bachelor’s degree Master’s degree Higher than a master’s

Survey Respondents 50 44 6

NCES 54.2 33.8 7.8


Age Range Less than 30 30–49 50–54 55 or older

Survey Respondents 13 55 15 17

NCES 19.3 52.2 11.4 17


In previous surveys, we asked respondents to select a gender; however, we have discontinued doing so because offering only a binary option is inconsistent with contemporary practices in surveying. Offering respondents the option to select options other than male/female would not allow us to compare the data we collect to NCES data, as NCES only offers a binary option. Given that the NCES data (which represents the entire teaching population in North Carolina) is similar to the demographic data collected from respondents, the respondents to the survey are representative of the teaching population in North Carolina; however, there are limitations to the ability to generalize the survey findings. Additional information about that matter can be found in Appendix H.

A46

Appendix H This appendix provides some guidance around interpreting survey data in the current report.

A47

INTERPRETING DATA Minimizing Survey Error

Survey research is a commonly used research method to determine what people are thinking, feeling, or doing.i However, survey research can be complicated and, if done poorly, yield inaccurate or imprecise results. A successful survey is based on sound research questions, accurate measures of the topics of interest, and a design that enables the generalization of research findings to the population of interest.ii Four types of survey error, however, can undermine the results of any given survey: measurement, exclusion, sampling, and nonresponse.iii,iv Measurement Error

Measurement error, also referred to as an error of observation, occurs when respondents give inaccurate or imprecise answers to survey questions. This may happen as a result of poorly worded questions (e.g., lengthy or double-barreled questions) or poorly designed surveys (e.g., unclear instructions or inadequate response options). By taking care in the design of survey items and the broader survey questionnaire, by having key stakeholders review draft items and the draft questionnaire, and by testing out items with potential respondents, HSG sought to minimize measurement error. Exclusion Error

Exclusion error occurs when (a) there are members of the population of interest who have no chance of being surveyed and (b) these excluded members of the population differ from included members of the population. For example, in an email survey of teachers, exclusion errors could occur if there were teachers whose email addresses were not active, teachers who did not check their listed email account, or teachers who were not on the email list at all (e.g., teachers who were newly hired). To the extent that these teachers differed from others regarding the survey topic of interest, exclusion error exists. Because all of the teachers in the Network Fellows’ professional learning networks (PLNs) have provided an email address, exclusion error in this survey is minimal. Sampling Error

Sampling error involves random differences occurring between sample estimates and true population values. Sampling error is unavoidable in sample surveys because only some population members are surveyed. Sampling error is often quantified by standard errors or margins of error (also referred to as confidence intervals), which provide information on the probability that any finding from a sample is due to chance (i.e., sampling error). Holding all else equal, increasing sample sizes generally reduces sampling error. Because HSG attempted to survey all of the teachers in the state by emailing the survey link to all teachers in PLNs, inviting teachers in PLNs to send the link to colleagues who may or may not be in a PLN, having state partners also send the link to teachers for whom they have access via an active email address, and publicizing the availability of the survey via social media, sampling error does not apply. In other A48

words, there is no statistical basis for calculating confidence intervals regarding the survey results. Nonresponse Error

Nonresponse error occurs when individuals do not respond to a survey or to particular questions on a survey, and when these individuals would have responded differently, on average, than those people who did respond. More specifically, nonresponse error is the product of: 1. The nonresponse rate, which is the percentage of the sample or population who do not respond to a survey (unit nonresponse rate) or to an item on the survey (item nonresponse rate); and 2. Nonresponse bias, which is the difference between the average respondent’s response and the average nonrespondent’s response for a given item. Thus, a survey can have a low response rate with little or no nonresponse error. This can occur if there are small or no differences between respondents and nonrespondents that are relevant to the survey’s topics of study. In contrast, when there are large differences between respondents and nonrespondents, it is possible for surveys to have high nonresponse bias even with high response rates.v There are many reasons why responses of respondents might differ from those of nonrespondents. Survey format matters. For example, given their greater technological literacy, younger teachers may be more likely to respond to an online survey than teachers nearing retirement age. If younger teachers differed from older teachers in their thoughts regarding a survey topic (for example, teacher pension reforms), then this difference could bias results if analyses did not take into account teacher age. Survey topics also matter. For example, in a survey of teacher compensation, teachers who feel strongly that they are underpaid might be more inclined to respond than teachers who are comfortable with their current level of compensation. As a result, such a survey could overstate teacher dissatisfaction regarding pay. When seeking to minimize nonresponse bias, it is important to consider what affects respondents’ likelihood of participating in a survey and how respondents might respond to specific survey questions.vi Because surveys can have low response rates and still have little to no nonresponse bias, response rates are not a good measure to judge the quality of a survey in and of themselves.vii Recent empirical evidence has shown that the relationship between response rates and nonresponse bias is weak, at best: • • • •

A comprehensive study using exit poll data found no statistically significant relationship between response rates and survey error.viii In a study of household surveys, response rates accounted for only about 11 percent of the variation in nonresponse bias estimates.ix Results from two identical national telephone surveys were similar, despite dramatically different response rates (61 percent versus 36 percent).x A national health survey that saw declining response rates over time also found declining nonresponse bias.xi A49

Because response rates are not a good indicator of nonresponse bias, investigations into the extent of possible nonresponse bias are important. There are a number of practical approaches that survey researchers can take to make such investigations. For example, researchers can compare demographics or other administrative data on survey respondents to those of nonrespondents, contrast survey results for early responders with those for late responders, and judge survey results against findings from an external data source.xii In each case, the more similar the results, the less substantial any nonresponse bias is likely to be. Conclusion

When assessing the quality of a survey, it is important to consider the total survey error, composed of measurement, exclusion, sampling, and nonresponse errors. HSG has taken steps to reduce total survey error via a rigorous, multifaceted approach to survey design and implementation.

i

Mitchell, M. L., & Jolley, J. M. (2010). Research design explained (7th ed.). Belmont CA: Wadsworth, Cengage Learning. Mitchell, M. L., & Jolley, J. M. (2010). Research design explained (7th ed.). Belmont CA: Wadsworth, Cengage Learning. iii Groves, R. M., Kalton, G., Rao, J. N. K., Schwarz, N., & Skinner, C. (Eds.) (1989). Survey errors and survey costs. New York: John Wiley & Sons, Inc. iv Dillman, D. A., Smyth, J. D., & Christian, L. M. (2008). Internet, mail, and mixed-mode surveys: The tailored design method. New York: John Wiley & Sons, Inc. v Langer, G. (2003). About response rates: Some unresolved questions. Public Perspective, 16–18. vi Groves, R. M., & Peytcheva, E. (2008). The impact of nonresponse rates on nonresponse bias: A meta-analysis. Public Opinion Quarterly, 72(2), 167–189. vii Gideon, L. (2012). Handbook of survey methodology for the social sciences. New York: Springer New York. viii Merkle, D., & Edelman, M. (2002). Nonresponse in exit polls: A comprehensive analysis. In R. M. Groves, D. A. Dillman, J. L. Eltinge, and R. J. A. Little (Eds.), Survey nonresponse (pp. 343–358). New York: Wiley. ix Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646–675. x Keeter, S., Miller, C., Kohut, A., Groves, R. M., & Presser, S. (2000). Consequences of reducing nonresponse in a national telephone survey. Public Opinion Quarterly, 64(2). xi Mokdad, A. H., Stroup, D. F., Giles, W. H., & Behavioral Risk Factor Surveillance Team. (2003). Public health surveillance for behavioral risk factors in a changing environment. Recommendations from the Behavioral Risk Factor Surveillance Team. MMWR. Recommendations and Reports: Morbidity and Mortality Weekly Report. Recommendations and Reports / Centers for Disease Control, 52(RR-9). xii Montaquila, J. M., & Olson, K. M. (2012). Practical tools for nonresponse bias studies [PowerPoint slides]. Retrieved from http://www.amstat.org/sections/SRMS/webinarfiles/NRBiasWebinarApril2012.pdf. ii

A50