Collect data regarding district or regional initiatives using rubrics, surveys, peer ... (e.g., demographic, enrollment,
Strategies for Measuring the Effectiveness of District & Regional Initiatives Michelle Robinette Director of Curriculum, Red Creek CSD
Sarah Vakkas Assistant Superintendent for Instruction, GST BOCES
Session Objectives Participants will be able to: • Identify strategies for setting district or regional goals that are measurable • Collect data regarding district or regional initiatives using rubrics, surveys, peer observations and walkthroughs – Understand how to use Google Forms to create efficient data collection tools – Instructional Rounds Process
The Data Wise Process
3
The Data Wise Process
Step 3 Step 4
Step 5
Step 6
How much time is spent on data? Summative District and State Assessments (aggregated, disaggregated, strand, item)
Data about people, practices, perceptions (e.g., demographic, enrollment, survey, interview, observation data, curriculum maps) Benchmark Common Assessments (e.g., end-of-unit, common grade-level tests reported at item level) Formative Common Assessments (e.g., math problem of the week, writing samples, science journals, other student work) Formative Classroom Assessments for Learning (e.g., student self-assessments, descriptive feedback, selected response, written response, personal communications, performance assessments)
Annually
2-4 times a year Quarterly or end of unit
1-4 times a month
Daily - Weekly
How much time is spent on data?
Example Flow of the Funnel: Focus Area: ELA 3-8 (esp. Special Education subgroup) Priority Question: What factors are contributing to students performing below the region on the 3-8 ELA Assessments, but still showing success compared to the region on the HS English Regents? Learner-Centered Problem: Students are struggling to answer multiple choice questions correctly on 3-8 ELA Assessments. [Dig Deeper through Student Work Protocol] Problem Of Practice: We have phased in use of the NYS 38 ELA Modules which do not contain opportunities for students to practice multiple choice questions. 7
Looking at Student Work Protocols for looking at student work are abundant. The important piece is to design tasks/questions that will help answer your Priority Question and dig into uncovering a Learner-Centered Problem.
Choosing Student Tasks Locate tasks to give to students across a whole grade level that can be used to help look into the issues identified: – If MC is more of an issue, use MC questions (or if CR is more of an issue, use CR... match tasks to the concerns) – If you have concerns with particular standards at your grade level, choose tasks that address those standards – Don’t “teach” to the task prior to giving it, give it to students “cold” to see what they can and can’t do (gives us more authentic data when we look at the student work) – Don’t grade the tasks, don’t even look at the student work if you don’t want to, wait until we meet and bring the stack of papers with you
Problem of Practice Some general considerations to think about: •Is what we are considering observable? Consider having teachers observe each other teaching lessons addressing the standards or skills in question to help identify potential problems of practice. •What if they aren’t ready for this? See First Classroom Visits and Observer as Learner handouts along with our template. •What is there that, if done by everyone, could serve as an umbrella for a number of teaching and learning strategies? •What can have the most positive effect on what students do, what teachers do and the quality of the work that students are10 producing?
Walkthroughs to Identify Problem of Practice or to Assess Progress Technology Integration
Co-Teaching Practices
11
Brainstorm Action Plan Learner Centered Problem and/or Problem of Practice:
Students are struggling to answer multiple choice questions correctly on 3-8 ELA Assessments, especially SWDs. Students are not being exposed to MC questions and strategies or being given opportunities to grapple with MC. Strategy/Task
When
Use of Edmodo Snapshot by ELA teachers (focus on multiple Beginning Fall 2015 choice by particular standards, either aligned with current teaching or based on individual student need for review/differentiation) Increase independent reading by students with opportunities to practice answering multiple choice questions based on text pilot use of LightSail and myON apps
LightSail - Fall 2015, myON - Spring 2016
Improve use of explicit instruction, differentiation strategies and co-teaching practices - provide ongoing, job-embedded PD
Throughout 2015-16 School Year
Data Source Shortterm
When
Baseline
Goal
Edmodo Snapshot (also LightSail and myON data as appropriate)
Ongoing throughout the year
Individualized
Use data to individualize and/or group students
Medium RtI Screening Data from FASTbridge -term
Jan & May
Oct: CBM Reading Jan 55%, May 60% at grades 2-6 had benchmark on each 52% at benchmark, subtest aReading grades 28 had 49% at benchmark
Walkthrough data on co-teaching strategies & tech integration
Quarterly summary
Sept/Oct: 63% of coteachers using 1Teach/1Assist, Of all teachers using technology, 52% were as a projector/display
2nd Semester: Less than 50% of coteachers using 1Teach/1Assist, Of all teachers using technology, less than 30% as a projector/display
Spring 2016 NYS ELA Assessment data by grade level and for SWD subgroup
Summer 2016
2015: Average of 27% proficiency across grades 3-8
2016: Average of 40% or more proficient across grades 3-8
Plan to Assess Progress
Longterm
Intended vs. Learned 1. What ways do you measure what your intended curriculum is vs. what is being taught in your classrooms? 2. What ways do you measure what is being taught vs. what is being learned?
14
GST BOCES Regional Data Team Analyze our data, report the findings and make recommendations for modifying, improving services, and creating innovative services throughout GST region. ● Utilizing the Data Wise process to support sustainability of our regional work. ● Regional Instructional rounds to highlight best strategies within the region.
Example Flow of the Funnel: Focus Area: ELA & Social Studies Priority Question: What factors are contributing to students performing below NYS average within ELA & SS assessments? (end of step 3) Learner-Centered Problem: Students continue to not show improvement in NYS ELA 3-8 and SS regents data. Our region is underperforming NYS averages for student growth in both of these areas. With the transition to CCLS and changes in curriculum there does not seem to be any noticeable patterns. (end of step 4) Problem Of Practice: Currently in ELA and SS there are not common practices or common language used within writing instruction. (step 5)
Brainstorm Action Plan Ideas Learner Centered Problem and/or Problem of Practice:
Currently in ELA and SS there are not common practices or common language used within writing instruction. (step 5) Strategy/Task
When
Three districts volunteered and we have identified classrooms within each district that achieve high performance based on 3 years of data and on 3 different data points: Constructed Response Reports, Student Growth Data, APPR scores.
Winter 2016
Complete Instructional Rounds in districts, Interview teachers afterwards to gather qualitative data, plan a day to process rounds.
February/March
Gather data on writing curriculum used across the region and within participating districts.
April 2016
Gather research on best instructional practices, John Hattie
March 2016
March 29th
Plan to Assess Progress ●
Short-Term: Review AIS data reports progress and enrollment Data Source When on student Baseline Goalevery 6 weeks. Baseline: Enrollment based on multiple data points from prior year Gather evidence of bestGoal: Spring 2016 Instructional Roundsthe Report findings with and/or last data meeting. Students are accessing AIS supports bydistricts Shortinstructionalfor strategies data researched and makecourse. recommendations being scheduled the class and making % and progress in their math term
●
from research and themes. for modifying, improving compare with I.R. services, and creating Medium-Term: course enrollment in regents math courses 11th throughout and findings ofIncrease best innovative in services practices in writing GST region. [Model Schools, 12th grade every semester. Baseline: Current student enrollment Goal: instruction. SIP, District PD, etc.]
Increase of 10% in regents math courses Algebra, Geometry, Algebra 2, Precalculus Walkthrough data from Fall 2016 Instructional Rounds Increased observation of Medium district administrators
data, District
instructional best practices
-term on the Regional Data walkthrough data prior within the districts that ● Long-Term: Review national clearinghouse reports and localreceived data on diploma Team to PD offerings PD offerings type for graduates and 9th-11th graduation plans annually. Baseline: School Counselors will meet with each 9th-12th grade student (10th/11th grade studentsInstructional w/ parents) to generate on diploma type. Goal: TBD once of Rounds Winterbaseline 2017 Instructional Rounds Increased observation Long2) data from Spring 2016 instructional best practices baseline(Phase is established term within the districts that received PD offerings
Instructional Rounds 1. Lay the Foundation: Review Data 2. Assemble the Right Team: a. Instructional rounds for chosen districts b. New to process and some 2nd year c. Ensured safety i. Teacher Support - non-evaluative, systems thinking 3. Clarify Your Purpose and Strategy: a. Instructional rounds training i. What makes good evidence: Observable vs. Inference ii. Lens was defined and T-chart used 4. Visit Classrooms a. Interviewed teachers afterwards 19
Ladder of Inference •
20
A Focus on Learning "The researchers then examined these schools to find what qualities they had in common. Of the five they found, two struck me as crucial: 1) If the students did not learn, the schools did not blame them, or their families, backgrounds, neighborhoods, attitudes, nervous systems, or whatever. They did not alibi. They took full responsibility for the results or non results of their work. 2) When something they were doing in the class did not work, they stopped doing it, and tried to do something else. 21 They flunked unsuccessful methods, not the children."
John Holt, How Children Fail (1964)
1. Do we alibi in our school? Yes? Evidence...No? Evidence... 2. When something that we are doing in class is not working...what do we do? i. Do we stop? Evidence… ii. Do we try something else? Evidence…
DataWise Training Online through Harvard University: https://www.edx.org/course/introduction-data-wise-collaborative-harvardx-gse3x
Contact Information Michelle Robinette Director of Curriculum Red Creek CSD
[email protected]
Sarah Vakkas Assistant Superintendent for Instruction GST BOCES
[email protected] 