Rule Development Workshop Transcript – October 11, 2017

Slide 1:
Good day, ladies and gentlemen and welcome to our second Rule Development workshop and webinar, with an opportunity for public input on draft rule language for Rule 6A-5.066 – Approval of Teacher Preparation Programs. This is Kimberly Pippin, Program Director with the Office of Educator Preparation, and I am joined by other Department staff as well.

Today’s workshop will focus specifically on the Florida Site Visit Framework that we propose to incorporate by reference into the rule. Because of its relationship with Florida Atlantic University, Stetson University, St. Petersburg College and the University of Florida as part of the three-year Centers of Excellence grant for improving Florida elementary education programs, Florida adapted a framework developed by Teacher Prep Inspection-US (also referred to as TPI- US) to be used as part of the continued approval site visit process.

Katie Moyer is joining me on the call today. Katie Moyer serves as the Chief Operating Officer for TPI-US, and manages the internal operations to support the delivery of reliable inspections and organizational growth.

So that everyone is aware, we are conducting this workshop both as a webinar via conference call and face-to-face. I would like to go through a few instructions for all of the different groups of people joining us today. For everyone’s information, today’s webinar and conference call are being recorded and the recording, the presentation materials and the transcript will be posted to our website within 10 days. It will be extremely helpful if you have the draft rule text and the proposed Florida Site Visit Framework available so you can follow along. These documents were sent to all individuals who are part of the teacher preparation program listservs, and any members of the public who requested copies on Tuesday, October 10th. For anyone who has not received the documents, please email Eileen McDaniel at Eileen.McDaniel@fldoe.org. She will send the documents within two business days.

For those of you in the room, a copy of the Florida Site Visit Framework is located at the back of the room.

Slide 2:
We are going to conduct our workshop in three sections. As a reminder, we conducted a rule development workshop on September 29 where we reviewed the proposed rule language. Today our focus is on the draft Florida Site Visit Framework.

The first part of our workshop today will be a brief review of the statutes that are the underpinnings to this framework and form the basis for the entire draft rule language. The law states that the State Board of Education shall adopt rules for continued approval of teacher preparation programs, which include the program review process. We will then review proposed
changes made to the rule subsequent to the workshop on September 29th, and the four components of the Florida Site Visit Framework.

After we have reviewed the Florida Site Visit Framework, there will be two parts that will be interactive. In Part Two, everyone who is participating, whether it is via the webinar or conference call OR in-person, can ask clarifying questions. This provides an opportunity for someone to make sure he or she understands properly what was said during the presentation, whether it was in the statute, draft rule text or in the Framework document. These questions – and also the comments that follow – will help the Department to improve the Florida Site Visit Framework and the draft rule text.

After the question and answer session has ended, we will go to Part Three, which is the actual public comments’ section for this rule development workshop. During this time, audience participants (whether physically participating, or participating by conference call or webinar) who wish to make a comment about any part of the rule, including the proposed documents
being incorporated by reference, will have an opportunity to do so. During this time there will be no responses, or questions, or debate.

Slide 3:
Before we actually begin, we wish to thank five teacher preparation program stakeholders who have been working with the Department as we reviewed and drafted rule language for 6A-5.066.

We wish to thank:
Dr. Gina Almerico from the University of Tampa; Dr. Amy Guerette from Florida State University;
Dr. Gloria Pelaez from St. Thomas University and who is also FACTE’s President-Elect; Dr. Amy Ringue from Daytona State College; and
Mr. Darran Town from the University of South Florida.

In addition, we would also like to thank FACTE conference participants, and other stakeholders who have contributed feedback and recommendations for strengthening the draft rule. Today, we will share some of the changes made to the rule as a result of clarifying questions and recommendations received.

Slide 4:
Let’s review again the rule adoption timeline.

As I mentioned, we are currently in rule development. We advertised on July 28, 2017, that we were seeking revisions to the rule.

On September 14, we advertised that we would conduct a Rule Development Workshop – and we conducted it on September 29.

Note that throughout August and September we have been gathering comments and recommended revisions for consideration for rule changes.

We also advertised that we will have a second Rule Workshop on October 11 – which brings us to today.

Public input and feedback will remain open up until this rule goes to the State Board of Education for possible adoption. Throughout this time, the public comment avenues are open to you and changes to the proposed language being presented to you today for your consideration will continue.

Later in October, after reviewing any additional public input or comment we have received, we anticipate that we will publish a final version which the Commissioner will present to the State Board of Education for consideration at its November 28 board meeting.

Slide 5:
Before we start discussing Rule 6A-5.066, Approval of Teacher Preparation Programs, let us first begin with a brief discussion of the authority for the rule under development, that is, the laws
that authorize state-approved teacher preparation programs.

Slide 6:
The laws that authorize the State Board of Education to adopt rules for the initial and continued approval of ALL state-approved teacher preparation programs permits the State Board to describe the program review process; the continued approval timelines; and the criteria for continued approval. As a reminder, it is the Commissioner of Education that determines the continued approval of all Florida state-approved teacher preparation programs.

The authority comes through statutes – and specifically the following:
•For our Initial Teacher Preparation Programs – often termed our “traditional teacher preparation programs” that usually terminate in a degree – the authorizing statute is Section 1004.04, Florida Statutes, entitled Public Accountability and State Approval for Teacher Preparation Programs. We refer to these programs as ITPs.
•Second, for our Educator Preparation Institutes or EPIs – these are post-baccalaureate programs that also prepare teachers and are offered by institutions and private providers – the authorizing statute is Section 1004.85, Florida Statutes, entitled Postsecondary Educator Preparation Institutes.
•And third, we have Professional Development Certification Programs, formerly called District Alternative Certification Programs, in which school districts and now charter schools and charter management organizations may offer these post-baccalaureate programs – the authorizing statute is Section 1012.56, paragraph 8.
Slide 7: So now, let’s turn to the draft rule language itself for your review and consideration.

Slide 8:
As a result of feedback, we have applied additional revisions to the draft rule language. These revisions are notated in purple font and include the following:

On page 6, under Section (3)(b)6, which pertains to the Annual Program Performance Report (APPR), this section was revised to allow providers 30 business day to review its APPR data and submit errors and omissions to the Department.

For Section (3)(c)1.a., of the Continued Approval Site Visit section, additional clarification was added to the draft rule text pertaining to critical teacher shortage areas.

Under Section (3)(c)3, the new language added to the draft rule pertains to the submission of the program’s self-assessment report, rather than the program evaluation plan (also referred to as IPEP, APEP, or DPEP) during the year of the provider’s scheduled site visit.

Under Section (3)(c)5, the rule revisions include an opportunity for providers to review and respond to the preliminary site visit report and submit clarifying information within five business days of receipt of its report. This process is similar to the rejoinder process provided to programs in the past.

Under Section (3)(c)6, we are proposing to use the term weak, rather than weakness.

Moving on to page 7 of the draft rule text, for Section (3)(d)1, regarding the Continuous Improvement Plan, the proposed language references that a provider has 30 business days to submit its plan to the Department. Additional specificity to the requirements of the
plan is also included in the draft rule language.

Regarding Section (3)(e)2, the weights of the Continued Approval Summative Score have been revised to 50% of the APPR Average Summative Rating; 20% of the Continued Approval Site Visit Rating, and 30% of Evidence of Programmatic Improvement Rating.

Slide 9:For continued approval of a state-approved program, a site visit of the program’s quality in four areas is conducted by the Department during the last year of its continued approval period. The site visit provides the Department and the provider with detailed insights into the quality of the state-approved teacher preparation program. The process is designed to support programmatic improvements and ensure that pre-service teachers are developed to support student learning from day one.
As we stated during the September 29, 2017, workshop, Rule 6A-5.066 incorporates a framework for implementing the continued site visit of state-approved teacher preparation programs. This framework will be used to assess the program’s quality in four areas. These areas are:

Quality of Selection
Quality of Content Knowledge and Teaching Methods
Quality of Clinical Placement, Feedback, and Candidate Performance Quality of Program Performance Management

These four areas are aligned to the four standards in the draft initial and continued approval standards guidelines. It is important to emphasize that the initial and continued approval standards guidelines set forth the statutory requirements that a program must respond to for either initial state-approval through an electronic folio application, or for continued state-

approval through the program’s annual program evaluation plan, referred to as IPEP, APEP, or DPEP; whereas, the site visit framework is the rubric used to conduct the qualitative review of the program in each of these same four areas.

 Slide 10: 
The Florida Site Visit Framework specifies the four judgment areas, indicators/criteria and requirements for the continued approval site visit. The draft framework is chunked into four sections – one section for each judgment area.

Noted on this slide in red font, is a proposal to use alternative language of Review Areas, rather than Judgment Areas. We ask that you consider both options and suggest which one is better or  propose different language.

Each area includes a context and rationale, which provides an explanation of the judgment area; essential questions; the potential sources of evidence that will be collected and reviewed during the site visit; and associated indicators/criteria. For each of the criteria under the judgment area, there is a scoring scale: Score of 4 is Strong; Score of 3 is Good; Score of 2 is Weakness; and Score of 1 is Inadequate. We are proposing to revise Weakness to Weak. Please consider both options when making a recommendation.

A description of how the score for the judgment area is determined is described on page 2 of the framework. It is important to note that the review team will utilize a plethora of data to determine the program’s quality in each of the four areas. For instance, to assess how well the program is preparing teacher candidates to differentiate instruction in the classroom, reviewers will triangulate evidence from observing the program’s course or courses where differentiation is taught with observations of student interns’ abilities to differentiate while teaching in the classrooms, as well as conversations with various stakeholders (including professors teaching the course(s), candidates and recent graduates, and cooperating teachers).

Slide 11:
Judgment Area 1, Quality of Selection, is described on page 3 of the framework. In addition, this page identifies the essential questions and potential sources of evidence reviewers will utilize to analyze how well the program is selecting quality teacher candidates for the teaching profession prior to program admittance.

You will notice that this specific area is for informational purposes only. So what does that mean? First, this means that the program and provider will not receive an o verall score for this area. Reviewers will review and analyze any evidence that the program provides for this area, and as a result, the provider will receive findings and recommendations on how to enhance the program in selecting quality teacher candidates. But again, no overall score or rating will be provided for this area.

Starting on page 4, the indicator and criteria for Quality of Selection are provided.  This area has one indicator – Selection – and five criteria – GPA, standardized tests, demographic representation of enrolled candidates and program completers, and the admission process.

Slide 12:
Page 7 of the framework focuses on the second area - Quality of Content Knowledge and Teaching Methods. This judgment area considers how well the program ensures teacher candidates acquire content knowledge and key teaching methods and skills needed to be an effective educator. The context and rationale for this area are provided on this page.

The essential questions that are being answered during the site visit and the sources of evidence for this area are explained on page 8. You will also notice that page 8 provides an explanation of the constraining criteria for Elementary Education program site visits. This means that in order for this overall judgment area to be scored as Good or Strong, the quality of literacy training delivered by the program to all teacher candidates must be good or strong.

There are three indicators under the area of Quality of Content Knowledge and Teaching Methods:
Indicator 2.1: Content Knowledge Indicator 2.2: Teaching Methods Indicator 2.3: Connections to Practice

The first indicator – Content Knowledge – spans from pages 9 - 12 of the framework. The type of program being reviewed determines which criterion is applicable under this indicator. For instance, if a secondary level program is being reviewed then the criteria on page 12, entitled (Secondary) Core Subject Area is only applicable for this indicator. The criterion labeled – Alt Cert – is only applicable for alternative teacher preparation programs where the candidate is employed in an instructional position in a school district and functions as the teacher of record.

The next Indicator – Teaching Methods – begins on page 13 and ends on page 15 of the framework. There are four criteria under this indicator.

The last Indicator – Connections to Practice – is found on page 16. There is only one criterion under this indicator.

Slide 13:
Page 17 explains Judgment Area 3 – Quality of Clinical Placement, Feedback and Candidate Performance. This area considers how the program selects placement sites, how well candidates are able to apply learned content in clinical experiences and the feedback practices used to support candidate development.

An idea of what evidence will be reviewed for this area is provided at the top of page 18. In addition, the constraining criteria for this judgment area is also specified on this page. The

constraining criteria - Quality of Written and Oral Feedback - which is found on page 22 – must be rated as good or strong in order for the entire judgment area to be rated good or strong.

Under Judgment Area 3, there are three indicators and each indicator has four criteria. Indicator 3.1 is Clinical Placement and is located on pages 18 through 20.
Indicator 3.2 is Observation and Feedback. This indicator starts on page 21 and continues to the top of page 23.
Indicator 3.3 – Candidate Performance starts on page 23 and ends on page 26.

Slide 14:
The last judgment area is Quality of Program Performance Management. The rationale and context for this judgment area is explained on page 27. This area focuses on the program’s use of high-quality data to continually improve the quality of the teacher preparation program and outcomes for all teacher candidates. You will see on page 28, the essential questions being answered and the likely sources of evidence that will be used to review the program’s processes in this area. There is only one indicator and five criteria under this judgment area. These are located on pages 29 through 31.

Slide 15:
That concludes the overview of what the laws state and the review of the Florida Site Visit Framework – which was Part One of today’s workshop. Now we will move to Part Two, which is to provide you an opportunity to ask cla rif yin g qu estions .

As a reminder, you may ask questions to clarify anything in the presentation or the draft text. If we are unable to provide an answer today, we will take your questions and get back with you for a response.

Slide 16:
. Again, this is the clarifying questions section. If you have comments regarding the rule, we will take them a little later. When we do get to the comment section, if there are many individuals who wish to comment, we may need to limit the time to allow as many as possible to comment before we run out of time. Remember that input may be provided at any time online at the address noted on this slide.

Slide 17:
So let us begin with those who are on the conference call line and then go to those who are in the room, and then to the chat feature on the webinar. For those who are in the room, please indicate to a Department member that you have a clarifying question so that when it is your turn, we can be certain to recognize you.

Operator, will you please compile the queue for questions and answers at this time.

Slide 18:
That concludes Part 2. We are now ready to move to Part 3 where you will have an opportunity

to make comments about any aspect of the rule, including the Florida Site Visit Framework. As a reminder, we will not respond comments during this time, nor is this time for a debate. Your comments will be considered as the draft rule language continues to evolve.

Slide 19:
As you think about the input you might provide, please keep these guidelines in mind: We are seeking input from you on how to improve the rule language, such as:
Suggestions for changes or agreement with specific text in the rule (for example., keep this…, delete this…, add this…, change this to read…, etc.).
To the extent that you can provide wh y you believe the change is needed, please provide a reason. This will help us with context and help us track that same issue in other parts of the rule.
We are also seeking comments on whether the rule text actually accomplishes what the law and/or presentation indicates is intended.

Slide 20:
So, with the time remaining, let us turn to comments. We will begin with those on the conference call line and then we will go to those in the audience today. If there are comments in the chat room, we will read those aloud as well. Please remember, if time is limited, we will limit the amount of time for your comments to no more than two minutes to ensure that all comments are heard.
Remember, you can also add comments to the website we indicated earlier. Operator, please compile the queue for comments.
Comments from the audience? Comments from the chat feature on the webinar?

Slide 21:
This concludes our Rule Development Workshop for today. Thank you for your participation throughout – and have a good day.