February 2006 Volume 2, Number 2 Instruments and Procedures for Implementing Early Childhood and Family Support Practices t o o l s CASE Coaching Practices Rating Scale for Assessing Adherence to Evidence-Based Early Childhood Intervention Practices Introduction This CASEtool includes a description of the development and use of the Coaching Practices Rating Scale. The scale is useful for assessing practitioner use of and adherence to evidence-based coaching practices in early childhood intervention (Rush & Shelden, 2005a, 2005b; Rush, Shelden, & Hanft, 2003). This paper includes a brief overview of coaching practices, a description of the scale, and an explanation of how to use the scale to determine the extent to which a practitioner uses the characteristics of coaching as part of his or her work with a family member or in supporting a colleague. The reader is referred to Rush and Shelden (2005b) for more in-depth information on the evidence to support this practice and a description of coaching practice indicators. Sources of information related to the characteristics and consequences of coaching practices are described in Rush and Shelden (2005a). Coaching Practices Coaching is an adult learning strategy that is used to build the capacity of a parent, caregiver, or colleague to improve existing abilities, develop new skills, and gain a deeper understanding of his or her practices in current and future situations (Hanft, Rush, & Shelden, 2004; Rush et al., 2003). As part of early childhood intervention, coaching is conceptualized as a particular type of capacity-building help giving practice that supports people in using existing abilities and developing new skills to attain desired child and family outcomes (Dunst & Trivette, 1996; Dunst, Trivette, & LaPointe, 1992; Rappaport, 1981; Trivette & Dunst, 1998). Additionally, Dathan D. Rush M’Lisa L. Shelden This CASEtool includes a description of the development and use of the Coaching Practices Rating Scale for assessing practitioner use of and adherence to evidence- based coaching practices in early childhood intervention. The scale is used to determine the extent to which a practitioner uses the characteristics of coaching as part of his or her work with a family or a colleague. A practitioner can use the scale for self-reflection or joint discussion to assess how his or her coaching practices are consistent with evidence-based practice indicators. CASEtools is an electronic publication of the Center for the Advanced Study of Excellence in Early Childhood and Family Support Practices, Family, Infant and Preschool Program, J. Iverson Riddle Developmental Center, Morganton, NC. CASE is an applied research center focusing on the characteristics of evidence-based practices and methods for promoting utilization of practices informed by research. Copyright © 2006 Center for the Advanced Study of Excellence in Early Childhood and Family Support Practices CASE coaching promotes self-reflection and refinement of current knowledge and skills by the person being coached. The intended outcome of coaching is improved competence and mastery of desired skills of the person being coached (Doyle, 1999). In early childhood intervention programs, practitioners use coaching to build the capacity of family members to promote their child’s learning and development (Trivette & Dunst, 1998, 2000). As a result, children’s primary caregivers gain competence when a coach supports them in blending new or existing knowledge, skills, and experience in ways that support and strengthen child learning and development. Coaching is “not telling people what to do, [but] giving them a chance to examine what they are doing in light of their intentions” (Flaherty, 1999), p. xii). In this way, early childhood practitioners who use coaching encourage action, reflection, and sharing of information based on the parent’s intentions and current level of knowledge and skills necessary to promote the child’s participation in everyday family, community, and early childhood settings (Bruder & Dunst, 1999; Hanft et al., 2004). Definition of Coaching Based on a research synthesis of coaching practices, coaching is defined as: An adult learning strategy in which the coach promotes the learner’s ability to reflect on his or her actions as a means to determine the effectiveness of an action or practice and develop a plan for refinement and use of the action in immediate and future situations (Rush & Shelden, 2005b). Coaching can be used to strengthen current knowledge, promote the acquisition of new skills, and support continuous self-assessment and learning of parents and colleagues regarding how to promote and support child learning and development. Coaches create a supportive and encouraging environment in which the learner (parent, colleague, etc.) and coach jointly examine and reflect on current practices, apply new skills and competencies in the context of feedback, and problem-solve challenging situations. The goal of the coach is to assist the person being coached in acquiring the competence and confidence to engage in action, self reflection, self correction, and use of new skills and strategies in other situations as appropriate (Flaherty, 1999; Kinlaw, 1999). Coaching Characteristics A research synthesis of coaching practices studies (Rush, 2003) identified five practice characteristics that promote the use of newly learned practices or improvement of existing skills. The coaching characteristics identified in the research literature were: (1) Joint planning, (2) observation, (3) action/practice, (4) reflection, and (5) feedback. Joint planning occurs as a part of all coaching sessions, which typically involves discussion of what a parent intends to do between coaching sessions and how the parent will use the information discussed or skills that were practiced. Observation refers to opportunities where: (a) the practitioner directly observes an action by the parent, which provides an opportunity for reflection and discussion or (b) the parent observes modeling by the practitioner where the practitioner builds upon what the parent is already doing and demonstrates the use of new strategies. After modeling occurs, the practitioner and parent discuss how the parent’s actions or new strategies match the parent’s intent and/or what research informs us about child learning. The characteristic of action provides opportunities for the learner to use the information discussed with the coach or to practice newly learned skills either during or between coaching sessions. Reflection occurs following an observation or action, and provides the parent an opportunity to analyze current strategies and refine his or her knowledge and skills. Feedback occurs after the parent has the opportunity to reflect on his or her observations, actions, or participatory opportunities to practice new skills. As part of feedback, the practitioner may affirm the parent’s reflections and/or add information to deepen the parent’s understanding of the practice being discussed. Coaching Practices Rating Scale The coaching model for supporting families and colleagues described by Hanft et al. (2004) and Rush et al. (2003) and the characteristics identified from a coaching practices research synthesis (Rush, 2003), were used to develop the Coaching Practices Rating Scale. The purpose of the scale is to determine the extent to which a practitioner is adhering to and using coaching practices with either families or colleagues to strengthen competence and confidence in using evidence-based early childhood practices. The items on the Coaching Practices Rating Scale were developed based on the findings of a recent research synthesis on how people learn (Bransford et al., 2000; Donovan, Bransford, & Pellegrino, 1999). A item pool was developed by a work group of practitioners at the Family, Infant and Preschool Program who were examining how coaching could be used to sup port parents’ and practitioners’ use of natural learning environment practices (Dunst, Hamby, Trivette, Raab, & Bruder, 2000; Dunst, Trivette, Humphries, Raab, & Roper, 2001). The group examined each item for consistency with the evidence, to eliminate redundancy, and to ensure that items covered all five of the characteristics of both coaching practices and how people learn (Bransford et al., 2000; Donovan et al., 1999). The first version of the Coaching Practices Rating Scale was reviewed by two individuals familiar with the coaching literature who provided feedback regarding item content. This information was used by the work group to make changes to the scale items. The scale was then piloted with six early intervention teams in two states. Individual items were modified based on the feedback from the team members. The Coaching Practices Rating Scale and instructions for use are included in the Appendix. Administering the Scale The scale is completed based on participation in or observation of a single coaching session or a series of coaching interactions between a practitioner and family member, between two practitioners, or between a supervisor and practitioner. For each item, the person scoring the coaching session(s) indicates how often the practice was used by the practitioner using the following scoring system: 0 = No opportunity to measure or use the practice, 1 = Use of the practice none of the time, 2 = Use of the practice some of the time, 3 = Use of the practice about half of the time, 4 = Use of the practice most of the time, 5 = Use of the practice all of the time. A rating of zero (0) indicates that the rater was unable to observe the behavior described or no opportunity occurred in a coaching session(s) to rate the item, whereas a rating of one (1) indicates that an opportunity for use of the behavior occurred, but was not used by the practitioner. The Coaching Practices Rating Scale can be used in a number of ways. First, practitioners can use the scale in a self-administration format to reflect on the extent to which his or her interactions with a parent or colleague were consistent with the characteristics of the practice. The scale can also be used to assess and reflect on coaching practices between a practitioner and family member, two practitioners, or a supervisor and practitioner. This type of reflection consists of identifying examples of one’s coaching practices within these venues and determining the extent to which the practices were consistent or inconsistent with the coaching practice indicators and why. Practitioners can use this information to identify targets of change they might make to strengthen their coaching skills and to ensure their practices are more consistent with the coaching indicator described by the scale items. Second, the Coaching Practices Rating Scale can be used by supervisors or colleagues following observation of a coaching session to assist a practitioner to reflect on his or her coaching practices. The supervisor or colleague can use the scale items as benchmarks for providing feedback about the observation. Follow-up discussions can assist the practitioner in developing a plan to change or improve his or her practices in ways more consistent with the characteristics of coaching. Third, the Coaching Practices Rating Scale can be used for program evaluation purposes. Programs managers can use the scale to collect and analyze data regarding the extent to which staff members are using the characteristics of coaching in their interactions with families and to monitor adherence over time. Conclusion Coaching is a strategy for improving practices that are consistent with available evidence on adult learning. The Coaching Practices Rating Scale can help practitioners determine the extent to which their interactions with colleagues or family members are consistent with coaching practices. The Coaching Practices Rating Scale can be useful to practitioners and program managers or directors for knowing if their implementation of the practices is both consistent and used frequently enough to promote the intended changes in the learner. This instrument is important because it makes an objective assessment of the extent to which the characteristics of the practice are used and how variations in use of the practices may be related to the expected benefits or consequences. Whereas other literature and research related to coaching provide descriptions of two or three of the characteristics of coaching (Donegan, Ostrosky, & Fowler, 2000; Flaherty, 1999; Hendrickson, Gardner, Kaiser, & Riley, 1993; Kinlaw, 1999; Kurtts & Levin, 2000), this instrument measures all of the characteristics described in various research studies. The Coaching Practices Rating Scale provides a way to measure multiple characteristics to allow a more precise assessment of which coaching characteristics matter most. In addition to the discussion of which features of the practice of coaching are and are not important, data collected using the scale should provide further empirical evidence about adult learning in general. This would contribute to further assessment and expansion of the knowledge base related to the characteristics of how people best learn. References Bransford, J. D., Brown, A. L., Cocking, R. R., Donovan, M. S., Bransford, J. D., & Pellegrino, J. W. (Eds.). (2000). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press. Bruder, M. B., & Dunst, C. J. (1999). Expanding learning opportunities for infants and toddlers in natural environments: A chance to reconceptualize early intervention. Zero to Three, 20(3), 34-36. Donegan, M. M., Ostrosky, M. M., & Fowler, S. A. (2000). Peer coaching: Teachers supporting teachers. Young Exceptional Children, 3(3), 9-16. Donovan, M. S., Bransford, J. D., & Pellegrino, J. W. (Eds.). (1999). How people learn: Bridging research and practice. Washington, DC: National Academy Press. Doyle, J. S. (1999). The business coach: A game plan for the new work environment. New York: Wiley. Dunst, C. J., Hamby, D., Trivette, C. M., Raab, M., & Bruder, M. B. (2000). Everyday family and community life and children’s naturally occurring learning opportunities. Journal of Early Intervention, 23, 151-164. Dunst, C. J., & Trivette, C. M. (1996). Empowerment, effective helpgiving practices and family-centered care. Pediatric Nursing, 22, 334-337, 343. Dunst, C. J., Trivette, C. M., Humphries, T., Raab, M., & Roper, N. (2001). Contrasting approaches to natural learning environment interventions. Infants and Young Children, 14(2), 48-63. Dunst, C. J., Trivette, C. M., & LaPointe, N. (1992). Toward clarification of the meaning and key elements of empowerment. Family Science Review, 5(1/2), 111-130. Flaherty, J. (1999). Coaching: Evoking excellence in others. Boston: Butterworth-Heinemann. Hanft, B. E., Rush, D. D., & Shelden, M. L. (2004). Coaching families and colleagues in early childhood. Baltimore: Brookes. Hendrickson, J. M., Gardner, N., Kaiser, A., & Riley, A. (1993). Evaluation of a social interaction coaching program in an integrated day-care setting. Journal of Applied Behavior Analysis, 26, 213-225. Kinlaw, D. C. (1999). Coaching for commitment: Interpersonal strategies for obtaining superior performance from individuals and teams. San Francisco: Jossey-Bass. Kurtts, S. A., & Levin, B. B. (2000). Using peer coaching with preservice teachers to develop reflective practice and collegial support. Teaching Education, 11, 297-310. Rappaport, J. (1981). In praise of paradox: A social policy of empowerment over prevention. American Journal of Community Psychology, 9, 1-25. Rush, D. (2003). Effectiveness of coaching on adult learning. Synthesis in preparation. Rush, D., & Shelden, M. (2005a). Characteristics and consequences of coaching. CASEmakers, 1(9), 1-3. Available from http://www.fippcase.org/casemakers/ casemakers_vol1_no9.pdf Rush, D., & Shelden, M. (2005b). Evidence-based definition of coaching practices. CASEinPoint, 1(6), 1-6. Available at: http://www.fippcase.org/caseinpoint/ caseinpoint_vol1_no6.pdf Rush, D. D., Shelden, M. L., & Hanft, B. E. (2003). Coaching families and colleagues: A process for collaboration in natural settings. Infants and Young Children, 16, 33-47. Trivette, C. M., & Dunst, C. J. (1998, December). Family- centered helpgiving practices. Paper presented at the 14th Annual Division for Early Childhood International Conference on Children with Special Needs, Chicago, IL. Trivette, C. M., & Dunst, C. J. (2000). Recommended practices in family-based practices. In S. Sandall, M. E. McLean, & B. J. Smith (Eds.), DEC recommended practices in early intervention/early childhood special education (pp. 39-46). Longmont, CO: Sopris West. Authors Dathan D. Rush, M.A. is Associate Director, Family Infant and Preschool Program and Investigator, Center for the Advanced Study of Early Childhood and Family Support Practices, J. Iverson Riddle Developmental Center, Morganton, North Carolina. M’Lisa L. Shelden, Ph.D. is Director, Family Infant and Preschool Program and Investigator, Center for the Advanced Study of Early Childhood and Family Support Practices, J. Iverson Riddle Developmental Center, Morganton, North Carolina. Appendix, continued Think about the coaching conversations for the time period covered. For each practice indicator, note how the practice was used. Coaching Practice Indicators Description of Practice 1. Acknowledged the learner’s existing knowledge and abilities as the foundation for improving knowledge and skills. 2. Interacted with the learner in a nonjudgmental and constructive manner during coaching conversations. 3. Identified with the learner the targeted skills and a timeline for the coaching process. 4. Developed with the learner a plan for action/practice necessary to achieve targeted skill(s) following each coaching conversation. 5. Observed the learner demonstrate knowledge and understanding of the targeted skill(s) or practice(s). 6. Observed the learner’s use of the targeted skill(s) or practice(s). 7. Created opportunities for the learner to observe the coach and/or others model the target skill(s) or practice(s). 8. Promoted use of multiple opportunities for the learner to practice implementation of the targeted skill(s) and practice(s) (e.g., role plays, in context). 9. Used both planned and spontaneous opportunities to strengthen the learner’s knowledge and skills. 10. Asked probing questions to examine the learner’s knowledge and abilities. 11. Prompted learner reflection on his/her knowledge and use of the targeted skill(s) and practice(s) compared against research-based practice standards. 12. Provided feedback about the learner’s knowledge and skills following the learner’s reflection on his/her performance. 13. Provided and/or promoting access to new information and resources after the learner reflects on his/her performance. 14. Engaged the learner in reflection on the usefulness, effectiveness, and need for continuation of the coaching process. Copyright . 2006 • Family, Infant and Preschool Program • All rights reserved. May be reproduced for practice purposes. Appendix, continued Coaching Practices Rating Scale Administration Procedure The Coaching Practices Rating Scale is used to determine the extent to which a practitioner uses the characteristics of coaching as part of his or her work with a family member or in supporting a colleague. A practitioner can use the scale for self-reflection or joint discussion to assess the extent to which his or her coaching practices are consistent with evidence-based coaching practice indicators. The scale items are based on the characteristics of how people learn and coaching practices. The characteristics related to how people learn include acknowledging and building on a learner’s existing understanding and interacting with the learner in a nonjudgmental way. The characteristics related to coaching include the use of joint planning between the learner and coach, observations by and of the learner, participatory action on the part of the learner, promoting the learner’s reflections on his or her actions, and feedback by the coach based on the learner’s reflections. The scale can be used by practitioners as a self-assessment tool to reflect on the extent to which his or her coaching practices are consistent with the coaching characteristics. The scale can also be used by supervisors or peers based on an observation of a practitioner’s coaching interaction(s) to assist the practitioner in reflecting on his or her coaching practices. When used for this purpose, the supervisor or peer can also use the scale to provide feedback related to the observation and the practitioner’s own reflections related to the interaction as well as to assist the practitioner in developing a plan for changes to make his or her practices more consistent with the characteristics of coaching. The scale can be used for individual self-assessment following a single or series of coaching interactions. When used for supervision or peer-to-peer support, completion of the scale follows an observation of a coaching session between a practitioner and family, two practitioners, or supervisor and practitioner. Program managers can use the scale to evaluate the extent to which staff members are using and adhering to coaching practices. The scale includes space to record the name of the practitioner conducting the self-assessment or being observed, the name of the person observing the practitioner and completing the assessment (if applicable), the date(s) of the interaction(s), and the period of time covered by the ratings. The time period may include a specific date if the self-assessment or observation is based on only one coaching interaction or the period may be the number of sessions and timeframe (e.g., four coaching sessions). Each item is rated using the following scale: 0…..No opportunity to measure or use the practice/ not applicable 1….. Use of the practice none of the time 2….. Use of the practice some of the time 3….. Use of the practice about half of the time 4….. Use of the practice most of the time 5….. Use of the practice all of the time When using the scale for conducting a self-assessment or reflection on the observations of a colleague or supervisee, the rater should do the following: (a) review the items and consider the opportunities that were provided for the practice to occur and the frequency with which the person being rated actually demonstrated the practice, (b) note the specific circumstances regarding when and how the practices were used, (c) identify missed opportunities for using the practices, (d) consider reasons why practices could not be assessed, (e) identify areas for improvement, (f) develop a plan for improvement for use during interactions with the person being coached in the interaction(s) documented on the scale as well as interactions with other persons coached by the practitioner. This information is recorded on the reverse side of the Coaching Practices Rating Scale.