ECUR 809 Assignment #3

Assignment #3; ECUR 809; Evaluation Assessment

TIPS for Residents; College of Medicine, University of Saskatchewan

Engaging Stakeholders:

TIPS for Residents is a program delivered by Educational Support & Development in the College of Medicine, University of Saskatchewan. It became a required course for residents upon the approval of the Faculty Council. Its purpose is to prepare residents to teach medical students (undergraduates) effectively. A secondary goal is to help residents to be, and feel, more confident about teaching and learning presentations they are required to do. Thus, the Undergraduate Medical Education Department is the primary stakeholder. Because of the commitment of residents in this program, postgraduate medical educators are involved and interested in the delivery and success of TIPS. Finally, the creators of TIPS for Residents, the facilitators of the workshops, and Educational Support & Development in general all have a vested interest in TIPS for Residents. Thus, the stakeholders to consider with this program include the Postgraduate Dean, Postgraduate Program Directors, Director of Undergraduate Medical Program, residents (represented by the Chief Residents), medical students, and staff and faculty in Educational Support & Development involved in creation and delivery of the program. All of the stakeholders (except for medical students) have been contacted and asked to provide input regarding the following three questions:

1. What are the primary goals and objectives of the TIPS for Residents program?
2. How would you measure/determine success in these goals/objectives?
3. What information emerging from a program evaluation of TIPS for Residents would you find to be the most valuable?

Some responses have been collected and others will continue to be gathered to help create a focus and direction for the program evaluation.

Focus:

The purpose of the program evaluation is to determine the success of TIPS for Residents and address possibilities for improvement. As such, this will be an outcomes evaluation, concentrating on goals and objectives. While the satisfaction of all stakeholders is of importance, the primary users of the program are the residents who are participants in the workshops. The direct beneficiaries are medical students who should experience more successful learning as a direct result of better resident teaching. A more complete description of the program can be found in the logic model to follow (assignment #4). The key questions to ask have to do with the residents’ knowledge of teaching and skill improvement as a result of TIPS for Residents. Are residents becoming better educators and becoming more confident in teaching as a result of TIPS for Residents? What aspects of TIPS are helping in this regard? Are there ways to alter the program to improve residents’ skill and knowledge as educators?

Data Collection:

Ideally, it would be most beneficial to discover if their teaching skills improve as a result of the program but this would be logistically extremely difficult to measure. However, if the quality of resident teaching is improving, it should be revealed by interviewing medical students and getting their feedback about strengths and weaknesses in this regard. Insight could also be provided by interviews with Program Directors and other faculty to see if presenting at Rounds has improved as a result of the program. A pre-test before the workshops and then post-test shortly after the program, administered to residents, would also help indicate the effectiveness of TIPS for Residents. The feasibility of doing a pre-test and post-test would have to be determined with input from the stakeholders as well as considerations of budget allowances. If this would not be possible, post-test alone would help to understand the effectiveness of the program. Self-assessment data from residents would be valuable to gain insight into how they see their knowledge and skill as educators as a result of TIPS for Residents. Existing sources of data include workshop evaluations completed by residents over the past three years. In sum, data collection would be in the form of interviews with Program Directors, faculty, and medical students; pre and post-test surveys/questionnaires of residents including some self-assessment; and existing data from resident workshop evaluations. It would also be valuable to interview facilitators of the workshops; however, I am currently the primary facilitator so objectivity could be an issue. Some of the above data could also be collected with focus groups, again, with consideration for budget availability and stakeholder preference.

Analysis and Interpretation:

All stakeholders will be provided with results of the program evaluation and be able to give input into how to best interpret the results. Ultimately, Educational Support & Development would then take any suggestions and incorporate them into the planning and implementation of TIPS for Residents. Marcel D’Eon, Director of Educational Support and Development is the primary decision maker and is the person to consult in all matters pertaining to the program. Goals of the program evaluation would be to determine what is working well (so as to continue) and what needs to be modified and/or improved. In addition, Educational Support & Development would be conducting the research and collecting the data. As mentioned, some of this has already been collected in the form of the workshop evaluations. Krista Trinder is Research Assistant with Educational Support & Development and would likely have an important role in conducting the research and interpreting results. Krista is currently in the process of applying for funding for a program evaluation of TIPS for Residents so the result of that will go a long way in determining the feasibility of the above proposed measures of data collection. In addition, the methods will involve time commitments from stakeholders and medical students and, therefore, their accessibility and involvement will also be considerations. The timeline for completion of the program evaluation is also dependent on the results of the request for funding assistance.