Cheryl Poth

Assistant Professor; Centre for Research in Applied Measurement and Evaluation

Educational Psychology
6-110 A Education North Building
cpoth@ualberta.ca

Research Interests and Professional Background

As a faculty member in a research-intensive Centre for Research in Applied Measurement and Evaluation in the Measurement, Evaluation and Cognition (MEC) program area, Dr. Poth conducts research, teaches, and supervises graduate students in the areas of program evaluation, mixed methods, assessment and health sciences education. See below for details related to each area.

Program Evaluation: 

Advancements in the field of evaluation continue to emerge in terms of practice and research. As a new approach, developmental evaluation represents a radical shift from traditional evaluation approaches in that conducting evaluative inquiry is not predicated on pre-establishing evaluation goals, time constraints, or a detached role for the evaluator (Patton, 2010). The developmental evaluator is charged with stimulating discussions and using evaluative logic that in turn facilitates data-informed decisions supportive of ongoing organizational and program development.

Dr. Poth is an active program evaluator having been involved in a variety of organizational settings including Faculty of Health Sciences, local school boards, provincial governments, early childhood education centres, and internationally in Guyana and Uganda. Dr. Poth’s specific research interest is in the how developmental evaluation can promote evaluation use within dynamic contexts. Organizational theories informed by complexity science provide an innovative framework in which to anchor this research.

Dr. Poth is a member of the Canadian and American professional evaluation organizations and regularly contributes to the annual meetings and publications of the Canadian Evaluation Society (CES) and the American Evaluation Association (AEA).  She is also the University of Alberta’s co-representative to the Consortium of Universities for Evaluation Education, which is a collaborative partnership among universities, government, and the Canadian Evaluation Society for the purpose of building capacity in graduate-level education and training in Canada. Graduate students enrolled in the evaluation program focus often participate in an internship offering practical evaluation experiences. During the past two years, students have interned at Alberta Innovates, Evaluation and Research Services, and Community and University Partnerships.

EDPY 615 Program Evaluation is a doctoral course offered every fall (and on occasion in winter term) with the aim of introducing course participants to the complexity of social and program evaluation as a consultative process.

Interesting news: The profession is undergoing a significant change with the recent approval by the Canadian Evaluation Society National Council of the Professional Designation project and endorsement of competencies for evaluation practice (see http://www.evaluationcanada.ca/site.cgi?en:5:6). New evaluators seeking designation will require two years of evaluation experience, completion of graduate level education, and to demonstrate competencies across five domains.

Representative Publications & Presentations

Poth, C., Howery, K., & Pinto, D. (2012). Addressing the challenges encountered during a developmental evaluation: Implications for evaluation practice, Canadian Journal of Program Evaluation, 26, 39-48.

Pei, J., Poth, C., Pinto, D., Hutchinson, M., & Gordon, J. (2012). The Collaboration Project. Report for Alberta Education, Edmonton, AB.

Howry, K., Pinto, D., & Poth, C. (2011). Final Year Three Findings: “Making a Difference”: Developing a Shared Vision Through Technology Leadership Project. Report prepared for BlackGold School District, Leduc, AB.

Poth, C. (2011). The untapped potential of developmental evaluation in teacher education programs. In T. Falkenberg and H. Smits (Eds.) The question of evidence in research in teacher education in the context of teacher education program review in Canada (2 vols.) (pp. 205-214). Winnipeg, MB: Faculty of Education of the University of Manitoba.

Poth, C., Pinto, D.*, & Howry, H. (2011, November). Examining critical episodes of a developmental evaluation: Unpacking the progressions involved in relationship building and capacity development. Paper presented at the annual meeting of the American Evaluation Association, Anaheim, CA.

Poth, C., Ross. S., Georgis, R., Donoff, M., Humphries. P. Steiner, I. (2010, November).  Enhancing evaluation quality and use for dynamic organizations: Lessons learned from a developmental evaluation of a competency-based medical education innovation. Paper presented at the annual meeting of the American Evaluation Association, San Antonio, TX.

Poth, C. (2010, October). Tapping into evaluation’s potential for supporting organizational development and learning. Invited keynote address. Annual meeting of the Canadian Evaluation Society, Ontario Chapter, Toronto, ON.

Howery, K., Poth, C., & Lejeune, A. (2010, May). Embracing the 21st century?: A case study of web 2.0 tools in a developmental evaluation. Paper presented at the annual conference of the Canadian Evaluation Society, Victoria, BC.

DeLuca, C., Poth, C., & Searle, M. (2009). Evaluation for learning: A cross-case analysis of evaluator strategies. Studies in Educational evaluation. 35, 121–129.

Poth, C, & Shulha, L (2008). Encouraging stakeholder engagement: A case study of evaluator behaviour. Studies in Educational Evaluation, 34, 218-223.

Poth, C., & Stanbury, H. (2008). Queen’s University Inter-professional patient-centred Education Direction (QUIPPED) Evaluation Findings. Final Report to Health Canada, Ottawa, ON.

The use of mixed methods as a research vehicle is not new; quantitative and qualitative data have been collected for many decades. The field of mixed methods continues to gain attention and acceptance among researchers having been described as the third methodological movement by various researchers (e.g., Tashakkori & Teddlie, 2010). What is emerging is understandings that mixed methods researchers must possess not only a background in qualitative and quantitative research methods but also knowledge of the current mixed methods research literature.

Dr. Poth brings to the field of mixed methods expertise in a variety of qualitative and quantitative research methods as well as experience as the mixed methods practitioner for research teams. Her interests are related to maintaining validity and reliability in research designs through examining procedures for data collection, for analysis, and integrating results.

Dr. Poth is a member scholar of the International Institute for Qualitative Methodology, and is an editorial board member for the Journal of Mixed Methods Research. Dr. Poth is a regular contributor to the annual meeting of Mixed Methods Researchers and to the Department of Educational Psychology graduate student Mixed Methods reading group.

EDPY 697/604 Mixed Methods Research is a doctoral course offered every second year (most recent offering winter 2011) with the aim of introducing course participants to the knowledge and skills required for undertaking a mixed methods study.

Interesting news: As mixed methods research evolves as a field, granting agencies and journals are beginning to reflect this shift and value the approach. An International Association for Mixed Methods Researchers is in the planning stages but in the meantime the new AERA SIG focused on Mixed Methods is a great avenue for research dissemination.

Representative Publications & Presentations

Poth, C. (in press). Exploring the role of mixed methods practitioner within educational research teams: A cross case comparison of the research planning process, Special issue of “Mixed Methods Research in Education” International Journal of Multiple Research Approaches.

Turner, S. R., White, J., & Poth, C. (in press). Learning the CanMEDS roles in a near-peer shadowing program: A mixed methods randomized control trial, Medical Teacher.

Poth, C. (2012, May). Assessing the impact of a team instructional approach: A mixed methods study of effective large class learning environments. Paper presented at the 8th International Congress of Qualitative Inquiry, Urbana-Champaign, IL.

Poth, C. (2011, June). Rethinking how we teach graduate-level methods courses: A progressive framework for mixed methods education. Paper presented at the 7^th annual Mixed Methods conference, Leeds, UK.

Poth, C. (2011, February). The unique contribution of the mixed methods researcher. Invited guest lecture. University of Nebraska, Lincoln, NE

Poth, C. (2010). Examining the Contributions of a Mixed Methods Exploratory Sequential Design: A Qualitative Perspective (pp. 525-539). In M. Pourkos & M. Dafermos (Eds.), Qualitative Research into Social Sciences: Epistemological, Methodological and Ethical Issues. Athens: Topos.

Poth, C., Pei, J., Rasmussen, C., & Henneveld, D. (2010, July). Validity in embedded mixed methods designs: Implications for practice. Paper presented at the 6th Mixed Methods International Conference, Baltimore, MN.

Poth, C. (2009, 2010, 2011. June). Mixed Methods Design to Dissemination. Workshops presented at Thinking Qualitatively, International Qualitative Institute, Edmonton, AB.

Alberta Education (2007) reports the assessment of student learning as one of the most complex responsibilities teachers undertake and requires the application of professional judgments. Many school boards have implemented policies seeking a balance of assessment for learning (also called formative assessment) along with assessment of learning (also called summative assessment). Together, a balanced framework has been found to empower students as a partner in their own learning. This shift in practice requires developing knowledge, skills, and attributes for a new classroom assessment approach.

Dr. Poth builds on her experiences as a classroom teacher in the Canadian public and private school systems, and large scale assessment analysis in her research. In addition, Dr. Poth spent 5 years teaching internationally and served as an administrator of an overseas educational program. Dr. Poth’s interests include exploring the development of pre-service and beginning teachers’ classroom assessment practices. She is also interested in the relationship between assessment practices that support learning, inform instructional practice, and communicate achievement and student motivation. A current grant from the Teaching and Learning Enhancement Fund is allowing Dr. Poth to examine the learning environment in large-class, multi-section courses including the link between curriculum, instruction, and assessment.

Dr. Poth is a member of Canadian and American professional educational research organizations and regularly contributes to the annual meetings of the American Educational Research Association (AERA) and Canadian Society for Studies in Education (CSSE).

EDPY 303 Educational Assessment is a required undergraduate course within our teacher education program and is multiple sections are offered on campus both fall and winter terms. Collaborative programs also offer off-campus options. As coordinator of the EDPY 303 Educational Assessment course, Dr. Poth and course instructors introduce pre-service teachers to the current assessment practices using a team instructional approach.

Interesting news: A team teaching approach has emerged as an effective means of enhancing the student learning environment and providing coordination among multi-section large class courses. A team teaching approach represents a collaborative educational milieu wherein multiple educators assume responsibility for the organization, instruction and supervision of students’ learning experiences (Zhou, Kim & Kerekes, 2011). What is emerging from our study is that the learning environment is enhanced not only for students but also for teaching assistants and instructors.

Representative Publications & Presentations

Luhanga, U., Leighton, L., & Poth, C. (2012, June). Beyond conceptual boundaries: Using student feedback to enhance teaching and learning in higher education. Paper presented at the annual meeting of the Society for Teaching and Learning in Higher Education, Montreal, QC.

Reidel, A., Poth, C., & Luhanga, U. (2012, May). A mixed methods design exploring underlying reasons for student assessment preferences. Paper presented at the annual meeting of the Canadian Society for Studies in Education, Waterloo, ON.

Moore, E., Chu, M., Poth, C., & Daniels, L. (2011, March). The impact of a team instructional approach: The perspectives of students, teaching assistants, and instructors involved in multiple sections of large classes. Poster presented at the annual Festival of Teaching and Learning, Edmonton, AB.

Poth, C., & Moore, B. (2011, February). Optimizing course resources: Rethinking the role of the teaching assistant as a member of an instructional team. Workshop presented Centre for Teaching and Learning, Edmonton, AB.

Poth, C., Janwadkar, S., & Georgis, R. (2010, May). Preparing Teachers to Support and Measure Learning: A Western Canadian Comparison of Initial Teacher Education. Paper presented at the annual meeting of the Canadian Society for the Study of Education / Canadian Educational Research Association, Montreal, QC.

Lejeune, A., Poth, C., Daniels, L. (2010, May). Examining Pre-Service Teachers’ Knowledge, Skills, Attributes, Experiences, and Goals Related to Formative and Summative Classroom Assessment. Paper presented at the annual meeting of the Canadian Society for the Study of Education / Canadian Educational Research Association, Montreal, QC.

Klinger, D. A., Rogers, W. T., Anderson, J. O., Poth, C., & Calman, R. (2006). Contextual and school factors associated with achievement on a high-stakes examination. Canadian Journal of Educational Research, 29, 771-797.

Poth, C. (2006, May). Student and school indices in Ontario data sets and impact on grade 6 achievement. Paper presented at the annual conference of the Canadian Society for the Study of Education, Toronto, ON.

Health Sciences Education

Innovative approaches for health science education are constantly emerging. Among the most recent are competency based medical education and inter-professional educational learning opportunities. Specifically, medical education has undergone a shift towards competency-based models of education due to calls for greater accountability in all aspects of the profession (Frank & Danoff, 2007). Competency-based medical education (CBME) approaches physician training by focusing on outcomes, emphasizing abilities, de-emphasizing time-based training, and promoting greater learner-centeredness (Frank, Mungroo, Ahmad, Wang, DeRossi, & Horsley, 2010). Recent research highlights the need for feedback related to progress towards achievement of competency as critical for supporting learning (Teztlaff, Dannefer, & Fishleder, 2009).

Dr. Poth is involved in health sciences education innovations as a methodologist and evaluator. Previously she was the principal evaluator for a 3-year inter-professional education initiative at Queen’s University funded by Health Canada. Currently she is collaborating with the Department of Family Medicine on the development and implementation of their Competency-Based Achievement System (CBAS). CBAS is an innovative framework that emphasizes assessment for learning in the development of competences for Family Medicine residents. The primary tool for learning and assessment in CBAS is formative feedback. Dr. Poth has also served as a research methodologist with the Health Sciences Council at the University of Alberta in the development of instruments to measure medical communication skills.

Interesting news: The new Master of Education in Health Sciences Education (MEd in HSE) program aims to extend the knowledge and skills of health sciences educators in the areas of educational pedagogy, educational research, and interprofessional leadership within the collaborative context of communities of practice of professional educators.  

Representative Publications & Presentations

Turner, S. R., White, J., & Poth, C. (in press). Learning the CanMEDS roles in a near-peer shadowing program: A mixed methods randomized control trial, Medical Teacher.

Turner, S. R., White, J., & Poth, C., (in press). 12 Tips for Developing a Near-Peer Shadowing Program, Medical Teacher.

Turner, S. R., White, J., Poth, C., & Rogers. W. T. (2012). Preparing students for clerkship: A resident shadowing program, Academic Medicine, 87, 1288-1291.

Ross, S., Poth, C., Donoff, M., Papile, C., Humphries, P., Stasiuk, S., & Georgis, R. (2012). Involving users in the refinement of the competency-based achievement system: An innovative approach to competency-based assessment, Medical Teacher, 34, e143-e147. doi: 10.3109/0142159X.2012.644828.

Ross S., Poth C., Donoff, M., Humphries P., Steiner I., Schipper S., Janke F., & Nichols D. (2011). Program description: The competency-based achievement system (CBAS): Using formative feedback to teach and assess competencies with family medicine residents. Canadian Family Physician, 57, e323-e330. doi: 10.3109/0142159X.2012.644828.

Turner, S., Poth, C., & White, J. (April 2011).  A novel Program for first-year medical students to prepare for clinical training by shadowing residents. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

Ross, S., Poth, C., Donoff, M., & Humphries, P. (2009). Monitoring, adapting, and evaluating a competency-based assessment framework in medical education through participatory action research, American Educational Research Journal, 55(4), 549-552.

Broers, T., Poth, C., & Medves, J. (2009). What’s in a word: Understanding “interprofessional collaboration” from the student’s perspective. Journal of  Research in Interprofessional Education, 1, 3-9.

Courses Taught

• EDPY 303        Educational Assessment    
• EDPY 615        Program Evaluation
• EDPY 604        Mixed Methods Research
• EDPY 501        Introduction to Methods of Educational Research