Key Performance Indicators (KPI) Ontario colleges are mandated by the provincial government to collect and report performance data to measure how well they are meeting the needs of students, graduates and employers. The Key Performance Indicator (KPI) initiative is designed to ensure that the needs of above stakeholders remain accountable, effective and responsive. The following are a few of our highlights from the provincial government’s latest KPI survey of Ontario colleges, released on February 28, 2024. We are committed to providing our students with quality learning experiences and support in finding pathways to jobs and productive citizenship. Graduate Employment Employer Satisfaction Graduate Satisfaction Graduation Rate 88.6 per cent 95.0 per cent 76.6 per cent 65.8 per cent For more information, visit www.collegesontario.ca HOW ARE KPI RESULTS CALCULATED? Four KPIs are used to measure performance: Graduate satisfaction Employer satisfaction Graduate employment Graduation rate Data for graduate and employer satisfaction as well as graduate employment are collected by an external survey consultant through telephone surveys. Graduation rate is calculated annually by each college. WHAT GRADUATES ARE CONTACTED FOR THE GRADUATE SATISFACTION SURVEY? WHEN AND HOW? As per an agreement with the Ministry of Colleges and Universities, all colleges provide Forum Research, a consulting firm, with a list of the official graduates for each graduating term. For the 2022-23 reporting period our graduating terms were: Summer 2021 Fall 2021 Winter 2022 Across the three completed graduate survey periods, more than 42,036 interviews were completed out of a total graduate population of about 138,158. For Durham College, 1,233 graduates were surveyed out of a total graduate population of 5,484. Approximately 2,053 employers completed the survey. Of these, 30 had employed Durham College graduates. Graduates are surveyed by phone six months after graduation. These parameters were developed by the province-wide KPI Steering Committee, which is made up of college representatives and ministry personnel and is consistent across all colleges. In addition to being asked about satisfaction with the programs and services experienced while at Durham College, graduates are asked about their employment status, which provides information for the Graduate Employment KPI including if they have chosen to further their education. WHAT EMPLOYERS ARE CONTACTED FOR THE EMPLOYER SATISFACTION SURVEY? As a part of the graduate satisfaction survey, graduates are asked for permission to contact their direct supervisors. If permission is granted, every effort is made by the consulting firm to contact employers within five weeks after completing the survey. Employers are not contacted without the consent of graduates. The purpose of the employer satisfaction survey is to assess the performance of the college, not the graduate. HOW IS THE GRADUATION RATE CALCULATED? As defined by the Ministry of Colleges and Universities, the graduation rate is based on tracking individual students who entered a program of instruction in a particular enrolment reporting period and assessing how many of those individuals completed the program a number of years later. For each program of instruction, students who transfer out are removed from the group that is being tracked, and those who transfer in are added. The normal program duration used to establish a student’s program completion time frame equals approximately 200 per cent of the normal program duration. For example, the 2022-23 graduation rate is based on students who started one-year programs in 2020-21, two-year programs in 2018-19, and three-year programs in 2016-17, and who graduated by the 2021-22 academic year. HOW VALID/RELIABLE ARE THE SURVEYS AND CORRESPONDING RESULTS? The surveys used in the KPI data collection process were developed from surveys previously used within the college system. Minor adjustments were made to select surveys by the province-wide KPI Steering Committee to meet KPI objectives. The surveys meet conventional standards with respect to satisfaction surveying or opinion polling. The third-party consultants who were contracted to gather the data brought both experience within the college system and expertise in survey design to the planning tables. Sample size plays a significant role in the reliability and validity of data. Small sample sizes can sometimes result in chance findings that are not accurate. Sample size for program level, by definition, is always smaller than the overall college level. Caution is urged in interpreting data at the program level. WHAT IS THE MARGIN OF ERROR FOR THESE SURVEYS? At the college system level: The graduate sample had a margin of error of +/- 0.5 per cent, 19 times out of 20. The employer sample had a margin of error of +/- 3.1 per cent, 19 times out of 20 Across the three completed graduate survey periods, more than 42,036 interviews were completed out of a total graduate population of about 138,158. For Durham College, 1,233 graduates were surveyed out of a total graduate population of about 5,484. More than 2,053 employers completed the survey. Of these, 30 had employed Durham College graduates. WHAT SCALE WAS USED AND WHY? All decisions regarding survey design, instrumentation, etc. were made by the province-wide KPI Steering Committee with input from the contracted research firms. It was decided to use a five-point scale, where: Five equaled very satisfied Four equaled satisfied Three equaled neither satisfied or dissatisfied Two equaled dissatisfied One equaled very dissatisfied Such five-point scales are often used in instances where satisfaction is being measured. Only results registered as very satisfied and satisfied were considered and/or reported as positive results. HOW WERE THE KPI FIGURES REACHED? Each survey is comprised of many questions to provide valuable feedback to the colleges on how they might improve programs and services. Embedded in each survey are questions that are referred to as capstone questions – key questions that ask respondents to give their overall opinion about the central issue. These capstone questions are used to calculate the KPI figures. In selecting/designing the surveys, the general goal is to gain a comprehensive picture of user satisfaction. Each question is designed to measure slightly different aspects of respondents’ experience, so the use of capstone questions focusing on the overall satisfaction level, was adopted. This is a standard procedure for these types of surveys. WHAT DOES DURHAM COLLEGE DO WITH THESE RATINGS? The Office of Research Services, Innovation and Entrepreneurship (ORSIE) is responsible for the analysis of KPI data. Each program is provided with KPI Graduate and Employer outcomes related dashboards that include current year results as well as five-year trends. Particular emphasis is placed on aspects identified as important to students. Each program reviews survey results when considering improvements that will further enhance the student experience. HOW CAN I FIND OUT MORE ABOUT THE PERFORMANCE OF SPECIFIC PROGRAMS? All Ontario post-secondary institutions are required to provide students with information on: Graduation rates Graduate employment rates While common methods have been used to calculate the program-related information provided by each college, there may be some differences between how universities and private colleges calculate their information. For instance, graduate employment rates for college graduates are based on surveys completed six months after graduation, while university graduate employment rates are based on surveys completed six months and two years after graduation. For more information: OSAP performance indicators collegesontario.ca