Since 1998, Ontario colleges have been mandated by the provincial government to collect and report performance data to measure how well they are meeting the needs of:
The Key Performance Indicator initiative is designed to ensure that the needs of above stakeholders remain:
The following are a few of our highlights from the provincial government’s latest KPI survey of Ontario colleges, released November 19, 2018.
We are committed to providing our students with quality learning experiences and support in finding pathways to jobs and productive citizenship.
|86.5 per cent||94.3 per cent||78.1 per cent||72.5 per cent||68.1 per cent|
For more information: www.collegesontario.ca
How are KPI results calculated?
Five KPIs are used to measure performance:
- Graduate satisfaction
- Employer satisfaction
- Graduate employment
- Student satisfaction
- Graduation rate
Data for graduate and employer satisfaction as well as graduate employment are collected by an external survey consultant through telephone surveys.
Student satisfaction surveys are completed in-class by colleges across the province and the results are then compiled by an external survey consultant.
Graduation rate is calculated annually by each college.
What graduates are contacted for the graduate satisfaction survey? When and how?
As per an agreement with the Ministry of Training, Colleges and Universities, all colleges provide Forum Research, a consulting firm, with a list of the official graduates for each graduating term.
For the 2017-18 reporting period our graduating terms were:
- Summer 2016
- Fall 2016
- Winter 2017
Across the three completed graduate survey periods, more than 47, 250 interviews were completed out of a total graduate population of about 100,750. For Durham College, more than 2,260 graduates were surveyed out of a total graduate population of about 4,414.
Approximately 4,760 employers completed the survey. Of these, 226 had employed Durham College graduates.
More than 130,000 college students in their second semester or later completed the Student Satisfaction Survey, including 5,781 from Durham College.
Graduates are surveyed by phone six months after graduation. These parameters were developed by the province-wide KPI Steering Committee, which is made up of college representatives and ministry personnel and is consistent across all colleges.
In addition to being asked about satisfaction with the programs and services experienced while at Durham College, graduates are asked about their employment status, which provides information for the Graduate Employment KPI including if they have chosen to further their education.
What employers are contacted for the employer satisfaction survey?
As a part of the graduate satisfaction survey, graduates are asked for permission to contact their direct supervisors. If permission is granted, every effort is made by the consulting firm to contact employers within five weeks after completing the survey. Employers are not contacted without the consent of graduates.
The purpose of the employer satisfaction survey is to assess the performance of the college, not the graduate.
How is the graduation rate calculated?
As defined by the Ministry of Advanced Education and Skills Development, the graduation rate is based on tracking individual students who entered a program of instruction in a particular enrolment reporting period and assessing how many of those individuals completed the program a number of years later.
For each program of instruction, students who transfer out are removed from the group that is being tracked, and those who transfer in are added.
The normal program duration used to establish a student’s program completion time frame equals approximately 200 per cent of the normal program duration.
For example, the 2017-18 graduation rate is based on students who started one-year programs in 2015-16, two-year programs in 2013-14, and three-year programs in 2011-12, and who graduated by the 2016-17 academic year.
How valid/reliable are the surveys and corresponding results?
The surveys used in the KPI data collection process were developed from surveys previously used within the college system. Minor adjustments were made to select surveys by the province-wide KPI Steering Committee to meet KPI objectives.
The surveys meet conventional standards with respect to satisfaction surveying or opinion polling.
The third-party consultants who were contracted to gather the data brought both experience within the college system and expertise in survey design to the planning tables.
Sample size plays a significant role in the reliability and validity of data. Small sample sizes can sometimes result in chance findings that are not accurate. Sample size for program level, by definition, is always smaller than the overall college level. Caution is urged in interpreting data at the program level.
What is the margin of error for these surveys?
At the college system level, for year 2016-17:
- The graduate sample has a margin of error of +/- 0.2 per cent, 19 times out of 20.
- The employer sample has a margin of error of +/- 0.6 per cent, 19 times out of 20.
- The student sample has a margin of error of +/- 0.2 per cent, 19 times out of 20.
Across the three completed graduate survey periods, more than 47, 250 interviews were completed out of a total graduate population of about 100, 750. For Durham College, more than 2,260 graduates were surveyed out of a total graduate population of about 4, 414, with a margin of error of +/- 1.6 per cent, 19 times out of 20.
More than 4, 760 employers completed the survey. Of these, 226 had employed Durham College graduates. The margin of error is +/- 7.3 per cent, 19 times out of 20.
More than 130, 000 college students in their second semester or later completed the Student Satisfaction Survey, including 5,781 from Durham College, with a margin of error of +/- 0.8 per cent, 19 times out of 20.
What scale was used and why?
All decisions regarding survey design, instrumentation, etc. were made by the province-wide KPI Steering Committee with input from the contracted research firms. It was decided to use a five-point scale, where:
- Five equalled very satisfied
- Four equalled satisfied
- Three equalled neither satisfied or dissatisfied
- Two equalled dissatisfied
- One equalled very dissatisfied
Such five-point scales are often used in instances where satisfaction is being measured.
Only results registered as very satisfied and satisfied were considered and/or reported as positive results.
How were the KPI figures reached?
Each survey is comprised of many questions to provide valuable feedback to the colleges on how they might improve programs and services.
Embedded in each survey are questions that are referred to as capstone questions – key questions that ask respondents to give his or her overall opinion about the central issue. These capstone questions are used to calculate the KPI figures.
In selecting/designing the surveys, the general goal is to gain a comprehensive picture of user satisfaction. Each question is designed to measure slightly different aspects of respondents’ experience, so the use of capstone questions focusing on the overall satisfaction level, was adopted. This is a standard procedure for these types of surveys.
What does Durham College do with these ratings?
The Office of Research Services, Innovation and Entrepreneurship (ORSIE) is responsible for the analysis of KPI data. Each program and service/facility area is provided with a KPI report card that includes current year results as well as five-year trends. Particular emphasis is placed on aspects identified as important to students. Each program and service/facility area is responsible for completing an improvement plan that will further enhance the student experience.
How can I find out more about the performance of specific programs?
All Ontario post-secondary institutions are required to provide students with information on:
- Graduation rates
- Graduate employment rates
- Ontario Student Loan default rates for government-funded post-secondary programs
While common methods have been used to calculate the program-related information provided by each college, there may be some differences between how universities and private colleges calculate their information.
For instance, graduate employment rates for college graduates are based on surveys completed six months after graduation, while university graduate employment rates are based on surveys completed six months and two years after graduation.
For more information: