Tool: Assessing Your Occupational Skills Training Program

(Based on Mark Friedman's Result-Based Accountability Model)

Use the tool below to set goals for success and plan for data collection, analysis, evaluation, and improvement.

Indicators of Success

Determine Your Quantifiable Goals for Each Indicator
For example, success indicators might include:

For Recruitment/Enrollment
___ # of youth the program will recruit
___ % of recruited youth who will be determined eligible for training
___ % of eligible youth who will attend orientation
___ #/% of youth attending orientation who will be enrolled in training

For Classroom-Based and/or Field-Based Hands-on Occupational Skills Training and Industry-Recognized Certification
___ #/% of participants who will attend (%) of classroom occupational skills training
___ #/% of participants who will complete classroom-based occupational skills training
___ #/% of participants who will take a credentialing exam
___ #/% of participants who will attain industry-recognized certification

For Internships/Apprenticeships
___ # of internships/apprenticeships the program will develop
___ # of internship/apprenticeship supervisors the program will orient
___ #/% of internship/apprenticeship sites program staff will monitor
___ #/% of participants who will be placed in internships/apprenticeships
___ #/% of participants who will successfully complete internships/apprenticeships
___ #/% of employers who will rate their experience with participants as satisfactory

For Academic Support
___ #/% of participants w/o a high school diploma who will obtain a diploma or GED
___ #/% of participants enrolled with a diploma/GED who will improve literacy/numeracy skills by (grade level or score)

For Professional Development
___ #/% of participants who will attend (%) of professional development sessions
___ #/% of participants who will increase level of job readiness by a score of < >

For Employment and/or Postsecondary Education
___ #/% of participants who will successfully complete training
___ #/% of participants who will be employed by the company that provided their internship/apprenticeship
___ #/% of participants completing training who will be employed in the industry in which they were trained at six weeks, at three months, at six months
___ #/% of participants completing training who will enroll in postsecondary education
___ % of stakeholders will will report that their experience was satisfactory

For Follow-Up
___ % of overall stakeholders who will be satisfied with the program will be <‰>
___ #/% of participants who will be contacted following training at six weeks, at three months, at six months
___ #/% of participants who will be employed in the industry at one month, at six weeks, at three months, at six months
___ #/% of participants who will enroll in at a postsecondary institution or training program

Data to Collect to Measure Success

Identify Data That Will Allow You to Measure Success Indicators
Examples of data to collect that will help measure success:
For Recruitment/Enrollment
# of youth recruited
# of recruited youth determined eligible for training
# of eligible youth who attended orientation
# of youth enrolled for training

To Collect During Phase 3 and 4
% of participants who attended
% of participants who completed classroom-based occupational skills training
#/% of participants who took credentialing exam
#/% of participants who attained industry-recognized certification
# of internships/apprenticeships the program developed
# of internship/apprenticeship supervisors the program oriented
#/% of participants placed in internships/apprenticeships
#/% of participants who successfully completed an internship/apprenticeship
#/% of employers who rated their experience with program as satisfactory
% of participants who attained a high school diploma or a GED
% of participants with a diploma/GED upon enrollment who increased literacy/numeracy
#/% of participants who successfully completed professional development sessions
#/% of participants who increased their job readiness by a score of ___
#/% of participants who successfully completed training
#/% of participants employed by the company in which they completed an internship/apprenticeship
#/% of participants who were employed in the industry at six weeks, three months, six months, one year
#/% of employed program graduates who continue to receive support services
#/% of participants who enrolled in postsecondary education or training

To Collect During Phase 4
% of overall stakeholder satisfaction

Methods for Analyzing Data

Plan How You Will Analyze Your Data to Determine Success Examples:

Number of youth recruited will be calculated as number of youth submitting an application.
Percentage of participants who successfully complete professional development will be calculated as the number of participants attending at least 85 percent of the professional development sessions who attain a score of (#) on the job readiness assessment tool.
Percentage of participants who successfully completed internships/apprenticeships will be calculated by the number of reviews that indicate that participants performed at a satisfactory or better level divided by the number of final performance reviews completed by supervisors.
Stakeholder satisfaction will be measured by the number of stakeholders, including youth-serving organization and program partner staff and internship/apprenticeship supervisors, who found the overall experience to be satisfactory as reported on satisfaction surveys.

Process for Assessment/Evaluation/Improvement

Examples:

Determine your occupational skills training program success by comparing your goals for the success indicators with the outcomes of your data analysis.

For example: at the end of phase 3, use the data to determine what percentage of participants completing classroom-based occupational skills training attained the industry-recognized certification.
If you did not successfully reach your goals, brainstorm potential factors and carefully analyze data for clues. Make adjustments likely to produce better outcomes.

For example: If you met your goal for the number of participants who successfully completed classroom-based occupational skills training but did not meet your goal for the number attaining the industry-recognized certification, examine the occupational skills curriculum to see if it aligns with the skills required for a credential. If indicated, adjust the curriculum to better align and then test the changes made. If you still do not achieve the desired outcome, examine other factors that might affect the participants' ability to attain the credential, such as instruction techniques, training facilities and equipment, length of training or time between completing training and testing, and/or literacy or numeracy skills of participants who attained the credential versus those who did not.
Continue data collection, analysis, evaluation, and improvement with each training cycle.