Regulatory authorities such as FDA and ISO require organizations to maintain certain standards in processes and quality to stay compliant. These requirements are sometimes complex to un-derstand and implement. Complying with requirements specified by these standards involves making changes to several processes and controls in an organization. Organizations invest enor-mous time and money in meeting these compliance requirements. It becomes essential for people involved to understand the taxonomy, requirements and inter relationships of the compliance requirements. This competence in employees also helps organizations increase their operational efficiency.
All employees associated with quality management activities and compliance audits must be trained in implementing and maintaining these required standards.
Awareness, proficiency and process related practices are usually accomplished through trainings in-house and external. An average 17% of an enterprise's total budget is spent on training of employees and workers. Often however, the impact and return of this investment is barely measured, tracked or analyzed for improvement. Effective training leads to effective employees which in turn leads to organizational effectiveness. Thus, a way to measure effectiveness of trainings through metrics serves as a good proxy for organizational effectiveness.
Measuring every training program for effectiveness and results is achieved through defining and tracking of training metrics. A set of such metrics presented together to provide an overall pic-ture of trainings and their outcomes becomes a trainings scorecard for the organization. A com-prehensive learning and development scorecard helps organizations measure the effectiveness of trainings and in turn efficiencies achieved by employees. It helps in tracking and decision making to improve the quality of training and to align every training initiative with the organi-zational goals and objectives. Each effective training contributes to the overall organization's effectiveness. Thus, improving a training's effectiveness is directly improving an organization's effectiveness.
Compliance trainings related to Food and Drug Administration (FDA), International Organization for Standardization (ISO), and Occupational Safety and Health Administration (OSHA)are typically geared towards creating compliance awareness, training on new processes and creating profi-ciency in systems. Since these areas span the organization and exist across multiple depart-ments, it is a good idea to start designing a scorecard with a top down approach. One can start from the vision and mission of the organization with strategic objectives defined for each de-partment. These then become the anchors for training initiatives. Every training program must be aligned to its respective department's objectives and in turn to the organizational goals. Once identified, every training's defined outcome can be mapped to one or more of the de-partment's objectives. This exercise helps in aligning trainings to organizational goals. To illus-trate, an organization in aerospace may have a strategic objective this year to upgrade to the new ISO 9001:2015 certification. However, this certification comes with new requirements such as High Level Structure (HLS) clauses which must be met. Senior management then breaks this down into departmental goals. Thus, for departments, trainings become a means to achieve some of these goals. They may internally or with help of the learning and development de-partment, on-board a training program for the year to meet these goals. A scorecard with train-ing metrics will help the department to measure and evaluate against these set goals as is the case with ISO 9001:2015. Measuring effectiveness of the trainings will also measure organiza-tional performance. Deploying the right training is one thing, but training the right set of peo-ple is even more critical. Measuring training outcomes also ensures that training is imparted to the right groups of people. A well-designed scorecard should provide for measuring, tracking and improvement of these program outcomes. This will help decision makers in realigning train-ing programs or individually address execution level concerns such as instructor quality, con-tent quality or delivery mechanisms.
A course, once conceptualized should have defined objectives that can be measured, tracked and reported on a regular basis. A training management system can be employed to collect metrics and report data for each course. These metrics can then be rolled up to a program lev-el, which in turn roll up to a department and eventually to the organization level. Thus, a rela-tionship between a training effectiveness and an organizational effectiveness are established. Management must be cognizant of the fact that not every outcome can be quantified and measured. These are particularly true for programs around organizational culture.
Once a training is set up with objectives defined and aligned to the organization, applicable metrics must be defined for each individual training. For example, a course like "FDA Regula-tions for marketing OTC drugs" will have an outcome such as "after completion of this course, all employees of marketing will have knowledge on FDA Regulations for OTC drugs". However, measuring if this outcome is achieved through training is important. Outcome can be assessed through end of the training exams and feedback surveys to determine the training content and delivery effectiveness. A training management system can efficiently streamline assessment da-ta collection and analysis, which will help in continuous improvement of training programs. This achievement score may later be required as part of compliance or to measure organiza-tional efficiency achieved through this training.
There are several popular frameworks that can be employed to define training metrics. One such popular framework is Donald Kirkpatrick's four level evaluation model (Kirkpatrick, Donald L. "Evaluating training programs: Evidence vs. proof." Training Dev J (1977)). The four levels are Reaction, Learning, Behavior and Results. Below is a list of training metrics that every training program must measure and track at the least, to evaluate the relevance, individual develop-ment and organizational impact. In addition to learning evaluation metrics, few cost analysis metrics are also listed for budgeting efficiencies. However, this list is not exhaustive for every scenario and should be treated as the minimum required set up before adding domain specific metrics.
Training Quality Metrics:
Training metrics evaluate the efficiency of the training delivery. Content quality with respect to relevance and organization, instructor quality measuring the knowledge of the instructor and ability to teach, logistics efficiency and overall delivery quality are few examples. These are typ-ically recorded through post training feedback gathering. Tracking employees' responses and deriving insights from all trainings for a given module or instructor will help in measuring quali-ty and will render for continuous improvement. In some cases, an organization may choose to administer a survey after a regulatory audit by authorities such as ISO and FDA to determine how the trainings that have been availed have helped the employees through the audit phase.
Learning Quality Metrics:
Metrics under this category analyze the attainment of objectives that were planned for the training. These are usually collected through assessments, post training projects and feedback on knowledge increase. Before and after results are collected through pre-and post-assessments and surveys. Trainees will indicate what new they have learnt from the training and how their understanding of the subject has improved from the training program. Root cause analyses that highlight employee or operator errors can also be used to determine if a training program needs to be modified or improved.
This category of metrics evaluates the behavioral changes achieved through the trainings. At-tainment of results that were originally planned for the training are collected and analyzed. If a training program was conducted to familiarize a new Quality management system, metrics used here would be related to the adoption of the tool across targeted user groups.
Lastly, total cost of training is the most important metric to measure how much money is in-vested for the training through direct and indirect expenditure. This includes, cost of material, instructor's time, resources used and other operational expenditures such as travel and board-ing. For some sectors, opportunity cost that measures lost income of the employee must also be included to measure the investment. Every other metric is measured against this to evaluate effectiveness of the training program.
Return on investment is another direct metric that is more relevant in operational efficiency improvement programs. It is defined as a savings achieved through the training divided by the investment on the training. If a training is conducted on using an effective Quality and Risk Management system, ROI can be measured by determining the cost savings achieved through risk avoidance and mitigation for a specified period.
As each training is ultimately geared towards increasing organizational efficiency, effectiveness of trainings is directly linked to an organization's performance. A well-defined approach to rec-ord, analyze and report training metrics through a scorecard is a way to ensure an organization's optimal use of resources. Once a scorecard is put together with these basic metrics and other domain specific measures, the scorecard must be maintained on a regular basis for it to become the driver for analysis and decision making. Learning and development in an organization in-volves considerable effort via time, money and labor. Tracking effectiveness of trainings regular-ly helps an organization to continue good programs and make improvements where possible. An effective training management system administered as a part of integrated Quality Management system will enable automated tracking and monitoring of training metrics and help in deploying effective trainings.