Hendon Publishing - Article Archive Details
Training Course Evaluations: What We Don't Know
Written by Roger Overholt
While most law enforcement agencies document training that officers have received, it is often difficult for an agency to demonstrate the impact the training had on an officer’s behavior or the agency’s performance. Many law enforcement agencies cannot validate the return they have received for their training investment.
To ensure the value of law enforcement training courses, the agency should be able to demonstrate that a behavioral change has occurred as a result of the training or a performance gap has been addressed. This can be achieved by conducting training evaluations, which require a four-level process. The four-level processes will involve trainers, personnel and supervisors. When done correctly, the evaluations will measure behavioral and performance change.
Law enforcement trainers are taught advanced principles and techniques to educate adult learners. While some trainers may not recognize the terms andragogy (adult learning) and pedagogy (juvenile learning ), they are well versed in the techniques used to teach police officers the many skills needed to ensure an officer’s success. Law enforcement training courses must address a variety of competencies that officers need to police in today’s modern and highly technical society.
Most trainers conduct post-training evaluations of courses they teach but do not evaluate performance improvements that occur as a result of newly acquired knowledge being applied by the officer in the work environment. Law enforcement administrators, supervisors and trainers should be familiar with Donald Kirkpatrick’s four levels of training evaluations, which are often used by private companies to evaluate their training programs.
Donald Kirkpatrick’s model of four levels of evaluation can be utilized to measure the accomplishments of training. Law enforcement trainers are taught the importance of conducting evaluations at the conclusion of each training course. Post evaluations conducted at the end of a training course do not usually properly measure whether new knowledge or skills are being transferred to an officer’s daily performance or if an organizational improvement has occurred as the result of the training. Follow-up phases of training evaluations should be conducted in order to measure behavioral change and performance improvement. Kirkpatrick’s model measures the student’s reaction to the training, learning accomplishments measured in knowledge or skills, behavioral change on the job, and the impact of the training. These four levels of evaluation can be completed with a cooperative effort by an officer’s trainer and his or her supervisors. By conducting the four levels of evaluation, the trainer and the organization have a better understanding of the impact of the training, if the training goal has been met, and the return of the investment that was expended to conduct the training.
The first level evaluation should be designed to critique the instructor, the facility and the participant’s interest in the subject. The summative (end of course) evaluation can be used as a formative (course development and improvement) evaluation for future classes. This evaluation will measure the student’s reaction to the course and the instructor. Trainers and supervisors should ensure that instructors are using adult learning techniques and methods.
Information obtained from the level one evaluations should be reviewed by the instructor, training supervisor, and when possible, a command level supervisor. To effect a behavioral change, the student must feel that he or she is in a learning environment and is receiving information or developing a skill that is valuable to him or her personally. Level one evaluation should include questions that address the personal value of the training and the student’s perception of the training environment.
While the end goal of training is to enhance skills and knowledge of an employee to improve work performance, the employee must have buy-in as to the personal value of the training. The instructor and supervisors should use the information obtained from the summative evaluation to continue to improve the course for future participants and to measure the participants’ interest and reaction to the instructor and the training methods being used.
The second level evaluation is designed to measure the transfer of knowledge or skill. This is often accomplished by having the student complete a post-test at the conclusion of the course. The post-test will evaluate information or skills presented during the course of training. A pre-test may be given to the student prior to the course being taught so the results of a post-test can be compared to demonstrate a transfer of knowledge.
A pre-test should be designed to measure the skills or knowledge of a student in the topic being taught prior to the course of training being conducted. The pre-test may be a written test, a physical demonstration of skills, or a class discussion relating to the topics being taught during the course of instruction. In some incidents, it may be necessary to document an individual’s knowledge of a subject with a pre-test in order to measure the level of knowledge the individual student obtained as a result of a training course.
An individual assessment is important in many of the dangerous and high-liability areas of law enforcement. An assessment of the individual student’s ability should be conducted for firearms, defensive and pursuit driving, physical arrest techniques, and other high-risk areas of training.
If a pre-test to post-test comparison reveals that a student has not acquired the knowledge or skill being taught, the instructor should recommend additional training or other corrective actions. The instructor and the employee’s supervisor should evaluate the training using information gathered from the first and second level evaluations to ensure the course of training is conducted in a manner consistent with adult learning. When an employee fails to retain new knowledge or learn a new skill, course evaluations should be reviewed to look for possible causes.
Both the first and second level evaluations are simple to conduct as they can be required as part of the training course and should be returned to the instructor by the student prior to the completion of the course. The first and second level evaluations are the most common type of evaluations conducted by law enforcement trainers. The evaluations are important to assess the training course but do not evaluate the impact the training had on the officer’s performance.
Level three and four evaluations are more difficult to perform but provide the most valuable information to the trainer, the officer’s supervisor and the officer. Level three and four evaluations will measure the effect of training on a participant’s behavior and the return on the training investment.
Level three evaluations should be designed to measure the transfer of newly obtained knowledge or skills into work performance. The level three evaluation gives the trainer and the organization a tool to determine if information presented in a training course has resulted in behavioral change. As a result of training, a behavioral change should transfer into performance by the student. The newly acquired or enhanced skill should result in a positive return for the training investment made by the organization.
The level four evaluation measures the results of performance improvement that occurred after training. The purpose of the level four evaluation should be to determine if training has improved performance and resolved performance gaps. The level four evaluation should compare officers’ performance after the completion of a training course to performance prior to the instruction.
For example, if the course goal was to reduce the number of police pursuits in which an officer is involved, without level three and four evaluations, the trainer and the organization would not have a measure of the effectiveness of the training.
A simple and effective way to conduct level three and four evaluations is to have supervisors monitor and document officers’ performance or require officers to complete a performance document in a specified period after the completion of a training course. The trainer and the officers’ supervisors should develop a form that will allow information necessary to evaluate the officers’ performance to be captured with a small amount of time investment.
The documentation should become a part of the officer’s training file. This type of documentation can be used to show that a change has occurred in the officer’s behavior and performance as well as the impact the training had on the agency. Trainers must recognize that the transfer of knowledge is only a success when it is applied to an officer’s work performance.
In order to ensure that law enforcement training is successful, administrators and trainers must be willing to dedicate the resources necessary for conducting multiple-level training evaluations. By conducting proper evaluations, behavioral and performance changes can be measured and performance gaps can be recognized and addressed. The officer, the organization and the trainer can benefit from the information gained from conducting multiple level evaluations. Trainers and supervisors should develop a system for reporting both improvements and deficiencies as a part of the evaluation review process.
After the completion of a training course, levels one through four evaluations are necessary to determine if an organization has received a positive return on resources invested. It becomes easier to justify training and the associated cost of training to an agency’s governing body and citizens of the community when the agency has documented a positive return for the expenses related to training its personnel.
Law enforcement trainers are highly skilled professionals who are constantly seeking ways to improve both training methods and the professionalism of law enforcement. With a small amount of additional resources and time investment, trainers working with department supervisors and personnel can greatly improve the impact training courses are having on the individual’s and the organization’s performance. Properly conducted evaluations can ensure core problems are addressed, officer safety is increased, and liability exposure of the officer and the organization is decreased.
Major Michelle Jones has been with the Morristown Police Department since August 1998. She is currently assigned to the Administration Division where she is responsible for statistical information gathering and analysis, assisting with the budgeting process, personnel allocation and grant writing. Jones holds a Master of Arts degree in Organizational Management from Tusculum College.
Roger D. Overholt has 27 years of law enforcement experience. He holds Master’s Degree in Education and a Bachelor’s Degree in Organizational Management. He is a graduate of the FBI National Academy and the Tennessee Law Enforcement Executive Development School. He currently serves as Police Chief for the Morristown, Tenn. Police Department.
Published in Law and Order, Oct 2011
Rating : 7.0
Click to enlarge images.