Common errors strip the interview of its validity.
The interview is a critical part of the hiring process and is one of the few times, if not the only time, that the applicant's job-related skills are assessed. The written examination may have measured cognitive ability and, in some cases, personality traits. However, the interview is aligned to measure critical skills such as interpersonal orientation, customer service, problem-solving, verbal communication, etc. Despite the unique and valuable contribution of oral interviews, they are often the most poorly executed part of the entry-level hiring process.
In order for an interview to be effective, it must be both valid and reliable. Validity is a matter of designing questions that assess skills that are job-related and essential to police officer performance. Reliability is a matter of designing and deploying the interview in such a way that candidates can be accurately assessed.
There are a number of ways in which the validity and reliability of interviews are routinely damaged. Following are the most common errors that affect an interview's quality. The vast majority of these errors strip the interview of its reliability / accuracy. Certain errors, such as the first two, also harm the validity of the interview.
Lack of an Assessment Plan
There are a number of skills that you could be trying to assess: interpersonal skill, conflict resolution, customer service, decision making, prioritization, communication, etc. The goal of a good oral interview is to measure the skills that are most critical to the job. This requires that you first identify the essential job skills you are trying to measure and then design your questions to measure these job skills. Too often questions are drafted without any consideration regarding what you aim to assess.
Asking the Wrong Questions
Once you know what you are trying to measure, the questions become the critical mechanism to get you there. Often, agencies are asking questions that fail to measure important skills. Questions such as "Why do you want to be a police officer?" or "What have you done to prepare for a career in the law enforcement field?" generally have little value. Ask yourself, would answers like, "I am interested in a well-paid career with a good pension" or "I thought long and hard about being a police officer and then filled out a job application" be poor responses?
The real question is, what are these types of questions telling you about the candidates' job skills? In reality, they do not provide much reliable information. A good question will place the applicant in a complex and job-related scenario where the applicant's response offers great insight into various skills.
There are a handful of "time-honored" questions that many agencies ask that have relatively little value. Maybe more importantly, most candidates expect them and have found answers to these questions on the Internet. If you "Google(tm)" "common police officer interview questions," you will likely see a lot of the questions that you have used in the past or use currently. Questions such as, "Why do you believe you will make a good police officer?" or "What have you done to prepare for a career in law enforcement?" are not the best questions for reliably measuring essential job skills and will often result in introducing unwanted error into the rating process.
Asking Questions with Obvious Answers
Interviewers often ask questions that have obvious answers. Considering that the candidate is always attempting to provide what he s he believes is an intelligent answer, coupled with what he / she believes you are looking for, questions that have obvious answers simply produce socially desirable, less-than-candid responses.
The ideal strategy is to use complex scenario-based questions that require the candidate to process details and present a complex response. The candidate will be challenged to come up with a reasonable response to the question and will have little mental energy to expend trying to figure out what answer you are "looking for."
Too Few Questions, Too Little Time
One of the strategies that is employed when there are high numbers of candidates to interview is to reduce the amount of time spent with each candidate by reducing the number of questions and the time allowed to ask and answer these questions. The fewer questions that are asked, the less information you receive and the fewer the job skills you can assess.
A worthwhile interview will include five to eight complex questions and allow for approximately 20 to 30 minutes of time with each candidate. If this cannot be accomplished, it may be worth considering alternatives to the interview. There are tools available to replace interviews when candidate numbers make interviews impractical. These will be discussed later.
Lack of Rating Guidelines
Some interviewers operate with the following mentality: "I will know a good answer when I hear it." This approach does not work when a team of interviewers are evaluating candidates together. In order for an interview to achieve a reliable outcome, the interview questions must have associated criteria that define ideal responses.
This "answer key" of sorts allows raters to hold candidates to a uniform standard. In addition to these rating guidelines, assessors should have a standardized system for converting rating criteria into a numeric score. This process may be as simple as giving a specific numeric value based on the number of rating criteria achieved.
Interviewers without Training
Interviewers will often administer the interview without any training. Reviewing concepts such as rater error and improving reliability by establishing scoring consensus rules are critical elements that make the interview process both fair and accurate. An interview is successful to the extent that the interviewers rate like performances with like scores. This can only happen when interviewers are playing by the same rules and using the interview tool in the same manner. Rater training is a means of establishing standardization and identifying methods to improve process reliability.
Assume that three interviewers each observe a performance and then rate the performance on a seven-point scale. One rater gives the candidate a "3" and the other two raters give the candidate a "6." The interviewers average the three ratings and the candidate gets a score of "5." Is that fair? Was the candidate's skill level a 5? Or was it a 3? Or was it a 6? One performance cannot be accurately described as demonstrating three different skill levels. A reliable interview will seek to identify the proper score for the specific performance.
In order to achieve a reliable outcome, assessors need to discuss their scores and work toward consensus - or an agreement concerning the proper score. Averaging scores is a poor practice because averaging always pulls disparate scores to a center point. This center point is not necessarily the accurate score for a candidate; it is simply a statistical midpoint between multiple points of disagreement. A process of consensus turns disagreement into agreement, thereby creating accuracy.
Enhancing the Scope of the Interview
There are a number of strategies that can be employed to increase the effectiveness of your interview process by broadening its scope and increasing its realism. First, you can incorporate a writing exercise. Writing skills are frequently cited as one of the greatest weaknesses of law enforcement officers and these skills are critically important for competent report creation and documentation.
Require the candidate to arrive for the interview 30 minutes prior to the time you anticipate interviewing him / her. Allow the candidate approximately 30 minute to draft a written report that explains what skills he / she possesses that he / she believes will make her an excellent law enforcement officer. There are numerous other topics that would work well - the primary goal is to produce a writing sample that can be assessed for clarity and professionalism.
You can also include a presentation as part of the interview. Officers will often present before community groups, media or school children. Requiring them to make a brief presentation as part of an interview process is practical and it allows for an accurate assessment of interpersonal style, organization and communication skills. The presentation, interview questions and writing sample can all be scored immediately following the interview process.
Alternatives to Oral Interviews
As was mentioned previously, it is often difficult and impractical to administer interviews to large groups of candidates. When the group reaches a certain size, the security and reliability of the interview can be so compromised that an interview process is no longer feasible. In such cases, there is an alternative assessment tool that can assess similar job skills in a much more efficient manner. Situational judgment tests are paper-and-pencil tests that assess job skills using job-related scenarios. In fact, these tests can contain the same scenarios that are used in oral interview questions.
The manner in which the multiple-choice answer options are created and scored is the key to deploying a successful situational judgment test; however, if done properly, these tests can produce results that are very similar to the interview and much more efficient. These tests have been underutilized, likely as a result of people's affinity for oral interviews - people tend to appreciate the face-to-face interaction. However, if you have 100 or 1,000 candidates to interview, a good situational judgment test may be the perfect solution.
Chad Legel is the President of Selection Works, a human-resources consulting firm specializing in public safety testing. Legel may be reached at email@example.com
Photos by Mark C. Ide.