Standardization and Evaluation: Training's Evil Twins
Tate, Robert, Phi Kappa Phi Forum
In a previous column, which appeared in the spring 2008 issue of The Forum, I focused on the necessity for businesses to develop and maintain robust training programs. In addition to training, however, any organization that wants to succeed must also implement training's evil twins, standardization and evaluation.
I say evil with my tongue firmly imbedded in my cheek, but standardization and evaluation move beyond training by encompassing aspects of analysis, enforcement and if necessary, punishment--punishment in that if you don't make the standard, you're finished.
In the flying world, standardization in safety results in annual checkrides. In my 25 years of flying, this standardization, while admittedly creating consternation and many a restless night, continues to serve as a linchpin to safety and mission effectiveness. During these evaluation flights, everything the pilot does is watched by the evaluator. Having been on both sides of the checkride fence, much like a Christmas present, it is always better to give than to receive.
What a pilot is graded on during their checkride, and the number of checkrides per year, depends on the type of aircraft they fly. Although civilian and military requirements differ greatly, while flying NATO Airborne Warning and Control System (AWACS) aircraft, for example, I received at least two checkrides per year. One checkride was a four-hour simulator event from hell in which I was tested on dozens of normal and emergency procedures. The other consisted of actually flying the airplane. Once I became an instructor pilot, annual instructor requirements had to be graded as well.
[FIGURE 1 OMITTED]
Each of these "tests" consisted of mission planning, ground operations, all phases of flight including takeoffs, landings, approaches, air refueling, post-flight, and debrief. In all, 30 or more items were watched by an evaluator pilot, then graded Level 1, or Pass; the less-enjoyable Level 2, Pass but Needs Work, or a failing Level 3. A Level 3 in certain critical items such as "Judgment" or "Safety" meant an automatic Fail for the entire ride. Failure for any reason also meant mandatory retraining, then a "recheck" in which the pilot successfully executed the failed event or even performed the entire checkride over again.
As a standardization and evaluation pilot, my primary instrument outside of the checkride was the Trend Analysis Tool (TAT). Here, every item graded on every checkride was logged. If I had 30 pilots in my squadron, some 60 or more annual checkrides would be reflected in the TAT. …