USAF Aviation Safety Program Gap Analysis Using ICAO Safety Management Guidance
By Ostrowski, Karen E | |
Proquest LLC |
These requirements are further defined within ICAO Order 9859, Safety Management Manual, and are attributable to "training organizations . . . aircraft operators, approved maintenance organizations, organizations responsible for type design and/or manufacture of aircraft, air traffic service providers and certified aerodromes" (ICAO, 2013, p. 15). Several countries have already successfully established their own aviation safety management programs and continue to comply with ICAO regulations (Stolzer, Halford & Goglia, 2008).
U.S.
ICAO and
While this type of analysis is still needed today, SMS emphasizes proactive and predictive safety processes to identify hazards before reactive safety techniques are necessary (
To identify areas of compliance and noncompliance, this research performed a longitudinal capability gap analysis in 2012 and again in 2013 on the state of the USAF aviation safety program as compared to ICAO SMS standards and operated under the premise that full adherence would enhance the safety program, save lives, and reduce injuries and costs.
The results of this research suggested that the 2012 USAF aviation safety program was 40.6% compliant, 22.8% partially compliant and 36.6% noncompliant with ICAO safety management reference criteria, while the 2013 program was 64.8% compliant, 25.3% partially compliant and 9.9% noncompliant.
Literature Review
Publicly available USAF aviation safety-trend data and mishap-cost estimates were identified to reinforce the need for programmatic evolution from legacy safety program directives. While the review of literature did not reveal extensive peerreviewed research specific to the USAF safety program, quality, nonpeer-reviewed information was available. According to USAF Safety Center (2012), USAF flew 35 different models of aircraft for a total of 11.9 million flight hours from FY2006 to FY2011. Regulations specific to flight operations dictate that an aviation safety mishap investigation take place whenever damage to
*Class A = any unplanned occurrence in which the direct costs exceed
*
*Class C = direct costs of
*Class E = nonreportable events, but are investigated for mishap prevention purposes (
USAF Mishap Data
From FY2006 to FY2011, USAF experienced 119 Qass A mishaps and 441
In 2006 and 2010, the USAF experienced its lowest ever Class A mishap rates of 0.90 and 0.71 per 100,000 flight hours, respectively (Kreisher, 2006; USAF Safety Center, 2012). While the Class A rates of fiscal year 2011 climbed slightly, two fatalities occurred, equating to approximately one death for every 1 million flight hours flown (USAF Safety Center, 2012). Zero mishaps and fatalities is an admirable goal, but entities such as ICAO suggest failures and operational errors will occur in aviation, and no activity incorporating humans can be "guaranteed to be absolutely free from operational errors and their consequences" (ICAO, 2013, para 1.1).
After they reanalyzed 124 USAF aviation mishaps from 1992 to 2005, Gibb and Olsen (2008) recommended a program change and improvements to the operational risk assessment process through quality assurance and line-oriented safety audits focusing on latent error mitigation strategies. Such strategies are inherent in an SMS-based program and are being implemented on a limited basis with USAF.
USAF Mishap Policy
USAF's mishap prevention program is outlined in Air Force Instruction (AFI) 91-202 (USAF, 2011). This document specifies the overall approach that USAF will take with regard to general mishap prevention and directly addresses aviation safety in 11 of its 154 pages. After 1998, the document was unchanged until revisions were incorporated in 2011, 2012 and 2013. With the 2013 update, the aviation safety section now includes several proactive safety initiatives and the document also introduced SMS structure. In addition to AFI 91-202, its source document, Air Force Policy Directive (AFPD) 91-2 was updated in 2012 from its 1993 version, collectively demonstrating a new commitment to safety after programmatic stagnation.
Several higher-level safety policies are currently being rewritten to include SMS principles within
Safety Management Benefits
USAF places great emphasis on the resultant yearly mishap statistics, not unlike those mentioned previously. Liou, Yen and Tzeng (2008) recognize that this type of forensic data can provide useful insight, but caution organizations against relying solely on such data. One concern associated with an incident-rate-based safety system is that a mishap must occur before such a system can react (Liou, et al., 2008). Instead, safety professionals may choose to consider how an organization's safety process is applied since its variability may ultimately result in mishaps. An SMS reduces this variability by emphasizing safety risk and quality management in order to prevent mishaps before they can occur (Panagopoulos & Bond, 2011).
Shifting USAF's focus from a reactive, operationally focused safety program to one that reduces process variability will substantially enhance its safety culture and operational outcomes. Bottani, Monica and Vignali (2009) suggest that companies that incorporate an SMS exhibit higher performance in defining safety goals and communicating them to employees, updating risk data and risk analysis, as well as identifying risks and defining corrective actions, and employee training. These organizational safety changes may aid USAF in changing its established safety culture.
In 1994, a lack of organizational oversight allowed a pilot to perform dangerous aerial maneuvers resulting in four fatalities and the destruction of a B-52 aircraft (Trimble, 2010). Sixteen years later in 2010, a USAF C-17 crashed due to a similar lack of organizational oversight. These events suggest that simply changing regulations or firing culpable individuals is not enough to prevent similar mishaps from recurring (Trimble, 2010). A robust SMS seeks to ensure that quality safety processes are in place to continue risk management and feedback processes to reduce the likelihood of preventable mishaps. Without the quality tools and feedback processes inherent within an SMS, a traditional safety program is more likely to repeat past mistakes.
Research Questions
*As of
*As of
*Based on longitudinal research, is the USAF aviation safety program progressing toward an SMS as defined by international civil aviation?
Methods
ICAO and
A gap analysis is inherently subjective; however, this research attempted to minimize these effects by incorporating three researchers, each performing independent analyses, then collaborating afterward on disagreements. USAF was not asked to endorse this preliminary research; therefore, no official resources were provided to the research team other than documentation freely available to the public. The documentation, however, includes the same regulations and policies USAF uses to enforce its initiatives and is considered an accurate assessment of safety management regulatory implementation.
USAF did not allow access to its personnel or facilities; therefore, practical field implementation of USAF safety management regulations was not assessed within the body of this research. This type of research would require official USAF endorsement and acceptance of public disclosure within a non-USAF-controlled study. This type of research would be beneficial to safety management implementation and is recommended later in this article. Safety management, its processes and implementation are applicable to a wide range of disciplines and are not exclusive to aviation or USAF. While there are certainly limitations in generalizing gap analysis results, the authors believe the difficulties of large-scale safety management limitations may be generalized to other large organizations regardless of discipline.
This research compared the regulatory state of the USAF aviation safety program, as of
This research considered regulatory policy, procedures and safety oversight functions applicable to USAF organizations directly responsible for implementing and conducting flying operations. Each gap analysis question was individually evaluated by the research team members by reviewing applicable supporting information from ICAO references and USAF regulations. This process resulted in ratings of compliant, partially compliant or noncompliant.
Once individual analyses were complete, inconsistent ratings were discussed, researched and reevaluated until unanimous resolution was attained. Between the two ICAO gap analysis versions, the overall structure of four major components and 12 elements remained the same and allowed for descriptive statistics and direct high-level comparison of the two USAF program states.
Results
A gap analysis of 101 ICAO (2009) criteria was used to evaluate the USAF aviation safety program, based on regulatory review in
*Component 1: Safety policy and objectives;
*Component 2: Safety risk management;
*Component 3: Safety assurance;
*Component 4: Safety program.
Analysis & Discussion
This research suggests that while USAF is not attempting to specifically adhere to ICAO criteria, many of its current written initiatives demonstrate an estimated 40.6% (n = 41) compliance as of 2012 and a 64.8% (n = 46) compliance as of 2013. Perhaps more important, the noncompliance decreased from 36.6% (n = 37) to 9.9% (n = 7) during the same time frame. During the course of this research, ICAO decreased its total criteria from 101 to 71 items; however, the organization appears to have generally accomplished this task by combining similar criteria, then reframing the questions, or by placing the detailed information as compliance reference material to broad-based questions. USAF's aviation safety program appears to be maturing toward compliance with accepted international safety management principles.
The SMS compliance rates of private industry are not readily known since
Component 1: Safety Policy & Objectives
Individual elements within Component 1 included management commitment and responsibility, safety accountabilities, appointment of key safety personnel, coordination of emergency response planning and SMS documentation. From 2012 to 2013, the USAF aviation safety program strengthened its commitment to safety management principles; several areas of Component 1 were strengthened, but areas of possible future improvement remain. In the 2011 analysis, USAF regulations did not link aviation safety goals to safety performance indicators, targets or action plans; however, the 2013 analysis revealed that the Ar Force Safety Management System (AFSMS) now included those links within its framework and newly established proactive safety programs.
Most areas rated in partial compliance only required relatively small changes to be in full compliance. For example, ICAO SMS standards include multiple requirements to be communicated within organization-wide safety policy. While USAF commanders are required to communicate their safety expectations, no minimum requirements are established. This provides commanders with great latitude to implement safety policy; however, it also allows for safety generalities and oversights to permeate throughout the organization in cases where leaders are not well-versed on current safety methods.
The most notable area of noncompliance was the omission of an organization-wide AFSMS implementation policy, as recommended in ICAO guidance and currently employed by
From 2012 to 2013, the USAF aviation safety program experienced considerable growth by releasing guidance directing an AFSMS. A program that was once based on reactive safety principles now includes proactive safety techniques, similar to those adopted by international civil aviation. Minor regulatory changes would bring the USAF program closer to full SMS compliance; however, an overall implementation strategy should first be addressed.
Component 2: Safety Risk Management
Individual elements within Component 2 included hazard identification and safety risk assessment and mitigation. This component showed the most regulatory improvement over the course of this study, from 52.4% fully or partially compliant in 2012 to 100% in 2013. In 2012, the USAF risk management program was managed by the safety directorates, arguably with more emphasis on ground-based safety as opposed to aviation safety. At the time, USAF aviation safety staff members were formally trained in courses emphasizing incident investigation, rather than proactive hazard and risk mitigation.
In 2013, the USAF risk management program transitioned from one that was somewhat outdated and governed by safety-based regulations to one that includes industry-accepted best practices and is governed by special management regulations. The act of placing the risk management program within the special management regulations clearly placed hazard identification, risk assessment and mitigation as a process fully integrated into all personnel duties.
As opposed to safety personnel assessing hazards and supplementing the process with voluntary methods, the entire process is managed by all personnel and validated by safety specialists. Computer-based risk management courses are required for all personnel, and program managers must attend classroom instruction.
USAF could strengthen this program further by establishing a refresher training frequency or, at minimum, retracting its grandfather clause requiring personnel to take the course only once. The USAF risk management program is almost fully compliant with ICAO SMS standards, only requiring minor regulatory changes to clearly establish applicability to USAF mission-essential task listing creation and subsequent change management.
Component 3: Safety Assurance
Individual elements within Component 3 included safety performance monitoring and measurement, management of change and continuous SMS improvement. The overall section was assessed as 19.2% compliant in 2012 and 44.4% compliant in 2013; however, during the same time frame, items rated as noncompliant dropped from 42.3% to 5.6%.
Within the 2012 assessment, no formal aviation safety guidance was available to establish proactive criteria for safety process change. At the same time, USAF had not established proactive processes to evaluate how well its safety program was working and instead focused on reactive safety incident rates.
The newly established 2013 AFSMS and overhauled risk management program set forth principles to discover aviation safety program performance flaws before necessitating reactive safety. The challenge, however, will be to design an implementation strategy and audit process to allow for the collection and meaningful interpretation of vast amounts of hazard, risk assessment and risk controls data not previously captured to this degree. This facet reemphasizes the importance of a strategic AFSMS implementation plan as mentioned in Component 1.
Several items in the safety performance monitoring and measurement section were rated as partially compliant. Proactive safety monitoring could be bolstered by including Class E event review processes and further refining AFSMS performance indicators. Targets associated with performance indicators should be frequently reviewed to ensure relevant metrics in a dynamic environment. Partially compliant items were also assessed within the change management section. USAF clearly established how change management applied to aircraft and material but did not formally implement a process to manage the hazards and risk inherent in headquarters-level operational aviation planning as it relates to change management.
Component 4: Safety Program
Individual elements within Component 4 included training, education and safety communication. An SMS is substantially different from a traditional aviation safety program, as it is datacentric and relies heavily on an informed safety culture. According to regulatory guidance and a review of all available USAF safety course syllabi, USAF has not established AFSMS education for its leaders and personnel involved with the implementation of the AFSMS, nor is there evidence of organization-wide awareness strategies. To foster AFSMS awareness and integration into daily activities, USAF should consider developing educational programs for all levels of personnel, likely as part of its implementation guidance. As such guidance is established, USAF would also benefit from creating a medium for sharing SMS information from related activities in other branches of the military that choose to adopt a safety management program for their aviation as well as groundbased activities.
Conclusion
USAF is not mandated to employ international civil aviation safety management standards; however, these same standards may be useful in identifying new areas for improvement. The USAF aviation safety program was measured against ICAO SMS criteria in
As of 2012, USAF had not formally adopted safety management principles into its aviation safety program, and this was evident within the first part of this analysis and possibly the historic USAF aviation mishap rates. As of 2013, USAF began its own AFSMS; this effort is greatly reflected within the second analysis and shows a strengthening regulatory commitment to international practices. The introduction of the AFSMS marked a substantial change in USAF aviation safety philosophy. This research made no attempt to evaluate the practical implementation of USAF safety measures, but rather was limited to a regulatory policy and document review.
While this analysis indicated that the 2013 USAF aviation safety program was more compliant with ICAO SMS criteria as compared to its program in 2012, several obstacles must be overcome before this program is realized. The AFSMS program will mature over the coming years, and it may not be feasible for public entities to research the practical implementation of this effort; however, it may be possible to observe long-term USAF aviation mishap safety trends and note 2013 as a changing point in preventable mishaps. If made possible by USAF, subsequent studies could compare mishap rates against actual AFSMS implementation criteria, as assessed by field research, to verify that the initiative enhances the overall safety program, saves lives to a greater extent, and reduces injuries and costs.
IN BRIEF
*A safety management system (SMS) approach to aviation safety is being implemented by member states of the
*A longitudinal gap analysis compared the regulatory states of the
*The results of this research suggested that the 2012 USAF aviation safety program was 40.6% compliant, 22.8% partially compliant and 36.6% noncompliant with ICAO safety management reference criteria, while the 2013 program was 64.8% compliant, 25.3% partially compliant and 9.9% noncompliant.
Historically, aviation safety analyzed past mishaps during a specified period to identify causal factors and anticipate future occurrences within a process commonly termed reactive safety.
Visit www.asse .org/psextra for a comprehensive explanation of the gap analysis criteria for each SMS element.
SMS emphasizes proactive and tive safety processes to identify hazards before reactive safety techniques are necessary.
Disclaimer
The views and conclusions contained in this article are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the
References
Adamshick, M.H. (2007) Leadership and safety climate in high-risk military organizations (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3260384).
Aircraft mishap investigation course. (2013). Retrieved from www.afsec.af.mil/organizations/edu cation/amic.asp
Aviation safety program management course. (2013). Retrieved from www.afsec.af.mil/organiza ti ons/e ducati on/as pm. asp
Bottani, E., Monica, L. & Vignali, G. (2009). Safety management systems: Performance differences between adopters and non-adopters. Safety Science, 47(2), 155162. doi:10.1016/j.ssci.2008.05.001
Gibb, R.W. & Olson, W. (2008). Classification of
Helmreich, R.L.,
ICAO. (2009b). Safety management manual (2nd ed.) (ICAO Order No. 9859).
ICAO. (2013). Safety management manual (3rd ed.) (ICAO Order No. 9859).
Jennings, J. (2008, June). Human factors analysis and classification: Applying the
Kreisher, O. (2006, Dec.). Toward zero mishaps.
Liou, J., Yen, L. & Tzeng, G. (2008). Budding an effective safety management system for airlines.
O'Connor, P. (2008, June). HFACS with an additional layer of granularity: Validity and utility in accident analysis. Aviation, Space and Environmental Medicine, 79(6), 599-606.
Panagopoulos, I. & Bond, S. (2011). Improving flight safety in combat training: A step forward. In 2011
Stolzer, A.J., Halford, C.D. & Goglia, J.J. (2008). Safety management systems in aviation.
Trimble, S. (2010). USAF safety culture slammed.
USAF. (2008). Safety Investigations Reports (AH 91204).
USAF. (2011). The
USAF Chief of Safety. (2009,
USAF Safety Center. (2012). Fiscal year 2006-2011 aircraft mishap statistics [Data file]. Retrieved from ww w. af sc. af.mil/ organiz ati ons/a viati on/ aircraftst a tistics/index.asp
*
Copyright: | (c) 2014 American Society of Safety Engineers |
Wordcount: | 4868 |
Advisor News
Annuity News
Health/Employee Benefits News
Life Insurance News