It is often mentioned that air traffic is expected to double in the next 15 years. That growth, combined with rapid technological changes, the increasing complexity of the aviation system and innovative approaches proposed by the aviation industry, presents challenges to the traditional regulatory approach, which is mostly based on the establishment of prescriptive requirements.
In the fourth edition of the Safety Management Manual (SMM), Chapter 4, Safety Performance Management, outlines the importance of developing safety objectives that stem from the identification and understanding of the top risks being faced by the State or service provider. Safety performance indicators (SPIs) and safety performance targets (SPTs) are then derived from the safety objectives and are the main tools for monitoring and measuring safety performance.
For a more accurate and useful indication of safety performance, lagging SPIs, that measure events that have already occurred, should be combined with leading SPIs, that focus on processes and inputs that are being implemented to improve or maintain safety. The figure below is from the 4th edition of the SMM and shows the links between lagging and leading indicators.
These tools should help us to measure if we are achieving our safety objectives at the service provider, State, regional and global levels. ICAO has put in place an Indicators Catalogue to support the sharing of SPIs being used by States and industry and to support the standardization of the indicators through the use of the form provided. In addition, ICAO has developed the Safety Information Monitoring System (SIMS), a web-based safety data and information system comprised of different applications which generate indicators, to assist States in the processing and analysis of safety data to monitor safety performance. States interested in joining SIMS are invited to express their interest to their accredited Regional Office. Interested industry stakeholders can become partners to SIMS through the ICAO project-based partnership programme.
These activities related to safety performance management should be supported by internal and external audit processes that monitor compliance with safety regulations, the foundation upon which safety management is built, but that also assesses the effectiveness of individual processes and activities as well as the SSP or SMS overall. The tools used to conduct audits typically provide a checklist to be used for the review of each requirement from a binary perspective – Yes/No, Satisfactory/Not Satisfactory. An example question from the Universal Safety Oversight Audit Programme (USOAP) Protocol Questions (PQs) is shown below.
To assess whether something is effective, however, requires a performance-based approach which actually reviews whether the activity or process is achieving its intended result. An added benefit of this type of review is that it allows each organization the flexibility needed to implement SSP or SMS in a way that works for their own organization. In addition, it is important to determine if the various processes and activities are appropriately linked (e.g. link between the safety risk management process and the monitoring of safety performance indicators) to enable the achievement of the overall safety objectives of the organization. Although it is usually much easier to implement a prescriptive requirement as well as to audit its implementation, we will not reap the benefits of safety management if treat it as another set of prescriptive requirements.
One approach initially developed by the Safety Management International Collaboration Group (SM ICG) in the SMS Evaluation Tool (Version 1.0 – 1 April 2012) is to provide for the evaluation of the maturity of the SMS which provides four levels to be considered by the auditor/assessor: Present (P), Suitable (S), Operating (O) and Effective (E). The Management System Assessment Tool published by EASA in September 2017 has followed this model and includes “word pictures” to help the inspector determine the correct level.
ICAO is currently working on the development of a maturity evaluation system, and the technical tools to be utilized for evaluating the level of SSPs implementation, as part of the Universal Safety Oversight Audit Programme (USOAP).
Once the evaluation system and tools are developed, they will be made available to States and assessors will be provided appropriate training to promote standardization in how the tool is used.
ICAO has established the Safety Management Implementation (SMI) website to complement the 4th edition of the SMM and serve as a repository for practical examples and tools to support our diverse community in the implementation of SSP and SMS. Users who previously found the Appendices in the 3rd edition to be very useful will be pleased to learn that most of that content has been updated and posted on the SMI website. Additional practical examples and tools will be collected, validated and posted on the SMI website on an ongoing basis. I invite you to visit the website and consider submitting a practical example or tool showing how your organization is implementing safety management to share it with the rest of the aviation community. There is a form available for this purpose once you select “Start” and launch the website.
About the author
Ms. Elizabeth Gnehm is responsible for the Safety Management Programme at ICAO . Her duties include serving as the Secretary of the Safety Management Panel, which has developed proposals for the first and second editions of Annex 19 – Safety Management, and leading a team to support the implementation of State safety programmes (SSP) and safety management systems (SMS) worldwide. Ms. Gnehm spent nine years working in the ICAO Universal Safety Oversight Audit Programme (USOAP) as Technical Coordinator, Team Leader and Airworthiness Auditor and has participated in over 50 ICAO USOAP missions. Prior to joining ICAO in 2002, she worked for the US Federal Aviation Administration and the Boeing Company under various airplane programmes. She holds a BS and a MS in Aerospace Engineering from Texas A&M University.