What are ICAO State Safety Programme Implementation Assessments?

8,441
Overview of the development, evolution, and maturation

The State Safety Programme Implementation Assessment (SSPIA) is a performance-based activity that falls under the framework of the Universal Safety Oversight Audit Programme (USOAP) Continuous Monitoring Approach (CMA). Through the programme, ICAO assesses the level of maturity of a State Safety Programme (SSP) by conducting a systematic and objective review of the State’s implementation and maintenance of its SSP.

This activity follows six years of continuous efforts in evolving the approach used to better suit the nature of SSP implementation. Since 2015, while many States were progressing in SSP implementation, ICAO has been developing and maturing its methodology, processes and tools to conduct SSPIAs. To support these efforts, in 2018, the Monitoring and Oversight (MO) office recruited a Standards and Procedure Officer (SPO) to be responsible for the area of Safety Management within the USOAP CMA.


 

SSPIA roll-out and major milestones

Phase 0 of SSPIAs:

Initially, ICAO had planned to “audit” SSP implementation beginning in 2015, using a set of protocol questions (referred to as “SSP PQs”) that were introduced for this purpose in the 2014 edition of the PQs. In early 2015, it was recognized, however, that the “satisfactory vs. non-satisfactory” approach used in the legacy USOAP CMA activities (such as audits) was not adapted for evaluating States’ progress in implementing SSPs, given that implementation was “performance-based” in nature.

Considering the best practices in assessing safety management implementation, including the experience gathered in some States and regions when assessing the implementation of Safety Management Systems (SMS), it was decided to adjust the approach and conduct assessments, rather than audits, of SSP implementation with a few States, on a voluntary and confidential basis. This approach would benefit States (by providing them with feedback on their progress and achievements as well as opportunities for further enhancement of their SSP), USOAP (by progressively building a specific methodology for this new type of activity), and ICAO (by providing additional feedback on the challenges faced by States in SSP implementation).

During Phase O, five States received an SSPIA, in which the approach was changed from an audit to an “assessment”, using the SSP PQs contained in the 2014 edition of the PQs. The assessments reviewed the States’ progress and highlighted achievements and opportunities for enhancement. The approach was purely qualitative. No score was given to the States following the completion of the activity and the activity had no impact on the States’ “Effective Implementation” (EI) values.

At the same time, MO started to progressively develop new, more relevant “SSP-related PQs”. In 2018, MO provided feedback to the ICAO Secretariat, the Safety Management Panel and ICAO’s governing bodies on the performance of these activities, as well as the observed challenges faced by States in SSP implementation. Based on the experience accumulated and lessons learned from these missions, it was decided to amend the SSP PQs in a manner that would better suit this new type of activity, including the performance-based environment in which SSP is implemented.


The development of amended SSP-related PQs:

In 2018, MO started developing an amended set of SSP PQs, which were more performance-based and did not call for a “satisfactory vs. non-satisfactory assessment”. Each PQ was linked to one of the four SSP components (i.e. State Safety Policy and Objectives, State Safety Risk Management, State Safety Assurance, and State Safety Promotion), rather than the eight Critical Elements associated with the legacy USOAP CMA activities.

The “technical areas” for SSPIAs became distinct from those in USOAP legacy activities. In the 2018 edition of the SSP-related PQs, the LEG and ORG areas were removed and two new areas were introduced, namely “General Aspects” (SSP.GEN) and “Safety Data Analysis” (SSP.SDA), in addition to the sector-specific areas (SSP.PEL, SSP.OPS, SSP.AIR, SSP.ANS, SSP.AGA and SSP.AIG).


Phase 1 of SSPIAs:

In 2018, Phase 1 of the SSPIAs was officially launched under the USOAP framework, in which the SSPIAs were still conducted on a voluntary basis but were no longer confidential. Under Phase 1, the SSPIA report focused primarily on two aspects: the State’s achievements (which were shared with all States following completion of the SSPIA process) and “Opportunities for Enhancement” (which were only shared with the assessed State and highlighted aspects in which the State could make further progress).

Three States (Finland, Spain and the United Arab Emirates) received an SSPIA under Phase 1. The “achievements” part of their SSPIA reports were posted on the USOAP Online Framework (OLF) to inform all States of the outcomes of the SSPIAs and enable them to further explore any good practices observed in the assessed States.


The development of maturity levels assessment tool:

In 2020, ICAO established an SSPIA group of experts, comprised of eight experts from seven States (Australia, Canada, Finland, Singapore, the United Arab Emirates, the United States and Spain) and one regional safety oversight organization (European Union Aviation Safety Agency or EASA). They support MO in finalizing the SSPIA maturity level assessment tool and its related guidance material as well as in identifying additional areas of improvements for the proposed tool and associated guidance material. The outcome of this work was shared with the Safety Management Panel members and internally within ICAO, and a very positive feedback was received, along with proposed opportunities for enhancement. Upon reviewing the comments received, the maturity levels matrices were amended and the SSPIA assessment tool was finalized and posted on the OLF in December 2020.

As part of the assessment tool, five maturity levels were determined and criteria were developed for levels 2 and 3 for each PQ. The five determining maturity levels are:

  • 0: Not present and not planned;
  • 1: Not present but being worked on;
  • 2: Present;
  • 3: Present and effective; and
  • 4: Present and effective for years and in continuous improvement.

Phase 2 of SSPIAs:

Following the inception of Phase 2 which will no longer be voluntary nor confidential, the SSPIA becomes a more integral and coherent part of USOAP and will provide ICAO with a picture of how States are progressing in implementing and maintaining their SSPs. This phase, which would be quantitative, will be reflected in terms of maturity level for each assessed PQ by utilizing the newly developed assessment tool.

In 2021, MO reported to ICAO’s governing bodies on its preparedness to deploy Phase 2 of the SSPIAs, with its first mission to be launched once health, safety and travel restrictions are lifted.


Envisaged future steps:
Phase 3 of the SSPIAs:

In accordance with the concept of continuous improvement, MO will revisit the methodology, processes and tools of Phase 2 upon completion of the current three-year phase and will take action, as necessary, to refine them based on lessons learned and new development that will need to be reflected in the assessments.

What does MO do to support States’ preparation and readiness to undergo an SSPIA?

In support of States’ preparation for an SSPIA, MO has initiated two main measures:

  • SSPIA Workshop – Similar to the USOAP CMA workshops, the SSPIA workshop aims to provide valuable information to States on how the SSPIA is conducted, including its methodologies, processes and tools.
  • SSPIA Q&A Sheet – The Q&A sheet aims to provide brief answers to the most common queries that MO receives from States and this will be posted on the OLF shortly.

About the author

Ariel Weiss serves as a Standards and Procedure Officer – Safety Management (SPO-SM) in the Monitoring and Oversight office (MO). As SPO-SM, he leads the State Safety Programme Implementation Assessment (SSPIA) team under the Universal Safety Oversight Audit Programme – Continuous Monitoring Approach (USOAP-CMA).