< Previous8-16 Safety Management Manual (SMM) One SMS across multiple service providers 8.4.7.16 Organizations with multiple service provider certifications may choose to include them all under the scope of one SMS to capitalize on the benefits of SMS and better address interface aspects. The State regulator should consider the following when assessing the SMS of these parent organizations or the implementation of SMS requirements for service providers which are included in the scope of a wider SMS: a) Ensure that SMS monitoring policies and processes are consistently applied throughout the State, in particular where inspectors from different organizations within the regulator are responsible for the oversight and monitoring of different service providers: 1) there is evidence of management commitment for the consistent interpretation of regulations and application of oversight and monitoring; 2) all oversight and monitoring personnel have been provided standardized training; ideally including participants from different disciplines; 3) common policies, procedures, and auditing tools need to be developed and implemented when there are different oversight and monitoring organizations; 4) there is consistent and frequent communication between the responsible inspectors assigned to each service provider; 5) there are mechanisms in place which monitor the degree of standardization of the oversight and monitoring activities. Any issues identified should be addressed; 6) it is recognized that the service provider’s activities may be addressed by the SMS at the corporate (“parent”) level. This might include activities that require SMS and activities that are outside of Annex 19 applicability. 7) the parent organization has documented: i) its policies and procedures for how safety data and safety information is shared, communications are relayed, decisions are made, and resources are allocated across the different activity areas and, where applicable, with different regulatory authorities; ii) the roles and responsibilities associated with its SMS and the accountability framework for the SMS; and iii) the organizational structure and interfaces between different systems and activities in its system description. b) Ensure awareness that parent organizations holding multiple certificates - some of which include certificates from foreign regulators - may elect to implement one SMS across the multiple service providers. 1) Recognize that the scope of an SMS is clearly defined in the system description and details the individual activities. The service provider can demonstrate the compatible between their SMS processes and the corporate SMS. 2) Be aware that this scenario can induce additional challenges when the parent organization holds both domestic and international approvals, such as the acceptance of the SMS by different regulatory authorities. An agreement should be made with the other regulatory Chapter 8. State Safety Management 8-17 authorities on how oversight and monitoring will be shared, delegated or maintained separately (duplicated) where arrangements for SMS acceptance have not yet been established. Integrated management systems 8.4.7.17 Regulator should consider the following when assessing service providers which have integrated their SMS with other management systems: a) drafting a policy that clarifies the scope of their authority (they may not be responsible for oversight of the related management systems); and b) resources necessary to assess and monitor an integrated management system (this could include staff with appropriate expertise, processes, procedures and tools). 8.4.7.18 There are benefits for the service provider to integrating their SMS with other management systems. The integration should be completed to the satisfaction of the CAA, and in a way that the CAA can effectively “see” and monitor the SMS. Guidance for service providers implementing SMS as part of an integrated management system is available in Chapter 9. 8.4.8 Accident investigation 8.4.8.1 The accident investigation authority (AIA) must be functionally independent from any other organization. Independence from the CAA of the State is of particular importance. The interests of the CAA could conflict with the tasks entrusted to the AIA. The rationale for the independence of this function from those of other organizations is that accident causation can be linked to regulatory or SSP-related factors. Also, such independence enhances the viability of the AIA and avoids real or perceived conflicts of interest. 8.4.8.2 The accident investigation process has a pivotal role in the SSP. It enables the State to identify contributing factors and any possible failure within the aviation system, and generate the necessary countermeasures to prevent recurrence. This activity contributes to the continuous improvement of aviation safety by discovering active failures and contributing factors of accidents/incidents and providing reports on any lessons learned from analysis of events. This can support development of corrective actions decisions and corresponding allocation of resources and may identify necessary improvements to the aviation system. Refer to ICAO Annex 13 and related guidance for more information. 8.4.8.3 There are many safety occurrences that do not require an official investigation in accordance with Annex 13. These occurrences and identified hazards may be indicative of systemic problems. These problems can be revealed and remedied by a safety investigation led by the service provider. For information about service provider safety investigations, refer to Chapter 9. 8.4.9 Hazard identification and safety risk assessment General Guidance 8.4.9.1 One of the most important roles of aviation authorities is to identify hazards and emerging trends across the aviation system. This is often achieved by analysing safety data aggregated from multiple sources. The level of complexity and sophistication of a State’s SRM process will vary based on the size, maturity, and complexity of the aviation system in the State. General guidance on the SRM process can be found in Chapter 2. 8.4.9.2 Collection of internal and external safety data and safety information is essential to achieving an effective SSP. Non-complex aviation systems may produce limited data. In this case, collection and exchange of external data should be a priority. External data is often available from other States, such as: investigation reports; annual safety reports (including information and analysis on incidents); safety alerts; safety bulletins; safety studies; 8-18 Safety Management Manual (SMM) iSTARS; etc. At a regional level, ICAO groups (e.g., RASGs, planning and implementation regional groups (PIRGs), etc.) may also a good source of safety information. The State’s safety data collection and processing system (SDCPS) should include procedures for the submission of accident and incident reports to ICAO, which will facilitate the collection and sharing of global safety information. 8.4.9.3 The primary goal of SRM is to identify and control the potential consequences of hazards using the available safety data. The principles for SRM are the same for States and service providers. 8.4.9.4 Service providers have access to their own safety data. States have access to safety data from multiple service providers. Therefore, the State implementing common taxonomies to classify safety data it collects will greatly improve the effectiveness of the State SRM process. This also allows data gathered from multiple sources across different aviation sectors to be analysed more efficiently. The data analysis process inputs and outputs are depicted in Figure 20 below. Figure 20. Data-driven analysis programme 8.4.9.5 Inputs can be received from any part of the aviation system, including: accident investigations; service provider safety investigations; continuing airworthiness reports; medical assessment results, safety risk assessments; audit findings and audit reports; and safety studies and reviews. 8.4.9.6 When necessary, outputs or safety risk controls are applied to eliminate the hazard or reduce the level of safety risk to an acceptable level. A few of the many mitigation options available to the State include: airworthiness directives; providing input to refined oversight and monitoring of the service provider(s); amendments to certification, rulemaking or safety policies; safety promotion programme; facilitation of lessons learned workshops. The chosen action will obviously depend on the severity and type of issue being addressed. Chapter 8. State Safety Management 8-19 Hazard Identification 8.4.9.7 Hazard identification is predicated on collection of representative data. It may be appropriate to combine or aggregate data from multiple sectors to ensure a comprehensive understanding of each hazard. The process depicted in Figure 20 is equally valid for reactive or proactive hazard identification. Analysing the hazards identified during an incident or accident investigations is an example of a reactive methodology. A proactive one might include hazards identified during audits or inspections, or from mandatory reports. It could include being alerted to early signs of safety performance degradation from day-by-day system reliability monitoring. 8.4.9.8 Hazards exist at all levels in the State’s aviation system. Accidents or incidents occur when hazards interact with certain triggering factors. As a result, hazards should be identified before they lead to accidents, incidents or other safety related occurrences. 8.4.9.9 States are encouraged to appoint an individual or team to gather, aggregate and analyse available data. The State safety analyst should analyse data to identify and document potential hazards as well as corresponding effects or consequences. The detail required in the hazard identification process depends on the complexity of the process under consideration. 8.4.9.10 A systematic process should be developed to ensure effective hazard identification. It should include the following elements: a) access to the data sources necessary to support the management of safety risk in the State; b) safety analysis team with appropriate analytical skills and operational experience, and training and experience in a variety of hazard analysis techniques; and c) hazard analysis tool(s), appropriate for the data being collected (or will be collected) and the scope of aviation activities in the State. Hazard Identification Triggers 8.4.9.11 There are many situations where hazard identification should be initiated. Some of the major ones are: a) System design: Hazard identification starts before the beginning of operations with a detailed description of the particular aviation system and its environment. The safety analysis team identifies the various potential hazards associated with the system as well as impacts to other interfacing systems. b) System change: Hazard identification starts before introducing a change in the system (operational or organizational) and includes a detailed description of the particular change to the aviation system. The safety analysis team then identifies the potential hazards associated with the proposed change as well as impacts to other interfacing systems. c) On demand and continuous monitoring: Hazard identification is applied to existing systems in operation. Data monitoring is used to detect changes in the hazard situation. For example, hazard manifestation may be more frequent or more severe than expected, or the agreed mitigation strategies are less effective than expected. Continuous monitoring and analysis can be established with notification thresholds based on a set of critical items of interest. 8-20 Safety Management Manual (SMM) Safety Risk Assessment 8.4.9.12 General guidance on safety risk assessment can be found in Chapter 2. It should be noted that safety risk can be viewed and controlled across an aviation sector or a region. 8.4.9.13 There are many different tools to analyse data and use different safety risk modelling approaches. When selecting or developing safety risk assessment, States should ensure that the process works well for their environment. 8.4.10 Management of safety risks 8.4.10.1 Guidance on resolution of safety issues (CE-8) can be found in Doc 9734, Part A. 8.4.10.2 The objective of the management of safety risks is to ensure safety risks are controlled and an ALoSP is achieved. The appropriate State aviation authority develops, documents, and recommends appropriate safety risk mitigation or safety risk control strategies. Examples include: direct intervention with a service provider, implementing additional policies or regulations; issuing operational directives or influencing through safety promotional activities. 8.4.10.3 An evaluation of each proposed safety risk control should be performed as a next step. Ideal safety risk control candidates are cost effective, easy to perform, quickly implemented, effective, and do not introduce unintended consequences. Since most situations do not meet these ideals, candidate safety risk controls should be evaluated and selected based on balancing the attributes of effectiveness, cost, timeliness of implementation, and complexity. Once safety risk controls have been selected and implemented, they should be monitored and validated to ensure the intended goals have been achieved. 8.4.10.4 Many of the safety risk controls require action by service provider(s). States should direct the service provider(s) to accomplish effective implementation. States may need to monitor the effectiveness of the safety risk controls and their impact on service provider’s, and collectively, State’s safety performance. Safety risk mitigation approaches are outlined in Chapter 2. 8.5 COMPONENT 3: STATE SAFETY ASSURANCE 8.5.1 State safety assurance activities aim to assure the State that their functions are achieving their intended safety objectives and targets. Service providers are required to implement a safety assurance process as part of their SMS. The SMS assurance capability assures each service provider that their safety processes are functioning effectively, and they are on target to achieve their safety objectives. Similarly, State safety assurance, as part of their SSP, provide the State with assurance that its safety processes are functioning effectively and the State is on target to achieve its safety objectives via the collective efforts of the State’s aviation industry. 8.5.2 Surveillance activities and safety data/information collection, analysis, sharing and exchange mechanisms ensure that regulatory safety risk controls are appropriately integrated into a service provider’s SMS. This provides confidence that the system is being practiced as designed, and the regulatory controls are having the intended effect on SRM. States can collect aviation safety data/information from many sources, including through surveillance processes and safety reporting programmes. The data should be analysed at various levels, and the conclusions drawn from the analysis should be used as the basis for well-informed safety decision-making with regard to the surveillance activities and safety in the State’s aviation system. Chapter 8. State Safety Management 8-21 8.5.3 Surveillance obligations 8.5.3.1 Guidance on Surveillance obligations (CE-7) related to compliance monitoring can be found in Doc 9734, Part A. Prioritizing surveillance activities 8.5.3.2 A safety risk-based surveillance (SRBS) approach enables prioritization and allocation of State’s safety management resources commensurate with the safety risk profile of each sector or individual service provider. States gain experience and familiarity with each service provider by monitoring the steadily developing maturity of their safety assurance process, and in particular, their management of safety performance. Over time the State will accumulate a clear picture of the service provider’s safety abilities, particularly their management of safety risk. The State may choose to amend the scope and/or frequency of surveillance as their confidence and evidence of the service provider’s safety capability develops. 8.5.3.3 SRBS is most appropriate for organizations with a mature SMS. SRBS may also be applicable to organizations where SMS has not yet been implemented. The foundation of effective SRBS is reliable enough and meaningful data. Without reliable and meaningful data, it is difficult to defend adjustments to the surveillance scope or frequency. 8.5.3.4 States should develop or reinforce their data management capabilities to ensure they have reliable and comprehensive data upon which to base their (data-driven) decisions. Individual sector safety risk analyses may also allow the State to evaluate common safety risks that affect multiple service providers with similar types of operations (for example, short-haul airlines). This facilitates safety risk ranking among service providers within a specific aviation sector or across sectors, and supports the allocation of surveillance resources to sectors or activities with greatest safety effect. 8.5.3.5 Analyses at the sector level allows the State to view the aviation system in context: how the parts contribute to the whole. It empowers the State to identify which sector(s) will benefit from higher levels of support or intervention, and which sectors are the best candidates for a more collaborative approach. This gives the State assurance that regulation across the aviation system is commensurate and targeted at the areas with greatest need. It is easier to identify where changes to specific regulations are needed to achieve maximum regulatory effectiveness with minimal interference. 8.5.3.6 SRBS comes at a cost. It requires ongoing interactions between the State and the aviation community beyond compliance-based audits and inspections. An SRBS approach uses the safety risk profile of the service provider to adapt its surveillance activities. The output of internal reviews, analysis and decision-making within the service provider’s system becomes a targeted action plan addressing key safety risks and the mitigations that effectively address them. The analysis from both the State and the service provider define the priority areas of safety concern, and outline the most effective means of addressing them. 8.5.3.7 Importantly, safety risk-based surveillance may not necessarily reduce the amount of surveillance conducted or the resources, the quality of the surveillance and the quality of the interaction between the regulator and the service provider will however improve greatly. 8-22 Safety Management Manual (SMM) Service provider organizational safety risk profiles 8.5.3.8 States may wish to develop organizational safety risk profiles that are consistent across each aviation sector to support the process of modifying the scope and frequency of their surveillance activities. Such tools should aim to capture and aggregate information that should already be available for service providers and may include factors such as: a) the financial health of the organization; b) number of years in operation; c) turnover rate of the key personnel such as the accountable executive and safety manager; d) competence and performance of the accountable executive; e) competence and performance of the safety manager; (for more information about accountable executive or safety manager competence, see Chapter 9) f) results of previous audits; g) timely and effective resolution of previous findings; h) measures of relative level of activity (exposure to safety risk); i) indicators of the relative scope and complexity of the activities being performed; j) maturity of the hazard identification and safety risk assessment process; and k) Measures of safety performance from State safety data analysis and performance monitoring activities. 8.5.3.9 An example of a process that can be used to modify the scope or frequency of the surveillance of a service provider is shown in Figure 21. Figure 21. Safety risk-based surveillance concept Chapter 8. State Safety Management 8-23 8.5.4 Monitoring a service provider’s safety performance The State should periodically review each service provider’s SPIs and SPTs. The review should take into consideration the performance and effectiveness of each SPI and SPT. The review may show the need to make adjustments to support the continuous safety improvement. 8.5.5 State safety performance 8.5.5.1 For general information on safety performance management, refer to Chapter 4. Acceptable level of safety performance 8.5.5.2 States are required to establish the acceptable level of safety performance (ALoSP) to be achieved through their SSP. This can be achieved through: a) implementation and maintenance of the SSP and; b) implementation and maintenance of SPIs and SPTs showing that safety is being effectively managed. 8.5.5.3 The ALoSP expresses the safety levels the State expects of their aviation system, including the targets that each sector needs to achieve and maintain in relation to safety, as well as measures to determine the effectiveness of their own activities and functions that impact safety. ALoSP, then, is a reflection of what the State considers important and is agreed on by the State level aviation stakeholders. ALoSP should not be developed in isolation. Rather, it should be defined having regard for higher level strategic guidance (from the GASP, Regional Plans, etc.) and the safety objectives established in the SSP. Establishing the ALoSP 8.5.5.4 The responsibility for establishing the ALoSP rests with the State’s aviation authorities, and will be expressed through the set of SPIs for the State, sectors and service providers under their authority. The goal is to maintain or continuously improve the safety performance of the entire aviation system measurement process outlined in Chapter 4. This enables the State to understand how it is doing with regards to safety and to act to impact the situation when necessary. The acceptance of service provider’s SPIs and targets is part of this process. 8.5.5.5 ALoSP represents the agreement between all State aviation authorities of the expected level of safety performance that its aviation system should deliver and demonstrates to internal and external stakeholders how the State is managing aviation safety. It includes, but is not limited to, the expectations for safety performance for each sector and service provider under the State’s authority. Establishing an ALoSP does not replace or supersede a State’s obligation to abide by the Convention on International Civil Aviation, including the implementation of all applicable SARPs. 8.5.5.6 Figure 22 outlines the concept of ALoSP based on SPIs and SPTs. Further information on safety objectives, SPIs and SPTs can be found in Chapter 4 and the subsequent paragraphs. 8-24 Safety Management Manual (SMM) Figure 22. Acceptable level of safety performance (ALoSP) Safety performance indicators and safety performance targets 8.5.5.7 Meaningful SPIs should reflect the specific operational environment and serve to highlight conditions that can be used to identify how safety risks are being controlled. The State monitoring and measurement strategy should include a set of SPIs that encompass all areas of the aviation system for which the State is responsible. It should reflect both outcomes (e.g. accidents, incidents, regulatory violations) as well as functions and activities (operations where the safety risk mitigations in place performed as expected). This combination allows safety performance to be evaluated by not only what does not work (i.e. outcomes), but also with considerations of what does work (i.e. activities where safety risk mitigations produced the expected results). In practical terms, this approach encompasses the consideration of SPIs reflecting two distinct types of safety risks: a) Operational safety risks (depicted on the left-hand side of the diagram) focuses on conditions that could lead to an unwanted outcome. These are the conditions associated with accidents, incidents, failures and defects. Operational safety risk is essentially a by-product of the delivery of services. For this reason, SPIs focused on operational safety risk will be mostly linked – indirectly – to service providers’ SMS. These SPIs reflect mainly operational safety issues identified by the SRM process of service providers. The State’s SRM process may also be used as an input, reflecting operational safety issues across the State aviation system derived from aggregation of service provider operational safety risk SPIs. There will be frequently a one-to-many relationship between an operational safety issue and related SPIs. That is, one operational safety issue may be indicated by several SPIs. Chapter 8. State Safety Management 8-25 b) Process implementation safety risks (depicted on the right-hand side of the diagram) focuses on the means and resources necessary for operational safety risk to be managed. Management of safety risk from a process implementation perspective starts with the evaluation of ICAO SARPs implementation status (safety-related national laws and regulations), the implementation of SMS processes within the industry, and the implementation of SSP at the State level (which includes effective oversight and monitoring of the industry). If improvements in any of the above are necessary, the activities to achieve them should be planned, implemented and monitored, and adequate resources should be allocated for these activities. SPIs are then developed that allow tracking of the planning, implementation and/or effectiveness of the changes. SPIs focused on “process implementation safety risk” provide the State an alternative means other than strict compliance to monitor the adequacy of SMS institutional arrangement and implementation of SRM/safety assurance processes by service providers. These SPIs may also be established with reference to needed improvements, as shown by USOAP analyses and SSP continuous improvement activities. The results of USOAP audits, aggregation of SMS evaluations, and SSP continuous improvement information determine potential areas for improvement. These should be prioritized according to greatest benefit. This will contribute to improvement in the State aviation system’s safety performance. These SPIs should be distinct from operational safety risk SPIs. 8.5.5.8 SPIs for both the operational and process implementation safety risks become a key part of the State’s safety assurance process. The aggregation of operational safety risk SPIs and process implementation safety risk SPIs broadens the feedback source for establishing the State ALoSP. Periodic review of safety performance indicators 8.5.5.9 A periodic review is essential once the State SPIs are established. Initially the identification of the top safety risks is an activity aided by analysis based on historical data. However, the aviation system is dynamic and constantly changing. New safety issues may arise, processes within the State may change, and so on. Periodic review of State operational safety issues and processes supports the update and refinement of the State safety objectives and consequently the SPIs and SPTs. Periodic review of ALoSP 8.5.5.10 The senior management team responsible for the original ALoSP agreement should determine the continued appropriateness of the ALoSP. The periodic review of the ALoSP should focus on: a) identifying critical safety issues within aviation sectors, ensuring inclusion of SPIs that allow safety performance management in these areas; b) identifying SPTs that define the safety performance level to be maintained or the desired improvement to be achieved for relevant SPI in each sector, with a view to enhance safety performance management throughout the entire aviation system of the State; c) identifying triggers (if appropriate) when an SPI reaches a point that requires some action; and d) reviewing SPIs to determine whether modifications or additions to existing SPIs, SPTs and triggers (if appropriate) are needed to achieve the agreed ALoSP. 8.5.5.11 A result of the periodic review of the State’s top risks is a better understanding of the nature of each operational safety issue in as much detail as the data allows. The State should consider its hazards and their potential consequences at all levels of the State aviation system. It should also analyse how State processes (licensing, certification, approval, surveillance and so forth) contribute to SRM. Each operational safety risk is evaluated to identify the safety risk mitigations required. These actions are monitored through SPIs that measure their effectiveness. Next >