< Previous9-14 Safety Management Manual (SMM) Figure 25. Safety investigation decision process Chapter 9. Safety Management Systems (SMS) 9-15 Assigning an investigator 9.4.5.7 If an investigation is to commence, the first action will be to appoint an investigator or where the resources are available, an investigation team with the required skills and expertise. The size of the team and the expertise profile of its members depend on the nature and severity of the occurrence being investigated. The investigating team may require the assistance of other specialists. Often, a single person is assigned to carry out an internal investigation, with support from operations and safety office experts. 9.4.5.8 Service provider safety investigators are ideally organizationally independent from the area associated with the occurrence or identified hazard. Better results will be obtained if the investigator(s) are knowledgeable (trained) and skilled (experienced) in service provider safety investigations. The investigators would ideally be chosen for the role because of their knowledge, skills and character traits, which should include: integrity, objectivity, logical thinking, pragmatisms and lateral thinking. The investigation process 9.4.5.9 The investigation should identify what happened and why it happened and this may require root cause analysis to be applied as part of the investigation. Ideally, the people involved in the event should be interviewed as soon as possible after the event. The investigation should include: a) establishing timelines of key events including and the actions of the people involved; b) review of any policies and procedures related to the activities; c) review of any decisions made related to the event; d) identifying any risk controls that were in place that should have prevented the event occurring; and e) reviewing safety data for any previous or similar events. 9.4.5.10 The safety investigation should focus on the identified hazards and safety risks and opportunities for improvement, not on blame or punishment. The way the investigation is conducted, and most importantly, how the report is written, will influence the likely safety impact, the future safety culture of the organization, and the effectiveness of future safety initiatives. 9.4.5.11 The investigation should conclude with clearly defined findings and recommendations that eliminate or mitigate safety deficiencies. 9.4.6 Safety risk assessment and mitigation 9.4.6.1 The service provider needs to develop a safety risk assessment model and procedures which will allow a consistent and systematic approach for the assessment of safety risks. This should include a method that will help determine what safety risks are acceptable or unacceptable and to prioritize actions. 9.4.6.2 The SRM tools used may need to be reviewed and customized periodically to ensure they are suitable for the service provider’s operating environment. The service provider may find more sophisticated approaches that better reflect the needs of their operation as their SMS matures. The service provider and CAA should agree on a methodology. 9.4.6.3 More sophisticated approaches to safety risk classification are available. These may be more suitable if the service provider is experienced with safety management or operating in a high-risk environment. 9-16 Safety Management Manual (SMM) 9.4.6.4 The safety risk assessment process should use whatever safety data and safety information is available. Once safety risks have been assessed, the service provider will engage in a data-driven decision-making process to determine what safety risk controls are needed. 9.4.6.5 Safety risk assessments sometimes have to use qualitative information (expert judgement) rather than quantitative data due to unavailability of data. Using the safety risk matrix allows the user to express the safety risk(s) associated with the identified hazard in a quantitative format. This enables direct magnitude comparison between identified safety risks. A qualitative safety risk assessment criterion such as “likely to occur” or “improbable” may be assigned to each identified safety risk where quantitative data is not available. 9.4.6.6 For service providers that have operations in multiple locations with specific operating environments, it may be more effective to establish local safety committees to conduct safety risk assessments and safety risk control identification. Advice is often sought from a specialist in the operational area (internal or external to the service provider). Final decisions or control acceptance may be required from higher authorities so that the appropriate resources are provided. 9.4.6.7 How service providers go about prioritizing their safety risk assessments and adopting safety risk controls is their decision. As a guide, the service provider should find the prioritization process: a) assesses and controls highest safety risk; b) allocates resources to highest safety risks; c) effectively maintains or improves safety; d) achieves the stated and agreed safety objectives and SPTs; and e) satisfies the requirements of the State's regulations with regards to control of safety risks. 9.4.6.8 After safety risks have been assessed, appropriate safety risk controls can be implemented. It is important to involve the “end users” and subject matter experts in determining appropriate safety risk controls. Ensuring the right people are involved will maximize the practicality of safety risk chosen mitigations. A determination of any unintended consequences, particularly the introduction of new hazards, should be made prior to the implementation of any safety risk controls. 9.4.6.9 Once the safety risk control has been agreed and implemented, the safety performance should be monitored to assure the effectiveness of the safety risk control. This is necessary to verify the integrity, efficiency and effectiveness of the new safety risk controls under operational conditions. 9.4.6.10 The SRM outputs should be documented. This should include the hazard and any consequences, the safety risk assessment and any safety risk control actions taken. These are often captured in a register so they can be tracked and monitored. This SRM documentation becomes a historical source of organizational safety knowledge which can be used as reference when making safety decisions and for safety information exchange. This safety knowledge provides material for safety trend analyses and safety training and communication. It is also useful for internal audits to assess whether safety risk controls and actions have been implemented and are effective. Chapter 9. Safety Management Systems (SMS) 9-17 9.5 COMPONENT 3: SAFETY ASSURANCE 9.5.1 Annex 19, Appendix 2, 3.1.1 requires that service providers develop and maintain the means to verify the safety performance of the organization and to validate the effectiveness of safety risk controls. The safety assurance component of the service provider’s SMS provides these capabilities. 9.5.2 Safety assurance consists of processes and activities undertaken to determine whether the SMS is operating according to expectations and requirements. This involves continuously monitoring its processes as well as its operating environment to detect changes or deviations that may introduce emerging safety risks or the degradation of existing safety risk controls. Such changes or deviations may then be addressed through the SRM process. 9.5.3 Safety assurance activities should include the development and implementation of actions taken in response to any identified issues having a potential safety impact. These actions continuously improvement of the performance of the service provider’s SMS. 9.5.4 Safety performance monitoring and measurement To verify the safety performance and validate the effectiveness of safety risk controls requires the use of a combination of internal audits and the establishment and monitoring of SPIs. Assessing the effectiveness of the safety risk controls is important as their application does not always achieve the results intended. This will help identify whether the right safety risk control was selected and may result in the application of a different safety risk control strategy. Internal audit 9.5.4.1 Internal audits are performed to assess the effectiveness of the SMS and identify areas for potential improvement. Most aviation safety regulations are generic safety risk controls that have been established by the State. Ensuring compliance with the regulations through the internal audit is a principle aspect of safety assurance. 9.5.4.2 It is also necessary to ensure that any safety risk controls are effectively implemented and monitored. The causes and contributing factors should be investigated and analysed where non-conformances and other issues are identified. The main focus of the internal audit is on the policies, processes and procedures that provide the safety risk controls. 9.5.4.3 Internal audits are most effective when conducted by persons or departments independent of the functions being audited. Such audits should provide the accountable executive and senior management with feedback on the status of: a) compliance with regulations; b) compliance with policies, processes and procedures; c) the effectiveness of safety risk controls; d) the effectiveness of corrective actions; and e) the effectiveness of the SMS. 9.5.4.4 Some organizations cannot ensure appropriate independence of an internal audit, in such cases, the service provider should consider engaging external auditors (e.g. independent auditors or auditors from another organization). 9-18 Safety Management Manual (SMM) 9.5.4.5 Planning of internal audits should take into account the safety criticality of the processes, the results of previous audits and assessments (from all sources), and the implemented safety risk controls. Internal audits should identify non-compliance with regulations and policies, processes and procedures. They should also identify system deficiencies, lack of effectiveness of safety risk controls and opportunities for improvement. 9.5.4.6 Assessing for compliance and effectiveness are both essential to achieving safety performance. The internal audit process can be used to determine both compliance and effectiveness. The following questions can be asked to assess compliance and effectiveness of each process or procedure: a) Determining compliance 1) Does the required process or procedure exist? 2) Is the process or procedure documented (inputs, activities, interfaces and outputs defined)? 3) Does the process or procedure meet requirements (criteria)? 4) Is the process or procedure being used? 5) Are all affected personnel following the process or procedure consistently? 6) Are the defined outputs being produced? 7) Has a process or procedure change been documented and implemented? b) Assessing effectiveness 1) Do users understand the process or procedure? 2) Is the purpose of the process or procedure being achieved consistently? 3) Are the results of the process or procedure what the “customer” asked for? 4) Is the process or procedure regularly reviewed? 5) Is a safety risk assessment conducted when there are changes to the process or procedure? 6) Have process or procedure improvements resulted in the expected benefits? 9.5.4.7 In addition, internal audits should monitor progress in closing previously identified non-compliances. These should have been addressed through root cause analysis and the development and implementation of corrective and preventive action plans. The results from analysis of cause(s) and contributing factors for any non-compliance should feed into the service provider’s SRM processes. 9.5.4.8 The results of the internal audit process become one of the various inputs to the SRM and safety assurance functions. Internal audits inform the service provider’s management of the level of compliance within the organization, the degree to which safety risk controls are effective and where corrective or preventive action is required. 9.5.4.9 CAAs may provide additional feedback on the status of compliance with regulations, and the effectiveness of the SMS and industry associations or other third parties selected by the service provider to audit their organization and processes. Results of such second and third-party audits are inputs to the safety assurance Chapter 9. Safety Management Systems (SMS) 9-19 function, providing the service provider with indications of the effectiveness of their internal audit processes and opportunities to improve their SMS. Safety performance monitoring 9.5.4.10 Safety performance monitoring is conducted through the collection of safety data and safety information from a variety of sources typically available to an organization. Data availability to support informed decision-making is one of the most important aspects of the SMS. Using this data for safety performance monitoring and measurement are essential activities that generate the information necessary for safety risk decision-making. 9.5.4.11 Safety performance monitoring and measurement should be conducted observing some basic principles. The safety performance achieved is an indication of organizational behaviour and is also a measure of the effectiveness of the SMS. This requires the organization to define: a) safety objectives, which should be established first to reflect the strategic achievements or desired outcomes related to safety concerns specific to the organization’s operational context; b) SPIs, which are tactical parameters related to the safety objectives and therefore are the reference for data collection; and c) SPTs, which are also tactical parameters used to monitor progress towards the achievement of the safety objectives. 9.5.4.12 A more complete and realistic picture of the service provider’s safety performance will be achieved if SPIs encompass a wide spectrum of indicators. This should include: a) low probability/high severity events (e.g. accidents and serious incidents); b) high probability/low severity events (e.g. uneventful operational events, non-conformance reports, deviations etc.): and c) process performance (e.g. training, system improvements and report processing). 9.5.4.13 SPIs are used to measure operational safety performance of the service provider and the performance of their SMS. SPIs rely on the monitoring of data and information from various sources including the safety reporting system. They should be specific to the individual service provider and be linked to the safety objectives already established. 9.5.4.14 When establishing SPIs service providers should consider: a) Measuring the right things: Determine the best SPIs that will show the organization is on track to achieving its safety objectives. Also consider what are the biggest safety issues and safety risks faced by the organization, and identify SPIs which will show effective control of these. b) Availability of data: Is there data available which aligns with what the organization wants to measure? If there isn’t, there may be a need establish additional data collection sources. For small organizations with limited amounts of data, the pooling of data sets may also help to identify trends. This may be supported by industry associations who can collate safety data from multiple organizations. c) Reliability of the data: Data may be unreliable either because of its subjectivity or because it is incomplete. 9-20 Safety Management Manual (SMM) d) Common industry SPIs: It may be useful to agree on common SPIs with similar organizations so that comparisons can be made between organizations. The regulator or industry associations may enable these. 9.5.4.15 Once SPIs have been established the service provider should consider whether it appropriate to identify SPTs and alert levels. SPTs are useful in driving safety improvements but, implemented poorly, they have been known to lead to undesirable behaviours – that is, individuals and departments becoming too focused on achieving the target and perhaps losing sight of what the target was intended to achieve – rather than an improvement in organizational safety performance. In such cases it may be more appropriate to monitor the SPI for trends. 9.5.4.16 The following activities that can provide sources to monitor and measure safety performance: a) Safety studies are analyses to gain a deeper understanding of safety issues or better understand a trend in safety performance. b) Safety data analysis uses the safety reporting data to uncover common issues or trends that might warrant further investigation. c) Safety surveys examine procedures or processes related to a specific operation. Safety surveys may involve the use of checklists, questionnaires and informal confidential interviews. Safety surveys generally provide qualitative information. This may require validation via data collection to determine if corrective action is required. Nonetheless, surveys may provide an inexpensive and valuable source of safety information. d) Safety audits focus on assessing the integrity of the service provider’s SMS and supporting systems. Safety audits can also be used to evaluate the effectiveness of installed safety risk controls or to monitor compliance with safety regulations. Ensuring independence and objectivity is a challenge for safety audits. Independence and objectivity can be achieved by engaging external entities or internal audits with protections in place - policies, procedures, roles, communication protocols. e) Findings and recommendations from safety investigations can provide useful safety information that can be analysed against other collected safety data. f) Operational data collection systems such as FDA, radar information can provide useful data of events and operational performance. 9.5.4.17 The development of SPIs should be linked to the safety objectives and be based on the analysis of data that is available or obtainable. The monitoring and measurement process involves the use of selected safety performance indicators, corresponding SPTs and safety triggers. 9.5.4.18 The organization should monitor the performance of established SPIs and SPTs to identify abnormal changes in safety performance. SPTs should be realistic, context specific and achievable when considering the resources available to the organization and the associated aviation sector. 9.5.4.19 Primarily, safety performance monitoring and measurement provides a means to verify the effectiveness of safety risk controls. In addition, they provide a measure of the integrity and effectiveness of SMS processes and activities. 9.5.4.20 The State may have specific processes for the acceptance of SPIs and SPTs that will need to be followed. Therefore, during development of SPIs and SPTs the service provider should consult with the organization’s regulatory authority or any related information that the State has published. 9.5.4.21 For more information about safety performance management, refer to Chapter 4. Chapter 9. Safety Management Systems (SMS) 9-21 9.5.5 The management of change 9.5.5.1 Service providers experience change due to a number of factors including, but not limited to: a) organizational expansion or contraction; b) business improvements that impact safety; these may result in changes to internal systems, processes or procedures that support the safe delivery of the products and services; c) changes to the organization’s operating environment; d) changes to the SMS interfaces with external organizations; and e) external regulatory changes, economic changes and emerging risks. 9.5.5.2 Change may affect the effectiveness of existing safety risk controls. In addition, new hazards, and related safety risks may be inadvertently introduced into an operation when change occurs. Hazards should be identified and related safety risks assessed and controlled as defined in the organization’s existing hazard identification or SRM procedures. 9.5.5.3 The organization’s management of change process should take into account the following considerations: a) Criticality. How critical is the change? The service provider should consider the impact on their organization’s activities, and the impact on other organizations and the aviation system. b) Availability of subject matter experts. It is important that key members of the aviation community are involved in the change management activities. This may include individuals from external organizations. c) Availability of safety performance data and information. What data and information is available that can be used to give information on the situation and enable analysis of the change. 9.5.5.4 Small incremental changes often go unnoticed, but the cumulative effect can be considerable. Changes, large and small, might affect the organization’s system description, and may lead to the need for its revision. Therefore, the system description should be regularly reviewed to determine its continued validity, given that most service providers experience regular, or even continuous, change. 9.5.5.5 The service provider should define the trigger for the formal change process. Changes that are likely to trigger formal change management include: a) introduction of new technology or equipment; b) changes in the operating environment; c) changes in key personnel; d) significant changes in staffing levels; e) changes in safety regulatory requirements; f) significant restructuring of the organization; and g) physical changes (new facility or base, aerodrome layout changes etc.). 9-22 Safety Management Manual (SMM) 9.5.5.6 The service provider should also consider the impact of the change on personnel. This could affect the way the change is accepted by those affected. Early communication and engagement will normally improve the way the change is perceived and implemented. 9.5.5.7 The change management process should include the following activities: a) understand and define the change, this should include a description of the change and why it is being implemented; b) understand and define who and what it will affect, this may be individuals within the organization, other departments or external people or organizations. Equipment, systems and processes may also be impacted. A review of the system description and organizations interfaces may be needed. This is an opportunity to determine who should be involved in the change. Changes might affect risk controls already in place to mitigate other risks, and therefore change could increase risks in areas that are not immediately obvious; c) identify hazards related to the change and carry out a safety risk assessment, this should identify any hazards directly related to the change. The impact on existing hazards and safety risk controls that may be affected by the change should also be reviewed. This step should use the existing organization’s SRM processes; d) develop an action plan, this should define what is to be done, by whom and by when. There should be a clear plan describing how the change will be implemented and who will be responsible for which actions, and the sequencing and scheduling of each task; e) sign off on the change, this is to confirm that the change is safe to implement. The individual with overall responsibility and authority for implementing the change should sign the change plan; and f) assurance plan, this is to determine what follow up action is needed. Consider how the change will be communicated and whether additional activities (such as audits) are needed during or after the change. Any assumptions made need to be tested. 9.5.6 Continuous improvement of the SMS 9.5.6.1 Annex 19, Appendix 2, 3.2 requires that… “the service provider monitor and assess its SMS processes to maintain or continuously improve the overall effectiveness of the SMS.” Maintenance and continuous improvement of the service provider’s SMS effectiveness is supported by safety assurance activities that include the verification and follow up of actions and the internal audit processes. It should be recognized that maintaining and continuously improving the SMS is an ongoing journey as the organization itself and the operational environment will be constantly changing. 9.5.6.2 Internal audits involve assessment of the service provider’s aviation activities that can provide information useful to the organization’s decision-making processes. The internal audit function includes evaluation of all of the safety management functions throughout the organization. 9.5.6.3 SMS effectiveness should not be based solely on SPIs; service providers should aim to implement a variety of methods to determine its effectiveness, measure outputs as well as outcomes of the processes, and assess the information gathered through these activities. Such methods may include: a) Audits; this includes internal audits and audits carried out by other organizations. b) Assessments; includes assessments of safety culture and SMS effectiveness. Chapter 9. Safety Management Systems (SMS) 9-23 c) Monitoring of occurrences: monitor the recurrence of safety events including accidents and incidents as well as errors and rule-breaking situations. d) Safety surveys; including cultural surveys providing useful feedback on staff engagement with the SMS. It may also provide an indicator of the safety culture of the organization. e) Management reviews: examine whether the safety objectives are being achieved by the organization and is an opportunity to look at all the available safety performance information to identify overall trends. It is important that senior management review the effectiveness of the SMS. This may be carried out as one of the functions of the highest-level safety committee. f) Evaluation of SPIs and SPTs; possibly as part of the management review, it considers trends and, when appropriate data is available, can be compared to other service providers or State or global data. g) Addressing lessons learnt; from safety reporting systems and service provider safety investigations. These should lead to safety improvements being implemented. 9.5.6.4 In summary, the monitoring of the safety performance and internal audit processes contribute to the service provider’s ability to continuously improve their safety performance. Ongoing monitoring of the SMS, its related safety risk controls and support systems assures the service provider and the State that the safety management processes are achieving their desired safety performance objectives. 9.6 COMPONENT 4: SAFETY PROMOTION 9.6.1 Safety promotion encourages a positive safety culture and helps achieve the service provider’s safety objectives, through the combination of technical competence that is continually enhanced through training and education, effective communications and information-sharing. Senior management provides the leadership to promote the safety culture throughout an organization. 9.6.2 Effective safety management cannot be achieved solely by mandate or strict adherence to policies and procedures. Safety promotion affects both individual and organizational behaviour, and supplements the organization’s policies, procedures and processes, providing a value system that supports safety efforts. 9.6.3 The service provider should establish and implement processes and procedures that facilitate effective two-way communication throughout all levels of the organization. This should include clear strategic direction from the top of the organization and the enabling of “bottom-up” communication that encourages open and constructive feedback from all personnel. 9.6.4 Training and education 9.6.4.1 Annex 19 requires that “the service provider develop and maintain a safety training programme that ensures that personnel are trained and competent to perform their SMS duties.” It also requires that “the scope of the safety training programme be appropriate to each individual’s involvement in the SMS.” The safety manager is responsible for ensuring there is a suitable safety training programme in place. This includes providing appropriate safety information relevant to specific safety issues met by the organization. Personnel being trained and competent to perform their SMS duties, regardless of their level in the organization, is an indication of management’s Next >