Air transportation safety recommendation A06-08

Reassessment of the responses to Aviation Safety Recommendation A06-08

Value of statistical life

Background

On 30 May 2000, the pilot of a Cessna 177B Cardinal was attempting to take off from a grass airstrip at Calling Lake, Alberta. The aircraft struck trees during the initial climb, struck the ground, and burst into flame. The two occupants were exposed to smoke and flame for some time. One occupant was fatally injured due to the effects of fire, and one sustained serious burns.

The accident investigation identified fuel system crashworthiness as a small-aircraft safety deficiency. In light of that finding, the Transportation Safety Board of Canada (TSB) identified a need to examine the extent to which fuel system crashworthiness and other safety deficiencies contribute to the risks associated with post-impact fire (PIF) in otherwise survivable accidents, and to consider the risk control options available to mitigate those risks.

According to TSB data from 1976 to 2002, PIF occurs in approximately 4 per cent of accidents involving small aircraft; however, these accidents account for approximately 22 per cent of the fatalities and 11 per cent of the serious injuries caused by all accidents. Overall, 6.2 per cent of fatal injuries and 3.8 per cent of serious injuries that resulted from small-aircraft PIF accidents were related to fire. Because small aircraft have a higher accident rate with a correspondingly greater number of PIF accidents, more defences are required to mitigate the risk of PIF. Considering these factors, the Board identified the need to conduct a Safety Issues Investigation.

The Board concluded its investigation and released report SII A05-01 on 29 August 2006.

Board Recommendation A06-08 (29 August 2006)

The report submitted to the TSB on the process of economic analysis into risk control options for mitigation of PIF risks identified that the United States guidelines on economic analysis and cost-benefit analysis (CBA) are commendable; however, the $3 million value of statistical life (VSL) figure currently used by the Department of Transportation and the Federal Aviation Administration (FAA) is low relative to recent empirical estimates. The original CBA for Notice of Proposed Rule Making (NPRM) 85-7A used a VSL of $1 million. Because numerous cost-effective technological advances to eliminate PIF have been developed, and given that benefits are directly proportional to the value chosen for VSL and to the effectiveness rate of the PIF risk control measures, the calculated benefits may be greater and the costs proportionately lower if recent higher empirical VSLs were applied to the original CBA. Using the comprehensive PIF database assembled during this investigation, it is possible to calculate the expected benefits in lives saved by preventing PIF. Using Canadian PIF statistics, the expected present-discounted benefits over the lifetime of an aircraft are several thousand U.S. dollars, and that value is sufficiently large that a detailed CBA may be warranted for specific PIF risk control option technologies.

Therefore, the Board recommended that:

Transport Canada's responses to A06-08 (20 November 2006 and 29 January 2007)

In its 20 November 2006 response, Transport Canada (TC) did not make reference to the deficiency described in Recommendation A06-08.

In its dated 29 January 2007, TC states that it does not select the VSL but merely uses a VSL established by Treasury Board Secretariat (TBS) for use in its regulatory related CBA. TC has contacted TBS to discuss Recommendation A06-08 and TBS staff has expressed interest in reviewing the VSL. TC states that it is interested in participating in a TBS-led interdepartmental review of the VSL. TC's response suggests that the TSB may wish to participate in such a review of VSL rates and calculation methodology.

Additionally, TC states that, because the VSL is not solely an aviation issue, it would be inappropriate to approach the FAA regarding changes to the VSL. TC advises that the FAA is aware of TSB's SII A05-01 and that TC is in regular contact with the FAA and will relay any information regarding FAA's intentions to the TSB.

Board assessment of the responses to A06-08 (12 September 2007)

As TC's response dated 20 November 2006 contained no action or proposed action that would reduce or eliminate the risks associated with this deficiency, TSB staff sought an update of TC's position to Recommendation A06-08.

TC's updated response does not indicate TC's position with respect to the core of Recommendation A06-08; that the CBA used in FAA's NPRM 85-7A be revised in accordance with suggested criteria. Instead, it focuses on the issues related to a single criterion: the current VSL rates. Additionally, there is no mention of revising the CBA in light of the benefits of the Canadian PIF statistics or in consideration of the newest advances in PIF prevention technology.

As far as working with foreign regulators, TC indicates that it will not take any extraordinary measures in its dealings with the FAA to advance the merits of Recommendation A06-08 and makes no mention of contacting other foreign regulators. Essentially, TC's action plan is limited to working with TBS to review the VSL.

Because TC's responses propose action that may reduce but will not substantially reduce or eliminate the deficiency, the response to Recommendation A06-08 is assessed as Satisfactory in Part.

Transport Canada response to A06-08 (15 January 2010)

TC's latest response states that it uses the VSL provided by the TBS for cost benefit analysis purposes. These numbers were updated in 2004 and are continually updated by TBS.

Board assessment of the response to A06-08 (28 July 2010)

TC's latest response repeats information previously provided to the TSB regarding the source for the VSL that it uses for cost benefit analysis. In keeping with its previous responses TC has again failed to comment on the core of Recommendation A06-08; that the CBA used in the FAA's NPRM 85-7A be revised in accordance with suggested criteria. Furthermore, the response is once again silent with respect to the aspect of revising the CBA in light of the benefits of the Canadian post-impact fire statistics or in consideration of the newest advances in post-impact fire prevention technology. Essentially, TC's response is limited to receiving the VSL from the TBS on a regular basis.

Meanwhile the post-impact fire experience remains high. Preliminary research reveals that since 01 January 2005 the Board has recorded 99 occurrences that involve a post-impact fire phase/event combination. The majority of these occurrences involved aircraft weighing less than 5700 kg. A word string search of the NTSB database for the same timeframe turned up records of over 300 general aviation accidents where post-impact fire is referenced.

Given that this is the second response in which TC has failed to address the risks associated with the deficiency identified in Recommendation A06-08 it would appear that TC has in fact declared that it plans no further action with respect to the deficiency identified in Recommendation A06-08.

Because TC's response indicates that no action has been taken or proposed, that will reduce or eliminate the deficiency, the response to Recommendation A06-08 is assessed as Unsatisfactory.

Board review of A06-08 deficiency file status (11 October 2012 as amended on 03 April 2013)

The Board notes that a review of the TSB's Aviation Safety Information System reveals that the PIF experience continues. From 01 January 2005 to 09 October 2012, the TSB recorded 93 occurrences of aircraft weighing less than 5700 kg that included a PIF phase/event combination.

The Board believes that the risks identified in Recommendation A06-08 have not abated and remain substantial. The Board therefore determined that these risks deserve a renewed effort by TSB to improve its data gathering of PIF-related events, and that TC should take appropriate safety actions to mitigate the risks associated with Recommendation A06-08 in order to reduce injuries and death associated with PIF.

Transport Canada's response to A06-08 (13 December 2012)

TCCA is aware of the FAA work in the 1980's leading to NPRM 85-7A, titled 14 CFR Part 23 Airworthiness Standards: Crash-Resistant Fuel Systems, issued on 28 February 1990. The NPRM was subsequently withdrawn on 28 February 1999. The NPRM 85-7A withdrawal notice stated that the FAA had completed a revised economic evaluation of the safety recommendations as a result of the comments received and concluded that the costs of the proposed changes were not justified by the potential benefits, in addition to identifying some technical questions that remained unanswered.

As the TSB has asserted, some estimates of VSL have increased in the intervening years. However, TCCA would observe that the costs of aircraft modification have also increased in this period, by a factor that may have outpaced the VSL. Therefore, it may be unlikely that a revised economic analysis would yield a different outcome at this time. Consequently, TCCA would be unable to satisfy CDSR criteria in putting forward new rulemaking on this subject. However, work in other areas of GA safety is progressing that is likely to have a more holistically positive effect.

It has been noted that, despite all the advancements in design standards since the 1960's, the FAA observes that only 7% of the general aviation fleet has a modern set of applicable standards in its basis of certification. As many as 93% of general aviation airplanes in operation today have a basis of certification dating back to Civil Air Regulation (CAR) 3, designed in the 1950's through 1970's.  The main barrier to the introduction of newer airplanes to the GA market is cost.  Substantial fleet renewal with newer model airplanes, meeting the latest design standards, is necessary to obtain significant improvement in GA safety.

A holistic approach to renewing the GA sector, and its safety, is presently underway with the FAA's Part 23 ARC. An international group of aviation authorities and industry are currently redesigning the requirements for “Part 23” airplanes in an attempt to accomplish two goals: improve safety by a factor of two, and reduce costs to certify by half.  The reduction of costs is necessary to enable the GA fleet renewal that would be necessary to obtain a fleet-wide safety improvement; the safety of the existing legacy and aging fleet of GA airplanes cannot be expected to improve with time.

In particular, the hope for the reorganization of Part 23 would be to facilitate the introduction of lifesaving technologies in GA airplanes in an attempt to reduce fatal accidents by half (i.e. loss of control (stall/spin close to ground), controlled flight into terrain, and engine mismanagement, collectively contributing to 50% of all GA accidents).  A secondary objective is to reduce type certification (and production certification) costs by half.

FAA, TCCA, EASA, ANAC-Brazil and other aviation authorities are focussing on this vision for general aviation for the next 25 years, a vision that offers an economically viable means to have an overall improvement in GA safety.

Federal Aviation Administration's response to A06-08 (27 November 2012)

TSBC safety recommendation A06-08 asked that Transport Canada, together with the FAA, revise the cost benefit analysis for Notice of Proposed Rulemaking (NPRM) 85-7A, using Canadian statistics. Although the FAA did not respond to this recommendation, it should be noted that the FAA withdrew the NPRM on December 30, 1999. Regardless of the merits of substituting Canadian cost benefit analysis, the NPRM was withdrawn for a number of reasons, thus rendering the safety recommendation moot. Since there has been no effort to revive the NPRM, the FAA also considers this safety recommendation "Closed," with no further activity planned.

Board assessment of Transport Canada's response to A06-08 (06 March 2013)

Transport Canada provided a collective response to recommendations A06-08, A06-09, and A06-10. Paragraphs 1 and 2 of the response pertain to A06-08; the remaining paragraphs identify the existence and goals of the FAR 23 ARC.

FAA NPRM 85-7A on Airworthiness Standards; Crash Resistant Fuel Systems was published on 28 February 1990 and withdrawn on 30 December 1999. As stated in the Transport Canada response, the FAA had completed a revised economic evaluation as a result of the comments received and concluded that the costs of the proposed changes were not justified by the potential benefits.

The TSB notes that NPRM 85-7A originally included a summary of the cost-benefit analysis, and that the FAA believed the proposed regulation to be cost-beneficial based on those calculations. It is known that 17 commenters responded to the NPRM and that several disagreed with the economic evaluation; however, a revised cost-benefit analysis is not provided with the withdrawal notice. ^{Footnote 1} Therefore, the reason for the withdrawal of NPRM 85-7A as it pertains to the cost-benefit analysis is not adequately explained.

The Transport Canada response also states that the FAA identified some technical questions that remained unanswered. A review of the NPRM 85-7A withdrawal notice determined that the following additional concerns had been submitted by commenters:

• The reliability of self-closing devices in fuel lines and the effects of uncommanded operation of such devices need to be determined.
• A definition of “survivable crash” is needed.
• There was objection to the mandating of flexible bladder tanks instead of specifying an objective test for fuel tanks.
• The application of the proposal to previously type-certificated, newly manufactured airplanes vs. newly type-certificated airplanes needed to be clarified.
• It was suggested that the proposed standards should be applied to newly manufactured airplanes on an individual basis rather than on an overall basis, as it appeared some aircraft types were more prone to post-crash fires. However, the FAA does not selectively apply airworthiness standards to specific airplane models.
• The proposals did not adhere to the recommendations made by the General Aviation Safety Panel (GASP II) committee.   

These concerns are surmountable. Therefore, based on the information contained in this response, Transport Canada appears not to recognize the risks associated with post-impact fires or the potential to mitigate those risks.

Transport Canada agrees that estimates of VSL have increased in the intervening years and points out that the costs of aircraft modification have also increased in this period, by a factor that may have outpaced the VSL. Transport Canada goes on to state that based on this it may be unlikely that a revised economic analysis would yield a different outcome at this time, and that consequently, TCCA would be unable to satisfy the CDSR criteria in putting forward new rulemaking on this subject. These statements are made in general terms, without quantification.

Transport Canada states that one of the goals for reorganization of Part 23 is to facilitate the introduction of lifesaving technologies in small airplanes in an attempt to reduce fatal accidents by half; a secondary objective is to reduce type certification (and production certification) costs by half. Both of these goals fit with the intent of Recommendation A06-08, as any changes to FAR 23 requirements will require a cost-benefit analysis to be completed, using the latest VSL. Reducing type certification costs by half may have a significant positive impact on cost-benefit analyses specific to post-impact fire countermeasures.

Since the last Transport Canada response to A06-08 was received on 28 July 2010, Transport Canada appears not to have taken any action to support revision of the cost-benefit analysis of NPRM 85-7A by the FAA.

Board assessment of the Federal Aviation Administration's response to A06-08 (06 March 2013)

Note: the FAA had not previously responded to TSB Recommendation A06-08.

FAA NPRM 85-7A on Airworthiness Standards; Crash Resistant Fuel Systems was published on 28 February 1990 and was withdrawn on 30 December 1999. While the FAA response to A06-08 states that NPRM 85-7A “was withdrawn for a number of reasons”, the NPRM 85-7A withdrawal notice states the NPRM was withdrawn due primarily to comments received regarding the cost-benefit analysis. ^{Footnote 2} Seventeen commenters responded to the NPRM; several disagreed with the economic evaluation and believed that either the benefits had been overestimated, costs had been underestimated, or both.

The TSB notes that NPRM 85-7A originally included a summary of the cost-benefit analysis, and that the FAA believed the proposed regulation to be cost-beneficial based on those calculations. A revised cost-benefit analysis is not provided with the withdrawal notice; therefore, the reason for the withdrawal of NPRM 85-7A as it pertains to the cost-benefit analysis is not adequately explained.

The FAA's response to A06-08 mentions, but does not clarify, the other reasons for withdrawal of the NPRM. A review of the NPRM 85-7A withdrawal notice determined that the following additional concerns had been submitted by commenters:

• The reliability of self-closing devices in fuel lines and the effects of uncommanded operation of such devices need to be determined.
• A definition of “survivable crash” is needed.
• There was objection to the mandating of flexible bladder tanks instead of specifying an objective test for fuel tanks.
• The application of the proposal to previously type-certificated, newly manufactured airplanes vs. newly type-certificated airplanes needed to be clarified.
• It was suggested that the proposed standards should be applied to newly manufactured airplanes on an individual basis rather than on an overall basis, as it appeared some aircraft types were more prone to post-crash fires. However, the FAA does not selectively apply airworthiness standards to specific airplane models.
• The proposals did not adhere to the recommendations made by the General Aviation Safety Panel (GASP II) committee.   

While these concerns are both administrative and technical in nature, all appear to be surmountable. Therefore, based on this response, the FAA appears not to recognize the risks associated with post-impact fires or the potential to mitigate those risks.

The NPRM 85-7A withdrawal notice also indicated the FAA intended to conduct further internal study. The TSB recently requested an update regarding any FAA internal study pertaining to NPRM 85-7A; no response was received.

It is also noted that NPRM 90-24 on Airworthiness Standards; Crash Resistant Fuel Systems in Normal and Transport Category Rotorcraft was issued on 27 September 1990. ^{Footnote 3} This NPRM was the rotorcraft equivalent of NPRM 85-7A. NPRM 90-24 resulted in a requirement for crash resistant fuel systems to be incorporated into helicopters certified after November 1994. The improvements were similar in nature and concept to those proposed for small production fixed-wing aircraft in NPRM 85-7A. One economic factor that may have swayed the withdrawal of NPRM 85-7A and the success of NPRM 90-24 was the state of the small fixed-wing manufacturing economy during the period these NPRMs were being deliberated. Small fixed-wing production rates dropped about 90% during the late 1980s, compared to production rates from the mid-1970s to the early-1980s, largely due to product liability costs. One reaction was the complete shutdown of the Cessna small aircraft production line from 1986 to 1996. Following approval of the General Aviation Revitalization Act of 1994 (GARA), small aircraft production rates increased, but never to the levels seen a decade earlier. The economic impact of the manufacturing decline is believed to have been proportionally less in the helicopter industry, which may have supported the more favorable cost-benefit result for post-impact fire countermeasures in helicopters. Small fixed-wing aircraft manufacturers did not want to increase manufacturing costs for any reason during this period and comments received by the FAA regarding NPRM 85-7A may have reflected that, more so than those received regarding NPRM 90-24. From an economic perspective, it may have simply been a bad time for NPRM 85-7A deliberations to take place.

While the FAA is unwilling to act on Recommendation A06-08, based on the withdrawal of currently outdated NPRM 85-7A, it is known that the FAA convened a 55-member FAA Part 23 Aviation Rulemaking Committee (ARC) in August 2011 to look at simplifying the Part 23 regulations. The goal is to improve small aircraft safety by a factor of two and to reduce certification costs by half. There is strong international involvement, including Transport Canada and EASA. The TSB recognizes it may be easier to mandate post-impact fire countermeasures into FAR 23 regulations in the future if the FAR 23 regulations are simplified and certification costs are reduced as a result of Part 23 ARC work. This possibility is not identified in the FAA response to Recommendation A06-08.

The FAA has made no effort to revise or reconsider the cost-benefit analysis for NPRM 85-7A; as well, no further FAA activity is planned.

Board assessment of the Transport Canada, and Federal Aviation Administration responses to A06-08 (06 March 2013)

The Board believes that the risks identified in Recommendation A06-08 have not abated and remain significant. The Board therefore determined that these risks deserve a renewed effort by TSB to improve its data gathering of PIF-related events. In addition, the Board believes that Transport Canada and the FAA should consider appropriate safety actions and applicable cost-benefit considerations, including increased VSL, to mitigate the risks associated with Recommendation A06-08, in order to reduce injuries and death associated with post-impact fire. These considerations are especially significant at this time, considering the expected positive impact of the FAA Part 23 ARC work that is currently in progress and the ensuing adjustments to cost-benefit analyses as they pertain to improvements to reduce the risk and impact of post-impact fire.

However, the FAA has made no effort to revise or reconsider the cost-benefit analysis for NPRM 85-7A; as well, no further FAA activity is planned. In addition, since the last Transport Canada response to A06-08 was received on 28 July 2010, Transport Canada appears not to have taken any action to support revision of the cost-benefit analysis of NPRM 85-7A by the FAA.

To date, no action has been taken or proposed that will reduce or eliminate the deficiency.

Therefore, the responses in aggregate are assessed as Unsatisfactory.

Federal Aviation Administration response to A06-08 (14 November 2013)

In its response dated 14 November 2013, the Federal Aviation Association (FAA) restated its response from 27 November 2012 and considers the recommendation as closed.

Transport Canada response to A06-08 (26 November 2013)

In its response dated 26 November 2013, Transport Canada states that the FAA considers Recommendation A06-08 closed. With no further activity planned, Transport Canada will not be revising the cost-benefit analysis for Notice of Proposed Rule Making 85-7A. No further updates will be provided.

Board assessment of the responses to A06-08 (02 April 2014)

The TC and FAA responses from November 2013 contained no action or proposed action that would reduce or eliminate the risks associated with this deficiency. There has been no indication since July 2010 that any action will be pursued by TC or the FAA in response to this recommendation.

Because TC’s response indicates that no action has been taken or proposed that will reduce or eliminate the deficiency, the response to Recommendation A06-08 is assessed as Unsatisfactory.

Next TSB action

The assessment determines that there is a residual risk, but no further action is planned, and continued reassessment will not likely yield further results. This recommendation will not be reassessed on a regular basis. However, occasional reviews will be conducted to see if any dormant recommendations should be reactivated and/or reassessed. The Board may also reassess a dormant recommendation at any time if actions have been taken that significantly reduce the residual risk.

The deficiency file is assigned a Dormant status.