STS-128: SSP FRR creates opening rationale for IFR acceptance

by Chris Bergin

The LO2 Ice Frost Ramps (IFRs) will be modified to mitigate against foam liberation – pending design verification – from ET-133 onwards. However, ET-132 has gained the opening round of “rationale for acceptance” via the Space Shuttle Program (SSP) Flight Readiness Review (FRR), which will be verified by a Safety Panel and up for approval at the Agency FRR next week.

STS-128 Processing Latest:

Processing continues to track a NET (No Earlier Than) launch date of August 24 – though publicly NASA continue to contradict KSC Launch Integration milestones by noting August 25, pointing to a potential allowance for an additional day of processing flexibility.

Such flexibility is helpful, as per processing timeline refinements, such as Discovery’s preparations to head into S0024 Hyper Propellant Load operations, which were delayed by 12 hours.

“Yesterday, due to range requirements for the upcoming Delta launch, planning changed to enable working S0024 testing on 3rd shift last night, then switch to S5009 on 1st shift today to complete required range safety checks. S0024 will then pick up again this afternoon,” noted Wednesday processing information on L2.

“S0024 Hyper Propellant Load: Call to stations was performed at 2200 last night. RCS (Reaction Control System) driver test is complete. SRB hydrazine fill operations are scheduled to pick up later today following S5009 RSS testing.

“Main Propulsion System (MPS) helium signature test and ball seal leak checks have been completed, less SSME (Space Shuttle Main Engine) throat plugs and blank off plate removals.”

Discovery’s pad flow has encountered a small number of issues – perfectly normal, and expected for such complex operations – and usually dealt with by the time they have been reported to the NTDs (NASA Test Directors) that closely follow the flow.

“Fuel QD (Quick Disconnect) MD225 failed interface leak checks. The nose seal was R&R’d and the leak checks were successfully re-performed,” added processing information.

“EMU (space suits) functional is complete and the airlock is closed out for flight. GH2 recharge at the GH2 storage battery failed samples and will pick up again today. The leaking GSE (Ground Support Equipment) valve that was constraining OMBUU (Orbiter Mid Body Umbilical Unit) GH2 leak checks was replaced and the leak checks are complete. S0009 is now complete.”

D1aWork on replacing hardware in the left hand Solid Rocket Booster (SRB) is continuing to progress to plan – a great achievement by the Kennedy Space Center (KSC) engineers, given this procedure has never been undertaken at the pad before. The hot fire of the replaced Hydraulic Power Unit (HPU) has now advanced from the third shift, to the first shift on Friday.

“LH SRB HPU update: A replacement HPU lube oil servicing cart was obtained yesterday and lube oil servicing is now complete. HPU hotfire is now scheduled for Friday 1st shift.”

Meanwhile, the LOX storage tank replenishment at the pad was completed on Tuesday, with four waves (20 tankers total) offloaded, ahead of ET-132 being loaded with the propellant on launch day.

SSP FRR (ET-132 LO2 IFRs):

A huge amount of documentation has been reviewed by the SSP FRR managers, as they work towards a final meeting (Agency FRR) next week.

The majority of the documentation notes confidence in the key issue being discussed – the foam liberation events observed during STS-127’s ride uphill.

D1bEndeavour’s tank – ET-131 – suffered numerous foam loss events. However, it was the liberation from one of its LO2 IFRs that has required additional analysis.

With checks on Discovery’s tank – ET-132 – carried out in the Vehicle Assembly Building (VAB) prior to rollout to Pad 39A, a good understanding on the health of the one specific ramp in question (Xt 718) has been created.

The question heading into the FRR related to whether this ramp is good to fly without modification, or if rollback to the VAB should be called.

Numerous pages of the ET FRR presentation – acquired by L2 – outline the previous incidents with foam loss from the LO2 IFRs, and the work in evaluating the state of ET-132 (and ET-133 – set to fly with Atlantis on STS-129).

However, the most interesting element to the presentation is the overview of what could be expected if the ramp does liberate foam for a third time in succession during STS-128’s launch – using STS-127’s findings as an example.

“Failure Mode Assessment: STS-127 XT 718 LO2 IFR loss most likely caused by void/delta pressure failure mechanism. Physics-based void/delta pressure analysis suggest larger divots likely to occur (late in flight),” noted one of 28 FRR presentations acquired by L2.

“Divot/no divot analysis shows earliest possible time for release is ~> 260 seconds MET (Mission Elapsed Time) at the depth estimated from imagery (~2.5”) and void size consistent with the maximum expected (slot width = 1.4”).

“Divoting time determined assuming void pressure/stress intensity increases as a function of TPS (Thermal Protection System – Foam) temperature vs. time. Smaller, more shallow voids could divot earlier (<1” depth).

“Larger than expected voids could also divot earlier (>1.5” width) Umbilical well imagery of final crater also possibly explained as a result of successive, smaller void/delta pressure divoting combined with popcorning/erosion.”

D1dThis shows that the liberations are likely to occur – if at all – at a stage of the flight where the aerodynamic forces would result in a no danger to the orbiter’s TPS, in the event the liberation(s) strike the vehicle.

Foam losses prior to SRB sep (below MET 120 seconds) are usually more of a concern, as the debris hits the atmosphere, slows down, resulting in the orbiter racing into the decelerating debris.

These findings, coupled with the likeliness an IFR foam loss would break up into yet smaller pieces – or liberates from the ramp in small pieces – further reduce the “threat” to the orbiter.

“Rationale for Acceptance: Terahertz NDE (Non Destructive Evaluation) performed at KSC provides confidence in XT 718 IFR. Observations consistent with process performance expectations. One possible void indication identified and assessed (1” diameter cylinder ~ 2” – 3” depth). Void delta pressure analysis performed and shows divoting not expected,” added the FRR presentation.

“NDE results conservatively correlated with flight performance – No divoting observed at locations with void-type indications. Previous test/flight experience shows foam loss most likely to occur in multiple, smaller pieces resulting in appearance of larger crater.

“LH2 tank IFR thermal/vacuum test results show successive popcorning-type events resulting in appearance of larger crater.”

In summary, the ramp is unlikely to liberate – given it only has one small potential void via the NDE scans – and if it does liberate, such an incident would occur late in flight, and would unlikely result in one main piece of foam flying loose.

D1eThe result of the FRR findings calls for a slight increase in the risk assessment, but to accept it for flight – pending a review via a Safety Issue Briefing, which will be held during Thursday’s Program Requirements Control Board (PRCB) meeting.

“Hazard Report T.02, Loss of ET TPS Risk/Cause Matrix for the LO2 IFR TPS being revised for Cause 9, Manually Poured Foam Application Process Deficiencies. Safety Issue Briefing (per NSTS 22254) planned due to incremental increase in risk for STS-128 (8/13/09 PRCB),” concluded the main FRR presentation.

“Overall classification of T.02 unchanged – Accepted Risk. Recommend closure of STS-127-FRR-002 action item. TIM actions and implementation of recommendation to be worked as PRCB or SICB Directive.”

The only issue with raising the risk numbers, is that could ultimately become a topic of interest to the Agency FRR – an executive body – members. This is observed in one of the FRR presentations that appears to couple the risk element of the evaluations with a call for all tanks – including ET-132 – to be modified.

“Certification of the terahertz NDE process, retrofit of as-built tanks by removing the inboard and outboard portion of the PDL ramps and replace with manual BX application for ET-132 and subsequent builds, and/or performing cryogenic cyclic testing with certified NDE techniques to assess the effects of cycling on the IFRs would provide increased confidence and reduce the risk impact.”

D1rLaunching with an increased risk – albeit small – would not be a problem, with previous issues with the tanks, including those seen with the LH2 IFRs ahead of STS-121, raising the risk to the ‘red’ area of probable/catastrophic on the often-used matrix – which the current LO2 IFR issue is not bracketed in (yellow risk – see slide on the left hand side).

The STS-121 Agency FRR was “lively” as a result, with the Marshall Space Flight Center Safety Engineering Review Panel (MSERP) dissenting over the first change to the LH2 Ice Frost Ramps on the risk assessment.

“The integrated SSV (Space Shuttle Vehicle) risk assessment associated with loss of IFR foam has shown the risk to be ‘red’, or probable/catastrophic,” they wrote, concerned about the ramps liberating foam during ascent.

NASA’s Chief Safety and Mission Assurance Officer Bryan O’Connor and then Chief Engineer Chris Scolese concurredwith moving the risk up to “probable/catastrophic”, along with Propulsion Systems Engineering and Integration (PSE&I), who noted: ‘Catastrophic outcome could easily happen in the life of the program.

‘The risk should be classified as ‘probable’ and is considered by PSE&I to be unacceptable,’ on their LH2 IFR Risk Assessment position.

With large-scale dissent at the FRR, the then NASA Administrator Mike Griffin stepped in to make the final call not to stand-down and launch – based on TPS damage not being a Loss Of Crew event with Safe Haven. Scolese and O’Connor signed off on the FRR, but “with remarks” on their position.

Ultimately, launched without major incident during STS-121.

Forward Plan:

Should the PRCB and ultimately the Agency FRR approval the recommendation to close ET-132’s IFR concern, and thus allow flight to proceed without the need for a rollback, the plan is to modify the tanks – from ET-133 onwards.

“Status/Forward Plans: Assessing potential to implement NDE at MAF or KSC for subsequent builds. Planning to inspect all stations with horizontal pours (7 locations). Developing option to retrofit as-built tanks to improve debris performance,” added the FRR presentation, again listing notes seen by last week’s PRCB – and previously reported by this site.

“Plan to remove in-board and out-board portion of PDL ramps and replace with manual BX spray. Outboard ramp BX spray previously validated in horizontal orientation and successfully flown on aft 3 stations (PAL ramp removed tanks).

D1c“Zero voids identified during validation. Eliminates PDL material where largest voids can occur. Design verification assessment in-work – No issues expected. Minimal / no change in structural temperatures and FS expected. Aft stations less critical for ascent heating environments. No additional wind tunnel / cryogenic cycling testing expected

“LH2 Tank IFR BX extension previously test-demonstrated to 1.25 x maximum dynamic pressure. BX extensions previously test-demonstrated during LH2 tank IFR thermal/vacuum test program

“Planning to perform work at KSC in checkout cell (vertical). Spray process validation plan in-work to ensure KSC ergonomics are addressed.

“Targeting all 7 horizontal pour locations on ET-133. SE&I provided ET with ranked list based risk analysis for prioritization.”

Further work will be carried out on tanks through to at least ET-138 at the Michoud Assembly Facility (MAF), pending the outcome on the potential for the shuttle program to be extended with additional flights.

L2 members: Documentation – from which the above article has quoted snippets – is available in full in the related L2 sections, now over 4000 gbs in size.

Related Articles