STS-134: Launch slips to NET May 8 as APU troubleshooting finds LCA fault
Endeavour’s critical – and final – mission to the International Space Station (ISS) has been delayed until at least May 8, following an engineering evaluation into her Auxiliary Power Unit 1 (APU-1), which suffered from a heater problem, breaking Launch Commit Criteria rules. The investigation found the root cause relates to a hybrid driver inside the (Load Control Assembly), requiring the lengthy process of removing and replacing the box. UPDATE NOW NET MAY 10.
Endeavour remains sat on Pad 39A for what was initially a 72 hour scrub turnaround, in order to protect a Monday launch date – although managers made it very clear that there was a fair amount of potential the launch date would slip further.
The STS-134 countdown was proceeding in a relatively nominal fashion on Friday, with activation of the orbiter’s power-generating Fuel Cells one of the final tasks on the S0007 Launch Countdown operation task sheet, prior to the Mission Management Team (MMT) approval to begin tanking operations on ET-122.
A morning NASA Test Director (NTD) report (L2) did list an item of interest during the Fuel Cell Pulse Purging operation, citing a potential constraint to resuming the count at T-9, although post scrub information notes this would not have become a problem, having been cleared as a Material Review (MR).
The loading of ET-122 – a veteran tank, refurbished and modified since being damaged during Hurricane Katrina – was completed without issue, during which time an unrelated pressure issue on the Right OMS (Orbital Maneuvering System) fuel tank breached LCC limits.
This issue was soon resolved, as an engineering solution employed the well-known crossfeed capability of the two OMS systems, interconnecting the high pressure tank with its relative on the Left OMS tank, equalising the pressure back to within nominal parameters.
With the OMS tank pressure solved, and ET-122 heading into stable replenish, all appeared to be going well, with only a minor heater note added to the Interim Problem Report (IPR) log, something not totally unexpected, given the stack was becoming much colder as ET-122 filled up with the cryogenic propellant.
“(APU) Heater B 183 and 184 trending low,” was the first mention of the IPR per L2’s launch day coverage, followed by a note that controllers were wanting to see if switching on heater A, then heater B, from cockpit would narrow down the issue – part of the preplanned mitigation procedure for verifying heater condition.
“Auxiliary Power Unit 1 Fuel Test Line Temp V46T0183A1 is 44 deg f s/b 45 deg F or above. Violates LCC APU-14,” was the official issue report (L2) from the NTD for the issue (acquired by L2 at 10am Eastern).
“System: APU 1 Fuel Service Line Temp V46T0184A1 is also trending down. A preplanned procedure exists to verify heater function. Requires cockpit access.”
With two technicians and one engineer sent to the pad to access the flight deck and make the required switch throws – which also included checks between internal and ground power sources as part of the root cause evaluation – the problem was narrowed down to one of the two strings used by the heaters (String B).
“Auxiliary Power Unit switched to the A heater string via ground command and the temperature is now trending up,” added L2 information. “Still need to verify heater B function via cockpit switch.”
Unfortunately, the B string still showed problems, an unacceptable condition due to the flight rules requiring both strings to be working prior to launch, resulting in the scrub.
The reason for this is safety driven, as much as a problem being spotted on orbit would lead to several mitigation procedures to avoid a near-immediate end to the mission.
The safety element relates to the APUs being a hydrazine-fueled, turbine-driven power unit that generates mechanical shaft power to drive a hydraulic pump that produces pressure for the orbiter’s hydraulic system. There are three separate APUs, three hydraulic pumps and three hydraulic systems on an orbiter.
Electrical heater systems and insulation thermally control the system above 45 degrees F to prevent fuel from freezing and to maintain required lube oil viscosity. Insulation is used on components containing hydrazine, lube oil or water to minimize electrical heater power requirements and to keep high surface temperatures within safe limits on the turbine and exhaust ducts.
The concern of a heater failure causing the hydrazine line to become thermally shocked would hold the potential of the line cracking and a leak forming. The leaked hydrazine is a flammability risk during re-entry, potentially resulting in a fire in the aft of the orbiter – a highly undesirable scenario.
STS-134 Specific Articles: https://www.nasaspaceflight.com/tag/sts-134/
Following the scrub, controllers in the Firing Room carried out a couple of additional tests, to see if they could gain any additional data on the failure signature, prior to draining the tank. Although the tank drain only requires a few hours to complete, the residual propellant needs to boil off, prior to inerting and safing the tank for engineers to access the orbiter.
With the tank inert and safe by around midday on Saturday, access was set up around one side of the orbiter, allowing engineers to gain access into the aft of Endeavour for a “hands on” evaluation of the APU and related hardware.
Engineers first tested the series of thermostats associated with the hydrazine line, by spraying them with freeze mist to check their operation. The thermostats are a key part of the system, as they provide the data for the heaters to keep the system in a nominal temperature environment.
“The APU heater tank/fuel line/H 2 O sys 1A, lB, 2A, 2B, 3A, 3B switches on panel A12 operate the thermostatically controlled heaters located on the corresponding APU fuel system and water system,” noted a technical overview of their role in the system.
“The fuel tank, fuel line and water line heaters for each auxiliary power unit are divided into redundant A and B systems for each unit. For example, for APU 1, 1A and 1B, the 1A switch controls the A heaters and the thermostats provide automatic control. Only one set of heaters is used at a time. The 1B switch controls the 1B heaters and the thermostats provide automatic control.
“The APU fuel tank and line heater thermostats maintain the temperatures between a nominal 55 F and 65 F. The water system heater thermostats maintain the temperatures between 80 F and 90 F. The off position of each switch removes power from the respective heater circuits.”
The results of the tests with the freeze mist showed some of the thermostats to be working in a nominal fashion, while some showed signs of issues.
Had the problem been specific to just the thermostats, it may have been possible to carry out an in-situ repair without the removal of any major hardware, and potentially in time for a Monday or Tuesday launch attempt.
However, when the fault with the thermostats was traced up the fault tree, engineers found it tied into a problem with what are known as hybrid drivers in the Aft (Load Control Assembly) LCA, a control box which was feared to be one of the leading candidates for the problem, one which requires several days to fix.
“Each LCA contains hybrid drivers, which are solid-state switching devices (no mechanical parts) used as logic switches and for low-power electrical loads of less than 5 amps,” added technical information on the LCA hybrid drivers.
“When the drivers are used as a logic switch, several control inputs are required to turn on a load. The hybrid drivers are current protected by internal fuses. Hybrid relays requiring multiple control inputs are used to switch three-phase AC power to motors.”
The only viable solution is to remove and replace the LCA box, prior to a two-day retest ahead of being able to re-enter the countdown.
Also convoluting matters is the May 6 launch of an Atlas V carrying SBIRS GEO 1 from Cape Canaveral, thus holding priority on the range, placing Endeavour into a provisional NET (No Earlier Than) May 8 launch date scenario, with a large amount of work required based on range and ISS conflicts. Should engineers fail to finish the work in time for May 8, May 10 will be the next launch opportunity.
A more set foward plan on the launch date scheduling will follow on Monday or Tuesday. UPDATE: Now May 10 NET – see:
(All images via L2, with addition to Larry Sullivan of MaxQ Entertainment/NASASpaceflight.com. Extensive coverage is being provided on the news site and forum, driven by the L2 special sections with further articles to follow).