Teams at launch pad 39A at the Kennedy Space Center, FL have completed operations to install a replacement Aft Load Controller Assembly (ALCA) into one of Endeavour’s aft avionics bays. Meanwhile, the ALCA that failed during the April 29th launch attempt for Endeavour is undergoing testing to isolate the cause of the failure and enable engineers and program managers to eventually clear Endeavour for launch.
New LCA Installed into Endeavour:
Following the scrub of Endeavour’s STS-134 mission just after 12-noon on Friday, April 29th, engineers across the agency began assessing the data gathered on Endeavour’s APU-1 B-channel heaters and their associated power systems.
After initially backing the S0007 launch countdown to T-11hrs and holding, teams gained entrance into the aft of Endeavour Saturday afternoon and quickly began a troubleshooting plan to isolate the nature of the fault within the heater system.
Initially focusing on the thermostat itself, teams quickly ruled it out as the cause of the issue – pointing instead to ALCA-2 in one of the Endeavour’s aft avionics bays. By Sunday afternoon, engineers had isolated the failure as being internal to the ALCA-2 and all options to avoid R&Ring (Removing and Replacing) the ALCA-2 were ruled out.
Teams then completely backed out of Endeavour’s launch countdown and powered down the vehicle after removing the onboard Fuel Cell reactants and depressurizing tanks within Endeavour.
Prior to the ALCA-2’s removal from Endeavour, a series of additional troubleshooting measures were carried out to verify as much as possible that the ALCA-2 was indeed the cause of the failure seen on launch day.
To this end, as noted by engineering sources (L2), Technicians “Performed additional troubleshooting: Operation 100- Demate, inspection and push test of 55P320 complete.” As confirmed by the same day’s NTD (NASA Test Director) update, this op was performed with nominal results.
Additionally, “Operation 110 – Continuity measurements taken through the HR116B/HR117B/HR118B and HR14B/HR15B – was completed. Resistances were 11 Ohms and 17 Ohms respectively. Isolation measurements were performed between all pins associated with the 4 heater circuits within that connector. Measurements were taken on both the heater and LRU sides of the circuit.”
This operation confirmed that the heater circuitry and continuity was nominal and that the heaters themselves were capable of functioning properly.
After this was verified, the teams moved on to Operation 120 where they applied at 28v external power supply to power up the heaters and confirm their proper operation. During this test, electrical currents (and the current draw itself) through the heaters was monitored and determined to be nominal per measurements obtained in Operation 110.
With the heaters confirmed to be in nominal condition, the teams then initiated Operation 130 to test the output voltage of ALCA-2. This was accomplished by “using the commands to operate the drivers” within the ALCA.
Thus, according to engineering sources, “Troubleshooting completed to date verified no output from drivers on ALCA2. The resistance and isolation readings performed on the heater circuits looked nominal. PR (Process Report) to be generated to R&R the ALCA2.”
With these tests completed and verified, removal of ALCA-2 from Endeavour began at 1815 EDT Monday, May 2. Five hours 15mins later, at 2330 EDT, all but three anchoring bolts had been removed from ALCA-2.
During the removal, Operation 140 began to “locate the four splices to be removed for hi-pot (high potential) testing.” It took engineers almost 3hrs to locate all four splices for the thermostat wiring. To avoid further time loss in the future, the splices were all “red tagged”.
However, slack margin in the splices was discovered to be marginal at best. “Tech could not find enough slack to reterminate splices 414 and 415. USA-OEL is aware that new disposition is required to add an extension wire to reterminate those splices.”
By 0300 EDT Tuesday, May 3, removal of the ALCA-2 was complete, revealing minor cold plate damage behind the unit.
As engineering sources stated, “Initial cold plate inspection indicated [area] of concern. Mold impression of this area will be taken after 3:00 AM for further evaluation.”
STS-134 Specific Articles: http://www.nasaspaceflight.com/tag/sts-134/
Adding to this report, Tuesday morning’s NTD (NASA Test Director) update (L2) identified the area of concern on the cold plate as a “location of raised metal.”
Evaluations of the cold plate coupled with an Engineering Review Board decision to completely disconnect all ordnance on the Endeavour/STS-134 stack prior to the installation of the new ALCA delayed the installation process until early Wednesday morning.
Ordnance disconnects were carried out during 2nd shift (1600-0000 EDT) Tuesday. Following the removal of the failed ALCA-2, thermostat R&R was also undertaken inside Endeavour.
The cold plate repair was also undertaken and the results accepted by Quality Control.
The replacement ALCA box was mounted to the repaired cold plate and fastener torques were completed to specifications.
By that time, all ALCA-2 connector mates had been completed less 55P320 which was still needed for hi-pot (electrical safety testing used to verify electrical insulation) testing following the completion of splice work to the heaters.
By 0725 EDT Wednesday morning, three of the seven splices were complete and the other four were in work – two of which were “KSC (non-design) splices for added wire due to lack of slack issues.”
Failed ALCA-2 box investigation:
Following its removal from Endeavour’s aft, the failed ALCA-2 box was brought to the NASA Shuttle Logistics Depot (NSLD) for failure testing and analysis.
According to engineering sources, “The in-coming functional testing on the removed ALCA 2 was completed with errors; however, there were some issues obtaining a printout to verify the errors.”
A manual test with breakout boxes was then performed. This test verified “a 0.04 Vdc on pins *V and *X on J10, should have been approximately 27 Vdc.”
This confirmed troubleshooting data performed while the box was still installed on Endeavour and “indicated a failed component within the LRU (Line Replaceable Unit).”
“Module A49 was removed and bench testing of the SRU verified the Type III driver (the upstream common driver) was bad.” This confirmed the suspect condition/failure mode but did not confirm if the driver failed “under load.”
Likewise, Wednesday morning’s NTD status update noted that “Analysis of ALCA2 at the NSLD indicates a major ‘short’ occurred external to the ALCA. A fusible link internal to the ACLA was replaced and the ALCA powered up nominally.”
By Wednesday morning, the Engineering Review Board (ERB) had created a series of forward steps. According to the ERB, “The NSLD will be performing additional failure analysis on the driver inside the ALCA…. Depending on what they find, they might be able to confirm whether the ‘short’ experienced by the system is inside or outside the ALCA.”
Engineering was also directed to identify any and all times that ALCA-2 was powered up in the OPF following STS-130 and out at Pad-A for STS-134 processing. Connector mate/demate history will also be identified to “determine if we possibly performed a hot mate/demate during troubleshooting,” notes the ERB.
Furthermore, the ERB determined that if the NSLD is unable to isolate an internal failure of the ALCA-2, a hi-pot/Megger test on “wiring that could be associated with causing a ‘short’ into the ALCA” and “hi-pot/Megger testing on the heaters themselves” will have to be performed.
(All images via L2, with the installation shot via NASA TV. Extensive coverage is being provided on the news site and forum, driven by the L2 special sections – which includes a large APU-1/LCA troubleshooting area, from which this article was created from. Further articles to follow).