Engineers are continuing to work on IPR-47 (Interim Problem Report) – a leaky flange on Discovery’s crossfeed line plumbing – after a decision was taken to replace the two internal seals on the hardware. Earlier, torque checks found no issues with the system’s bolts, prior to the leak inadvertently ceasing via a pressure change in the crossover line. However, managers have opted to replace the seals.
The hardware in question relates to the crossfeed lines, which connect the left and right OMS (Orbital Maneuvering System) propellant lines at a point between the tank isolation valves and the bipropellant valves.
Each crossfeed line has two crossfeed valves arranged in parallel to provide redundant paths for propellant flow, allowing an OMS engine in one pod may be fed with propellant from the other pod – an allowance for the contingency of balancing the propellant weight in each pod, or if an OMS engine or a tank fails.
Although not deemed to be part of the root cause, the right-side OMS pod was removed after Discovery’s STS-131 mission to allow for repairs on a helium isolation valve in the Right Reaction Control System (RRCS). Notably, after the pod was re-installed in July, leak checks were classed as normal and no obvious loss of pressure was noted.
The leak was first noticed last Tuesday, when engineers literally smelled a fishy odor coming from Discovery’s aft – a sign of monomethyl hydrazine (MMH) fuel vapor in the area. Listing the problem as IPR-47, engineers checked the most obvious areas the leak may have been coming from, such the associated Air Half Coupling (AHC) flight cap, or Quick Disconnect (QD).
“MD679 flight cap bleed is complete. No bubbles noted and the center of the cap moves freely (no trapped pressure). Flight cap will be removed. We will attempt to clean the AHC to flight requirement and install a new cap for flight. If the AHC does not clean up, we will install a temp flight cap and pressurize the crossfeed using aft and pod purge, and repeat the process,” noted full engineering logs of the repair effort carried out so far (L2).
After checking the removed flight cap, only to find hardly any signs of a fuel leak contamination, engineers moved on to the flange area to see if that was the source of the leak.
“Technicians removed the MD679 flight cap and detected a trace reading at the source. TVC’s (Toxic Vapor Concentration) of the flight cap itself were 0.0 PPM. Technicians continued to obtain reading around the crossfeed-pod flange area,” added the logs.
“Engineering directed technicians to remove the insulation from the flange. When insulation was uncovered it was found to be contaminated with residual fuel. Insulation was removed and bagged as solid waste and routed to the fuel storage area.
“Flange was cleaned and fuel wipe was placed underneath for spill containment. TVC’s were obtained in the area approximately 2′ from the flange with an indication of 60 PPB. Technicians returned and noted small wet spot on fuel wipe approximately 1/2” in diameter. Light film of liquid was noted on the bottom of the flange at the 6 o’clock position.
“TVC’s were taken again and indicated 60 PPM at the source and breathing zone was 0.0 PPM. Using an Interscan no readings were obtained from the 8 o’clock to 4 o’clock position on the flange. Only readings were obtained at the 7 o’clock to 5 o’clock positions.”
With the flange confirmed as the source, the engineering effort picked up again on Monday night – with managers deciding to place the draining of the OMS tanks on hold until “high end” torque checks were carried out on the flange hardware. It was hoped – but not expected – that the bolts may have simply required an additional amount of torque to seal the leak.
“Space shuttle technicians found a small hypergolic fuel leak at Discovery’s right-hand crossfeed line flange located in the crossfeed area of the aft on October 12,” noted a managerial review of the issue (L2). “Following an engineering review board Monday afternoon, crews will perform a torque check of the six flange bolts (tighten the six bolts) and evaluate the system for leaks.
“If the bolts are found to be at the maximum torque value, technicians will drain all four fuel propellant tanks and the crossfeed. They then will disconnect the right-hand fuel crossfeed flange, perform an inspection and replace the primary and secondary seals (total of two seals).”
The engineering logs (L2) outlined the re-torque effort, giving an insight into what the engineer – who entered the aft via the 59-64 door, a triangular door located on the aft bulkhead of the orbiter – was confronted by as he tightened the bolts.
“On station to work flange bolt re-torque. Initial data prior to re-torque: TVC’s 0.5 ppm fuel at source and 0.2 ppm fuel 5 feet away. Condensate drip approximate every 4 minutes. FWD purge air circuit one set to 65 degrees,” the engineer reported, prior to noting all of the bolts were confirmed at their maximum torque. However, there was a surprise once the crossfeed was pressurized immediately afterwards.
“Flange re-toque complete, data during re-torque: C4 closed. Crossfeed press between 26 to 24 psia. No movement (all six bolts) at 105 in/lbs. FWD purge air circuit one set to 80 degrees. C4 open. Crossfeed press between 120 to 118 psia. TVC’s 12 ppb fuel at source, 0.0 ppm fuel breathing zone. No condensate drip in 25 minutes.”
With the engineer leaving the aft and noting he hadn’t seen any further condensation drips, the aft was revisited at around 7am local on Tuesday. Now the readings were indicating both 0.0 at source and 0.0 in breathing zone – showing the leak had indeed ceased.
“Last night the crossfeed pressure was dropped to 24 psia by adjusting the purge air down to ~70 deg. Previously the flange leaked at 1 drop every 4 minutes. Litmus paper also indicated pH of 9 indicating mostly fuel. The fuel odor was noticeable at 20 ft easily,” overviewed an engineering synopsis on the engineering logs.
“A retorque was done, first at 85 in-lbs; then at 105 in-lbs. No movement was made at either setting. As expected the leak slowed down at the lower pressure. However, when the crossfeed valve was opened the crossfeed line was subjected to 100 psid increase possibly seating the primary seal.
“There was no immediate increase in the TVC reading at this higher pressure. 2 hours later the leak is still not noticeable. I was able to get next to the doghouse door where previously it was inaccessible. The tech who had watched the fuzz leak no longer could identify it. Still in question, however, is the integrity of the secondary seal.”
STS-133 Specific Articles: http://www.nasaspaceflight.com/tag/sts-133/
With the problem apparently solved by the pressure change in the feedline, the Engineering Review Board (ERB) discussed their options, ranging from flying “as-is” to erring on the side of caution and pressing on with the replacement of the flange seals.
Initially, managers decided to move forward with a second test relating to the pressure in the lines, in order to gain additional data on the integrity of the seals. This is called a “mass spec” test.
“The ROMS fuel leak source has been identified as the ROMS fuel crossfeed flange. During troubleshooting last night, when the crossfeed pressure was dropped, the leak rate slowed as would be expected. The bolts on the flange were retorqued but no movement was detected by the technician,” noted a JSC memo late on Tuesday (L2).
“When the crossfeed valve was opened, the crossfeed line was subjected to 100 psid increase and the leak was no longer detectable. The theory is that lowering and increasing the line pressure allowed the primary seal to reseat (there is a primary and a secondary seal on this flange).
“The forward plan was being discussed at the time of the Center Director Pre-FRR (Flight Readiness Review). The preliminary plan is to lower the pressure in the line, remove the test port cap (test port is between primary and secondary seal), and check for leakage to check the integrity of the primary seal. The secondary seal will be checked by pressurizing the test port with He and performing a mass spec of the flange.
“If necessary, the flange can be inspected, polished, and the seals R&R’d at the pad with no anticipated impact to the November 1st launch date. Meetings on this topic continue.”
However, soon after, a managerial decision was taken to cancel the mass spec test, in order to conserve the contingency time which remains in Discovery’s pad flow. Engineers were ordered to immediately move forward with the preparations for draining of the OMS tanks, in order to allow for the changeout of both the primary and secondary seals on the flange, prior to backtracking to the original pad flow testing for leaks in the system.
“Flash: SCAPE (Self-Contained Atmospheric Protection Ensemble (suits used while performing task such as hydrazine loading)) operation begins at 2000 requires pad clear. IPR-47 mass spec for Wed 10am cancelled,” engineers were informed. “OMS deservice preps in work.”
UPDATE 1: OMS Tanks offload scheduled for Thursday, to be followed by seals R&R.
UPDATE 2: OMS drain still to be confirmed by late Thursday. Problems with Ground Support Equipment (GSE) noted, but yet to be confirmed by NTD. Engineers had expected the drain to occur as late as Friday, and pre-empted the four day contingency allowance is “tight”.
UPDATE 3: Offload complete just after 5am Friday. Seal R&R expected Sunday.
Providing the replacement and retest of the seals on the flange proceed without issue, Discovery will still remain on track to make her November 1 launch date. The worse case scenario would involve another leak being detected, from the flange hardware itself.
Such repair work has never been carried out at the pad before, and usually requires rollback and destacking in the Vehicle Assembly Building (VAB). Even if the work could be conducted at the pad, pressure would be placed on what is a short launch window for Discovery in November.
“There are four contingency days left in Discovery’s schedule that will be used to resolve this issue and support the targeted Nov. 1 launch date,” added a managerial overview. “If replacement of the seals is needed (as now confirmed), the work also should not impact the targeted Nov. 1 date.”
Further updates will be posted in the update thread, this article, with all related materials updated live into L2.
(Lead Photograph: Larry Sullivan, NASASpaceflight.com and MaxQ Entertainment.)