The push to return Discovery back to launch operations has begun, with initial troubleshooting via five test cycles of the LH2 inboard fill and drain valve at the pad complete, following issues with the PV12 (pneumatic valve) during STS-128’s second launch attempt. The goal is to gain confidence the issue is instrumentation – as opposed to the valve suffering from damage – otherwise the launch will be postponed to October.
STS-128 Processing Latest:
Following Tuesday’s scrub, called for by the MPS-04 Launch Commit Criteria (LCC) relating to the cycling of the Fill and Drain Valve, engineers have been waiting for the External Tank to inert – following its draining of propellants.
“S0007 Launch Countdown. Launch was scrubbed last night due to an LCC violations. The team is working 48 hour Scrub Turnaround operations for a Launch attempt on Friday, August 28th at 0017 EDT. Scrub Turnaround operations are proceeding well,” noted Wednesday processing information on L2.
“Critical vehicle safing and securing is completed. LO2/LH2 system safing/securing is complete. LH2 boil-off in work.”
The tank needs to be inerted of its residual propellants, in order to reduce the hazards for engineers at the pad.
“IPR 77 (Interim Problem Report): (Lack of a closed indication on PV12) Update: LH2 I/B Fill/Drain troubleshooting will begin this afternoon once LH2 boil-off and inerting is complete,” added processing information. “Troubleshooting plan is to perform multiple dry cycles of the valve to test if the failure is instrumentation or hardware. The results of the troubleshooting will dictate the forward plan.
“Next attempt (with a 48 hr scrub turnaround): Friday Aug 28th, 00:17 Eastern. Tyveks due Saturday (no need to roll RSS back. Have not looked at them with binoculars to see any issues – will do that to make sure no upcoming problem). PRSD (Power Reactant Storage and Distributation): Need to top off on Sept 1st. Have range thru the 30th. Will be purged/inerted by 5:30 PM Eastern on Wednesday. “
Work will also be conducted on one of the Tail Service Masts (TSM), following the observation of a leak signature during the countdown. This issue was expected to clear once the tank had entered stable replenish, and thus was not serious enough to cause a scrub in the event of no other issues.
“IPR 78: TSM leak detector troubleshooting is scheduled for this afternoon also,” added the processing notes. “He (Helium) leak checks will be performed using a helium leak detector on all flexhose to hard line interfaces of the 8 inch fill and drain line as well as all fill and drain pressure sensing line connections.”
Another issue was noted on the LH2 TSM, when a leak detector failed on the T-0 umbilical. The T-0s are located inside the TSMs – and the leak detector will be changed out.
“IPR 79 was picked up by HGD for LD28 failing off scale low. The leak detector monitors LH2 T-0 TSM umbilical cavity. Will R&R the transmitter in MLP (Mobile Launch Platform 43A.”
Update: The five cycles of the valve have been completed at the pad – with the data to be evaluated on Thursday.
PV12 Issue and Troubleshooting (preview of cycling and managerial discussion):
The LH2 Inboard Fill and Drain Valve 50V41PV12 (PV12 for short) is located in the orbiter’s Main Propulsion (Liquid Hydrogen) System inside the Aft Fuselage of Discovery. These 8-inch internal diameter Bi-Stable Pneumatic valves are classed as Criticality 1 hardware.
There are four of these valves inside the MPS, with the LO2 system housing the PV9 LO2 Outboard Fill/Drain Valve and PV10 LO2 Inboard Fill/drain. The LH2 System houses the PV11 LH2 Outboard Fill/Drain and the failed PV12 LH2 Inboard Fill/Drain.
The requirement for the nominal operation and cycling of the valve mainly relates to the ability to offload propellant in the event of – for instance – a pad abort, where detanking needs to take place in a timely manner. Offloading could still occur via the alternate PV11 route, but would take twice as long – which would be undesirable.
“Description of anomaly: Cycled the LH2 I/B Fill and Drain Valve (Pneumatic Valve PV12) closed to go into reduced fast fill. Have both an Open and Close solenoids on the valve. When cycled the valve closed, lost the open position but never picked up the closed,” noted memos acquired by L2.
“A few years ago we had a valve failure where the driver-to-driver galled and the valve stuck. Don’t want one of these valve to stick closed and not be able to drain the tank. The LCC is clear – if lose the close indication, open the valve with unnecessary cycling and drain the tank and evaluate the data from that point on, which is what was done.
“Got the valve open and got into drain position. Have since started looking at the valve performance data. The timing of the valve on the attempted cycle to closed and to the open gives the appearance the valve cycled correctly. The valve probably made it to the closed position but something made the indicator not pick up the closed position.”
A possible failure scenario is galling of the “Rack and Pinion” actuator gearing inside the Al Alloy body of the valve, simply due to cyclic fatigue/wear and tear over many cycles – which is a candidate, given the PV12 valve was set to be replaced after Discovery’s return from STS-128, due to the end of its life cycle.
The problem could also be an electrical failure, caused by the controlling solenoids failing to actuate to “Open” or “Close” the valve – but this is highly unlikely since the Aft Fuselage has remained closed off from any human traffic, as noted by one MPS engineer who provided notes to L2.
The hope is the failure was related to instrumentation – such as a failure to properly register the cycling of the valve – as opposed to the valve becoming stuck via “galling”, which would require the valve’s replacement.
“R&R of the valve requires Rotation of the RSS (Rotating Service Structure), then Aft Access and Aft Platform installation to get to the big valve just inside the AFT fuselage,” noted the MPS engineer on L2. “Then they have to remove foam from around the flanges (2) and of course they need a certified replacement valve.
“Once the valve is R&R’d they must fully leak test the new valve, perform timing tests, leak check the 750 psi Pneumatic lines and verify both internal leak checks and seat leak checks, plus LH2 System Decay tests.”
Should the valve require replacement, a spare is available on site.
“Have lost confidence in the closed valve position indicator. Once get the tank drained and safed, it should be safe to cycle the valve to try to understand the valve’s health. Perhaps compare to OPF (Orbiter Processing Facility) test performance. Look at pneumatics, timing, decay data,” added notes on L2.
“Plan to purge and inert (after boil off) and then do the valve cycling at ambient. No FOD (Foreign Object Debris) concerns. On time and cycle, this is the 16th flight of this valve and would be the last flight per time and cycle requirements. Have a spare valve and a spare actuator (1 of each).”
So far, engineers believe the issue is instrumentation, which would good news for the ability for Discovery to launch prior to the end of the window (end of the month).
“Data on failure indicates this is a position indication anomaly, not an actual valve cycle failure. Key data supporting this position (manifold pressures, temps, haz gas data, reg pressures, valve initial move times) and the valve’s cycle time and LPAT data were reviewed,” noted memos on L2 – sent during the forward plan meetings.
“The valve will be cycled 5X to verify no evidence of cycle time degradations (and determine if the PI is still not functioning). Also a LH2 manifold decay test will be run with the valve closed to verify no seat leakage.
“We are also looking at options to perform a seat leak test, although there are concerns this would be a pretty gross leak check (lots of GSE (Ground Support Equipment) leak sources) and could introduce moisture into the system (and require reperformance of dew points).
“Both the cycle times and decay test must have ‘successful’ outcomes to support a 48 hour scrub (i.e., this is a position indication failure, not galling of the valve or something (like contamination) preventing the valve from cycling fully closed).
“KSC is scheduling fall-back options to enter the aft and perform testing to determine if full valve replacement or valve actuator replacement are necessary (this will obviously be longer than 48 hours).”
Options based on Outcomes:
At present, engineers can turn Discovery around for a Friday morning launch attempt, providing the cycling of the PV12 valve proceeds without issue, along with good leak decay checks. Based on positive Wednesday evening testing – that appears to be hopeful. All other options rule out a launch until October, due to the range cutout at the end of the month.
“Option 1: 48 hr scrub with no further work on valve except ambient valve cycling (5 times) and decay checks – assumes results ok for launch. Have good history that valve performs about the same in ambient and in cryo,” one JSC memo outlined.
“Option 2: Go into the aft with platforms for LPAT (Low Pressure Actuator Test) and discover have an indicator problem. Would be able to make a Sept 3rd launch (ignoring window constraints).
“Option 3: Go into the aft with platforms for LPAT and discover have to change out valve. With valve change-out, would be Sept 5th launch date (assumes no pyro disconnects) (ignoring window constraints).”
The decision point is expected at the Mission Management Team (MMT) meeting at noon Eastern time on Thursday.
“KSC will get into valve troubleshooting (ambient cycling) at 17:30 Eastern on Wednesday. Will have the data by NLT (No Later Than) 20:30 Eastern Wednesday,” another memo noted. “In parallel will be doing S0007 (Launch Countdown) work for the 48 hr turnaround. Already have scheduled an MMT for 11 Central, noon Eastern Thursday.
“Plan is to build the pass/fail matrix and flight rationale completed, with a tagup with OPO (Orbiter Project Office) (where) we will talk the test procedure, the pass/fail criteria matrix, and the flight rationale (including implications of ambient versus cryo, post MECO (Main Engine Cut Off) dump if the valve is galled, why the testing indicates its PI and not galling or contamination/leakage, etc.).
“We’ll probably meet Thursday morning before MMT to discuss the test results and if we are ok to fly or not.”
Further notes will be added as troubleshooting progresses.
L2 members: Documentation – from which the above article has quoted snippets – is available in full in the related L2 sections, now over 4000 gbs in size.