Engineers still believe they only have an instrumentation problem with the LH2 PV12 Inboard Fill and Drain Valve on Discovery, but further data is required – resulting in a 24 hour hold to STS-128. Testing on Wednesday night – involving five cycles of the valve – has aided engineering confidence the valve is undamaged, but flight rationale is still being worked.
STS-128 Processing Latest:
Discovery was in a scrub turnaround stance, following the second launch scrub relating to the valve issue that caused a breach of the MPS-04 Launch Commit Criteria (LCC).
“Orbiter: OV-103 / ET-132 / BI-139/ RSRM 107 (Pad A): S0007 Launch Countdown Scrub Turnaround operations continue. The launch team is (now was) continuing to work towards an attempt (early on Friday),” noted processing information on L2, ahead of the 24 hour delay.
“S0007.300 LCD Scrub Turnaround: ET LH2 boil-off was completed at 1336L yesterday. LO2 and LH2 tank replenishment was completed last night. COMM system activation was completed at 0030L this morning. Ascent switchlist is scheduled for 0700L. Air/GN2 changeover to support ET loading is scheduled for 1100L. ET LO2/LH2 loading is scheduled to begin at 1457L.”
Along with the valve, a smaller issue – relating to a leak in the Tail Service Mast (TSM) – required engineering work ahead of the third launch attempt. With no leaks detected during checks on Wednesday, further work has been deferred to post-launch.
“IPR 78 LH2 TSM leak detectors update: He Leak checks were completed on all flexhose to hard line interfaces of the 8 inch fill and drain line as well as all fill and drain pressure sensing line connections and no leaks were detected,” added processing information.
“The IPR will be transferred to an MLP (Mobile Launch Platform) IPR (Interim Problem Report) for additional troubleshooting post launch. He leak checks will be performed using a helium leak detector.”
Also associated with the TSM – an issue with a T-O Umbilical Leak Detector (LD) has resulted in its successful replacement.
“IPR 79 LD28 failed off scale low update: LD transmitter was R&Red and retested successfully. The IPR was transferred to Pad A GSE (Ground Support Equipment) IPR and will be kept open for evaluations and upgraded and closed post launch.”
Two new IPRs – one relating to a data stream, and the other requiring the changeout of one of the middeck payloads – are both no issue for the countdown.
New IPR 81 to Pad Electrical for an ECS secondary SCADA (Supervisory Control and Data Acquisition) server that rebooted. (Engineers) reported console readings not updating real time, but hardware seemed to be functioning nominally. Readings recovered after reboot. IPR transferred to a Pad A IPR,” added processing information.
“New IPR 82. Glacier mid-deck payload temperature toggling out of limits. Troubleshooting determined that Glacier R&R is required, and will be reperformed.”
The launch was moved to a 11:59pm Eastern as a result of the additional evaluation of data.
“The STS-128 Launch attempt for tonight has been cancelled to allow the team additional time to review valve cycle timing data and develop additional rationale for flight,” noted the Mission Evaluation Room (MER) memo, acquired by L2.
“Will be a 7:00 pm CDT OPO (Orbiter Project Office) meeting to discuss the status of the data review and flight rationale. The next launch attempt will be tomorrow Friday Aug 28th at 10:59 PM Central.”
Five successful cycles of the valve took place on Wednesday night, boosting confidence the problem is instrumentation, and not a problem with the valve itself – such as damage via life cycle spalling, resulting it the valve jamming.
Despite this, work still has to be conducted on understanding the root cause of the failed indication on the VPI (Valve Position Indicator) during the second launch attempt’s countdown, thus to allow flight rationale to be accepted by the Mission Management Team (MMT).
“IPR 0077 LH2 In-Board Fill/Drain PV12 update: Troubleshooting is CTD (Countdown). The valve and valve position indicator (VPI) operated nominally throughout the dry cycling testing and all leak checks were within specification,” confirmed engineering notes on Thursday morning.
“An ERB (Engineering Review Board) was held last night to discuss the troubleshooting results. The engineering community is still evaluating previous VPI failures, the need for any further troubleshooting under cyro conditions, and the course of action if PV12 failed to open and we had to detank through PV13.”
Once all the data has been successfully gathered, managers will discuss the failure, the consequences, and the forward plan for the valve.
“MPS LH2 Inboard Fill and Drain Valve Closed Position Indication Failure During STS-128 Launch Attempt #2: Problem Description: During the transition to Reduced Fast Fill, the inboard fill and drain valve (PV12) closed position indicator did not come on when the valve was commanded closed,” noted one of several pre-MMT presentations, all available on L2.
“This was a violation of LCC MPS-04. A 48 hour (min) scrub was declared and LH2 proceeded with boil-off operations. The LCC was revised in 2004 to not allow a repeat cycle attempt to attain the closed position indication.
“Based on a concern with galling failure mode within the valve driver mechanism. Previous LCC allowed for a repeat valve cycle attempt given the following conditions were met: No Sustained reg-out demand occurs. No sustained increase in Aft Haz Gas helium concentration. Initial move time <5.0 seconds. All the above parameters were satisfied.”
As per usual with engineering discussions, understanding the negative outcomes of a valve that can’t be cycled with confidence – and more so can’t be confirmed as fully closed – were also outlined in the presentation.
“Concern: Launching with a valve that is not in the fully closed position is a Crit 1 failure mode. Hydrogen subsequently leaks into the reverted (empty) 8 inch fill and drain line during or following terminal count.
“Trapped hydrogen boils-off and subsequently leak back into the manifold through the same leak path. Gas bubble is ingested by the SSME (Space Shuttle Main Engine) and results in SSME uncontained failure.
“Exposure at SSME start is most severe. Launching with a valve that is not fully closed is a Crit 1R failure in loss of redundancy to overboard leakage (with the outboard valve PV11). Subsequent de-tank affects in requiring drain through the 4” line.”
Also part of the concerns with unsuccessful cycling of the valve is the ability – or lack of confidence in the ability – to open the valve.
“Question: If we launched and the Inboard LH2 Fill/Drain failed to open as part of MPS Dump Start, what are the impacts?” asked the Orbiter Project Office presentation, pre-empting questions at the MMT.
“Uphill: If the Inboard F/D failed to open at MPS Dump Start (MECO + 2 minutes), we would call to take the Backup RTLS Dump Valve switch on Panel R2 to OPEN for 3-6 minutes after Dump Stop (per Flight Rule A5-205), then have the crew take the switch to CLOSE and then GPC (General Purpose Computer).
“This would allow some extra residuals to vent through the 1.5” Backup Dump Valves, and the automated vacuum inerts at ~Dump Stop + 25 minutes and MM106 would remove the rest of the residuals from the LH2 manifold. There should be no further mission impacts. We do get another inert through the Backup Dump Valves as part of the nominal Entry sequence, so any remaining residuals would be removed at that point.
“TAL (Trans Atlantic Abort): If the Inboard F/D failed to open at MPS Dump Start (MM 303), you would have 1.9 pounds of extra LH2 residuals in the manifold at touchdown, resulting in an Expedited Powerdown/Mode V egress. Per the nominal TAL/RTLS dump sequence, the Inboard closes 20 seconds after Dump Start anyway in support of the pressurized LH2 dump, so the overall impact to residuals is fairly minor.
“RTLS (Return To Launch Site Abort): Same impacts as TAL, however you would have 2.9 pounds of residuals at touchdown.
“All Cases: If we had a subsequent, independent Backup RTLS Dump Valve failure post MECO, the LH2 manifold relief valve would provide a relief path to prevent a manifold overpressure condition. You would have to have a 3rd independent failure of the relief path (either relief valve or upstream isol valve) in order to get into a Crit 1 overpressure condition. I consider this to be extremely unlikely, since you have to have 3 independent failures of Crit 1.”
However, such undesirable scenarios are based on events that are even more unlikely, thanks to the data already acquired via the scrub and the subsequent testing – which shows the valve acting in a nominal manner, further suggesting the error being seen on console is instrumentation.
“STS-128 Data Supporting VPI Anomaly: Limited set of vehicle instrumentation available to demonstrate the valve successfully cycled fully closed. Upstream (Orbiter inlet) and downstream (LH2 manifold) pressures. LH2 manifold temperature. MPS 750 pneumatic reg pressure responses to demand. Haz Gas aft helium concentrations. Valve response timing (initial move time),” added the main MMT presentation.
“Data comparisons between the first and second loading and between previous loadings were performed and show nothing anomalous with these parameters associated with the VPI anomaly.
“Low Pressure Actuation Test (LPAT) results for this valve show no indication of pronounced galling. Valve historical cycle times have not changed.”
Also, the vast database of the shuttle’s history books show that galling – where the valve hardware becomes damaged – is a rare occurrence. Historical records also show that “switch position anomalies” have occurred in the past – meaning Discovery’s condition would not be unusual.
“Historical Failures of MPS Fill and Drain Valves: Switch Position Anomalies: Improperly installed actuator end cap shims resulted in excessive play in the drive shaft and loss of contact with the switch cam,” added one of the MMT presentations.
“Conditions: Short end cap screw Mod. Bent switch arm caused by improper assembly. Original vendor work or during end cap screw mod.”
However, despite the data pointing towards instrumentation as the culprit, managers are not rushing into launching Discovery without the full understanding of the root cause, and more so, acceptable flight rationale to proceed towards launch.
L2 members: Documentation – from which the above article has quoted snippets – is available in full in the related L2 sections, now over 4000 gbs in size.