“The supply fans are located in the remote air building along the pad perimeter,” Blackwell-Thompson said in a media teleconference on April 3 after the scrub.
“We have redundant fans, and what they do is they provide air pressurization to different areas of the pad. This one in particular that had an issue feeds the PTCR (Pad Terminal Connection Room) and it feeds the Mobile Launcher itself along with the ECS (Environmental Control System) Room that is underneath the pad.”
The fans provide a continuous supply of air-conditioning to the different areas of the pad and Mobile Launcher but are placed at a higher speed during hazardous operations like cryogenic propellant loading in order to prevent a gas leak from getting into those areas. The primary fan had been running at that higher speed for several hours when a circuit breaker trip stopped it from running.
A troubleshooting team was sent to the pad but was unable to immediately resolve the issue, so it was decided to defer troubleshooting that problem and switch to the redundant supply fan. Unfortunately, that fan then also failed.
“It was a different fault, it was on the VFD (variable frequency drive) itself and the only way that they could see that was to go back into the pad; they did not have visibility into that remotely,” Blackwell-Thompson said.
Credit: Nathan Barker for NSF.
(Photo Caption: The engine/booster service platform is seen in April parked in position away from the vehicle for the launch day part of the Wet Dress Rehearsal. The platform fits in the flame hole of the Mobile Launcher to provide access to the RS-25 engines and the aft end of the Solid Rocket Boosters. It is lowered and moved away from the flame trench the day before launch prior to tanking.)
The April 3 attempt was scrubbed before any propellant loading started and the countdown was recycled back to T-6 hours 40 minutes and holding for a possible 24-hour turnaround while troubleshooting of the supply fans continued overnight.
The MMT again met early on April 4 for its pre-tanking briefing and went through the overnight work; the team at the pad was able to get both supply fans operating, with one at about 80% capacity for the attempt on April 4.
The MMT again gave a go to begin tanking the vehicle, but operations were delayed by an outside issue. “Just before our briefing completed, we did find that there had been an issue with the GN2 (gaseous nitrogen) supply to the center,” Blackwell-Thompson said.
The vendor that supplies gaseous nitrogen to the space center was unable to provide that supply for several hours; the broad outage was reportedly caused by a failure with a vaporizer that converts nitrogen that is stored in liquid to gas.
SLS uses gaseous nitrogen as a purge gas in the sections of the rocket around the propellant tanks when the vehicle is in the process of fueling.
The three human orbital launch site in the United States. Left to right: ULA’s SLC-41 for Atlas V/Starliner, SpaceX’s LC-39A for Falcon 9/Dragon, and NASA’s LC-39B for SLS/Orion. (Credit: Nathan Barker for NSF)
The nitrogen outage lasted long enough by itself that it would have consumed an entire launch window and scrubbed a launch attempt on that day; however, hours behind their timeline once GN2 was restored, the launch team was able to begin the tanking process.
The first commodity to be loaded was liquid oxygen into the Core Stage; the oxidizer is pumped from its storage sphere at Pad 39B to the Mobile Launcher and eventually the vehicle. Due to how cold the cryogenic propellants are, initially a low flow rate of propellant chills down the facility and vehicle lines to start the loading process, so they aren’t shocked by the sharp, several-hundred degree drop in their temperature from ambient temperatures to liquid oxygen temperatures.
In addition to chilling down the ground supply lines, liquid oxygen is flowed into the engine section of the Core Stage to also chill down the Main Propulsion System (MPS) and RS-25 engine lines down to cryogenic temperatures before beginning to pump it up the two large-diameter “downcomer” feedlines into the LOX tank that sits over a hundred feet above the engine section.
The launch team ran into a similar problem experienced during the Wet Dress Rehearsal for the Core Stage Green Run in December 2020. Initially during the April 4 attempt, the system wasn’t keeping the liquid oxygen temperature within the limits set to transition from chilldown to the “slow fill” phase where the propellant is pumped through the engine section MPS lines, up the feedlines, and into the LOX tank.
The Artemis 1 Orion/SLS vehicle is seen on its Mobile Launcher at Launch Pad 39B on March 18, hours after it was rolled out from the VAB. Credit: NASA/Kim Shiflett.
Although the problem was seen during the first LOX propellant load at both facilities, the cause at KSC was different. “You have to worry about anti-geyser[ing], so it has unique characteristics that make particularly the initial part of the load fairly challenging,” Tom Whitmeyer, NASA’s deputy associate administrator for common exploration systems development, said in an April 5 media teleconference.
“They look at the different temperatures and flow parameters and sequencing and they worked around that. It fundamentally comes down to LOX is fairly dense; you have to pump it. It takes a while to pump it, and you really have to worry about thermal conditioning as you go through the process, particularly at the start of the flow process.”
At KSC, the team has to supply cryogenic propellants to both of the SLS liquid-fuel stages, and after a couple of hours they adapted and adjusted their process. “Our [configuration] supports not just a Core Stage load but an [ICPS] load as well,” Blackwell-Thompson added.
The Mobile Launcher has additional propellant lines that branch off and up to the ICPS umbilical where they enter the stage; although the loading process for ICPS had not started, a small amount of LOX had flowed into the vertical lines. As they were trying to control the LOX flow and temperatures, some of the warmer, standing propellant in the ICPS transfer line was mixing back in with the Core Stage flow.
“It was a small amount, but as you’re loading and as you throttle that pump speed back, some amount of that LOX that’s headed toward that upper stage skid is going to come back down into your main flow stream and that affects your temperatures. And that was fundamentally what the issue was here,” Blackwell-Thompson said.
Things are going well with our @NASAGroundSys team and wet dress rehearsal. A critical part of this exercise is giving the team the space to refine the countdown procedures and getting their muscle memory down so that when we are ready to launch, it’s like second nature for them. pic.twitter.com/OvdKJn0CmM
— Jim Free (@JimFree) April 14, 2022
After adjusting the LOX loading control parameters in real-time, the launch team was able to transition from chilldown to the slow-fill phase and then, once the propellant in the Core Stage LOX tank reached a certain level, to the fast-fill phase.
At that point, although the countdown was several hours behind and there wasn’t going to be time to complete the test, the team pressed into liquid hydrogen loading operations following the same “risk reduction opportunity” philosophy. The attempt was then scrubbed when it was determined that a pneumatic actuation panel on the Mobile Launcher was configured incorrectly and LH2 chilldown could not begin.
It was later determined that the manual hand valve was in the closed position and should have been opened before teams left the pad for tanking. “As part of our system verifications, we have enhanced the walkdowns and the verifications of each of those, not just on the LH2 side, but across our systems and we’ll be talking about that at the pre-test briefing [for the third WDR attempt],” Blackwell-Thompson said in the April 11 teleconference.
“We’ve also added some verifications — in this particular case there’s some things we can do to ensure that we are in the proper configuration, and procedurally we’ve added those in as well.”
NASA wants to understand WDR results and vehicle health before deciding on next steps
The modified countdown has the same planned schedule as the first two attempts; for the simulated launch day, now planned on April 14, the Mission Management Team held their pre-tanking briefing early in the morning with a targeted, initial T0 of 2:40 pm Eastern / 18:40 UTC.
However, teams were quickly behind schedule by about an hour due to another gaseous nitrogen (GN2) issue at the pad that prevented switching the rocket from purge air to purge GN2.
Although a timeline with details of some countdown milestones was finally published by NASA on March 31 after requests for information that began at least as far back as 2017, the civilian space agency will continue to neither show nor tell the public about what happens during the WDR, citing export control policy.
As with the first two attempts, camera views and audio of the real-time situation as it happens will not be provided publicly. During the first two attempts, updates on social media and blogs were sporadic, especially when anomalies occurred, and the situation was unclear for long periods of time in between posts.
NASA is broadcasting a static camera feed from the pad perimeter, but the view only provides a broad perspective optimized for daylight; the stream has no operational audio and doesn’t show the things that move during the countdown, since those are also deemed secrets.
Credit: NASA.
(Photo Caption: One of the only images NASA has released of the Artemis 1 vehicle as seen from the top or near the top of the Mobile Launcher in March at Launch Pad 39B. Images, access, and details throughout the Artemis 1 launch campaign have been severely restricted by export control policy.)
With the ICPS excluded from significant parts of the third WDR attempt, the path forward after the test is unclear. The vehicle will be rolled back to the VAB for post-test inspections and maintenance, including on the ICPS check valve, but the agency and contractors will discuss whether to press ahead to a launch attempt or not.
During the April 11 teleconference, Whitmeyer said he wanted to at least see what happens with the remainder of the WDR test first.
“We really want to see how we do on Thursday [April 14], so we’re not trying to get ahead of that, and we want to get the vehicle back [in the VAB] and we want to inspect it; those are really important to us,” he said. As Chief Engineer for the SLS Program, Blevins noted that the test will be used to help anchor analytical models with flight hardware data.
“We call it Wet Dress, and so we’re focusing a lot on liquids and propellants here, [but] there are so many things that we’re checking and most of those answers that we’re looking to get aren’t ‘yes or no,'” he said. “They are ‘what are the temperatures at this interface in these conditions and how does that validate our model?’ — to expand that to the full flight envelope that we might launch.”
“And many of those questions will be answered for ICPS when we flow these cold propellants across the QD (quick-disconnect) to the point just getting to fast fill.”
Blevins also noted the test results themselves could also make decisions about the next step clearer to see, since the modified test will still cover everything except the second stage and they may learn something new that takes precedence over what they’ve already learned during their WDR work so far. “We’re ignoring a preponderance of the systems when we focus on ICPS that may tell us whether we need to do another Wet Dress or not,” he pointed out.
“If that missing [ICPS] data were the only missing data and every model acted perfectly, I think we would look long and hard at whether we needed to or not, but there’s a lot of information we’re going to be gathering on Thursday.”
Lead image credit: Nathan Barker for NSF.