The NASA Exploration Ground Systems (EGS) program is now working on collecting, analyzing, and applying lessons learned during the long Artemis I launch processing flow to streamline future launch campaigns, beginning with the launch of the first Artemis crew on the Artemis II mission in a couple of years. The first Space Launch System (SLS) rocket lifted off early on Nov. 16 from Kennedy Space Center (KSC) in Florida and sent an uncrewed Orion spacecraft to the Moon.
The launch team engineers staffing the Integration Console were in the middle of the troubleshooting and deliberations through all the Wet Dress Rehearsal and launch attempts in 2022. They are now coordinating the lessons learned effort at KSC, with one of the goals being to improve the “launch availability” of the Orion/SLS vehicle.
Applying lessons learned to improving launch availability
The EGS, Orion, and SLS Programs are continuing to review overall performance from the Artemis I mission. Once Artemis I cleared its Flight Readiness Review in August, it took three attempts to get to liftoff. One of the goals of the lessons learned effort is to reduce the number of attempts necessary to launch, increase the chances of liftoff on a given launch day, and improve the overall “launch availability” of the Orion/SLS vehicle.
“From a lessons learned perspective, we are taking that very seriously,” Anton Kiriwas, EGS Senior Technical Integration Manager and Senior Launch Project Engineer (LPE), said in a March 20 interview with NASASpaceflight. “I’m actually leading up for our program a very comprehensive set of lessons learned, going all the way back to the offline processing for Orion working through the processing side, but the main focus is on efficiencies.
“We certainly learned a lot about the systems that maybe in Shuttle we had just depended upon because they had been used so frequently that didn’t necessarily need a specific test or checkout. Well, [now] we’re adding those tests and checkouts because we want to find those things earlier [in processing] rather than once we get into a launch attempt, so we’ve added a number of kind of system and facility checkouts, facility being the big one that we’re really focused on for a couple of the issues that we had for Artemis I.
“It’s a lot of minor things like that, procedural updates that we think can improve things,” he noted. “We are working with the SLS Program, certainly, on the hydrogen leaks that we saw.
“I don’t know that that will be an Artemis II issue, but we are going to be looking for Artemis III, Artemis IV, on improving that so that we can move away from the kinder, gentler approach that we’ve taken, and try to get ourselves back to a more nominal cryo loading profile. We’re taking all of that into consideration, but at the same time trying to balance that with [the reality that] there’s a lot of work just to get back to Artemis II readiness.”
“We also learned quite a bit from an imagery perspective,” Tony Bartolone, EGS Launch Project Engineer, also noted in the interview. “We did have some issues with some of our cameras that we discovered after launch that we’re putting a concerted effort into correcting and making sure that, for Artemis II, we have better imagery coverage and more reliability from imagery assets so that we can collect a lot of the high-speed imagery that we unfortunately didn’t get reliably this launch.”
EGS is also coordinating a review of the launch commit criteria (LCC) to see what needs to be updated now that there is real launch and flight data for EGS, Orion, and SLS. “We are asking the owners of the existing LCCs to go back and take another look,” Kiriwas said. “It’s one thing to base your requirements on theoretical models and analysis; now that we’ve got some performance data, we [want] to go take another look.
“Certainly to make sure that we got them right the first time, but also to see what we can do about launch availability, [looking for] some of these requirements that may have been very restrictive based on the lack of real-world grounding of those models. We’ve now got a lot better picture of the performance of the vehicle, [so we’re asking] can any of those be expanded to give us a little more room in a launch countdown to prevent some of these LCCs that we may have had to write waivers for previously.
“And, we are seeing that in some cases — where we’re able to expand limits potentially to get us some more launch availability,” Bartolone added. “We’re also looking for can we commit things earlier [in the countdown] than we previously had.
“We were very conservative on the first flight of the vehicle, wanting to monitor things later [in the countdown] to make sure that we didn’t take any unnecessary risks. Now with that data that Anton mentioned that we have, that is definitely giving us a good opportunity to go back and look and see, is it really required for us to take, for example, a set of measurements down to engine start where we can potentially understand that that system has no dynamic changes after a minute and thirty seconds, for example, and we don’t have to go and risk possibly stopping launch for something that may not have any consequences.”
One area that got a lot of focus during the Artemis I launch campaign was Core Stage liquid hydrogen (LH2) propellant loading. After seeing leaks in the quick disconnects of the LH2 tail service mast umbilical, the launch team looked to adjust the LH2 loading procedure. “This was a change to the loading profile that we had done to minimize the number of temperature and pressure changes across that seal,” Kiriwas explained.
“So we took that original profile that we had planned, and we did extend it from a timeline perspective, and then we modified it from a procedural and a software standpoint. The first time that we did that, we had to go utilize a number of manual operations within the software, in some cases actually bypassing some of the automated parts of that software to enable the kinds of changes [we wanted to make].
“One of the downsides with automation is when you automate it, it can be difficult to ‘un-automate it,’ so the team spent a lot of time within our development lab prototyping how they would do that, and then we did it for the first attempt,” Kiriwas noted. “Going into the tanking test on [Sept. 21], though, we wanted to make sure that we were in the best situation from a team perspective, software perspective, procedure perspective.
“[After the second launch attempt], we took about three weeks to go rewrite the procedures [and] go actually update the software, finding the best balance between not wanting to change too much of the software and at the same time keep the most amount of automation we could for those sorts of changes that we were doing, and then get the team retrained. So, we actually ran several days of just non-stop training with the team, making sure we went through multiple [propellant] loads against a simulated environment.”
The end-to-end LH2 loading and thermal conditioning process was fully demonstrated in the Sept. 21 tanking test, and a change from Shuttle-era rules was instituted after the engineering community looked at hydrogen flammability tests performed early in the last decade.
“The one other thing that we went back and looked at very closely was our requirements for hydrogen concentration in this cavity where the umbilical meets the vehicle and went back and looked at all of our available test data that we have from previous lab tests that we had done with hydrogen back at the end of the Shuttle program and some other test data from industry,” Bartolone explained.
“Four percent had been kind of the line in the sand since I started back in the Eighties; it was just a given, and it was always unquestioned,” Phil Weber, who was the Lead Launch Project Engineer for EGS for Artemis I and has since retired, said. “But, our [hazardous] gas team had brought up that 4% for that particular location seemed very conservative, and like Tony just said, the engineering team had gone off and done series of field tests trying to ignite in an air environment, and they found that couldn’t make it ignite up to 16%, so Tony and the guys settled on 10%, giving yourself a 1.6 factor of safety.
“For that cavity, we were able to elevate the allowable limit up to 10%, and that got buy-in from John Blevins and the SLS team as well. He had his combustion lab expert go through and peer review all the analysis, and everybody concurred.”
After reviewing data from the tanking test and launch attempts, the 10% hydrogen concentration will be the limit going forward. Going into the tanking test, the higher concentration was allowed for a few minutes’ time, but in the future, there is no time or duration limit.
“We had initially talked about having that [higher limit] be only for a certain duration of time, but ultimately, with the review of the test data that was made available, we actually went away from that entirely,” Bartolone said. “[We] said as long as we stay under that 10% [concentration level], it doesn’t matter how long we’re between four and ten percent.
“That 10% has a lot of conservatism built in it, with 16% still not being able to sustain a flame based on the laboratory test data that we used to derive the 10% limit for the (launch commit criteria). That permanent change that we’re going to be using for Artemis II and subsequent is for both the TSM cavity for the LH2 as well as the intertank umbilical for the GH2 vent.”
Kiriwas added, “This is in an enclosed environment that is being purged with [helium], so it’s not like we’re allowing 10% into an oxygenated environment that would cause the fire triangle that we talk about sometimes. If that were to then escape into oxygen, we know from the physics that hydrogen will very, very rapidly disperse into well under 10% and also most likely under the 4% limit that we originally had.”
The rules on the allowable limit also take into account different circumstances, such as if not only hydrogen but also oxygen were to be detected in the umbilical cavity, that should theoretically only have the helium purge gas in it. “The 10% [limit] is with no oxygen detected,” Bartolone said.
“If we were to see an increase in oxygen, then the 10% limit would ratchet down to make sure we’re retaining the appropriate factor of safety to preclude any kind of explosive or flammability concerns there with oxygen present, but we never saw any oxygen present in any of the loadings before we made the change to the LCC or after.”
(Photo Caption: Ground-side umbilical plates at the Michoud Assembly Facility for the SLS Mobile Launcher. Boeing is the prime contractor for the Core Stage and also produces the plates that connect to the stage when it is stacked for launch.)
The hydrogen tail service mast umbilical connection was disconnected and reconnected multiple times during the Artemis I campaign. The groups responsible for the ground and vehicle hardware worked together to add tools to measure the fit between the ground and vehicle umbilical plates. SLS Core Stage prime contractor Boeing produces both sets of plates in New Orleans at the Michoud Assembly Facility, and new ground-side plates are expected for Artemis II along with the new Core Stage for the flight.
“We’re getting brand new plates with brand new QDs (quick disconnects) for Artemis II,” Bartolone said. “I don’t know how many mission sets they are producing, but for at least a few flights while we build up inventory, we’re going to be getting new plates and QDs with each Core Stage delivery.”
Weber noted, “The team working here with Core Stage and Boeing [have] come up with some tooling where we can do measurements on the QDs themselves to measure the compressive force in the bellows, so you’ll know if you have the right amount of compression into the seal to maintain a leak-tight configuration, because that was one of the things that Tony had a lot of experience on in Shuttle with some hydrogen leaks where they found that the QD itself wasn’t as strong, force-wise, to hold it together.”
“Back in Shuttle you may remember the GUCP (Ground Umbilical Carrier Plate) leaks that we had at the end of the program, we had them across a few of the different missions towards the end,” Bartolone explained. “We built some tooling back then to go measure the QD-poppet forces and the compression strength of the poppets and the springs in there, and so we actually leveraged that and built a tool to go investigate the strength of the poppets of the QDs that we have on Artemis.
“So we’re definitely applying lessons that we learned previously; different architecture, a little different interface, but it is still all applicable from a hydrogen perspective. Making sure you’re understanding those compressive forces on the interfaces is really key. We’re going to get different plates [and] different QDs, [so] it’ll likely have a little different spring force, so we’ll see how Artemis II performs.
“A tanking test for Artemis II is going to help us determine if we are still in a good position with our procedures and things for going into the next launch attempt for this mission,” Bartolone added.
Post-liftoff Mobile Launcher damage
The Mobile Launcher was expected to take a beating from the approximately 8.3 million pounds of liftoff thrust produced by the four RS-25 Core Stage engines but especially from the two Solid Rocket Boosters (SRB). The engines actually throttled up from 100% of rated power to 109% immediately after liftoff, and the SRB propellant is tailored to reach maximum thrust about 20 seconds later; the ML was designed with these forces in mind, but SLS still left the umbilical tower and launch platform with a few scars.
The elevators in the Mobile Launcher sustained more damage than expected by the blast effects of SLS ignition, liftoff, and climbout, but the launch also broke a pneumatic line on the Mobile Launcher supplying some of the gaseous nitrogen to different pad systems. “After launch, we lost our gaseous nitrogen supply, which delayed the flow of water that would have rinsed off some of the SRB residue early,” NASA EGS Program Manager Shawn Quinn said in a March 7 media teleconference. “Because of that, some of our pneumatic lines got corroded, and we’re right in the middle of replacing those.”
The gaseous nitrogen loss was different than the issues that delayed the Wet Dress Rehearsal from April to June. Kiriwas explained, “This was a specific loss of that nitrogen both for a valve from an actuation perspective and from a purge perspective that happened right on the Mobile Launcher due to the damage incurred at launch, as opposed to the one that we had had earlier was back at the supply side where they had lost that capability to supply that nitrogen to us.”
Bartolone added, “We were getting nitrogen to the pad; it’s just that it was not reaching the end items to be to actually affect the valve configuration changes we needed in order to activate fire suppression water or the safing of some of the key cryo systems after launch.” After liftoff, control and authority of the vehicle and flight systems automatically transfers to Mission Control at the Johnson Space Center in Houston, but EGS remains in charge of the ground systems for the Mobile Launcher and Pad 39B.
“[The] GLS (Ground Launch Sequencer) handles that immediate post-liftoff securing, but after that, it gets handed off to the individual subsystems,” Alex Pandelos, EGS Launch Integration Operational Project Engineer (OPE) and primary GLS engineer for Artemis I, said. “You still have the GTC, the Ground Test Conductor, that’s looking at the status of our ground systems immediately after liftoff, and you still have teams that are busy doing what we would normally consider non-critical safing. “They bubble up any issues, including this one, immediately to our Ground Test Conductor.”
The nitrogen supply was still flowing into the Mobile Launcher, but with the line break, it was leaking out from there into the air, which created additional complications in safing the systems. “That posed an extra hazard because we knew that nitrogen was flowing and we weren’t exactly aware where it was flowing to, and so that poses a safety hazard from an asphixiation perspective,” Kiriwas explained.
“It slowed things down,” Bartolone added. “Not knowing where the nitrogen was going, it delayed some of our post-launch pad entry work and trying to get folks out there and needing to make sure that obviously we maintained their safety. Sending them out there to go do any manual safing that had to happen post-launch that is always a part of our post-launch work to go do, but it added another layer of challenges that we were not anticipating.”
Mobile Launcher-1 is currently being refurbished following the Artemis I launch and is also receiving a set of upgrades and modifications to fully support crewed Orion launches beginning with Artemis II.
“We’ll start our Multi-Element Validation and Verification (MEV&V) at the pad this summer,” Quinn said in the March 7 telecon. “The Mobile Launcher is projected to be ready to start Artemis II processing in December of this year.
“Stacking ops will occur early in the first quarter of 2024, and then our integrated operations will start around June or July of next summer, leading to the targeted launch date in November 2024.”
(Lead image: Mobile Launcher-1 as seen during a recent KSC flyover. Credit: Michael Baylor for NSF.)