The first Space Launch System (SLS) Core Stage is now being readied for the program’s first launch at the Kennedy Space Center (KSC), but the large rocket’s first stop on the way to liftoff was the once-in-a-generation Green Run verification campaign. The vehicle spent over a year in the B-2 position of the B Test Stand at NASA’s Stennis Space Center in Mississippi, where prime contractor Boeing performed an exhaustive checkout of their first build.
The Green Run was expected to take most of 2020, but outside events added to the challenge for the Stennis team handling the big stage. Hurricanes and the largest global pandemic in a century created an extra dimension of complications on top of the large demands that an SLS Core Stage placed on the infrastructure at NASA’s primary rocket testing facility.
Unique test campaign concluded in April
The Core Stage is the all-new piece of the initial SLS Block 1 vehicle, which was designed around existing Space Shuttle liquid engines and evolved solid rocket boosters. It is also the figurative core of the vehicle and contains most of its complexity.
The Green Run campaign was planned to verify that the Core Stage design met the requirements set by the SLS Program, and the first flight article was shipped to Stennis to perform double duty as the Green Run propulsion test article. Over the years, the Core Stage Green Run evolved into a one-time only opportunity.
When NASA decided to cancel future Core Stage ground firings at Stennis in 2019, this Green Run became the only planned opportunity to explore the interactions between the complicated stage systems in a test environment.
After the test series was completed with a full-duration Hot-Fire test on March 18, the engines were refurbished and the stage was prepared for transportation to KSC for first launch preparations. With the Core Stage outdoors throughout the Green Run campaign, weather played a prominent role in the pace of some activities, including the installation and removal operations that bookended the stay of the stage.
(Photo Caption: Core Stage-1 is lifted out of the B-2 position of the B Test Stand on the evening of April 19, 2021, when calms winds presented a rare opportunity to perform the operation. Winds had to be five knots or lower at some points in the crane lift in order to safely remove the stage.)
“The requirement to remove the vehicle is a pretty low-wind condition, and we actually ended up getting very fortunate because we had had, I think, three weeks where there wasn’t a day that even met the condition,” Ryan McKibben, NASA Test Conductor for Green Run, said. “Then, when we were ready, it ended up falling very well within our timeline to remove the vehicle.”
“There are some tight clearances when you initial remove it…. So that’s where, if we got much of a crosswind, it could drive the vehicle in there, and we don’t want to damage anything there, so we really were just fortunate on the removal because [during] the install we ended up having to wait and had to punt on a few of the attempts that we did.”
The stage was removed from the test stand on the evening of April 19 by a crane attached to its top and lowered down over the tarmac. After a second crane was connected to the aft end of the stage, workers rotated the vehicle to horizontal where it could be attached to a ground transportation carrier.
After securing the stage to the carrier and rolling them onto NASA’s Pegasus barge, tugboats towed the barge out of the Stennis test stand area at daybreak on April 22, concluding an eventful 16 months. When asked about what stood out about Green Run, McKibben said: “It was the can-do attitude by everyone.”
“Boeing and NASA and S3 (Syncom Space Services) worked extremely well, side by side through the pandemic, through the weather impacts, through all the test cases; it really was impressive to see. From the janitorial staff, they ended up going to the 24/7 [schedule] and covering extra areas, they did whatever it took to get the job done,” he added. “So really it was the people and the attitude that the workers had.”
Final preparations in 2019
The departure near the end of April 2021 ended an over 16 month stay for the Core Stage at Stennis and wrapped up the test campaign that the space center had been preparing to support for years.
A multi-phase refurbishment of the B-2 position of the B Test Stand was approved by NASA in April 2012. Demolition work and restoration of the structure and systems was followed by buildout of new tower structures to accommodate the lengthy SLS sustainer stage and installation of new systems.
Activation of the renovated test stand and its new systems followed restoration and buildout. By the end of 2018, most of the subsystems from water to gases like nitrogen and helium to the cryogenic liquid oxygen (LOX) and liquid hydrogen (LH2) propellant were finishing up activation.
(Photo Caption: A view of the northern perimeter of the B Test Complex from Level 19 of the B-2 test stand position in December, 2018. The white-colored tubing to the right of the exhaust/water deluge pathway that extends out from the flame buckets of both B-1 (left) and B-2 (right) positions is where residual liquid oxygen was dumped in a LOX pond. While residual liquid hydrogen (LH2) was pushed back to propellant barges after the Green Run Wet Dress Rehearsal and Hot-Fire test cases, the more dense LOX was drained from the vehicle’s propellant tank and routed out to the pond.)
“I was the NASA Test Conductor, so before the vehicle got here I worked with our facility team to prep the stand and make sure all the systems would be ready to connect up with the vehicle,” McKibben explained. “We made sure all our flow rates for everything [were] where they needed them to be. We practiced LOX transfer and LH2 transfer; we had some temporary spools made up where we could simulate some backpressures and try to do that.”
“During [activation of] each of the systems, we would work with our instrumentation guys, and we talked to them about what type of pressure transducers or thermocouples or RTDs (Resistance Temperature Detectors) we would need in different systems. Additionally, we would work with our controls group so that we could remotely control a lot of the valves, our motor valves, the variable position valves, solenoids, heaters for ECS (environmental control system) and stuff like that.”
While Boeing was working around the clock at the Michoud Assembly Facility (MAF) in New Orleans in 2019 to complete final assembly of the Core Stage, Stennis conducted a “stress test” of the B-2 position at the test stand and all the infrastructure that would be needed to support the fueled Green Run test cases. “The stress test was [where] we had all our systems online at the same time, just like we would on test day,” McKibben said.
“Around the clock, just like what we would do for the dry volumes before we loaded [propellant], we start pumping in nitrogen there. The water plant would have the generators on and they would be bringing up the water.”
The test was conducted at Stennis December 13-14, 2019. “We simulated the countdown with all the gases flowing and everything else,” McKibben said. “Only the hydrogen burn-off igniters… we simulated an install, but we didn’t burn them because they’re kind of pricey; we had a limited set.”
“We ran through all our procedures, set up every system, made sure there wasn’t anything that we missed, made sure everyone knew their role before the vehicle even was here,” he added. “And we had orifices where we would connect to the vehicle; instead at the interface we would put in orifices so we could meet the right flow rate at the right pressures.”
Boeing completed production of Core Stage-1 at MAF just after New Year’s 2020, with rollout on January 8, 2020. Pegasus arrived with the stage at Stennis on January 12 after the relatively short, day-long trip by tugboats from MAF.
Early test cases interrupted by COVID, weather challenges
After the stage was installed in the stand, a modal test of the structure was performed at the end of January 2020, before preparations began for the first vehicle power up at Stennis. The Boeing, SLS, Stennis, and S3 team had a week or two worth of work left to do before the first power up when work everywhere stopped in mid-March due to the COVID-19 global pandemic.
“NASA internally was sending some emails, and of course it was in the media at the time, they were saying be prepared,” McKibben said. “Internally, there were meetings going on, and once the levels got to a certain point, then they had to pull the trigger and do what was safest for employees.”
From mid- to late- March until mid-May, Green Run work stopped. “There was a period where we only had one or two people come by the stand and just make sure everything was safe,” McKibben said.
(Photo Caption: Syncom Space Services (S3) employees Cheley Carpenter and Shelton Dunn work on Level 19 of the B Test Stand at Stennis in mid-May, 2020, when the initial wave of personnel was allowed to return to the test stand after work was suspended by COVID-19 in mid-March, 2020.)
During that time, safe work protocols were developed and personal protective equipment (PPE) was acquired to support resuming activities. One of the aspects of the new protocols was limiting the size of the team on-site.
“Boeing, NASA, S3, they all came up with their guidelines, and there was strict adherence to social distancing, mask wearing, cleanliness protocols to help minimize it,” McKibben said. “They did use contact tracing on the stand, and there was times where an entire shift of people [were exposed], but all the companies involved worked tirelessly together to support the mission.”
“If one of the contractors lost their shift of people, then they would shift around people from other areas that knew how to do the job to make sure they still supported it.”
The workforce at Stennis was augmented with hundreds of people who temporarily relocated to the area to support the crucial test campaign. “These weren’t just all local people working. We ended up [with people from] California, Texas, Florida, Alabama, Louisiana, Mississippi,” McKibben noted.
In response to the additional risks that COVID added to travel, many team members who had relocated to the Stennis area to support Green Run stayed put. “A lot of people sacrificed [when] they came out here, and they would minimize trips home, which was definitely a challenge for those people to work [away from home]. But they looked at [Green Run] the same way we have, where it’s kind of a once-in-a-career opportunity to test a new vehicle, especially one of this magnitude,” McKibben added.
The first powered test case was test case 2, which began in late June 2020. The test control center was staffed around the clock while the vehicle was powered, working in two, 12-hour shifts. “During the test cases, I’d be on console any time the vehicle was powered up, and one of [my] roles was writing all of the procedures to mate up with Boeing’s procedures for each of the test cases,” McKibben explained.
“Since it was two shifts, the bulk of the work was done on first shift, but second shift was also always there and always working through things. And that was actually quite important in case we hit a snag. They could work paperwork and work other issues overnight, and if things did go well, then they could keep working the paper as-is.”
Sometimes referred to as “non-conformances” (NC) and written up in problem reports (PR), McKibben said the issues that came up were tackled promptly. “The team performed really well,” he said. “On many of the test cases, there would be a component or something would have an issue, and that’s where it was nice to work with Michoud.”
MAF is within commuting distance from Stennis and also provided local support. “Our component shop ended up moving from Stennis to Michoud during the last couple of years,” McKibben noted. If there were no spares in the inventory on-site, then MAF would work to turn around repair work orders quickly.
“We would send them a part, and we [Green Run] had priority, and they would rework it or we would have critical spares of certain components on hand here,” McKibben said. “We might find an issue, and during the handoff [from one shift to another] we’d say ‘hey, we wrote a PR on this. We’re getting this component reworked’ or ‘this transducer is bad.’ And by the next shift it was already corrected and taken care of.”
(Photo Caption: Members of the Green Run test team on console in the Test Control Center during countdown operations for Hot-Fire 2 on March 18, 2021. Core Stage prime contractor Boeing, NASA Stennis, and Stennis contractor Syncom Space Services (S3) worked around the clock during the test cases throughout the campaign for the SLS Program.)
“So, in general, the site did extremely well, and everything that did pop up or was thrown at us… they recovered very quickly. I would do 12-hour shifts in the control center, and then I would pass that off to my backup and he would continue to man it from the facility side,” McKibben said. “So NASA had their test conductor, which was me and Jack, and then Boeing had their test conductors.”
Jack Conley, a member of the Mechanical Operations Branch team in the NASA Stennis Engineering and Test Directorate, was the Green Run NASA Test Conductor on second shift. “The conductor was in charge of coordinating within his own team to get all the their items wrapped up,” McKibben added.
However, as teams fell into a COVID work flow, the end of the summer 2020 for Stennis saw an exceptional number of tropical weather systems threaten the facility and the greater New Orleans area. A half dozen or so tropical storms and hurricanes interrupted Green Run work during the campaign.
“With the vehicle being here throughout the year, we had to deal with everything from hurricanes to freezing weather to the heat and humidity in the summer and lots of lightning storms as well,” McKibben noted. “It was probably the most active season I’ve seen. I’ve only been here for 13 years, but it’s the worst I’ve seen.”
Core Stage size a challenge for Stennis infrastructure
The stress test conducted in December 2019 demonstrated that Stennis could support the demands of the Core Stage, which required other test areas to pitch in to help. “It was definitely a challenge to handle due to the size of it,” McKibben said.
“We’re used to single engines, so all of our stuff is sized tank-wise and propellant-wise for one barge.” Barges are docked to plumbing at the A and B Test Areas to supply liquid oxygen and liquid hydrogen to storage tanks on the test stands. The Core Stage required all six barges for a single tanking.
“We might have two barges transferring [for a single engine test]. For example, at [the single engine] A-1 [test stand], the liquid hydrogen tank is a hundred thousand gallons, where on the [core] stage we’re looking at a half million [gallons],” McKibben said.