STS-93 at Twenty Years: “I had to hit the button!”

by Philip Sloss

Cutoff in last-second before engine start

At T-10 seconds the GLS tells the Orbiter RSLS it is “Go for Main Engine Start.” “As I pass through the ten-second milestone, that was ‘go for engine start’ and so then the sequencer flags the go to the vehicle,” Kennedy said.

“As it comes back and gives us the acknowledge we would pass through that point at ten seconds.” As Kennedy was beginning to announce that “GLS is go for main engine start,” Primary Hazardous Gas System Engineer Ozzie Fish quickly called “GLS give cutoff,” and Kennedy with her finger over the “execute” button quickly pushed it to stop the countdown and launch.

Credit: NASA.

(Photo Caption: A frame from a film camera showing the Radial Outward Firing Initiators (ROFIs) firing before engine ignition on a later Shuttle mission. The ROFIs were started at T-10 seconds and would run for at least eight seconds and as much as thirteen; typically, they would run out after ten seconds close to the moment of liftoff.)

When the pace of events allows more time, calls are typically made through a chain of command; with very little time to react in the last seconds of the countdown, GLS engineers were trained to listen for three words and maybe just the first one. “We’re waiting on any engineers who see an out of tolerance event and we were trained to [respond to the words] ‘GLS give cutoff,'” Kennedy said.

“And so the haz gas engineer Ozzie Fish calls ‘GLS give cutoff,’ I’m on it, and basically we got that cutoff in just milliseconds before engine start.”

Fish had seen a sudden spike in a reading from the hazardous gas system of the hydrogen concentration inside the orbiter aft compartment. The reading on his display came in at T-16 seconds, and the sudden jump in the detector reading to 640 parts per million (ppm) from a little above 100 ppm indicated that something was leaking highly flammable hydrogen inside the orbiter boattail.

“Little did we know at that time that we weren’t going to have a catastrophic event with that LH2 buildup in the aft,” Kennedy said. “We wanted to make sure we got that call in before the engines start just because of the situation and so I’m very happy that things worked out the way that it did.”

“Ralph Roe was the launch director at the time and the first time we had the crew loaded, the vehicle was fueled, the hydraulic APUs were started,” STS-93 Ascent/Entry Flight Director John Shannon recalled. “[We were] just about to do main engine start and the haz gas operator Ozzie Fish at the Cape called an abort and that was tough because he’s got to react to what he sees on his haz gas system because that’s obviously a critical system.”

“We were in a hazardous situation with the APUs running and that close to main engine start and trying to abort that late, so we had a lot of risk versus risk discussions,” Shannon added. “That’s why you have to make sure your launch commit criteria and the rules that you develop prior to launch are just religiously followed and there’s no room for interpretation.”

“Ozzie did exactly the right thing but that was a very stressful abort because you’re trying to deconfigure the system and safely shut everything down and then get the crew out and detanked,” he said.

Credit: Nathan Barker for NSF.

(Photo Caption: A Shuttle-era console in redecorated cabinetry on display in the Launch Control Center in 2017. Below the left-most screen is a programmable function panel, with two columns of three buttons. The prime GLS for a Shuttle launch countdown would configure two buttons for the possibility of a manual cutoff of the countdown, pushing one of them to arm the functionality after control of the launch was passed to the Orbiter at T-31 seconds and then being ready to push an execute button upon voice request. The GLS and the Orbiter RSLS could both automatically stop a countdown after T-31 seconds, which happened multiple times during 30 years of Shuttle mission operations; however, manual cutoffs were rarer and the one executed on the first STS-93 launch attempt was called the closest to liftoff.)

On the flight deck inside Columbia with her crew, Commander Eileen Collins was looking at something unique to her. “The countdown clock stopped at about T-8 seconds, and then it stated cycling to T-9 and back to T-8. I had never seen that in the simulator,” she said in an email.

“Then I heard the ‘cutoff’ call and we began our recycle procedures. This is something we train for. I was happy to hear (about an hour later) that the main engines did not light.”

“The standalone digital clock on the flight deck is not connected to the flight software so it continued counting down. (Incorrectly!),” Collins added. “I usually only looked at the CRT clock! The alternating numbers sent a huge [question] to my head…never saw that in the Sim. When we practiced scrubs in the sim, the clock would stop on a specific second, so it wasn’t always modeled correctly.”

“There’s three different kinds of cutoffs, there’s an RSLS hold, there is, of course, the manual GLS/LPS (Launch Processing System) hold, and there’s also an [RSLS] abort and the abort is after engine start,” Kennedy explained. The manual hold capability exists after the liquid main engines begin to start, but Kennedy reacted fast enough to stop the countdown before the first engine start command at T-6.6 seconds.

“The clock stopped at seven seconds and the engines fire at 6.6, so I was 400 milliseconds early,” she said, laughingly recalling what happened. “We did lots of data retrievals. In fact, my team that was there with me on that day, they ran a data retrieval and they turned around and they said ‘Way to go Miss Barbara! You got it in 400 milliseconds before that engine start!'”

Kennedy said it took one to two seconds for the system to fully react to the manual cutoff. “It will work through the software, safe the condition, shutdown, and stop the countdown clock. So in the priority order that needed to occur, the clock was not the priority, right?”

Safing the system, downstairs meeting

Once Kennedy pushed the “execute” button to cutoff the countdown, the orbiter RSLS gave control back to the GLS which started an automatic safing sequence. The sequencer keeps track of the countdown progress so it knows where to pick up a safing sequence if it becomes necessary.

“As you’re coming through the launch countdown that parameter continues to increment and so as soon as you have a safing indication that software moves over into the safing program and the first thing it checks is what that parameter’s value is, so it knows immediately where to go to in the safing sequence to back out everything you just set up,” Kennedy explained. “[We’re going] through the sequence of the safing module of the software, and so the APUs had to be shut down, the access arm (Orbiter Access Arm) had to be put back out for the crew to get out, and so all of those things happen in a sequence that’s called the safing sequence and that’s automatically initiated once the cutoff is given.”

“All the most important things needed to be done first for crew safety, and ground safety, vehicle safety, those were all sequenced in the proper order and then as you move down in the sequence other things like power, closing valves, and initiating drainback and those kinds of things would happen in an order that was pre-approved,” she noted.

Although it only takes a few minutes for the sequencer get through all the critical safing steps, Kennedy said: “It was an eternity! You just do it, [but] one second was an eternity that day, it truly was, and so it was quite an adrenaline pump that day to make sure we did the right thing and we did, so it was a good day.”

Credit: NASA.

(Photo Caption: A wide, fish-eye image taken in Firing Room 1 of the Launch Complex 39 Launch Control Center in 1998. The Integration Console is at the very back left from this point of view.)

“After the cutoff and the safing, which we had done many times through the Shuttle Program and through simulations, you’re basically just staying focused, staying focused, getting through safing or even after a launch there was another eight minutes of travel for the crew and the vehicle to Main Engine Cut Off eight minutes later, so it was always protocol for the test team to remain poised in functions that we performed until it was the appropriate time to go ‘Phew'”

After safing, Kennedy said: “We would back out of the sequencer, clear out the files for the mask and bypass for that day, and leave everything clean and terminated in the launch sequencer because it no longer needed to run, and so then they would initiate the cryo detank and of course the cryo team would be there to do that and then their relief would come in and continue to detank.”

While the External Tank was being drained and the scrub/turnaround procedures started, the launch team started taking a closer look at what had happened. “I want to say it was probably an hour after the safing and cutoff time we did some data retrievals and made sure we understood exactly what the sequencer was doing and also the haz gas folks wanted to meet with the senior managers,” Kennedy remembered.

“We all went down to the first floor and the room was packed and they were discussing exactly what happened, what the indications were, what the data retrievals [said]. I really wasn’t a part of the down-and-in research for the haz gas anomaly, but I had seen my director at that time, Tony Delmonte, and I said ‘Tony, I had to hit the button!’ and he goes ‘What?’ and so he was trying to absorb it all and then he walked into the meeting and then I left to go home and prepare for the next day to turn around, do it again.”

“I remember driving away from the gate that day, driving in my car on the way home going ‘Oh my goodness I hit the cutoff button!’ so basically it just kind of all hits you when you drive out the gate, what really happened,” she said.

Although it was a surprise, GLS operators were trained to be ready to execute the manual cutoff request, and Kennedy remembers seeing it in a simulation. “They did sim it on the GLS group,” she said.

“I was sitting second seat and George Thomas, an experienced GLS engineer there, he was asked to cut it off in a simulation. That occurred, so we were all like ‘I can’t believe they made you cut it off, George!'”

“Usually it would be an anomalous condition that would kick us out automatically, but George did actually have to hit the button one time in a simulation, so we all got to see that,” Kennedy added. “See? You’ve got to be ready.”

Cutoff postmortems

The spike in the hydrogen concentration turned out to be a false reading. “It was an incredibly unfortunately coincidence that it occurred at that point where we didn’t have time to get another reading before the engines started and it was at a point when the concentration in the aft was already increasing as it normally would and the additional increase just looked exactly like a hydrogen leak would look if it were real,” Grant Cates, Columbia’s Flow Director said the next day on July 21 in a countdown status briefing.

“The haz gas detection system, both the primary and the backup systems have what we call a vacuum ion pump, it draws the hydrogen ions into the detector and on occasions those vacuum ion pumps can outgas or ‘burp’ is what we call them hydrogen ions back into the detector and then those ions will be subsequently be read again for a second time and it was that occurrence that happened to us at T-16 seconds.”

“It’s important to note the quickness of the operator in making that decision, calling a manual cutoff, and then the Ground Launch Sequencer operator reacting to that and halting the countdown — we came within milliseconds of starting the engines,” Cates added. “Had we started the engines, we wouldn’t be here today talking about a launch tonight, we’d be talking about a launch in August.”

“We would have had at least three weeks, probably more, to turn these engines around to go fly this mission.”

Readings from the leak detectors were one of the few that were not automatically monitored and enforced by the GLS during the terminal count. “In the case of the hazardous gas detection system, because it is very system operator sensitive, the operators make manual calls and prior to 31 (T-31 seconds) if this event had occurred, we would hold at 31, evaluate for two more minutes and see if it stayed below 600 parts per million,” Launch Director Ralph Roe said in a media briefing after the cutoff. (That procedure was used previously in the launch countdown for STS-80.)

“The requirement is 300 parts per million,” Roe added. “We have a manual call even prior to 31; because of the unique nature of the haz gas detection system it is not [automated] in GLS.”

Credit: NASA.

(Photo Caption: An image of the STS-93 engines taken by the Final Inspection Team in the evening on July 19, 1999, after the vehicle had been fueled for the first launch attempt. One of the ROFIs can be seen middle-right. The three Space Shuttle Main Engines in the orbiter started and shut-down “fuel-rich;” the ROFIs worked during the start sequence, burning excess hydrogen coming from the engines before they ignited to prevent a rapid ignition/explosion of a large gas buildup.)

With the countdown being stopped so late and so close to engine ignition, the launch was scrubbed as soon as Kennedy pushed the execute button to initiate the cutoff. At T-10 seconds the hydrogen burn-off igniters, also known as Radial Outward Firing Initiators (ROFIs), started; once started, they fully discharged and needed to be replaced to make another attempt.

The engine service platform would need to be brought in, but it couldn’t be raised into place until after the External Tank was inerted, so a 48-hour scrub turnaround was implemented. Draining of the LOX and LH2 from the tank took a few hours, but boil-off of residuals continued until almost 24 hours after the scrubbed attempt. The platform was moved into place early on July 21 for the four-hour-long task to reservice the ROFIs and the countdown resumed for the second launch attempt at the opening of the launch window early on July 22.

Related Articles