Wrapping Up 2010: Safety and Success with an eye toward the Future

by Chris Gebhardt
As NASA enters 2011, preparations are underway for an exciting new venture in space operations with the development of commercial transportation services to the International Space Station, the creation of a 21st Century launch facility at KSC, and development of a new Heavy Lift Launch Vehicle for NASA – all as the U.S. and world space agencies prepare to bid farewell to the Space Shuttle orbiter fleet after 30 full years of remarkable service to the science and medical communities.

*Click here for Part 1 of the 2010 Review*
*Click here for Part 2 of the 2010 Review*

Part 3:
SpaceX’s Falcon 9:  A New Direction for Unmanned Cargo Resupply for Station:

With Atlantis’s first last flight successfully completed in May, attention quickly turned to the launch campaign of a new rocket on the Cape Canaveral Air Force Station: the Falcon 9.

The brainchild – and the latest generation of the Falcon rocket family – of the SpaceX corporation, a private space company and one of two private companies to gain the coveted COTS (Commercial Orbital Transportation Services) contract with NASA, the Falcon 9 rocket has proven thus far to be nothing less than a smashing success.

While it is true that Falcon 9 has only flown twice, its first two missions were conducted within 6 months of each other – and each flight accomplished each and every one of its mission objectives and provided SpaceX with invaluable data on its actual in-flight performance.

Launching on Friday, June 4 at 14:45 EDT, the Falcon 9 rocket lifted off on its maiden voyage carrying a dummy version of the Dragon capsule – the capsule that will eventually be used (sometime this year) to launch supplies to the International Space Station.

The main objectives of the maiden Falcon 9 flight (the sixth overall flight of the Falcon rocket family) were to test the vehicle’s performance through ascent and validate its various systems in flight.

Despite some minor roll issues, the flight proved a tremendous success and a major milestone toward SpaceX’s fulfillment of their part of the COTS contract.

While the June flight marked the first flight of the rocket architecture, several other components and systems that are planned for use on the Dragon capsule once it enters service had already flown and been tested in space – most notably the DragonEYE navigation software/hardware flown on STS-127/Endeavour in July 2009 and a Dragon communications demo conducted during the STS-129/Atlantis flight in November 2009. 

DragonEYE is currently scheduled for re-flight on the upcoming flight of Discovery/STS-133 sometime this year.

With this highly successful test flight behind them, SpaceX took six months to prepare the next Flacon 9 rocket for launch from the Cape Canaveral Air Force Station, implementing several upgrades and improvements to the Falcon 9 from lessons learned from the maiden flight.

While the launch date for the second flight of the Falcon 9 moved to several different places, one thing remained constant: SpaceX’s commitment to a successful flight of the rocket.

On December 8, after a successful engine test firing earlier in the month, SpaceX succeeded in launching the second Falcon 9 rocket as well as the first Dragon capsule (a demo version) into Earth orbit.

Unlike the first Falcon 9 flight which did not release the dummy Dragon capsule into orbit, this mission successfully delivered and deployed the demo Dragon capsule into Earth orbit… where it competed two orbits of Earth before reentering Earth’s atmosphere and splashing down in the Pacific ocean off the coast of Mexico.

The reentry of Dragon marked the first time the United States Federal Aviation Administration had granted a spacecraft reentry license to a private company and marked SpaceX’s first roundtrip flight to space.

In 2011, SpaceX aims to accomplish two more demonstration flights before conducting the first-ever private-company docking of a spacecraft to the International Space Station.

STS-133 Launch Campaign: A Shining Example of Safety and Understanding:

As the summer months came to a close, and NASA’s immediate launch manifest came into alignment with STS-133 on Discovery moving ahead of STS-134 on Endeavour, preparations swung into full gear for the final launch campaign of Discovery.

But with these preparations came the long-coming realization that this would in fact be the last time Space Shuttle orbiter Discovery, the longest-serving and most-flown space vehicle in history, would be launched into space – a definite signal of the coming end of the Space Shuttle Program.

Following initial plans to launch the STS-133 mission NET (No Earlier Than) September 16, NASA decided to push the launch to NET November 1 in order to add three days to the mission timeline as well as two EVAs (Extravehicular Activities – or Spacewalks).

This was due in large part to a desire by both the Space Shuttle Program (SSP) and the International Space Station Program (ISSP) to maximize all the resources possible on the last remaining Shuttle flights.

STS-133 Specific Articles (click for numerous background content on the ET and relating to this article): http://www.nasaspaceflight.com/tag/sts-133/

A similar mission elongation and EVA addition was made to the STS-134/Endeavour flight, which now stands as a 14-day four EVA mission.

With a new target launch date in place, Discovery was set to roll to the VAB from her OPF for the final time on September 8 – ironically enough the anniversary of the first airing of the classic Star Trek TV series after which the Space Shuttle Enterprise is named.

However, a water main break at the Kennedy Space Center on the night of September 7/8 forced NASA to postpone Discovery’s rollover due to an inability to extinguish any fires that might develop on or near the OTS (Orbiter Transport System) – the 72-wheeled vehicle used to transport Shuttle orbiters from their OPFs to the VAB.

The water main break would, however, prove to be the first of several postponements and delays for Discovery along the way.

Nonetheless, the water main was quickly fixed and Discovery rolled over to the VAB on September 9. Stopping at the half-way mark, Discovery was displayed on the road between the VAB and OPF-3 for several hours to allow for group and individual photographs with her before her 39th voyage to space.

Discovery was then rolled the rest of the way into the VAB where she was connected to the hoisting/mating sling and lifted up into VAB HB-3 to be mated with ET-137, which was already mated to the SRB stack.

However, engineers quickly discovered a problem with an internal nut and the installation of one of the two large flight structural bolts between the External Tank and Discovery herself.

The nut in question was prepositioned inside the orbiter, with the bolts installed from the outside. When the bolt was moved into place, the internal nut turned and became loose – rotating 45 degrees off center.

With the bolt unable to be installed until the nut could be repositioned or replaced, an Engineering Review Board (ERB) was called, and engineers debated between setting up access to the aft compartment, removing Discovery from the stack for a repair once she was horizontal, or installing access platforms to Discovery’s aft compartment prior to the completion of hard mate (which is usually not allowed for safety reasons).

In the end, the ERB decided to press ahead and complete soft mate operations, thus allowing one engineer to enter Discovery’s aft compartment (though a safety waiver was processed for this unusual activity).

Once soft mate was complete, repositioning of the internal nut took approximately 12-hours and mating operations between Discovery and ET-137 proceeded without issue following the correction of this is issue.

But this would not be the only bolt issue incurred during mating ops in the VAB. During mating of the TSM (Tail Service Mast) LH (Left Hand) T0 quick disconnect umbilical to Discovery, “the TSM carrier plate foot impacted the smaller of the two bolts on the lower LH attach point hard enough to drive a preliminary gauge pin inspection at the location,” noted an NTD report on the issue.

“The gauge pin could not be fully inserted and the bolt is believed to be bent. Engineering is discussing the ‘go forward’ plan.”

Normally the TSM would have been demated, allowing for the bent locating bolt to be removed and replaced. However, engineers decided to drill the bolt out – being careful to ensure that any particulates from the drilling were carefully removed to prevent potentially contamination in Discovery.

The drilling of the bolt proved time-consuming with “the tail end drilled approximately 1.25″ deep, the head end approximately 0.75″ deep. Both are 5/32″ diameter.” The bolt was eventually removed and all potential problems surrounding the issue cleared.

With these issues resolved, Discovery was put through the wringer with a full-up integration test of the STS-133 vehicle. Once this test was complete and verified, preparations for Discovery’s rollout to LC-39A began.

On the evening of September 20, Discovery began her journey to Pad-A at 1923 EDT, arriving at the pad less than 6-hours later at 0115 EDT on September 21.

After arriving at Pad-A technicians began configuring Discovery for launch. As with her VAB mating flow, the pad flow was planned as an elongated flow, with the standard 22-days pad processing stretched to 27-days with 15-days of contingency built-in.

Unlike Atlantis’s pad flow toward STS-132, Discovery’s pad flow would prove to be anything but calm, with many anthropomorphizing Discovery and jokingly referring to the technical issues that arose as Discovery voicing her anger over her impending retirement.

The first pad issue occurred on October 12 when a leak was detected from an Air Half Coupling (AHC) on the Right Hand OMS pod of Discovery.

As noted at the time, “IPR 47 was picked up on fuel vapors on the RH (Right Hand) doghouse door on the right OMS (Orbital Maneuvering System) pod. They went in and looked, and there is a suspect QD issue there on a low point bleed QD used on the pod R&R. They will go in … and hopefully clean that QD up and solve that problem.”

The suspected QD/Cap, located in the area of the crossfeed system between the two OMS Pods, was accessed by engineers via the 59-64 door – a triangular door located on the aft bulkhead of the orbiter – ahead of an effort to remove and replace the flight cap and clean the AHC.

Nonetheless, the efforts proved unsuccessful in stemming the leak of toxic hypergolic vapors. With the cap and AHC ruled out as the cause, engineers pinpointed the issue to be with the crossfeed flange area.

In order to avoid a rollback and destack of Discovery, a one-of-a-kind plan was developed to drain both LH and RH OMS tanks and repair the flange seal at the pad – something that had never been done before.

However, the leak suddenly vanished shortly after the failed flight cap R&R after a 100 psid increase in the crossfeed line. This led to speculation that the pressure increase may have seated the primary seal which was the source of the leak.

Nonetheless, NASA decided to press forward with R&R of the both the primary and secondary seals, and engineers devised a forward plan and ended up finishing the seal R&R one day AHEAD of schedule. The OMS tanks were then refueled and no leaks were detected.

Completion of this outstanding work allowed the teams to maintain a positive margin to protect the targeted Nov. 1 launch date – a date that became official at the October 25 SOMD FRR.

However, another seal and AHC issue would force a postponement of launch operations to Nov. 3. The new AHC issue was documented at IPR-133V-0053. “New IPR-53. After OMS/RCS (Reaction Control System) stage 1 pressurization, when the GSE feeding the right OMS helium tank was isolated and vented, the vehicle side tank also vented down indicating MD515 AHC poppet was stuck open,” noted the NTD report at the time.

“Engineers bypassed the GSE (Ground Support Equipment) orifice and were able to get the poppet closed. The AHC/GHC were demated to determine cause of the stuck poppet and half of the GHC o-ring was found stuck inside the AHC.”

The o-ring was R&Red and mission managers convened the start of Discovery’s countdown on Sunday, October 31. All proceeded well in the countdown until the power up the Main Engine Controllers (MECs).

“IPR 58 to CCME (SSME Controller Engineer Console): While performing the bus redundancy test on the main engine controllers, CCME reported Engine 1 and 2 performed nominally, however, Engine #3 redundant controller did not activate.

“CCME was expecting approximately 0.5 amp increase on all three phases.  Data retrievals showed a very small current spike on Phase A and C, but nothing on Phase B. The controller has a built in ‘safe’ mode that when a phase is missing, it will shut itself down.

“The CCME decided to cycle the Phase B circuit breaker to determine if there was any contamination in the breaker that could cause this problem. Prior to starting the troubleshooting CCME noticed the redundant controller had been activated without any action by engineering or the SCO (Spacecraft Operator) in the crew module.
“Data retrievals showed the switch scan for the controller cycled on/off on its own, the same signature on Phase A and Phase C were noticed and then shortly after that Phase B came up which ultimately activated the redundant controller.”

In the end, the issue was eventually traced to a tripped circuit breaker that had cleared itself and was therefore good for launch.

With these issues finally resolved, NASA managers agreed to set Discovery’s earliest launch attempt for Thursday, November 4 with full knowledge that the weather forecast for that day was anything but desirable.  However, Mike Moses and Mike Leinbach both observed that it is not necessarily a wise decision to forego a launch attempt with more than 24hrs to go until liftoff as weather conditions can change quite rapidly.

To this end, the decision was made to proceed with countdown operations and make an assessment of the weather at the MMT (Mission Management Team) Tanking Meeting in the early morning hours of November 4. At this meeting it was decided that weather conditions were in fact unfavorable for launch and the chances of the weather being acceptable quite slim.

The decision was therefore made to postpone launch by 24-hrs and retarget for a Friday, November 5, launch. Since fueling operations on the ET (External Tank) did not begin on November 4, the decision to postpone the launch did not represent a scrub scenario, thus making the launch attempt on November 5 the first scrub of the STS-133 mission.

The following day weather conditions had greatly improved, and the MMT gave the go to begin fueling Discovery’s ET with nearly 536,000 gallons of Liquid Oxygen (LOX) and Liquid Hydrogen (LH2).

As fueling operations proceeded, all looked promising for an early afternoon launch. But those thoughts would be dashed as the infamous GUCP (Ground Umbilical Carrier Plate) began to show signs of improper sealing at the GUCP/ETCA (External Tank Carrier Assembly) interface.

Launch Team personnel began a standard series of ET GH2 (Gaseous Hydrogen) vent valve cycles in an attempt to reseat the GUCP/ETCA seal.

These procedures proved unsuccessful and the GH2 leak eventually breached the 44,000ppm red line LCC (Launch Commit Criteria) limit – spiking all the way to 60,000ppm, the maximum reading on the GH2 sensors.

With a clear violation of the LCC, Launch Director Mike Leinbach ordered a scrub to launch operations and an immediate draining of the ET, which by the time of the GUCP leak LCC violation was nearly completely filled.

As the launch team monitored the draining of the ET’s cryogenic propellants, and mission managers began preliminary meetings on the GUCP leak, OTV cameras on the launch pad captured the appearance of a significant crack in the TPS (Thermal Protection System) foam on Orbiter-facing side of the intertank portion of the External Tank.

Detailed measurements (from camera footage) were taken and the problem documented as thoroughly as possible without the presence of a human being on the launch pad due to the need to allow the residual cryo propellants in the ET to boil off before allowing engineering personnel access to the launch pad.

Once ET cryogenic boil off was complete, two sets of engineering teams began detailed assessments of the vehicle, with one team focusing on the GUCP leak and the other on the ET intertank foam crack.

Within days, teams had identified and fixed the misalignment of the GUCP which caused the GH2 leak – a fixed that was validated during the mid-December tanking test on ET-137.

The ET foam crack, which was found to be caused by the cracking of the underlying stringers on the intertank, is still an ongoing investigation – with NASA managers refusing to set and official target launch date for Discovery in order to reinforce that the launch schedule is not a consideration during the ET stringer crack investigation and discussion.

Coming to Terms with the End of an Era – Visible Milestones toward Retirement:

But with these impressive and inspiring highlights and achievements in 2010 came several milestones that reinforced what everyone already knew: the Space Shuttle Program was coming to an end, and with it the end of an era.

On September 27 the Pegasus barge was towed into the Turn Basin at the Kennedy Space Center, arriving from the Michoud Assembly Facility (MAF) in Louisiana with ET-122 (External Tank 122) – the final External Tank to be delivered to the Kennedy Space Center for flight on the Space Shuttle.

Following the completion and delivery of ET-138, the final ET in numerical sequence, MAF workers completed an unprecedented undertaking to restore ET-122 to flight status following the significant damage inflicted on the tank during Hurricane Katrina’s landfall on Louisiana in 2005.

Arguably, there could be no greater example than ET-122 and the workers at MAF for the skill, ingenuity, and passion the thousands of people throughout NASA have for investigating, understanding, and resolving the problems and issues faced on a daily basis.

To many, the restoration and completion of ET-122 for flight on the STS-134/Endeavour mission is nothing short of a engineering miracle, made possible only by the hard work and extreme dedication of the MAF workforce who not only restored ET-122 to flight status, but completed every single External Tank for every single Shuttle mission despite personal traumas and an uncertain future.

Just as the shipment of the final External Tank brought to a close one chapter of the Shuttle Program, another chapter began its end at the Kennedy Space Center as contract workers began the process of dismantling and demolishing the Fixed Service Structure (FSS) and Rotating Service Structure (RSS) at LC-39B.

Already altered for use by the Ares I-X rocket in 2009, launch pad 39B saw its final service to the Space Shuttle Program in April/May 2009 when it hosted Space Shuttle Endeavour/STS-400 during preparations for Atlantis’s flagship mission to the Hubble Space Telescope in May 2009.

Pad-B began its service to the Shuttle Program with the ill-fated Challenger/STS-51L flight on January 28, 1986 and hosted its final Shuttle launch on December 9, 2006 with the launch of Discovery and the STS-116 mission.

With the completion of STS-400 activities and the extremely successful test flight of Ares I-X, Pad-B’s current configuration stood in contrast to the forward-looking plan of the now-defunct Constellation Program as well as a building plan for LC-39B: the desire to render it a clean surface pad with the long-term goal of being able to support multiple vehicles and missions – an idea that fits into the overall KSC infrastructure update plan for a 21st Century launch complex.

With this in mind, controlled demolition of Pad B began in mid-September with deconstruction work commencing on the RSS.

While initial work estimates called for the complete removal of the RSS by the end of October and the complete removal of the FSS by the end of December, portions of the RSS are still standing at Pad-B as work progresses in a slow but steady manner.

Demolition work is now expected to be completed by the first half of 2011, thereby rendering Pad-B a clean pad for the first time in over 30 years.

Into 2011: The More Things Change, The More They Stay the Same:

In all, 2010 proved to not be the end of the Space Shuttle Program as many had expected and prepared for. With Discovery currently working toward a penciled-in February 3 launch at 0137 EST, Endeavour NET April 1 at 0315 EDT, and Atlantis (on STS-335/135) targeting June 28 at ~1530 EDT, the end of the Shuttle Program is still a few months away.

But at the same time, the final landing gear touchdown and wheels stop on whichever flight ends up being the final mission will not mark the end of the Program. The orbiters themselves will remain at the Kennedy Space Center for some time as engineers and technicians take the vehicles through Transition and Retirement and prepare them for their honored places as the centerpieces of space exploration and Shuttle Program displays at three museums around the country.

Just where each of the orbiters will eventually end up is something that has yet to be decided. While Discovery still appears to a shoe-in for display at the Smithsonian Air and Space Museum outside of Washington, D.C., a final decision on that and where Atlantis and Endeavour will end up is not expected until later this year.

Nonetheless, based on pre-STS-133 mission delays, it does appear that all three orbiters will now be retained at their home at the Kennedy Space Center until at least the first half of 2012, before continuing their mission of inspiration elsewhere in the United States.

(Images: Larry Sullivan, MaxQ Entertainment/NASASpaceflight.com, L2 Presentations, NASA.gov, SpaceX).

Related Articles