If the first half of 2009 for NASA demonstrated the agency’s unwavering attention to safety, NASA not only continued that trend – upholding the utmost safety standards in the Space Shuttle and International Space Station Programs – during the second half of the year but also placed itself and its various programs in a tremendous position for the years to come.
Highlighting the second half of 2009 for NASA was the resolution of the Gaseous Hydrogen leaks at the Ground Umbilical Carrier Plate on the Shuttle launch pad (which resulted in the successful launch of STS-127/Endeavour in July), the midnight ride of Discovery on STS-128 in August, the amazingly successful test flight of the Ares I-X rocket in October, the discovery of water on the moon in October/November from the Lunar CRater Observation and Sensing Satellite and the Lunar Reconnaissance Orbiter, and the logistics run of shuttle Atlantis to the ISS in November.
The Gaseous Hydrogen Leaks and the STS-127 – LRO/LCROSS Range Debate:
Kicking off the second half of 2009 for NASA were the launch campaigns for shuttle Endeavour’s STS-127 mission to deliver the Japanese exposed experiment facility to the ISS and the Lunar Reconnaissance Orbiter (LRO) and shepherding spacecraft Lunar CRater Observation and Sensing Satellite (LCROSS) on the Atlas V rocket.
Initially targeting a June 13 liftoff for STS-127, the main constraints facing NASA for Endeavour’s launch window were the scheduled June 17 launch of the LRO/LCROSS mission – a mission which only had a 3-day launch window before standing down until early July – and an imposing Solar Beta Angle Cutout with the ISS that began on June 20 and ran through July 10.
Given the Eastern Range’s need of roughly 48-hours to reconfigure all their tracking and communication equipment between the various vehicles launched from the Cape Canaveral area, Endeavour was constrained to a 3-day launch window in the early morning hours of June 13, 14, and 15 in order to allow LRO/LCROSS to launch on June 17.
This plan was thwarted, however, in the evening hours of June 12 as launch personnel were nearly finished fueling Endeavour’s External Tank (ET) with over 500,000 gallons of Liquid Oxygen (LOX) and Liquid Hydrogen (LH2).
Just before the LH2 tank of the ET entered stable replenish mode, launch personnel noted an abnormally high concentration of gaseous hydrogen around the Ground Umbilical Carrier Plate (GUCP) assembly – a system used to vent the extremely flammable gaseous hydrogen (caused by LH2 boil off inside the LH2 tank) safely away from the Shuttle stack.
Following the scrub of Endeavour’s first launch attempt, NASA mangers reported that the signature of the leak was “nearly identical” to the leak seen during the first launch attempt of STS-119 in March. Since the Removal and Replacement (R&R) of the GUCP Quick Disconnect seals fixed the leak back in March, NASA managers decided to repeat the R&R procedure after obtaining critical data on the area of the leak for a root cause investigation.
However, the R&R of the seals meant that launch of Endeavour would not be possible until the early morning hours of June 17 – two days past the end of Endeavour’s launch window because of the Eastern Range launch conflict with the scheduled June 17 launch of LRO/LCROSS.
Nonetheless, engineers pressed forward with the R&R of the GUCP Quick Disconnect seals while the Agency as a whole began discussions on which mission now had priority.
Ultimately, through cooperation between the Space Shuttle Program, the LRO/LCROSS team, and the Eastern Range, a compromise was reached in which Endeavour would be given a launch opportunity at 5:40a.m. on June 17 while the LRO/LCROSS team would proceed toward a late afternoon/early evening June 18 launch with the understanding that if Endeavour launched on June 17, LRO/LCROSS would have to be delayed until June 19 – the last day of the LRO/LCROSS’ June launch window – because of Eastern Range reconfiguration time constraints.
Per the terms of the agreement, should Endeavour launch on June 17, LRO/LCROSS would take an additional one day delay and launch on June 19. However, should Endeavour scrub for any reason prior to 12 midnight June 17 (a scrub on calendar day June 16), the Eastern Range would undertake a 36-hour reconfiguration effort to allow LRO/LCROSS to maintain its ~5p.m. June 18 launch date/time.
As it would be, the GUCP leak reoccurred during ET fueling on June 16/17 and the launch was officially scrubbed at 1:55a.m. EDT. Later, Shuttle Program managers stated that they would not try to launch Endeavour before July 11 to allow for an investigation into the GUCP leaks.
However, the scrub came nearly two hours after the midnight deadline for LRO/LCROSS on June 18. Nonetheless, the Eastern Range undertook the reconfiguration effort and completed these operations on time for the June 18 launch of LRO/LCROSS. That day, during the final minute of the launch window, the Atlas V rocket carrying LRO/LCROSS lifted off from its seaside launch pad, returning NASA to the moon.
In the days that followed the second scrub of STS-127, engineers focused their attention on a minor misalignment between the GUCP and the External Tank Carrier Assembly (ETCA), the component on the ET to which the GUCP is mated.
In the end, the ETCA was realigned using high fidelity 0.515″ alignment pins and the Flight Seal in the GUCP was replaced.
Once the R&R procedure was complete, a tanking test was performed on July 1 to verify the integrity of the repair and confirm that no leaks were present. The tanking test was a success and no leaks were detected.
This paved the way for the July 11 launch attempt of Endeavour, which was postponed prior to fueling because of extra time needed to clear all of Endeavour’s systems following a lightening strike within the Pad-A perimeter the previous day.
Launch was rescheduled for July 12 but was scrubbed at the T-9 minute and holding mark due to unacceptable weather around the Kennedy Space Center. The launch was further scrubbed on July 13 – again at the T-9 minute and holding mark – due to adverse weather.
NASA stood down on July 14 to replace the Tyvek covers on Endeavour’s Reaction Control System jets and retargeted launch for July 15. The countdown on July 15 was uneventful and at 18:03:10 EDT (6:03:10p.m.) Endeavour lifted off from Pad 39A – on the third Shuttle mission of the year – one day shy of the 40th anniversary of the Apollo 11 launch from the same pad.
In all, STS-127 was the longest Shuttle flight of 2009 lasting nearly 16-days. During the course of the mission, Chris Cassidy became the 500th person to fly into space, a record 13 people lived and worked on the ISS during Endeavour’s docked mission (thanks to the increase of the permanent ISS crew from three people to six people at the end of May 2009), five EVAs (spacewalks) were performed, the Japanese experiment exposure facility was installed to the Japanese Kibo module, and six P6 truss batteries were R&R on the ISS.
Endeavour and her seven member international crew landing safely at the Kennedy Space Center at 10:48a.m. EDT on July 31st on the mission’s first landing opportunity, completing a 15day 16hour 44minute 58second flight.
STS-128 – ET Foam Loss Investigation and PV-12 Fill and Drain Valve Discussion:
With the successful completion of STS-127, attention turned to the next Shuttle mission: the STS-128 logistics flight of Discovery scheduled for late-August.
STS-128’s ground processing flow proved to be extremely interesting, with engineers working through issues with a Solid Rocket Booster Check Valve Filter Assembly, an ET Ice Frost Ramp foam void indication, a Main Bus Controller anomaly, and an Ordnance Cable issue on the ET Vent Arm System.
However, it was the numerous foam liberations from the Intertank region on Endeavour’s External Tank during her July 15 launch that kick started an investigation into why the foam liberated – an investigation that once again highlighted NASA’s strict standards of safety when it comes to the Space Shuttle fleet.
While plug pull tests on Intertank foam from Discovery’s ET confirmed the integrity of the foam bond to that tank, NASA managers demanded that they have all the information possible before clearing Discovery for flight.
In fact, the standard one day Space Operations Mission Director (SOMD) Flight Readiness Review (FRR) at the Kennedy Space Center – the FRR that formally approves each mission for launch – required an additional half day to complete all the necessary discussions.
As such, the possible root cause scenarios for the foam loss coupled with the extremely positive results from over 100 plug pull tests on the backside of the Intertank region on Discovery’s ET gave NASA managers the confidence needed to proceed with launch.
In the end, Discovery was approved for liftoff on August 25 at 1:36a.m. EDT. The countdown was uneventful from a technical perspective; however, heavy rains and lightening over the launch pad and in the KSC area resulted in a scrub of the August 25 attempt at the T-9minute and holding mark.
Launch was reset for the following day. Nonetheless, during fueling operations of the External Tank, the inboard LH2 PV-12 fill and drain valve on Discovery did not close when it was commanded to do so.
Launch Director Pete Nickolenko followed the Launch Commit Criteria (LCC) to the letter, calling a scrub and ordering the draining of Discovery’s ET.
For a stuck “open” fill and drain valve, LCC prevent launch personnel from attempting to cycle the valve under cryogenic conditions (meaning in the presence of Liquid Hydrogen or Oxygen). The theory behind this LCC is that – if there is a serious problem with the fill and drain valve – the launch team does not want to put themselves into a situation where they cycle the fill and drain valve closed and then cannot get it open again.
While the launch team followed these procedures to the letter, there were indications from sensors – both upstream and downstream of the PV-12 fill and drain valve – that the valve had indeed closed when commanded, and that the “open” indication was a sensor error in the valve itself.
As such, engineers conducted multiple ambient (meaning without the presense of Liquid Hydrogen) cycles of the valve. During these tests, the valve performed nominally.
This allowed Shuttle mangers to approve a third launch attempt for Discovery 22-minutes after midnight on Friday, August 28 while they reviewed a potential revision to the current LCC document (actually a reversion to a previous LCC rule) that would allow them to cycle the PV-12 valve under cryogenic conditions should they receive a sensor indication that the valve failed to close on the Friday morning attempt.
Nonetheless, extensive engineering conversations took place in regard to this LCC change and the possible scenarios that could occur if they launched with a PV-12 valve that was only partially closed or a valve the became stuck “closed.”
“Based on a concern with galling failure mode within the valve driver mechanism, the previous LCC was amended to preclude valve cycling under cryo conditions,” noted a NASA processing report on the engineering analyses.
“Previous LCC allowed for a repeat valve cycle attempt given the following conditions were met: No Sustained reg-out demand occurs. No sustained increase in Aft Haz Gas helium concentration. Initial move time <5.0 seconds. All the above parameters were satisfied.”
In the end, NASA managers decided they needed additional time to review all the information and complete closeout paperwork on the PV-12 valve issue. To this end, launch of STS-128 was postponed 24-hours to 23:59 EDT (11:59p.m.) on Friday, August 28 – the second launch opportunity of the calendar day but technically a 24-hour delay to the launch.
The final countdown on August 28 was uneventful and the PV-12 valve “closed” when it was commanded to do so.
In fact, the countdown went exactly as expected and at 11:59:37p.m. on August 28 Space Shuttle Discovery lit up the midnight sky of Central Florida, becoming the only manned mission to launch on one calendar day and reach orbit on the next.
Two days later, Discovery docked to the International Space Station 25 years to the day after the launch of her maiden voyage: STS-41D. That day, Nicole Stott – who launched on Discovery – officially transferred to the ISS crew, becoming the final person to rotate up to the ISS on the Space Shuttle.
During the course of the docked mission, Discovery’s crew delivered a second treadmill to the ISS as well as numerous new science experiments and new science racks via the Multi-Purpose Logistic Module Leonardo.
After a highly successful mission, Discovery and her crew undocked from the ISS and received final Thermal Protection System clearance for landing. The first two landing attempts on September 10 were waved off due to bad weather at the Kennedy Space Center.
Discovery was ultimately cleared for landing on September 11 and performed the second Boundary Layer Transition DTO (Detailed Test Objective) during her reentry into the Earth’s atmosphere.
Discovery and her seven member crew landed safety at Edwards Air Force Base, CA at 20:53 EDT (8:53p.m.) on September 11, completing a 13day 20hour 54minute 55second mission.
Ares I-X – Proving Concepts for the Future of Manned Space Exploration:
It was a mission nearly 4-years in the making. The Vision of Space Exploration, set forth by President George W. Bush in January 2004, committed NASA to returning men to the moon and continuing on to Mars – an endeavor that required a new launch architecture system.
Unlike the Space Shuttle, which underwent its first integrated test flight with a crew onboard, NASA’s new Ares I crew transportation rocket was designed to have multiple test flights before launching a crew into orbit.
The rocket, a virtually identical model of the Ares I vehicle, was designed to test the avionics and control systems of the Ares rocket during the first stage of flight – a two minute propulsion period powered only by a single Solid Rocket Booster (SRB).
As the Ares I design evolved, the SRB grew from a four-segment SRB (as used on the Space Shuttle) to a five-segment SRB. The addition of a fifth segment to the SRB required changes to the SRB’s nozzle, changes that could not be implemented into the design of the Ares I-X test rocket in time for all the data the test flight was meant to gain to be useful to Ares I engineers.
Since the Ares I-X test was meant to demonstrate the ability and performance of the Ares I design, not simulate a full up 5-segment SRB thrust test – a fifth segment simulator was added to Ares I-X to give the rocket the appropriate height and to accurately simulate the airflow over the entire length of the vehicle during the 2-minute powered portion of the flight.
Receiving and integration of the Ares I-X’s Upper Stage Simulator began in early 2009 on the floor of VAB High Bay 4, with full up stacking of the vehicle (SRB and all) beginning in July in High Bay 3 – directly across from High Bay 4.
After stacking of the vehicle was complete in mid-August, nearly two months of integration testing and sensor diagnostics took place, ensuring that over 700 sensors and the avionics and control assemblies of the Ares I-X vehicle were working perfectly.
Then, one week after Shuttle Atlantis/STS-129 was rolled out to Pad-A, the Ares I-X vehicle emerged from the VAB in the early morning hours of October 20 – marking the first time since 1975 that a vehicle other than the Space Shuttle was rolled out of the VAB.
One week of launch pad operations and processing followed for Ares I-X – a timeframe that was increased from the original four day pad flow to accommodate the dual pad flow nature of I-X processing with the ongoing processing of Space Shuttle Atlantis on nearby Pad-A.
In fact, Ares I-X shared many of the pad workers who were also busy processing Shuttle Atlantis for STS-129 in November. As a result, these workers undertook the second dual pad flow of 2009 – accomplishing both flows in time for Ares I-X’s and Atlantis’ respective launch windows.
With the countdown beginning at 1a.m. on October 27 for a launch window of 8a.m. through 12p.m., the Ares I-X launch team configured the vehicle and, after waiting out the weather and a cargo ship in the SRB recovery zone, gave the go to resume the countdown and launch Ares I-X.
The countdown resumed from the T-4 minute and holding mark and counted down to T-2minutes 39-seconds – at which point Ares I-X weather officer Kathy Winters ordered a hold due to a violation of the triboelectrification weather rule.
The resulting hold mandated a recycle of the countdown to the T-4 minute and holding mark, a milestone that was accomplished in a timely manner. The launch team then waited on the weather, before eventually scrubbing the October 27 launch attempt and announcing that a second launch attempt would be made on October 28.
On the morning of October 28, the countdown once again began at 1.a.m. EDT for an 8a.m. through 12-noon launch window. During the course of the morning, the T0 time was continuously realigned to reflect changing weather conditions.
Finally, a go was given by the launch team to proceed with launch at 11:30a.m. EDT – in the center of a patch of good weather.
The countdown resumed at 11:26a.m. and proceeded without issue. At 11:30a.m. on the nose, the Ares I-X’s SRB ignited, propelling the vehicle off of Launch Pad 39B and off onto a due east trajectory from the Kennedy Space Center.
Two minutes later, the SRB burned out and successfully separated from the Upper Stage Simulator, parachuting into the Atlantic Ocean 130 nautical miles east of KSC.
Ares I-X managers and NASA hailed the test flight a smashing success. All five the flight’s primary test objectives were accomplished, giving NASA invaluable data on the performance of a single SRB first stage powered launch vehicle.
Despite the uncertain nature of the Ares I rocket’s role in whatever the future of NASA’s manned space endeavors becomes, the test flight of Ares I-X proved the concept of the Ares I first stage design and provided the engineering community with copious amounts of scientific data that can be used in the development of future U.S. launch vehicles.
Water on the Moon! – LRO and LCROSS’s Enormous Discovery:
On October 9, the LCROSS satellite and its Centaur Upper Stage impacted the moon’s south pole in the Cabeus crater. Their mission was to expose the unseen material beneath the surface of the Cabeus crater – looking for signs of water, of the past presence of water, beneath the moon’s surface.
Trailing its Centaur upper stage by four minutes, the LCROSS conducted hundreds of scientific observations of the ejecta plume created by Centaur’s impact before it too impacted the Cabeus crater, creating a second ejecta plume of denser material for analysis by the Hubble Space Telescope and Lunar Reconnaissance Orbiter.
On November 13, the preliminary results of the impacts were released by NASA, and the answer to the age old question of whether water currently exists on other bodies in our solar system was finally put to rest.
The answer was a resounding yes. Water exists in the permanently shadowed crater Cabeus on the moon’s southern pole.
“We’re unlocking the mysteries of our nearest neighbor and by extension the solar system. It turns out the moon harbors many secrets, and LCROSS has added a new layer to our understanding,” said Michael Wargo, chief lunar scientist at NASA Headquarters in Washington D.C.
In fact, the LCROSS results have shed light on the mystery surrounding the large concentrations of hydrogen observed on the moon in the past decade. Given the current evidence, NASA suspects that water may be far more prevalent and exist in much larger quantities on the moon than originally theorized.
But just how much water is on the moon is still unknown. “Multiple lines of evidence show water was present in both the high angle vapor plume and the ejecta curtain created by the LCROSS Centaur impact,” stated Anthony Colaprete, LCROSS project scientist and principal investigator at NASA’s Ames Research Center in Moffett Field, California. “The concentration and distribution of water and other substances requires further analysis, but it is safe to say Cabeus holds water.”
The results of the LCROSS experiment hold significant implications for the current NASA plan to establish a permanent, manned colony near one of the moon’s poles. If water is present in large quantities beneath the moon’s surface, future inhabitants of a lunar colony could make use of that water, providing valuable lessons on how to use natural materials/substances around the colony – lessons that could be applied to future manned missions into the solar system.
STS-129 – Stockpiling the Space Station:
Continuing on the theme of preparing for the future, 2009 came to a close for NASA with the highly successful flight of Space Shuttle Atlantis on the STS-129 mission.
After initial worries and conversations about range conflicts with an Atlas V and Delta IV rockets, both vehicles encountered launch delays – giving Atlantis the full duration of her short launch window in mid-November, a launch window that was constrained by a Solar Beta Angle Cutout from November 20 – December 5.
Officially approved for a launch on November 16, Atlantis’ processing teams saw double work with the dual pad flow of Atlantis and Ares I-X. After the successful launch of Ares I-X, work proceeded in earnest on Atlantis with installation of the Express Logistics Carriers (ELCs) -1 & -2 into Atlantis’ payload bay as Program engineers and managers worked through a Main Engine Ignition (MEI) overpressure issue.
The issue, relating to the acoustic environment present at the aft of the vehicle during SSME (Space Shuttle Main Engine) ignition was brought to the attention of Program managers when it was discovered that bolts used to attach the aft stinger pods to the OMS Pods might not be sufficient to counter the acoustic environment.
Through an aggressive engineering review of all information pertaining to the MEI acoustic environment and the stinger pod bolts, Program managers were able to clear Atlantis for flight after installation of several new sensors on Atlantis and the Mobile Launch Platform to obtain as much data as possible during the vehicle’s launch.
With launch officially approved, Atlantis’ countdown began on Friday, November 13 at 13:00 EST (1p.m. EST) for a targeted 14:28 EST (2:28p.m.) liftoff on November 16.
Atlantis’ countdown was uneventful and – with the lowest number of PRs (Problem Reports) in the history of the Shuttle Program – STS-129 lifted off at 14:28:09 EST November 16 on its first launch attempt, making Atlantis the only Orbiter to launch on her first attempt for both of her 2009 missions.
After docking with the ISS, Atlantis’ six member crew installed both ELCs onto the Integrated Truss Structure of the Space Station – thereby delivering nearly 30,000 lbs of spare parts that will enable the ISS to remain operational well into the next decade.
Three EVAs were also performed on STS-129 and Nicolle Stott transferred to the Atlantis crew before orbiter undocked from the ISS the day before the Thanksgiving holiday in the U.S.
After performing the customary late inspection of their vehicle’s Thermal Protection System (TPS), Atlantis’ crew enjoyed Thanksgiving on orbit before receiving final TPS clearance to land.
On Friday, November 27, Atlantis and her now seven member crew glided to a perfect landing at the Kennedy Space Center at 9:44:22a.m. EST, completing a 4.4 million mile mission in 10days 19hours 16minutes and 13seconds.
In all 2009 was a year of tremendous accomplishments for NASA and its dedicated workforce.
As Mike Moses, KSC Launch Integration Manager, stated “this was really about the teams and their ability to go above and beyond what we asked of them. When you think about the fact that we worked through the Flow Control Valve issue, the hydrogen leaks, and many more things and still flew five successful Shuttle missions, had two dual pad flows (three if you count the first STS-125/STS-400 flow last year), launched an experimental rocket, and mated Endeavour for STS-130 next year – all within one calendar year – that’s pretty amazing.
“It really shows the level of dedication of the teams out here and their ability to remain flexible.”