Spacewalkers complete first Pump Module replacement EVA

by Pete Harding

US astronauts aboard the International Space Station (ISS) have completed the first of the planned Extra Vehicular Activities (EVAs) to replace a failed Pump Module outside the orbital laboratory. The spacewalk, which carried the designation US EVA-24, ended slightly early, but made great progress on the overall objectives.

Pump Module failure background – Refer to LIVE COVERAGE:

The EVAs are necessitated by the failure of a component inside a pump belonging to one of the station’s two external cooling loops, which has rendered one loop unusable for cooling of internal equipment.

The space station’s External Thermal Control System (ETCS) consists of two separate cooling loops – known as loop A and loop B – which both work together to transport heat away from electronic equipment and toward the radiators, which in turn dissipate the heat into space.

The ETCS loops use ammonia fluid as a coolant, while the Internal Thermal Control System (ITCS) cooling loops inside the ISS use water, in order to prevent a potential leak of toxic ammonia inside the ISS.

The ITCS water loops transfer their heat to the ETCS ammonia loops via the use of an Interface Heat Exchanger (IFHX), which enables the external ammonia loops to transport the heat from the internal water loops to the external radiators. In addition, the external ammonia loops also provide cooling for all electronic equipment located outside the ISS.

The two ETCS ammonia coolant loops are not to be confused with the Photo Voltaic Thermal Control System (PVTCS), which are completely separate ammonia coolant loops outside the ISS which are used to provide cooling to solar array power generation equipment only.

For the ETCS, ammonia is pumped around each of the two loops by an externally located component known as the Pump Module (PM), with each loop having its own separate PM. Specifically, the current failure is in the Flow Control Valve (FCV) inside the Pump & Control Valve Package (PCVP) inside the loop A PM.

The FCV controls the temperature of the ammonia in the cooling loops by mixing cool ammonia that is exiting the radiators with warm ammonia that has bypassed the radiators. However, the failure of the FCV means that it has allowed too much cool ammonia to enter loop A, which caused loop A to drop to a temperature that is far too low (around -32 degrees centigrade) for safe cooling operation.

Maintaining stable ammonia loop temperatures (which is typically around 4 degrees centigrade) is important since if the temperatures in the external ammonia loops are allowed to get too low, then the water inside the IFHXs could freeze, causing the water pipes to crack and the IFHX to fail, which could potentially allow toxic ammonia from the external loops to enter inside the ISS.

Indeed, an investigation is already underway due to the possibility that water temperatures in the IFHX on the European Columbus module dropped below zero degrees centigrade shortly after the loop A FCV failure, although no indications that this actually occurred are present at this time.

As a result of the FCV failure, loop A was “de-integrated” from all IFHXs, essentially meaning that loop A was no longer able to cool internal loads (however cooling of external loads via loop A remains, since external loads do not have IFHXs). Some heat loads were shifted to loop B, which continues to operate.

However, since one cooling loop alone cannot handle all of the station’s needs, several items of electronic equipment had to be shut down due to lack of adequate cooling for them.

Z8These loads included one of the two Multiplexer/Demultiplexer (MDM) computers in the Node 2 module, as well as three DC to DC Conversion Units (DDCUs) also in Node 2. Since the DDCUs in question provide power to many loads inside the Columbus and Japanese Experiment Module (JEM), many pieces of equipment inside those modules was shut down too.

Over the past week, ground teams have been trying to raise the temperature of loop A back to nominal levels via several “hacks”, including use of heaters and manipulation of the Radiator Return Valve (RRV) in order to prevent too much cool ammonia (coming from the radiator) from entering into the loop.

However, since the RRV was originally designed only to either fully open or fully close, a software patch had to be designed and uplinked to the ISS in order to allow for finer control of the valve in order to achieve a “sweet spot” where ammonia temperatures were within the desired range.

Z116While ground teams did in fact successfully manage to raise the temperature of loop A back into the nominal range, ISS program managers ultimately decided to proceed immediately with the EVA Removal & Replacement (R&R) of the loop A PM and its failed FCV, rather than the alternative plan of temporarily re-integrating loop A back into the IFHXs, and instead conducting the EVAs in January.

This decision was taken due to the time-sensitive nature of the issue, with a high beta angle period due to occur from December 30 through to early January.

High beta periods are where the side-angle of the Sun relative to the ISS become too high (above 60 degrees), which leads to high temperature differentials on the outside of the station due to some parts of the ISS being in full Sun, and some being in full shadow, which makes EVAs hazardous.

Thus, as confidence in the RRV temperature management solution was not 100 percent at the time the decision was taken, if teams had elected to delay the EVAs to January and then encountered issues with the RRV temperature management solution, it would have been too late to conduct the EVAs, meaning loop A would have had to remain down for a longer period of time.

However, ISS Program Manager Mike Suffredini expressed confidence in a recent press briefing that had more time been available, a stable loop A temperate management solution involving the RRV and other means would likely have been found.

MSBU via L2 documentationLeaving loop A shut down long term is something which is not desired, since it leaves no redundancy in the event of either a Main Bus Switching Unit-2 (MBSU-2) or a loop B ETCS failure, although, per a Next Worst Failure (NWF) impacts overview document available on L2, options would exist in such a situation, such as using Russian cooling units to provide cooling to internal US Segment equipment.

The decision to conduct the PM R&R EVAs prior to the beta cut-out resulted in the delay of the Orbital Sciences’ Cygnus Orb-1 mission from 19 December to NET 7 January.

Z54Ironically, the Orb-1 Cygnus is carrying some internal cooling jumpers which would allow for better management of internal cooling loop temperatures in the event of an external cooling loop failure.

Such jumpers would have been extremely useful in the present situation, especially since the Node 2 Moderate Temperature Loop (MTL) has been running hotter than usual.

Thus, if the jumpers were aboard, it is possible that more equipment could be powered back-up due to lower temperatures in the internal cooling loops.

EVA-24 overview:

Three EVAs were planned to R&R the loop A PM, although it is possible that as little as two or as many as four could be required, depending on how easy or hard the R&R turns out to be.

A loop A PM R&R was performed fairly recently in space station terms, when in August 2010 spacewalkers Doug Wheelock and Tracy Caldwell-Dyson replaced the pump after the previous pump failed.

That resulted in three very difficult EVAs being performed, due to some very troublesome ammonia Quick Disconnect (QD) lines (line M3 in particular) not wanting to close their valves and then disconnect from the PM in order to allow it to be removed.

While those EVAs have resulted in some very valuable lessons learned, the reason that has not translated into less EVAs this time around is because the 2010 EVAs did not include the stowing of the failed PM outside the ISS – instead it was left in a temporary location with a later Space Shuttle EVA crew stowing it in its correct location.

Z14However, with the Space Shuttle now retired, that is no longer an option, and so for this PM R&R, the ISS crew must stow the failed PM themselves – meaning the same number of EVAs are being planned this time as were conducted in 2010, despite the improved knowledge of QD ops.

Due to its relatively recent replacement, the current loop A PM has only been in service for just over three years. Luckily, three spare PMs are already pre-positioned outside the ISS, and since only one component of the current PM is failed (the FCV), it may be possible to re-use the current PM in future by connecting a new, external FCV to it.

NASA astronauts Rick Mastracchio and Mike Hopkins conducted the EVA, with Mastracchio already having prior EVA experience due to his previous Space Shuttle flights, and Hopkins having no previous EVA experience, since he is making his first spaceflight after being selected as an astronaut in 2009.

The EVA was planned to last six and a half hours, beginning at 12:01 AM GMT. However, it ended after just five hours and 28 minutes, potentially due to Mastracchio apparently feeling unwell, as much as he didn’t want to go into detail over the loop.

Luca drinking water from helmet during EVA-23Mastracchio was wearing Extra Vehicular Unit (EMU) number 3010, and Hopkins was wearing EMU 3011 – which is the very same EMU as the one which sprung a leak during US EVA-23 in July, which filled the helmet of Italian astronaut Luca Parmitano with water.

Since then however, an extensive investigation has been performed, and while no root cause has yet been conclusively identified, findings thus far do support a theory that contaminants in the water used for suit cooling blocked the Fan-Pump Separator (FPS), causing the water to leak into the helmet.

Z9The FPS in suit 3011 has since been completely replaced, and the suit was subsequently re-tested with no recurrence of the leak, and thus suit 3011 is considered to be the most recently and extensively serviced suit on the ISS, hence the decision to use it.

Additionally, mitigations have been put in place to minimise risk in the event of a water leak into the helmets, such as the placement of Helmet Absorption Pads (HAPs) inside the helmets to soak up any water, and the placement of a “snorkel” inside the suits to allow the astronauts to draw air from the middle of the suits, this preventing drowning even with a flooded helmet.

Both suits performed well during the EVA.

EVA-24 procedures – Refer to LIVE COVERAGE:

US EVA-24 mainly focused on disconnecting the failed loop A PM from its location on the Starboard 1 (S1) Truss – including the infamously difficult ammonia QD disconnections. See detailed overview/proceedures presentation for EVA-24, available on L2.

Prior to the PM’s removal, ground teams shut down the loop A PM, since it currently remains powered as it is still providing cooling to external equipment. This means that some external power distribution equipment lost its cooling and needed to be shut down. However, an internal Lab-Truss Contingency Jumper (LTCJ) has been used to provide alternate sources of power to some equipment to reduce the impacts of the shut-downs.

Z11Upon egressing the Quest Airlock (A/L), the first order of business for EV-1 (Mastracchio) was to set up the Space Station Remote Manipulator System (SSRMS), which the astronauts will “ride” during the EVA, with the arm being controlled by Japanese astronaut Koichi Wakata from inside the ISS.

Upon his egress, EV-2 (Hopkins) began work to prepare the failed PM for removal, assisted by EV-1 upon completion of the SSRMS prep tasks.

The failed PM preps included the opening the Multi Later Insulation (MLI) “shroud” that protects it, releasing clamps from the PM fluid lines, and removing Spool Positioning Devices (SPDs).

Less than two hours into the EVA, both EV-1 and EV-2 moved on to by far the hardest task of the day – disconnecting the four ammonia lines from the failed PM.

The lines are designated M1, M2, M3, and M4, with M3 being the most difficult during the previous loop A PM R&R in August 2010.

Z3Disconnecting the ammonia jumpers involved manually pulling a lever to close the bail and shut a valve (to prevent ammonia loss into space via an open line), and then pressing a button and pulling the jumpers away from their connecting ports on the PM.

As demonstrated on the August 2010 EVAs, this process can require significant physical strength and manipulation from the spacewalkers, as the ammonia lines can be very resistant to bail closure and disconnection.

M4 and M3 were disconnected first, with both these lines then connected to a PM jumper box, and then lines M1 and M2 were disconnected.

EV-2 then left EV-1 to finish the QD ops alone, with EV-2 moving on to disconnect five electrical jumpers from the failed PM, and preparing the new PM, located on External Stowage Platform-3 (ESP-3) on the S3 Truss, for removal by opening up its MLI housing and removing Velcro straps.

Z4At one point, the EVA was 90 minutes ahead of schedule, allowing the spacewalking duo to move into the EVA tasks set for the next spacewalk. This resulted in the EVA-2 tasks of removing the failed PM from its home and placed into a temp stow position.

There was even potential to extend the EVA to move into tasks relating to preparing the spare PM for its installation. However, the decision was made to end the EVA early. Despite this, the spacewalking duo managed to get far more task completed that had been scheduled for what was to be a six and a half hour EVA.

Both spacewalkers ended the EVA by heading back to the A/L.

EVA-25 has since been moved back one day to December 24, which will be outlined in a future article.

(Images: via L2’s ISS Section – Containing presentations, videos, images and ungoing ISS status updates, with additional images NASA and ESA).

(Click here: – to view how you can support NSF and access the best space flight content on the entire internet).

*Please remember to share this article on social media using the below options. You have a responsibility to promote space flight content to your friends and family via one simple click*

Related Articles