Hubble returns to science operations on backup payload computer

by Mihir Neal & Haygen Warren

NASA teams have successfully resurrected the iconic Hubble Space Telescope after it first experienced problems with an onboard computer on June 13 and suspended all science operations. The telescope is now fully operational and the switch to the backup hardware — which included powering on the backup payload computer — was performed successfully on July 16. With all its science instruments back online, the collection of science data resumed on July 17. 

“Hubble is an icon, giving us incredible insight into the cosmos over the past three decades,” said NASA Administrator Bill Nelson. “I’m proud of the Hubble team, from current members to Hubble alumni who stepped in to lend their support and expertise. Thanks to their dedication and thoughtful work, Hubble will continue to build on its 31-year legacy, broadening our horizons with its view of the universe.”

On Sunday, June 13, at 20:00 UTC, Hubble’s main computer stopped receiving a “keep-alive” signal from the telescope’s payload computer. Once Hubble’s main computer recognized the payload computer halt, it immediately put the telescope into safe mode, where all science instruments were shut down and only essential telescope systems, such as power and communication systems, are operational.

Hubble teams in the control center at the Goddard Space Flight Center in Maryland recognized that Hubble was in safe mode and later attempted to restart the telescope the following day. However, the halt occurred yet again, and Hubble went back into safe mode.

Thankfully though, the payload computer to blame for this issue was built with redundancy. Thus, a second computer is available for use in orbit in the event that a failure occurs.

The Hubble team was able to pinpoint the underlying problem in the Power Control Unit (PCU) of the telescope’s payload computer. It contains a power regulator which ensures a steady voltage supply of five volts to the payload’s computer hardware.

The PCU and the payload computer reside in the Science Instrument Command and Data Handling (SI C&DH) unit, which controls, coordinates, and monitors Hubble’s science instruments. To check if the PCU is working optimally, a secondary protection circuit senses the voltage levels leaving the power regulator. If the voltage falls below or exceeds the allowable levels, this secondary circuit alerts the payload computers to cease operations and enter into “safe mode.”

Safe mode is a capability built into most modern spacecraft that pauses all non-critical functions when a technical issue occurs. This keeps the spacecraft in a stable configuration and prevents further damage while controllers troubleshoot the issue.

The teams conducted a series of multi-day tests including attempts to restart and reconfigure the computer and the backup computer. Although these tests were initially unsuccessful, they gave ample insight into the underlying problem, and the teams discerned that either the voltage level from the regulator is outside of acceptable levels (thereby tripping the secondary protection circuit), or the secondary protection circuit has degraded over time, yet another indication of Hubble’s growing age.

Without the ability to service the telescope and update its instruments, Hubble teams have had to work around Hubble’s various failures throughout the past 12 years. One of Hubble’s most recent failures occurred in March of this year.

The team was not able to reset the PCU using ground commands, so the decision was made to switch over to the backup SI C&DH unit, which contains the backup PCU. The reviews and testing procedures for the switch were completed and the go-ahead was given by NASA management. The switch was a multi-day process that started on Thursday, July 15, and was completed successfully the next day.

After a successful switch, the teams monitored the hardware, ensured everything was working as expected and started the process of recovering the science instruments out of their safe mode configurations. They ran various procedures and ensured that instruments were at stable temperatures. After some initial calibrations, normal science operations were resumed on Saturday, July 17.

Hubble is serviced for the final time on board Space Shuttle Atlantis on STS-125 – via NASA

Scheduled operations which were missed over the month that the spacecraft was in safe mode will be rescheduled.

This switch was similar to the one performed in 2008, which allowed Hubble to continue normal science operations after a Command/Unit Science Data Formatter (CU/SDF) module, also a part of the SI C&DH unit, failed. The final Hubble servicing mission, STS-125, which included NASA astronauts Megan McArthur, Scott Altman, Gregory C. Johnson, Michael T. Good, John M. Grunsfeld, Mike Massimino, and Andrew Feustel, replaced the entire SI C&DH unit, including the faulty CU/SDF module.

Hubble’s payload computer is tasked with coordinating and operating the many science instruments onboard the telescope. Currently, there are four operational instruments on Hubble, including the Advanced Camera for Surveys (ACS), the Cosmic Origins Spectrograph (COS), the Fine Guidance Sensor (FGS), and Wide Field Camera Three (WFC3).

This payload computer is a 1980s NASA Standard Spacecraft Computer-1 (NSSC-1) system located in Hubble’s Science Instrument Command and Data Handling Unit. The NSSC-1 system features four memory modules containing 64K of Complementary Metal-Oxide Semiconductor (CMOS) memory.

(Lead photo via NASA)

Related Articles