Examining the probable causes of Progress MS-04 failure
In the month and a half since the Progress MS-04 spacecraft was lost in a third-stage launch mishap on 1 December 2016, Russian federal investigators and Roscosmos have narrowed in on the two most probable causes for the third stage failure of the Soyuz-U launch vehicle. The investigation, which has taken longer than Russian officials wanted, has led to two potential causes for the failure – foreign object debris or improper manufacturing of the RD-0110 third stage engine.
The failure of the progress MS-04 mission to reach orbit followed a flawless countdown and what appeared at the time to be the beginning stages of a flawless ascent for the Soyuz-U rocket.
After rising from the launch pad at the Baikonur Cosmodrome, the Soyuz-U rocket performed its pitch maneuver and headed out on a trajectory to insert the Progress MS-04 resupply craft into a 51.6 degree inclination orbit en route for the International Space Station.
First stage performance was nominal, as was second stage.
Following second stage separation and third stage ignition, the initial moments of third stage flight also appeared to be going well until the T+ 382.2 second point (T+ 6 minutes 22.2 seconds), at which time communication with Soyuz-U suddenly became ratty – or of poor quality – and then ceased altogether 0.2 seconds later.
However, despite a complete loss of telemetry from Soyuz-U’s third stage and from Progress as well, no indication that a major malfunction had occurred came from Mission Control Center – Moscow (MCC-M), which continued to show nominal ascent graphics accompanied by real-time, nominal ascent timeline calls.
The first hint of a major mishap from inside MCC-M came seconds after the time when a nominal third stage burn would have ended (T+8 minutes 40 seconds), when controllers did not call out the spacecraft separation milestone – which would nominally have occurred 8 minutes 49 seconds after liftoff.
Shortly thereafter, conflicting reports confused matters further.
First, it was reported that Progress had indeed separated from the third stage and that telemetry indicated that Progress had begun to deploy its solar panels – before telemetry from Progress was lost.
Then came word that the third stage burn was not nominal and had ended prematurely.
This was followed by word that Progress MS-04 had been placed into a preliminary orbit, with controllers trying to determine exactly what orbit that was.
Then, word began to seep out that all telemetry from the Soyuz-U rocket had been lost 2 minutes 17 seconds prior to the point when a nominal third stage shut down would have occurred.
At the same time, word from a remote region in the Russian Federation appeared on social media, documenting first-hand accounts from residents in the Republic of Tyva (also spelled Tuva) who witnessed high-altitude explosions along the nominal ascent flight path of Soyuz/Progress.
Eventually, it was confirmed from Roscosmos that the launch of Progress MS-04 ended at T+ 383 seconds at an altitude of 190 km (118 mi) with a failure of the third stage of the Soyuz-U booster.
With this understanding and confirmation, it was realized, too, that telemetry transmission from Progress MS-04 itself had continued in the immediate aftermath of the third stage’s failure, accounting for the MCC-M-received data indicating that Progress had separated from the third stage and begun to activate its nominal post-separation programs – including deployment of its KURS antenna and solar panels.
Ultimately, the remaining parts of the Soyuz-U third stage and Progress reentered the atmosphere and impacted the ground in an unpopulated, remote mountainous area in Tyva approximately 3,500 km (2,200 mi) downrange from the launch site at Baikonur.
Investigation and outcome:
Over the initial days that followed the mishap, numerous reports made unsubstantiated claims to various theories as to the cause of the failure.
The one thing, however, that was immediately clear was that Progress MS-04 was the second Progress craft to succumb to a third stage anomaly at roughly the same point in flight.
On 24 August 2011, Progress M-12M experienced a third stage launch failure of its Soyuz-U booster at the 325 second point in the flight.
The cause of that failure was traced to a blocked fuel duct of the Blok I third stage’s RD-0110 engine. The blocked duct triggered the Soyuz-U’s computer to issue an AVD – Emergency Engine Cutoff – command via thrust termination of the RD-0110.
For Progress MS-04, the official State Commission investigating the mishap initially stated that a final report on the failure would be released by 20 December 2016.
By 10 December, the official State Commission confirmed that the received telemetry indicated that no AVD command was likely issued by Soyuz-U’s third stage computer before the abrupt termination of the flight.
Confusingly, though, was independent data from Progress and the last vestiges of telemetry from Soyuz-U’s third stage that revealed no warning signs of an impending mishap.
As the State Commission continued to review the confusing nature of the accident telemetry, the official date for closure of the investigation slipped from 20 December, to 30 December, before moving into January.
By 19 December, a preliminary assessment of the telemetry revealed that everything was nominal with the flight right up to the T+382.2 second mark – at which point telemetry interference from both the Soyuz-U rocket and Progress MS-04 began.
At the time that telemetry interference began, data indicated that all was nominal aboard Soyuz-U and Progress MS-04.
Complete interruption of telemetry from Soyuz-U followed at T+382.4 seconds and was never recovered.
At the same time, T+382.4 seconds, telemetry interference from Progress MS-04 continued and indicated that Progress’ computers received “separation contact closure” and began control system preparations for post-separation propulsive operations.
At T+384 seconds, full telemetry returned with Progress, with data showing falling pressures in the DMT11 and DMT21 thermal control lines.
Confirmation of the fall in pressures of both thermal control lines provided crucial insight into what was happening at the time.
The fall in DMT21 pressure confirmed that the Soyuz-U third stage’s RD-0110 engine had cutoff – though no telemetry received could confirm whether the engine was shutdown by an ADV command or another failure.
Conversely, the fall in pressure on the DMT11 line indicated that Thermal Control System loop No. 1 on Progress MS-04 had been breached.
Likewise, the immediate telemetry at T+384 seconds showed that the tank pressurization sequence in the Progress to fill its propellant lines with oxidizer and fuel had commenced and that Progress itself was physically separating from the third stage at that time.
Immediately thereafter, telemetry from Progress MS-04 continued for another 34 seconds, during which Progress completed pressurization of its propellant tanks – followed by no indication of decreasing tank pressure and the complete loss of data at T+418 seconds.
Based on the data received, by 28 December, Progress MS-04 was cleared of any involvement in the accident – as all data from Progress indicated that the cargo craft’s computers and systems were reacting to a failure of the Soyuz-U third stage and did not contribute to the failure.
By the beginning of January, the investigation was working through the verification of “improbable assumptions” needed to explain the telemetry that was received from the third stage and account for the emerging leading theory to explain the failure.
This emerging theory related to a failure of the third stage oxidizer tank from either unknown dynamic loads on the tank’s structure that were amplified by the improper assembly of the tank or from unexpected vibrations in the bearings of the oxidizer turbopump on the RD-0110 that then created out-of-tolerance loads on the aft bulkhead of the oxidizer tank – both of which could not be proven conclusively by the telemetry received.
By 11 January, Roscosmos officially completed the investigation, stating that the presence of foreign particles or the improper assembly of the oxidizer turbopump on the RD-0110 third stage engine led to a fire in the turbopump that subsequently caused the disintegration of the pump and the complete failure of the oxidizer tank.
If the accident was caused by the improper assembly of the turbopump, a misalignment (too close) of the pump’s sleeve and shaft, floating rings, and impellers could have precipitated an imbalance in the turbopump’s rotor, leading to increased vibrations that breached the limits of the oxidizer tank’s tolerance loads.
With a determination in place, the challenge now facing Roscosmos is the commonality between the soon-to-be-retired Soyuz-U (scheduled to make its final flight in February) rocket’s third stage RD-0110 engine – which is also used as the third stage engine for the Soyuz 2.1a rocket, used for cargo resupply missions to ISS, and the Soyuz-FG, used for crewed Soyuz missions.
At present, it is understood that the third stage for the final Soyuz-U rocket has been returned to its processing facility in Samara, Russia, for additional checks and processing.
Moreover, the prolonged effect of the failure and its investigative results are not completely known at this time.
Currently, the launch of the next Soyuz rocket is scheduled to be the MS-05 Progress resupply flight in February with the final Soyuz-U rocket.
At the time of the Progress MS-04 failure, the Progress MS-05 mission was set to launch on 1 February – though that has now been postponed until at least the 21st of the month.
Whether that mission will launch on the final Soyuz-U rocket or the Soyuz 2.1a variant is unknown at this time – though no official indication has been given that a launch vehicle change is in work.
Finally, while the failure of the Soyuz-U’s third stage is a mark on the rocket’s storied history, it should be noted that Soyuz-U is the longest serving rocket in history, with 44 years of continuous operations spanning 785 missions and a 97.3% success rate.
(Images: Roscosmos, Anatoly Zak, NASA, RSC Energia)