The aim of mission evaluation is to extract the maximum amount of information from each flight for use in managing and planning future missions and programs. Its products also go before the scientific community and the public.
It is not surprising that a complex vehicle, such as a spacecraft, experiences peculiarities and system problems that have not been considered previously. The prime task of the mission evaluation team during a mission, therefore, involves identifying and understanding these peculiarities and problems and determining what action should be taken. After a mission, the evaluation team also has responsibility for (1) flight anomaly investigation and resolution and (2) preparation and publication of the mission report.
For the mission evaluation, the many engineering disciplines involved in Apollo are represented by groups of specialists, each managed by a NASA team leader. In general, the specialists have experience with a particular system from the initial design conception through development and testing of the hardware. Consequently, they have an intimate knowledge of the system operation and limitations. The specialists are selected both from contractor and from NASA engineering organizations and work as an integrated team. Before a mission, the selected team participates in certain simulations with Mission Control Center personnel. This training serves to integrate the two organizations into a single operational unit.
The responsibility of each team leader includes coordination with other team leaders to ensure that any solution or recommended course of action for his system does not jeopardize other systems. The team leaders report to the evaluation team manager, who is responsible for the overall operation of the team. The team manager is assisted by senior contractor engineering managers who have immediate access to the company facilities to provide any necessary support. All inputs and requests by the team leaders are integrated and reviewed by the team manager and by contractor engineering management. The team manager interfaces with the Flight Director through the Apollo Spacecraft Program Manager. During a mission, the team mans assigned positions in the mission evaluation room (fig. 6-1) around the clock.
To support the mission evaluation team effort at the NASA Manned Spacecraft Center (MSC), the prime contractors maintain similar teams of specialists at their facilities for analyses, tests, and related activities. These contractor teams are also under the direct management of the mission evaluation team at MSC.
A typical example of the activity associated with these supporting teams occurred when a lightning discharge on Apollo 12 caused the loss of inertial reference (tumbling) of the inertial platform. In this instance, the support team at the Massachusetts Institute of Technology performed a test to simulate the conditions so that it could be understood how the potential discharge caused the platform to tumble. It was important to have rapid verification that the platform had not been damaged. A quick response was received by the evaluation team, and the mechanism that had caused the conditions in flight was determined.
Many other times during Apollo flights, the evaluation team has provided test information that was valuable in understanding a problem and determining the best course of corrective action. In carrying out its task, the team must be constantly aware of the total system performance as the mission progresses. The team monitors data that are received by the tracking stations, transmitted to and processed by the Mission Control Center, and displayed in the Mission Evaluation Room on closed-circuit television. In addition, air-to-ground voice communications are monitored.
The frequency of the Apollo flights demands that anomalies be quickly identified and resolved so that prompt corrective action may be taken. Analysis of the data for problems and anomalies must be compressed, therefore, into a relatively short period. Also, within this time frame, the anomalies must be analyzed to the extent that the  mechanism associated with the cause is clearly understood. Of course, anomalies which involve flight safety or which would compromise the mission require corrective action before the next flight.
The first problem is to identify the anomalies. Many anomalies are easily recognized because a component has failed to operate. The most difficult cases, however, appear when the data from the system are not sufficient for an understanding of all the normal operating characteristics. A typical example of this condition occurred during the Apollo 7 mission when the battery recharging characteristics fell below predicted levels throughout the flight. Preflight tests had been conducted at the component level, but an integrated test of the entire system, as installed in the spacecraft, had not been included. Post-flight tests, using the actual flight hardware, showed the same characteristics as those experienced in flight. A detailed analysis showed that the line resistance between components of the system greatly controlled the amount of energy returned to the battery. The corrective action for this anomaly was to require that integrated system tests be performed to establish overall system characteristics of each installation and thus ensure adequate battery recharging. In this case, if the total system operating characteristics had been established previously, there would have been no problem.
Sometimes, also, data will not support an accurate analysis of a problem. This situation occurs because of insufficient flight instrumentation or absence of recorded data. The mission evaluation team, in attempting to focus on the anomalous condition, must rely on the history compiled from previous missions and on the experience gained from tests and checkout and knowledge of the failure history.
After an anomaly is identified, the cause and the corrective action must be identified. The approach may be experimental or analytical or both.
The depth and the extent of the analysis vary considerably and depend on the significance of the problem. For example, on the Apollo 6 mission, a structural failure occurred in the adapter that holds the service module to the launch vehicle and also houses the lunar module during the ascent portion of the flight. This adapter consists of the largest honeycomb structure designed and developed for any application. Long-range photographs show that the structure lost the outer face sheet from the honeycomb-sandwich panel by explosive separation. (See fig. 6-2. ) Response to the effects was obtained from many measurements in the command and service module, lunar module, and launch vehicle.
In resolving this problem, four possible causes were investigated: structural dynamics of the launch vehicle, the dynamic loads of the lunar module, dynamic modes oŁ the adapter shell itself, and quality of manufacture.
The investigation first focused on an understanding of the coupled vibration modes and characteristics of the launch vehicle and spacecraft. Extensive vibration tests ruled out vibration as a cause.
Further tests and analyses indicated that the internal pressure of the sandwich panels could have caused the failure. If a large unbonded area existed between the honeycomb and the face sheets, then aerodynamic heating of the air and moisture entrapped in the panel could have caused a pressure buildup in the honeycomb, separating the face sheets in an explosive manner. The most probable cause of this condition was traced to the manufacturing process (the possibility of unbonded face sheets remaining undetected). To circumvent the problem in the future, the inspection procedures for the structure were changed. Also, cork was placed on the outer surface of the adapter to reduce aerodynamic heating, and small vent holes were drilled through the inner face sheet into the honeycomb to reduce internal pressure.
Analysis of this anomaly involved testing full-size equipment under dynamic and static conditions, performing many experimental tests of smaller test articles, and conducting extensive structural analysis at the various NASA centers and at many contractor organizations. This effort verified that the structural integrity of the adapter was satisfactory for subsequent missions and established that the failure was not caused by a design deficiency.
The other extreme of treating a problem concerns anomalies for which no corrective action is taken because of the nature of the failure. For example, on the Apollo 11 entry-monitor system, an electroluminescent segment of the velocity counter would not illuminate. A generic or design problem was highly unlikely because of the number of satisfactory activations experienced up to that time. A circuit analysis produced a number of mechanisms that could cause this failure, but there was no failure history in any of these areas. This case is a typical example of a random failure. The basic design concept of the spacecraft overrides such failures by providing alternate procedures or redundant equipment. Consequently, this type of failure does not demand corrective action.
Causes of anomalies involve quality, design, and procedures. The quality items include broken wires, improper solder joints, incorrect tolerances, improper manufacturing procedures, and so forth. The structural failure of the adapter on the Apollo 6 mission, previously discussed, illustrates such a quality problem.
System anomalies caused by design deficiencies can generally be traced to insufficient design criteria. Consequently, the deficiency can pass development and qualification testing without being detected, but will appear during flight under the actual operational environment. A typical example of a design deficiency is the fogging of the Apollo 7 command module windows between the inner surfaces of the three window panes. A post-flight examination showed the fogging to be a product of the outgassing of the room-temperature-cured sealing material used around the window. The design  criteria did not require the sealing material to be cured; curing would have prevented outgassing in the operating temperature and pressure environment.
Procedural problems in operating various systems and equipment are ordinarily corrected simply. For example, an incorrect procedure was used to chlorinate the crew's drinking water. This resulted in an improper mixing of the chlorine and water, giving the water a strong chlorine taste. The procedure was revised, and the water no longer had an objectionable taste.
While a mission is in progress, the scope of a problem must be defined before action is taken; that is, the problem must be isolated to the extent of establishing possible effects on the spacecraft systems. Every effort is concentrated on developing a procedure that permits the mission to continue despite the problem. Complete understanding and analysis of the problem frequently require post-flight data analysis. crew debriefings, or testing of the spacecraft hardware.
When lunar module hardware exhibits a problem, it may be returned for post-flight analysis and testing, provided the command module has stowage space available and the lunar module gear fits the space. For example, the color television camera that failed on the lunar surface during the Apollo 12 mission was scheduled to be left on the lunar surface but was returned to earth for post-flight testing.
When post-flight testing cannot be done because the hardware cannot be returned to earth (e. g., the lunar module and service module hardware), every effort is made to perform in-flight tests to understand and isolate the cause of the failure. A typical case occurred during the Apollo 12 mission when the high-gain antenna, mounted on the service module, occasionally exhibited reduction in signal strength. The evaluation team developed a test to isolate the anomaly to specific areas and components of the high-gain-antenna system. The test, conducted later in the mission, produced data which, combined with analysis, isolated the cause.
An additional search for anomalies is conducted when the command module is returned to the contractor's facility for a general inspection. Those systems and components identified as having a problem or failure are removed from the vehicle and tested to establish the cause, or tests are performed with the affected equipment in position in the command module. In general, these post-flight tests are limited to those required for an understanding of flight problems.
The concerted effort on anomalies during the flight continues after the mission until each problem is resolved and corrective action is taken. This activity requires close coordination and cooperation between the various Government and contractor groups. Emphasis is placed on prompt and exact analysis for the understanding and timely solution of each problem.
To accomplish this task, a problem list is maintained during and after each flight. This list contains a discussion of each problem, the action being taken to resolve it, the engineer or contractor responsible for completing the action, and the anticipated closure date.
 After the flight, the most significant anomalies are published in the 30-Day Anomaly and Failure Listing Report. This report identifies the anomaly, discusses the analysis, and identifies the corrective action to be taken. The Mission Report, which is published approximately 60 days after a flight, includes a section which discusses all significant anomalies and corrective actions. The less significant problems are discussed in the appropriate sections of the Mission Report. A separate report covers anomalies not resolved before publication of the Mission Report. The Mission Report serves as an historical record of the pertinent events of the mission and includes discussions of systems performance, crew activities, flight anomalies, and scientific experiments. Supplements to the Mission Report are published to present detailed analysis of the engineering systems, medical aspects, and scientific aspects of the mission.
The techniques used in Apollo for assessing systems performance reflect significant advancements over those used in previous manned programs. The method of handling flight anomalies, including the depth and extent of analysis, has been sufficient for the time and economy constraints imposed by the program. The Apollo concept has proved to be very effective in organizing many contractors and federal organizations into one central team for the real-time support and post-flight evaluation of each mission. These concepts enabled the Apollo Program to advance at the rate required to achieve the national goal of landing man on the moon before 1970.
Home - NASA
Office of Logic Design
Last Revised: February 03, 2010
Web Grunt: Richard Katz