NASA Office of Logic Design

A scientific study of the problems of digital engineering for space flight systems,
with a view to their practical solution.

The NASA ASIC Guide: Assuring ASICs for SPACE

Chapter Five: The Review Process

Objective:

To describe to an ASIC program manager the nature and importance of independent assessments of an ASIC program.

In an ASIC program, reviews start early, during requirements identification and specifications definition, and continue on through device fabrication, test, and characterization phases. The responsibility for arranging and coordinating reviews falls to the ASIC program manager. Without the feedback these reviews provide, there is little hope of avoiding second pass silicon and delivering an ASIC within time and budget constraints.

Unlike off-the-shelf VLSI devices, ASICs call for the user and designer to participate heavily in the early reviews. For an off-the-shelf VLSI device, all requirement, specification, design and production reviews take place in-house at the vendor's facilities. Users rarely are invited to participate. The user mainly becomes involved in the "back-end of the line," i.e., during final assembly, electrical testing, environmental screening, reliability testing, and characterization--even then only in a high-reliability procurement.

In an ASIC program, however, users and designers participate and often coordinate those early reviews, which include:

• specifications/requirements review
• implementation (schematic) review
• preliminary design review
• critical design review
• chip sign-off review (build-readiness)

In this chapter we will discuss the nature of these reviews, how to manage a review process, and major topics for review. We will also review areas that need special attention and deliverables.

Nature of Reviews

Review boards monitor the ASIC program through each of the nine major tasks: Setting requirements; ASIC Trade-offs; Vendor Selection; Partitioning; ASIC Implementation; Physical Layout; Manufacturing; Test and Characterization; and Part Acceptance. Each review evaluates the progress and future tasks of the ASIC program, provides constructive feedback to many aspects of the program, and identifies real and potential problems.

In Chapter 2 of this section, we discussed the need for communication between a number of disciplines during an ASIC program. Reviews provide the forum for constructive criticism of the ASIC program from a cross-disciplinary perspective. Representatives of each discipline bring their experience and viewpoint to design, tools limitations, test and testability issues, reliability, and packaging, etc. Each review develops communication networks that the ASIC team can tap into during the rest of program as well as for future ASIC designs. As an introduction to reviews, we will discuss:

• review meetings
• managing a review process

THE REVIEW MEETING

The format of ASIC reviews resembles that of other technical program reviews. In a typical ASIC review, a chair person is selected. A group of experts and peers appropriate to the review subject hear design engineers, vendor engineers, ASIC program managers, and other ASIC team members present status and detailed information about the ASIC program. Discussions between the experts and the ASIC team follow each formal presentation allowing an opportunity to examine significant issues. These discussions often reveal the need for more information, in which case the group determines appropriate sources and calls in those individuals.

During the meeting, the official review board experts note major issues as "action items." The review board may resolve the action items later in the meeting or may invest additional time after the meeting is over. Typically, the review is not considered closed until all the action items are satisfactorily resolved.

MANAGING A REVIEW PROCESS

"Unlike off-the-shelf VLSI devices, ASICs call for the user and designer to participate heavily in the early reviews."

The review process includes selecting personnel, tracking requirements, documentation, and review activities. Each of these processes contributes to the overall success of the ASIC program and needs attention.

The ASIC program manager should be responsible for all review activities. In the case of vendor-sponsored reviews (some PDRs and CDRs), the ASIC program manager still needs to verify that all important tasks for the review have been done. In either case the review tasks include:

• selecting a chair person
• selecting review topics
• determining the agenda
• deciding the depth of review
• determining appropriate board membership
• distributing the review materials
• setting the schedule
• arranging the review time and place
• establishing review action close-out procedures

Selecting Review Personnel

Each major review may require different board members. Don't underestimate the importance of selecting appropriate board members. You will rely on their expertise to achieve a first pass working silicon. Depending upon the review task, select representatives from the following areas:

• design
• product architecture
• customer engineering staff
• CAE/CAT support
• ASIC center personnel or other resident ASIC experts
• parts reliability
• quality assurance
• procurement
• ASIC vendor

We suggest limiting the number of individuals at any particular review to those with direct contributions--too many participants will make it difficult to complete the review. It is perfectly acceptable to call people into a review after it has begun or, if necessary, to hold an action item open until an individual outside the review has had a chance to analyze the information.

The essential people at any review are those who will be able to point out areas that still need work. For example, ASIC PDRs and CDRs involve heavy vendor interaction. Thus, at PDRs and CDRs, you should expect significant vendor attendance. However, when performing a specifications review, the board may only need vendor data for the feasibility of a specification implementation and, therefore, the vendor representatives need not attend.

When in doubt about the appropriate make-up of a review board, ASIC experts can provide advice about suitable board representatives. You need to identify people in your organization with the appropriate expertise and with the ability to constructively contribute in a review environment.

Review Documentation

Whatever the topic of review, write the agenda documents using clear language and a format that makes the topics easy to find. Send these documents to each board member ahead of the review, allowing them sufficient time to digest the information and provide suggestions in their areas of expertise. Please refer to Section One: Chapter 4: "Information Management" for discussions on documenting various aspects of an ASIC design.

Select or create a format for action items, which should include:

• review title
• review date
• action item number
• originator
• originator organization
• topic of item
• description of item
• recommendation for resolving item
• disposition information, including:
  • disposition status (accepted/rejected/taken under advisement, etc.)
  • disposition rationale
  • individual responsible for disposition
  • manager responsible for disposition
  • organization responsible for disposition
• any tracking data needed for the specific project

Review activities

Note: these activities apply to a review for an in-house design or a design done by a third party.

• Verify requirements: Requirements come from, and can be verified by, various sources such as your organization, your top-level project group, your system/subsystem group and/or the vendor. For instance, during a requirements or implementation review, a designer may present the DFT plan to demonstrate how testability requirements will be satisfied; if the vendor representative is present, he or she can vouch for its feasibility in the vendor's DFT methodology.

• As another example, during a requirements or implementation review the vendor may present the packaging information along with its qualification status, its availability, and price. The review board can then verify that these vendor specifications meet the design's requirements.

See Section One: Chapter 1 for more on the sources of requirements.

• Identify problems: The review process detects problems from many different perspectives. Representatives from different groups hone-in on problems that relate to their area of expertise, thus offering a thorough examination of the proposed ASIC.

• For instance, the architecture group may point to a possible incorrect mode of some operation; the design group may find some path being ignored in the path analysis program or a possible setup/hold time violation; the parts reliability group may provide input on DFT results; the vendor may point out some package specifications not being properly followed, etc.

• Locate cause of problem: Once identified the board then looks for the possible causes. The expert who detects a problem can usually find the cause. Occasionally, experts from other areas identify the cause. Certain problems may be easily diagnosed; other more complex ones may have to be taken off-line and addressed at a later date.

• Address concerns: While concerns do not necessarily represent problems, they point out issues that the experts believe have not been sufficiently addressed. Experts may address some concerns related to the areas of CAE tools usage, aspects of design, package issues, reliability, qualification, procurement, cost, and scheduling. This process step will generate some action items that will need attention at the appropriate review meetings.

• Make recommendations: Since review boards focus their expertise on the process, they provide excellent forums for constructive criticism. Years of experience in a particular area enable experts to provide insights or direct solutions to a problem. Because board members look at a problem from a variety of perspectives, they may provide more than one solution.

• Develop communication channels: Designers need to open communication channels to draw suggestions from the pool of experts to address particular aspects of an ASIC program. These channels provide some key contacts for the current program as well as for future applications.

The QML and QPL programs cover some of the review process and related topics, particularly relating to the vendor process qualification, test procedures, and screening. We encourage managers to look at these documents from the review perspective.

As a general note, document the review materials, distribute them well ahead of the scheduled time of review, familiarize yourself with board members early on and make sure that the appropriate experts can attend the review.

Major Topics for Review, and Their Deliverables

"Major reviews" refers to the specification review, implementation review, PDR, chip sign-off, CDR, and flight build review. Other important reviews include: technology; vendor and tools selection review (described in Section One: Chapter 2); statistical process control and in-process monitor review; revalidation review; drop-in review; review of all tests performed per various MIL-STDs; and failure analysis review. The QML program document, MIL-I- 38535, describes these reviews in detail.

Below we discuss each of the major reviews and their deliverables in chronological order.

SPECIFICATIONS REVIEW

The specifications review takes place after the initial round of identifying requirements and generating specifications. The review must check the following items:

• Completeness of specifications: Specifications have to cover all the requirement sources, such as the organization, the project, the system, and the vendor. At this time address requirements related to reliability, radiation hardness, DFT, flight class uniqueness, and other top-level concerns. Representatives from each source should participate in the review to make sure that the specification covers their requirements.
• Compatibility: Representatives from each source must also review the specifications for compatibility with existing design work as well as for compatibility with future applications of the device.

Again, make sure that representatives from each discipline covered in the specification offer their input.

The above review process should deliver well-defined, implementable, and verifiable specifications, as discussed in Chapters 3 and 4 of this section.

IMPLEMENTATION REVIEW

The ASIC design group usually conducts an implementation review. Representatives from other design groups whose specifications have direct influence on the ASIC design also offer critiques at this review, along with representatives from an ASIC center or a parts reliability group, who may critique DFT implementation, proper component (library) usage, etc.

• Specification implementation: The designers responsible for generating the ASIC specification attend this review, along with representatives from the systems architecture group not directly involved with this design, to make sure that all the original specifications are implemented completely in the ASIC design.

  For example, the "arms-length" architectural designers will go through each mode of operation; all the registers used; other special features and functions; a complete timing analysis through various levels of simulation; static timing analysis; and the worst-case analysis at the chip and board level, etc. Power dissipation and other thermal issues must also be addressed. The review board members must check for completeness and accuracy of the chip design against specifications, and they must critique the design efficiency.

• Other issues: This addresses areas of concern unique to a particular ASIC, based on its intended use.

  For example, when using DFT techniques such as LSSD, scan path techniques, etc., check for correct implementation. The ASIC manager looks to support groups to provide inputs on proper usage of: a radiation hardened library; use of the latest supported library version (or a reason why not); equivalent gate count usage from the point of view of routing; any electromigration violations reported; the environmental impact on the device's operation or reliability from radiation; temperature cycling; etc.

The implementation review process looks for an implementation plan that can deliver a design data base that can meet all the ASIC's specifications.

PRELIMINARY DESIGN REVIEW (PDR)

PDR usually takes place before the physical design starts. This milestone in the ASIC program draws representatives from a wide cross-section of disciplines including: the vendor, design groups, a CAE/CAT group, the ASIC center, and the product support groups.

The following points were extracted from an ASIC vendor's ASIC products design process manual, modified slightly to a more general form. These points assume that the ASIC design group or a third party design house will perform all the front-end design activities as described in Section One: Chapter 2 (design, schematic entry, logic and timing analysis, and test vector generation), and that the vendor performs place and route. We also assume that: the goal is first-pass silicon success; "silicon breadboarding" is not being done; silicon is not used to prove a design; and a design has been simulated until perfect.

For review purposes, a vendor may require view graphs and hard copies of all items listed below:

Vendor PDR Item List

• Part specification: Gives a detailed part specification review and resolve issues of any missing elements.

• Verify report: Provides a review of all CAE design reports and checking routines showing no violations. If violations remain, the customer and vendor must waive them before holding the PDR.

• DFT summary: Describes the DFT methodology used. These may include LSSD, full scan path or partial scan, BIST, scan-set logic or any other type of scan used internally and JTAG 1149.1 used for boundary or I/O scan. There should be no scan violations present for either the internal or external scan methodology used. Report fault simulation results, detailing the percentage of fault coverage, undetected and possibly detected faults. The fault coverage has to be equal to or higher than its requirement.

MIL-STD-883, Method 5012 provides an excellent approach for the unambiguous reporting of fault coverage. Though somewhat complex, this document deserves the ASIC designer's time to thoroughly understand it. Many vendors will claim to be compatible with the test method covered by this military standard, when they are not. Nonetheless, even when vendors are not compatible with it, this test method is very useful, especially when used to calculate coverage for a device with mixtures of RAM/ROM and other compiled macrocells and conventional logic. For more on DFT, see Section Three: Chapter 3.

• Clock distribution summary: Shows clock distribution structures from primary input pins to the internal registers. Show how this structure produces acceptable clock edge skews in the worst-case delay environments.

  The clock structures may be a balanced tree, a vendor supplied structure, or modified clock distribution network.

  Using vendors who have an excellent hard-wired clock distribution mechanism often makes generating the clock distribution circuitry a simpler process for designers. In this case, clock skew is represented in a simulation and timing verification environment, usually using Manhattan distances for skew calculations. The vendor then takes care of the detailed connections at an individual flip-flop level during generation of their own internal netlist and physical layout design.

• Logic simulation summary (using pre-layout estimated media delays): Explains the procedures used to verify that, when inputs and clocks are set at their limits, logic simulation results are correct.

• Static timing analysis (using pre-layout estimated media delays): Show how critical paths operate correctly given worst-case delay values. These paths may include shortest and longest paths, critical setup and hold paths, and clock paths. Note that the worst- case delay values may differ. For example, increased delays (from radiation effects, aging, etc.) may allow a marginal circuit design to work--therefore minimum values are "worst-case values." In another part of the same device decreased delays may allow another marginal circuit design to work--in this case maximum values are "worst-case values." There are even circuits where differing control signals coming into the same device have opposite worst-case values. Thus, the definition of worst-case values must be selected and justified on the basis of each critical path under analysis.

• Critical paths: Shows the circuit paths the designer has selected where the control of timing parameters is extremely crucial. Discuss the tests which verify these paths timing performance. Justify that this is a complete list or at least representative of all such critical paths in the device.

  Every ASIC vendor will permit controlling for a limited number of critical paths during device layout. For these paths, the control of timing parameters is extremely crucial and may require hand placing and/or routing.

• Test summary: Provides an overview of the types and quantity of test vectors (test routines represented at the I/O pin level, for every clock cycle) to be supplied at the critical design review.

  Every vendor uses some test program that extracts and reformats different simulation or test files, and formats them for the production tester used to test parts as they are built and screened. The output of this program is files that are used to test an ASIC design on a vendor's tester.

• Bonding diagram file and layout: Presents information on the pin assignments of the device package and from the die I/O pads to the internal package bond pads.

  The ASIC designer creates a file assigning each pad on the die to a package pin. The vendor may or may not allow multiple pads to be connected to a single package pin. During the assignment of signals and power to pads, the designer must take into account simultaneous switching of I/O signals, location of clocks and other noise sensitive signals, separation of internal and external ground rings/planes, etc.

• Package and other information: The designer presents information on the intended package, on thermal considerations, on special uses of the device (board level test support, etc.), and on any mechanical requirements (special lead trimming or lead frame, temporary packaging, use of the device as a bare die, etc.)

• Schematics and directory structure of design: The designer must also present a detailed gate level schematics and a directory structure, with all necessary files identified.

After completion of the PDR, when both the vendor and the customer have officially signed off the checklist, the vendor then should have all the necessary design data base ready and available as a deliverable for place and route. The ASIC designer and the ASIC manager should consider the ASIC design "perfect" at this stage, given the constraints of time and budget, and only need to double check the results of place and route before giving the go- ahead to produce the chip.

CHIP SIGN-OFF REVIEW

This formal sign-off review for an ASIC program resolves any outstanding issues from the PDR and allows place and route to begin. If the PDR has met all of its goals, this review may not be necessary. However, when working with a new vendor then this initial review may not suffice and the ASIC group will consider a chip sign-off review necessary. Some groups may call this chip sign-off review a "delta-PDR."

One vendor's example of this particular review requires the application engineer to sit down with an ASIC designer and go through a checklist to make sure that everything remaining from the PDR is accomplished. If the results of any of simulation or timing analysis reports do not satisfy the application engineer, then the designer may have to redo the items in doubt.

Some vendors pay a lot of attention to a detailed implementation if the design in question is a very asynchronous one. For example, some vendors, will run a dither program to point out any marginal paths in a design.

When the design is formally signed-off, it is ready for the physical layout process.

CRITICAL DESIGN REVIEW (CDR)

The CDR occurs after completing the physical design process to ensure that no violations of vendor design practice, design rules, etc., exist. If violations do exist, the designer and vendor must waive them jointly. An essential review, the CDR provides a go-ahead for production of the ASIC devices with great confidence in first-pass ASIC success.

Specifics of the CDR include:

• Part specification: Review changes in the part specification since the PDR process and resolve any outstanding issues.

• Design verification check: The vendor supplies the information for the designer to perform the following reports and simulation runs, again using post-layout or the actual capacitance and resistance numbers. After running these analyses on the post-layout version of the design, the designer presents the results. Typical design checks include:
  • design rules verification
  • static timing analysis
  • logic simulation/functional analysis
  The above three steps are bound to show some changes from the pre-layout numbers. The designer must perform a careful analysis to assure that these new timing values will not create any timing violations against specifications and present these results during the CDR.

  The vendor, sometimes with the designer's assistance, will resolve any significant deviation in timing either through manual place and route, or automatically through rerunning place and route tools.

• Test Simulation Results: The vendor should present this information at CDR and the designer should agree. After place and route, it is usually necessary to adjust the tester timing (strobing or event windows) since delay changes can shift expected test values.

• Critical Paths: The vendor must demonstrate how identified critical path timings were maintained during place and route. These paths are often laid out by hand and should not change after place and route. However if they were handled by assigning relative weights to nets and if any other net got assigned higher value by an oversight, a problem may arise. The vendor and designer should review the critical paths to assure that no negative changes have taken place.

• Schematics and directory structure of the design: The vendor and designer must both present how their configuration management approaches ensure a correct design.

  For example, it may not be possible to place and route within timing constraints, which may dictate some changes to the design. All, or nearly all, tests and analyses must be rerun on this modified design. The configuration management systems must be able to demonstrate that revision control ensures this has been done.

  Another example of properly working configuration management shows how the file names for pre- and post-layout files differ.

• Bonding diagram and package information: the vendor and the user/designer must review the package design against the final package specifications including pad placement, die orientation, package marking, and ordering status. Review any required changes to pinout and/or pad placement against vendor's package rules once again.
  • The designer must check the chip plot and chip dimensions. Additionally, when using a standard cell design, the ASIC designer must check the deliverables, i.e., layout verification results. The ASIC program manager should also revisit the fabrication schedule at this point.
  • Before releasing the design data base to mask making, the ASIC designer and the vendor must go through a formal post-layout sign- off checklist to make sure that all outstanding issues are resolved.

FLIGHT BUILD REVIEW

The flight build review, also called a build-readiness review, provides a final check on all aspects of an ASIC before building the part as a flight-quality device. To determine the reliability of the part, the board considers issues remaining from any earlier reviews, along with data gathered from the evaluation and characterization of engineering parts. In general, information governing any aspect of the ASIC is germane at this review. Since PDRs and CDRs, government qualification status, and other earlier work could not possibly anticipate all potential problems when an ASIC is actually manufactured and subjected to testing in its target system, the flight build review is essential to producing reliable space flight parts.

Often all remaining issues concerning ASIC flight-readiness have not been resolved at the time of the flight build review. The review board must then judge the chance of all open issues being satisfactorily resolved and a successful flight-quality ASIC produced. The membership of this review board must have the authority, knowledge, and experience to pass on this kind of judgment.

This review offers the first opportunity for the board to peruse information from a detailed system evaluation. If a problem arises during ASIC system integration, then a choice would be made at this point whether to allow changes in the design to accommodate the problem. Since this review also offers the first opportunity to examine an actual device, the reviewers commonly announce whether or not an ASIC qualifies as a first pass success.

Working first-pass silicon means much more than a lucky designer; it indicates that the ASIC program carefully followed a complete methodology. Working first-pass silicon means ASIC program sponsors can expect a much more reliable over-all design than one achieves under the schedule pressure of a second or third pass try since changes made late in a program under pressure receive correspondingly less review. The flight build review must take this into account when considering the history of the ASIC device.

Summary

• Plan for reviews at the beginning of an ASIC program.
• Don't underestimate the importance of selecting appropriate board members. Remember you will rely on their expertise to achieve first-pass silicon.
• Identify the participants for reviews early enough so that they may receive all necessary review material in time for their analyses.
• Work with the ASIC vendor's review methodology to reconcile your organization's goals for a particular review with those of the vendor.
• Build the reviews into the contract and the statement of work so that sufficient resources will be available from the vendor to properly support the reviews and action items generated from them.
• Take ASIC reviews seriously. Mask programmable devices such as standard cell and gate array ASICs cannot be changed as easily as printed wiring boards. By finishing appropriate work at the time of each review, the minimum amount of resources will be spent to get the job done. While this discipline may be new to many organizations, it has been proven time and time again to be an effective way to build reliable complex systems such as ASICs.

Now you may jump to:

• the next section: Section 2: "Vendor Evaluation"
• Table of Contents
• Index
• Acronym list
• Glossary

Home - NASA Office of Logic Design
Last Revised: February 03, 2010
Digital Engineering Institute
Web Grunt: Richard Katz