You may remember the days when Quality Assurance meant a simple review by the Engineer-of-Record immediately before sealing the drawings. That seal alone was the documented evidence of quality. Clients hired engineers for their expertise and expected them to get it right the first time. It was up to the engineers to develop habits (often unwritten) to ensure the quality of their work.
Today’s projects and today’s clients are more demanding. Projects are larger, more complex, and involve more disciplines. Clients establish shorter schedules while enforcing more requirements on the engineer. And the informal review by a senior engineer has been replaced by a far more rigorous and well-documented Quality Assurance (QA) program. Engineering firms of all sizes have responded by creating internal Quality Management Systems (QMS) that are implemented by teams of dedicated quality professionals. While the senior review is still at the heart of most engineering quality systems, today’s projects benefit from a far more robust and complete approach.
This is especially true on large bridge mega-projects. Clients typically establish specific expectations for a mega-project quality plan. While every project is different, certain key components are frequently part of these plans. Detailed checking procedures, Quality Control (QC) reviews, audits, and continuous improvement measures are some of these core quality components.
Detailed Checking
“The devil is in the details” certainly applies to structural design. “Minor” details such as a formula in a spreadsheet, the units used in a calculation, or a note on a drawing have caused significant problems in design and in construction. The detailed check ensures a second knowledgeable person agrees with each detail of the work. A check of each calculation and each drawing before starting a senior QC review can prevent costly rework in design and in the field.
Detailed checking can be performed and documented in a variety of ways. However, most effective checking procedures share certain characteristics:
- Comprehensive – Detailed checks are not spot checks. Rather, they include all calculations, drawings, specifications, reports, and any other engineering deliverables.
- Performed by the designer’s peers – Checking is typically done by an engineer with a similar level of experience as the designer – one who is capable of creating the design that is checked. Although this may be a senior engineer, the checker is most often a mid-level engineer.
- Documented with mark-ups of the deliverable – A record of the “conversation” between the designer and checker must be retained as the documentation of the detailed check. This generally consists of color-coded mark-ups (e.g. yellow highlighter indicates agreement, red edits indicated proposed changes) on a Check Print of the document. Check Print stamps are used to identify the designer and checker, and the dates they completed their checking tasks. The checking procedure defined in the quality plan for the project outlines the color-coding system and processes that must be followed to complete a detailed check.
The end result of the detailed check is a corrected document that is mutually acceptable to two individuals – designer and checker. This is the document that is provided to reviewers in the subsequent Quality Control process. Figure 1 shows the roles of each person in the detailed checking process.
Figure 1. Checking and review process.
Quality Control Reviews
Quality Control reviews provide a higher-level evaluation of the adequacy of the checked documents. Senior engineers with experience designing and managing the work being reviewed provide comments that are based on the project requirements and their own experience. The QC reviewer is someone with proven engineering judgment in the work. Unlike detailed checks, QC reviews are not focused on details such as mathematical correctness. They are meant to reveal issues with the overall design approach taken, to identify discrepancies between contract requirements and the work, and to discover signs of flaws that might have eluded the detailed checker(s).
Some projects document these QC reviews with mark-ups, similar to the process followed for detailed checks. However, this can cause confusion between the separate and distinct check and review processes. And direct mark-ups can be a less effective way to document the higher-level input provided by QC reviewers. A better way to document review comments is in comment/response tables. This format provides significant advantages that are better suited to the importance and formality of QC reviews:
- Each comment is tagged with a unique identifier
- A written response is provided for each comment
- Status codes (Agree or Disagree, Open or Closed, etc.) can be assigned to each comment/response
- Resolution between the designer and reviewer can be explicitly documented for each comment
- All of the information above can be sorted and filtered for tracking
- Multiple reviews can be consolidated into a single review database for improved oversight and reporting of the review process
See Figure 2 for an example template used to document review comments, responses, and final closure.
Figure 2. Review comment form.
Mega-projects often include several stakeholder groups and multiple disciplines beyond structures. This diversity of input usually requires multiple types of review to capture all relevant comments. The list below provides examples of the review types that are frequently found on mega-projects:
- Quality Control Review (or Discipline Review) – This review is discipline-specific – a senior bridge engineer reviewing bridge documents, for example. It is focused on the technical correctness and completeness of the documents within that discipline.
- Interdisciplinary Review – Senior experts from each discipline review other discipline’s work for potential conflicts with their discipline. This could be a senior roadway engineer reviewing bridge documents while the senior bridge engineer reviews roadway documents. The intent is to break through the discipline silos and ensure the complete set of design documents work as a unified whole.
- Quality Integration Review – While the Interdisciplinary Review evaluates consistency between disciplines, the Quality Integration Review evaluates consistency between design teams or firms. Most applicable on larger, multi-firm projects, this review reveals and corrects the inevitable uniqueness in design work produced by teams from different companies or in different geographic regions.
- Constructability Review – This review is performed by the contractor team on design-build projects. It gives the contractor an ability to improve the efficiency of the design given the planned means and methods of building the project. A maintenance review may be part of this review if the contract includes a long-term maintenance component.
Tools such as Bentley’s Bluebeam Revu™ allow for simultaneous reviews to be performed in a shared virtual environment. This approach is essential for collecting review comments and closing them in an efficient manner. The back-and-forth process between designer and reviewer is shown in Figure 1.
It is not unusual for mega-projects to require independent calculations, especially for structural analysis and design. Independent design calculations typically involve two separate design teams creating parallel models and designs based on a common set of given data. When independent designs are required, the teams must not coordinate or compare work until they have completed their separate designs. Only when each team is finished should the results be compared and reconciled. The independence of this approach, while costly, results in a robust design that was independently achieved by two teams.
Audits
Checking and reviews control the quality of the work product, and are generally defined as Quality Control (QC) processes. Quality audits assure process compliance and effectiveness. As such, they are part of the Quality Assurance (QA) process. Despite the image of IRS agents combing through your receipts, quality audits are essential and (slightly) less painful steps in producing complete design submittals. Even the most diligent designers, checkers, and reviewers can inadvertently omit a required quality step or piece of documentation. Since quality plan requirements are generally contract requirements, these process discrepancies must be corrected. In many cases, audits are required as the basis of a formal quality certification document that supplements the engineer’s seal.
QA auditors are generally independent of the design, checking, and review processes. Although they are usually engineers, audits can be performed by other persons who can interpret the quality requirements and engineering work product. Auditing is not a technical evaluation of the work product. It is a confirmation that the quality plan processes were fully implemented and adequate documentation exists. Auditors often document their findings on a checklist that contains questions such as:
- Were all documents detail checked?
- Were all required reviews performed?
- Were the appropriate personnel used for the design, checking, and review tasks?
- Were all resolved checker mark-ups and reviewer comments closed and incorporated into the final documents?
- Were any late changes made to the documents and not checked or reviewed?
- Does adequate documentation exist in the form of Check Prints and review comments/response forms?
The value of QA audits is most apparent when a process issue triggers additional checks or reviews that result in content corrections to the documents. QA audits are based on the notion that effective quality processes, consistently applied, improve the quality of the product. When designers, checkers, and reviewers understand their work will be audited, it encourages a consistent application of the quality processes. The old quality axiom applies: “What gets measured gets done”.
Continuous Improvement
Detailed checks, quality reviews, and quality audits will not happen by accident. The quality manager of a mega-project is responsible for a vast array of processes, as well as the quality of the product itself. In order to maintain a high level of quality and process compliance, the quality manager implements continuous improvement processes such as quality training, quality metrics, and corrective/preventative actions. This “quality management toolbox” motivates everyone to improve their work from project start to finish.
Structural designers cannot implement quality processes if they haven’t been made aware of their existence. Quality plan implementation starts with quality training. This training is generally most effective when presented in-person by the quality manager. However, large teams may rely on videoconferencing and conference calls as well, so remote designers can stay aware of the project’s quality processes. The quality manager should develop documents that can be used both during and after the training sessions – PowerPoint slideshows, process flowcharts, example documents, QC/QA directives, etc. Quality training attendance must be tracked and a short quiz should be used to ensure trainees have retained the important material. This is no small effort, but it is well worth it to avoid hearing this response in a quality audit: “But no one told me!”
Your car’s dashboard tracks driving metrics such as your speed, the amount of fuel left in the gas tank, and warnings about engine problems. As the driver, you use this information to determine when to slow down, when to fill your gas tank, and when to get your car serviced. Quality metrics serve a similar function to your car’s dashboard. Quality metrics such as the time in review, the number of staff trained, and the number of review comments received measure the quality health of the project. Long review periods might indicate poor quality of work entering review or an overzealous reviewer. Reductions in the percentage of staff trained on quality indicate a need to increase the frequency/reach of the training. Increases or decreases in the number of review comments might indicate a change in the depth of the reviews or the quality of the work itself. If the quality manager is not tracking these metrics, it is difficult to know when corrective actions are needed. Regular quality reports to project management, that contain these metrics, are easily interpreted and are powerful management tools.
Corrective/Preventive Actions (CPA) processes are used to diagnose and treat quality issues that occur on mega-projects. We are all human beings, and human beings make mistakes. The quality metrics described earlier can make it easier to see when a mistake becomes the sign of a larger trend. A complete QC/QA plan must have a proactive plan for mitigating the extent and severity of mistakes. CPA plans usually contain the following elements:
- Problem Definition – A clear problem statement must be defined to ensure there is no confusion about the issue at hand. A concise, specific problem statement is the start of the CPA process.
- Root Cause Analysis (RCA) – If we treat only the symptom and fail to address the root of the problem, we can expect the problem to continue. Good quality managers can help the project team drill down into the true source of a problem. It is this problem source – the root cause – that must be addressed to prevent the recurrence of the issue. Quality tools such as “Five Why’s” exercises or Fishbone Diagrams can help direct the RCA.
- Corrective Actions – These are actions taken to fix the issues already caused by the problem. They are backward looking efforts to make sure the final work is ultimately correct, even if rework and/or repair is needed to get there. Corrective actions could include resubmittal of design documents or retrofit designs.
- Preventative Actions – CPA’s are valuable learning experiences. While we cannot change what happened in the past, we can take mitigating steps to reduce the likelihood of future problems. These preventative actions are the forward-looking changes to processes, people, or both to maximize positive outcomes in the future. Examples of preventative actions might be changes to the quality plan, new technical directives, or changes in project personnel.
Summary
Clients have always expected that engineers work fast and design it right the first time. However, the days of relying on an engineer’s seal as evidence of quality are over. Expectations are higher than ever and now include formal QC/QA processes with complete documentation. The definition of failure has expanded beyond technical problems and now includes failure to follow the proper processes or provide the necessary documentation. In this environment, mega-project quality has grown into a more robust and formal element of overall project success. The checking, review, audit, and continuous improvement processes included in these projects ensure that engineers continue to exceed client expectations and deliver outstanding designs.▪
