Gaining Accreditation for an American Standards Laboratory

The Fluke Primary Standards Laboratory was accredited by The National Voluntary Laboratory Accreditation Program (NVLAP) in June 1995 and by the German accrediting body, Deutscher Kalibrierdienst (DKD), in January 1998.

This article describes the steps required to achieve that accreditation and includes both expectations and the actual benefits achieved. It also discusses the differences in the accreditation experience with U.S. and European organizations and the findings during the internal and external audits.

Overview

The Fluke Primary Standards Laboratory went through two accreditation processes — the first with NVLAP and the second with NVLAP and DKD — over a four-year period. It pursued these accreditations to satisfy its customers’ needs, reduce the number of onsite audits by customers, and, most importantly, to eventually eliminate the need to recalibrate its calibration products in Europe.

The reason for two accreditation processes resulted from the limited scope of its first accreditation. When it achieved its first certification by NVLAP in June 1995 after 13 months of preparation and examination, the Fluke lab staff discovered that NVLAP accreditation is system-specific. That means that it covers only those systems and procedures included in the application and specifically reviewed by the NVLAP audit team.

The lab also expected the NVLAP accreditation to be accepted by the European Cooperation for Accreditation (EA) and other international accreditation agencies. However, after about a year of waiting for that acceptance, it became apparent that acceptance was not going to happen in the near future.

So, the lab decided to pursue accreditation by an EA accrediting agency on its own. It also decided to expand the accreditation process to cover calibration performed in the Fluke factory as well as in the standards laboratory.

The second accreditation of the primary standards laboratory and its factory annex by both NVLAP and the DKD started in May 1996. It was completed 15 months later.

Why Seek Accreditation?

Fluke’s Primary Standards Laboratory is a modest-sized facility with a staff of 11 engineers and technicians. In addition to the traditional task of establishing and maintaining reference standards for the Fluke Corporation, the lab provides high-end DCV, ACV, DCL, ACL, and resistance calibration services to other design, production, and service groups within Fluke and to outside customers.

The impetus for achieving accreditation came from outside customers. Some customers planned on gaining accreditation themselves and notified Fluke that they, in turn, would be requiring accredited calibration services from their suppliers. If the lab were not accredited, it would lose that business.

The lab also assumed that accreditation would reduce or eliminate time-consuming audits by its users. Furthermore, Fluke believed that when its NVLAP accreditation was recognized by EA, costly recalibration of its products sold internationally — especially in Europe — would be eliminated. It operated on the assumption that mutual recognition between NVLAP and EA would be accomplished in a year or two.

Round 1 – NVLAP Accreditation

When it began preparing for NVLAP accreditation in May 1994, the Fluke Primary Standards Laboratory had already gained certification as an ISO 9001 facility. This meant that all of its processes were identified and documented, personnel were properly trained, procedures were in place for all calibrations, and the required organization and management were in place and documented.

In-House Assessment

Despite the lab’s good record for providing reliable, high-quality calibrations, when the existing quality manual was checked against ANSI/NCSL Z540-1 (which closely resembles ISO Guide 25), 69 gaps were identified.

These included:

• Policies and procedures. Several were missing or were poorly documented.
• Uncertainty analysis and documentation. Much of the lab’s uncertainty analysis did not conform with EA-R2, “The Expression of the Uncertainty of Measurement in Calibration.” It also did not have a systematic method for naming and filing the analyses.
• Control charting of standards. Control charting was practiced informally by key lab members. However, little documentation for the preparation and review of control charts existed.
• System documentation and verification. Many of the calibrations performed in the Fluke Standards Lab were automated or semiautomated by dedicated, custom-designed systems. While these systems worked well according to informal evaluation by lab personnel, many unfortunately did not include proper documentation. Descriptions of associated software and its verification were often missing or insufficient. And uncertainty analyses embedded in the software were sometimes not well-documented.

Identifying these gaps and resolving at least those required to successfully process the accreditation application took a four-person technical staff about five months to complete. Because accreditation-related tasks were added to the normal workload, workdays often extended into evenings and weekends.

Submitting the Application

At the end of the five-month period, in October of 1994, the Fluke application for accreditation was submitted to NVLAP. The application package included:

• A quality manual based on NIST Handbook 150 (a superset of Z540-1).
• A list of major equipment used in the standards lab.
• A description of the lab and its staff.
• A description of operations.
• A detailed list of the scope of accreditation.
• A Stage 1 fee.

Pre-Stage 2 Review

Several weeks after the lab submitted its application, NVLAP began its pre-Stage 2 interactions with the lab personnel. It started by reviewing the lab’s quality manual and identified several deficiencies, none of which were sufficient enough to delay the accreditation process. Most of the deficiencies involved items unique to the NIST Handbook 150, such as use of NVLAP certificates and logo.

In addition, NVLAP requested that the Fluke lab submit:

• A sample calibration procedure.
• A sample calibration certificate.
• Documentation of uncertainty analysis.
• A list of measurement standards for each measurand submitted in the scope of accreditation.

Shortly after this material was submitted, NVLAP scheduled an onsite audit in April 1995 and billed the Stage 2 fee and expenses.

Onsite Audit

The NVLAP audit team consisted of a senior program manager within the NVLAP organization who focused on the quality system as documented in the quality manual, and the head of the Electricity Division at NIST, who conducted the technical review. The audit began on a Monday morning promptly at 8:30 a.m.

The NVLAP senior program manager spent four days comparing the lab’s quality manual against the requirements of NIST Handbook 150. He also looked for evidence that the lab practiced its documented policies and procedures. The NIST representative interviewed members of the lab’s technical staff and metrology technicians. He observed technicians performing calibration procedures, reviewed several uncertainty analyses, and probed their authors for possible additional unmentioned sources of uncertainties.

Audit Findings

On Friday morning, nearly five days after the onsite audit began, the NVLAP auditors presented their findings at an audit review meeting. They recommended accreditation of the Fluke lab to the NVLAP review committee, provided that seven deficiencies were corrected. They also required successful completion of a calibration proficiency test consisting of the measurement of an SR104 standard resistor.

The Fluke Primary Standards Laboratory received its NVLAP certification on June 30, 1995, approximately 14 months after initiating the application, and received a final bill for its accrediting service.

During the final review discussions, Fluke lab personnel learned that the NVLAP accreditation was system-specific. That is, it issued accredited calibration certificates only for those systems and procedures included in the scope of application and reviewed by the NVLAP audit team.

Since that included only high-end calibrations, the lab was not authorized to perform accredited calibrations on low-end equipment unless it used the same procedures and standards as for the high-end equipment.

Accreditation Benefits

While one of the reasons for seeking accreditation was an anticipated demand from Fluke customers for accredited calibrations, the company initially experienced only an occasional such request.

Customers who had earlier expressed this need apparently delayed their accreditation plans because they couldn’t justify them financially. That demand did, however, start to pick up in 1998.

Initially, the number of customers auditing the Fluke Standards Laboratory remained unchanged by its accreditation status. Most auditors were not familiar with NVLAP or any other accrediting agency and were not dissuaded by the related accreditation documents.

After about nine months, this started to change. By 1998, the lab had few requests for customer audits.

The biggest disappointment was the lack of acceptance of NVLAP accreditation by the EA and other international accreditation agencies. Every year, Fluke lab staff members were told that a mutual recognition agreement (MRA) between NVLAP and EA would happen shortly. The date of that agreement was still uncertain five years after the lab’s initial accreditation.

Round 2 – Expanding the Scope of Accreditation

By the summer of 1996, it was apparent that EA recognition of NVLAP was not likely to happen in the near future. So, the lab decided to achieve direct certification by an EA accrediting agency.

At the same time, it decided that the accreditation should be extended to calibration performed by the Fluke factory. And it wanted to expand the scope of accreditation to include measurands now available in new Fluke products including capacitance, temperature, phase, and frequency.

Having decided to ask for accreditation by an EA member, the question became, which member?

Fluke quickly reduced the field to the three it was most familiar with:

• NPL/UKAS (England)
• NMi/NKO (Netherlands)
• PTB/DKD (Germany)

Of these, representatives of the NMi and PTB visited the Fluke Standards Laboratory to discuss accreditation in general, and specifically the challenge of how to accredit the test systems on the factory floor. Fluke ultimately decided on PTB/DKD because it is recognized internationally, including areas outside Europe that are important to Fluke business.

Fluke decided to work on a joint accreditation with the DKD and NVLAP to give the company acceptance across a wide area of the world. This would also provide an easy transition if and when an MRA was finally approved between NVLAP and EA.

Preparing for Expanded Accreditation

Considerable work organizing and documenting the factory annex and updating the quality manual to include the expanded capability and to meet DKD requirements had to be completed before applying for expanded accreditation.

Factory Annex

Prior to selecting the EA accrediting agency, a PTB representative visited the Fluke facility and discussed some of its accreditation challenges.

One particular challenge was how to accredit the test consoles in the remote factory (located about three miles from the standards lab). The PTB representative recommended making them a factory annex to the standards laboratory. The factory annex would operate under the management and quality system of the standards laboratory even though manufacturing technicians would operate the automated consoles.

This setup was adopted after much discussion between manufacturing, quality assurance, and standards lab management. It was not easy for manufacturing to allow a new entity into its production stream. Nevertheless, it was necessary to ensure independence for the calibration and reporting processes.

Another challenge for the remote factory annex was to establish and maintain the traceability of the factory test consoles through the standards laboratory. This was accomplished using the method of process metrology.

The organization of the factory annex and its operations also had to be integrated into the standards laboratory’s quality manual. This involved writing local area documents (METs) on process metrology, test system documentation, and a description of responsibilities and interactions between factory and standards lab personnel.

Expanding the Quality Manual

As mentioned earlier, Fluke planned to expand the scope of its previous NVLAP accreditation to include the capability to calibrate recently developed multifunction calibrators with new measurands such as capacitance, temperature, frequency, and phase. This required adding new material for the claims, supporting standards, uncertainty analyses, and calibration method descriptions to the related section of the quality manual.

It was also necessary to make the revised quality manual compatible with the DKD requirements as described in DKD-6. This document describes a well-designed, highly structured quality manual that meets the requirements of EN45001 (very similar to ISO Guide 25) and helps to establish links between claims, standards, and procedures.

In comparison, the Fluke Standards Lab’s existing quality manual was initially designed to satisfy ISO-9001. It was modified to satisfy ANSI/ NCSL Z540-1 and NIST Handbook 150 by patching and adding. It was complete and satisfied NVLAP requirements, but it was not pretty.

The DKD accepted quality manuals structured differently than DKD-6, but required paragraph-by-paragraph mapping. Fluke chose to add the unique requirements of DKD-6 to its existing manual and prepare the required mapping.

One rather significant difference between the NVLAP and DKD applications was the required level of detail in the documentation — especially in the quality manual. DKD required that the quality manual include claims, uncertainty analyses, descriptions of calibration methods, standards required for each claimed measurand, and corresponding calibration procedures. All of this material must be included in both the lab quality manual and the accreditation application.

In comparison, NVLAP also required most of this material eventually, but not with the application. Therefore, it was possible to process the initial application with NVLAP, get into its process queue, and then complete documentation on sections such as procedures and methods during the Phase 1 review.

The DKD methodology allows a thorough review prior to scheduling the onsite audit. However, it requires more initial preparation time. NVLAP’s onsite review takes more time since its auditors must review documents not previously submitted.

Preparation for the DKD and NVLAP joint accreditation application began in October 1996. The applications were completed in March 1997 after approximately 12 human-months of work.

DKD Application Documents

The procedure and documents required for the DKD accreditation application were described in the organization’s documents DKD-1 through DKD-6. The required materials enabled DKD to perform a thorough evaluation of the lab and its documentation prior to scheduling an onsite audit.

The application package included:

• A completed DKD application form as provided in DKD-2, “Accreditation of Calibration Laboratories — Criteria and Procedures.”
• An annex for each measurand in the claim. Each annex included the measurand, a brief description of the calibration method, standards to be used, and a listing of instruments to be calibrated.
• Calibration certificates of the reference standards for all measurands and ranges.
• Proof of education and training of the head and the deputy head of the laboratory.
• A quality manual per DKD-6 or with a mapping to DKD-6.
• An application fee.

This material was similar to the NVLAP application package. However, it differed in the structure and content of the quality manual and in the need to submit calibration certificates for all relevant reference standards.

Interactions Prior to Onsite Audit

DKD responded in April 1997, approximately one month after receiving the application from Fluke. It submitted a 15-clause DKD contract and arbitration contract to be reviewed and approved by the lab and it appointed auditors to the project who were also subject to the lab’s approval. The lab promptly approved the contracts and the suggested auditors.

A few months later, the lab received a list of several deficiencies in its quality qanual. The most pressing problem involved control of the document by the DKD. The manual consisted of 30 separate local area documents and standard operating procedures (SOPs) contained in a pair of large three-ring binders. Each document was controlled by a revision date and an approval page signed by the author and the metrology manager. DKD requested a “kernel” document that included material of first-order interest such as claims, a list of reference and working standards, and procedure lists.

The DKD manual reviewer also asked that links between claims, standards, and procedures be added as modeled in DKD-6. The lab created the kernel document patterned according to DKD-6 and also included lists of uncertainty analyses, METs, and SOPs deemed of special significance. Approval initials were to appear on the footer of every page and any change to this document must be submitted to the DKD along with relevant supporting documents.

In addition to the deficiencies in the quality manual, some of the documents required by DKD-6, including some system documentation and a few uncertainty analyses, were missing. These were not complete when the application was submitted but the lab was allowed to submit them as they became available.

After about nine human-months of effort, all the deficiencies described above were satisfied. DKD gave its approval by scheduling an onsite audit in December 1997. DKD and NVLAP auditors agreed to coordinate a joint audit at this time to minimize the total time and expense required.

NVLAP officially acknowledged the Fluke Standards Lab’s application in July 1997 and assigned the same auditors as for the first NVLAP accreditation. The documentation submitted by the lab was accepted by NVLAP and only required that a few missing pieces be submitted. The lab provided NVLAP with the same modifications requested by the DKD. Both agencies were eventually given identical documentation packages.

Onsite Audit

The technical review started even before the auditors arrived onsite. All four auditors had reviewed certificates issued by NIST on the lab’s reference standards for the claimed measurands.

Then, bright and early on a Monday morning in early December 1998, the four-member team of auditors arrived at the lab. Because they were coordinating their efforts, the two DKD auditors focused on the technical aspects of the laboratory and the NVLAP auditors focused on quality aspects. Practically, however, all four shared much of the ensuing review.

Technical Audit

The technical audit began with the claims and linked standards and procedures presented in the recently completed core document (MET 12). Each family of claims was carefully compared with the associated uncertainty analysis. The authors or lab manager were often asked to justify the assumptions, and asked whether they had considered other potential contributions to uncertainty.

The PTB representatives strongly emphasized the practicality of the claims. The best claims often were determined by limitations of anticipated test instruments rather than by the capability of the test standards. This resulted in doubling the claimed uncertainty of the Josephson Array System, for example.

Most of the competency evaluation of key laboratory personnel took place during the technical review described above. Laboratory technicians were not questioned. This was a marked change from the initial NVLAP accreditation audit during which each technician was observed and questioned by the technical auditor.

One of the NVLAP auditors participated at times in technical discussions, but spent most of his time comparing the quality manual with the requirements of the NIST Handbook 150. All deficiencies that he had identified in the 1996 accreditation were checked for proper fixes. He again looked for clues that showed whether or not the lab actually practiced the processes described in the quality manual and documented those few occasions where that was not the case.

In some areas, the technical review was particularly stringent. For example, one of the NVLAP auditors had recently calibrated a 792A Fluke transfer standard at the PTB. This standard was purchased from Fluke by a large laboratory in South America, used for a year, and then sent to the PTB for recalibration. The NVLAP auditor compared the original calibration data — provided to the customer by the Fluke factory — with data he collected at the PTB. Differences were compared with the present claims for the laboratory.

Technical staff members were given a proficiency test with 1000 Volt TVC which had been characterized at the PTB and measured at NIST for AC/DC difference. Lab technicians were asked to measure its AC/DC difference, which fortunately agreed well with those made by NIST. Unfortunately, NIST and PTB disagreed significantly at some frequencies above 1 kilohertz (kHz).

Img. 2: Fluke is only one of a handful of commercial organizations operating Josephson Array standards to represent the volt in their metrology operations.

Systems Review

All of the lab’s calibration systems associated with its accreditation claims were reviewed carefully for calibration methods, software verification and control, and the related uncertainty budget. This review resulted in system improvements and the elimination of a few faults.

There was some discussion and questions from both PTB and NVLAP auditors regarding the lab’s process metrology — a method for linking the traceability chain between the standards lab and the remote factory test consoles. The team also visited the factory annex located approximately three miles from the standards laboratory.

Factory Annex Environment

Considerable attention was paid to how the environment of the factory annex is controlled and monitored.

The factory annex area is set apart in the factory by wide blue tape around its perimeter. Equipment storage cabinets and test consoles are also placed to limit access to the area. A facility computer controls several temperature sensors and a few humidity sensors distributed throughout the area. NVLAP was less than satisfied, however, by how long it might take to observe an out-of-tolerance condition and documented that deficiency in its audit report.

The onsite DKD audit, which began Monday morning, ended three days later. Both the NVLAP and DKD auditors summarized their findings in handwritten reports which stated that both were satisfied with the laboratory, its documentation, and its key personnel and would recommend accreditation for the lab and its factory annex, provided that certain deficiencies identified in their reports were resolved.

NVLAP listed four deficiencies in the quality system. PTB identified four documents of claims and eight quality manual deficiencies that required correction. No technical deficiencies were found.

Img 3: The accredited Fluke Factory Annex is set apart in the factory.

Comparing NVLAP and DKD Accreditation Audits

The auditors from these two agencies were similar in many ways. Both groups were professional and thorough. Both were tough, but fair. And both displayed cordial, friendly, and helpful attitudes throughout the process.

In many ways, the audit was invigorating, motivating, and helpful rather than a stressful, fearful event. However, the approach of the two agencies did differ in several ways:

Scope of Accreditation

Perhaps the biggest difference between the two agencies is that NVLAP’s accreditation is system-specific and DKD accredits for the best claims of which the laboratory is capable. This means that with NVLAP accreditation, only the systems specified and audited during the accreditation process can deliver accredited calibrations.

For example, the Fluke Standards Lab is accredited to calibrate high-end equipment for AC and DC Volts with a J Array System and precision 792A AC/DC transfer system respectively. It is not allowed, however, to issue an accredited calibration certificate on a low-end digital multimeter unless it is calibrated with the J Array and 792A systems rather than a less powerful multifunction calibrator. In this case, the J Array and 792A systems are the lab’s best systems for DCV and ACV respectively.

The lab is authorized by DKD, however, to issue accredited certificates on lower accuracy calibrations, using whatever standards are appropriate, provided the procedure, equipment and standards used are in accordance with the lab’s quality manual. Future audits by DKD may check to verify that the lab adheres to these provisions.

Quality Manual Review

Both agencies conducted a thorough review of the Fluke Standards Lab’s quality system against their quality standards. The DKD review was very structured with careful attention to the linking between claims, standards, and procedures.

Technical Review

Both agencies performed a thorough technical review. NVLAP seemed less structured with a focus on people, including metrology technicians. Proficiency testing played a minor role.

The DKD review was very structured and included comparisons of actual calibration results with those made by other laboratories, including the PTB. DKD evaluated only the laboratory’s technical staff, not its metrology technicians.

Certificate of Calibration

NVLAP had no specific form for its certificates other than that required in NIST Handbook 150. This document includes a list of the required content and places restrictions on the use of the NVLAP name and logo.

DKD has similar requirements and restrictions. However, it offers little flexibility in its reports and its recommended report template must be used. This requirement is driven, in part, by the desire to keep all accredited reports issued by EA members nearly identical.

Summary

NVLAP and the DKD issued the Fluke Primary Standards Lab and factory annex accreditation certificates approximately 15 months after the process was initiated.

In the years after accreditation, the lab experienced a growing demand for accredited calibrations. Its dual accreditation with NVLAP and the DKD satisfied a high percentage of this demand.

Another valuable benefit of this certification was the relationships that it helped establish between Fluke Primary Standards Laboratory staff and metrologists at NIST and the PTB. Easy access to that expertise continually helped the Fluke lab design, document, and provide high-quality, reliable calibration services.

References

• Ray Kletke, “Maintaining Traceability at Remote Sites With Process Metrology,” NCSL Proceeding, p531, 1995.
• P. Simonson and K.-E. Rydler, “Level Dependence of AC-DC Transfer Devices,” Conference on Precision Electromagnetic Measurements Digest, July 1998, pp375-376.
• U. Pogliano, G. C. Bosco, V. D’Elia, “Measurement System for the Voltage Dependence of High Voltage AC-DC Thermal Converters,” Conference on Precision Electronmagnetic Measurements Digest, July 1998, pp377-378.
• M. Klonz, T. Spiegel, H. Laiz, E. Kessler, “1000-V-Resister for AC-DC Voltage Transfer,” Conference on Precision Electronmagnetic Measurements Digest, July 1998, pp379-380.
• J. Kinard, T. Lipe, C. Childers, “Comparison of High Voltage Thermal Converter Scaling to a Binary Inductive Voltage Divider,” Conference on Precision Electronmagnetic Measurements Digest, July 1998, pp381-382.
• Ray Kletke, “Accreditation Experiences of an American Company,” NCSL Proceeding, 1999. Presented by David Deaver, Metrology Manager, Fluke Corporation.