Scope of Accreditation, Z540.1 & Z540.3

[USMC kalibrater]

You are right, it doesn't improve the quality of the calibration, it does however improve the quantification of the measurement.  In some cases and to more companies than you might think, that can be of critical import.
As far as customers over paying for service, you should do some homework.  Calibration service dollars, when adjusted for inflation, have been becoming more and more powerful over the last 40 years. In other words calibration service is cheaper than ever.
The fact that a company will pay for service they don't need on equipment they don't need calibrated is the fault of the customer, not the metrology industry.

[ck454ss]

The fact that a company will pay for service they don't need on equipment they don't need calibrated is the fault of the customer, not the metrology industry.

I would have to disagree with that statement.  I do not want to pay for a service I dont need but because of a blanket requirement I have to pay for it due to my customers needs.  Do I really give a crap, or my customer, that a Fluke 87 DMM my maintenance people use to measure 480 or 115VAC for LOTO or for any other measurement they take fixing a machine has an uncertainty/quardband of whatever...no I don't but to get my pretty piece of paper I have to get it to do business with my customer.  The "metrology machine" as I like to call it has gone the way of college professors and there experiments instead of real life manufacturing with common sense.  While I may get better quantification of my measurements does it really matter?  In most cases no because if I have a robust quality system in place and proper spc being used I should know well before a measurement device reaches a point where the uncertainty or even guardbanding becomes an issue on my product.

[griff61]

The fact that a company will pay for service they don't need on equipment they don't need calibrated is the fault of the customer, not the metrology industry.

I would say that it's the fault of a calibration industry that has abdicated its responsibility to properly inform and support its customers. We say that we are the experts, but then balk at informing or talking responsibility for a dysfunctional system. It seems that we're happy to just cash the check sometimes.
I would lean towards agreeing that there are far too many statisticians in calibration now. The returns seem to diminish towards zero.

[USMC kalibrater]

I hear what you are both saying griff and ck but I will argue
1) The fact that customers pay for accredited service on equipment that does not need it is the fault of the customer.  It is the customers responsibility to understand their own quality system and determine the best method to support their own equipment, much like it's on the customer to determine intervals.  Should we, as metrologists provide guidance? Sure, but ask the ones that do how often their advice is followed, then ask the ones who respond positively how accurately the advice was followed.  Companies (a large majority of them do it) overspend on Metrology services because they do not want the risk of an NC, they simply prefer not to "complicate" the process or, and to a lesser extent, just don't understand their own quality system.  Remember customers requirements drive our requirements..you can blame 17025 on the auto industry and Z540 on Uncle Sam.
"In most cases no because if I have a robust quality system in place and proper spc being used I should know well before a measurement device reaches a point where the uncertainty or even guard banding becomes an issue on my product." ...because an engineer has already performed guard banding calculations for this process....remember they have a TUR to achieve in their test as well.
2) You argue that companies are wasting money calibrating items that don't need calibrated or do not need calibration with full uncertainties and/or guard banding because it is a waste of money (to which I largely agree but disagree with you about where in  fact the blame falls).  Then you call a fire mission on the "over use" of uncertainty and guard banding practices that properly define and quantify measurements .  This seems more self-serving than practical considering your position over calibration.  It would seem to me that these practices, using your model, should be more prevalent since we would only be calibrating instruments that were actually providing some manner of quantifiable measurement.  In some cases, such as  your lab standards, especially if you are in a thermo-dynamic lab or mechanical standards lab, uncertainty can become quite important.  (we all know this just stating it to make a point)  I would  say that it is of great import for a technician to have the capacity to look at an uncertainty and determine where that number came from and to transfer that uncertainty to next lower level.  To a large extent (a very large extent) technicians do not understand and there in lies the issue.  This is where most companies (especially third party labs) take issue with the current standards.  Metrology is largely statistics gents...  you may not see it at your level or where you work.
3) "The returns seem to diminish towards Zero.  I would argue that this has much more to do with how a business is being run than anything else.  The third party industry has whore'd itself out so much that most companies charge less than my mechanic.  If you look at prices now and discount them back about 40s years we are charging significantly less (almost 45%) less than we did.  In some cases its closer to 70% less...meanwhile OEMS have held constant.   We look at OEM pricing and laugh at how expensive it is, meanwhile I was out at Fluke in Everett not that long ago and their service center was bursting at the seems with work, Ive been told that you will find the same thing at R&S in Maryland and at several of the Keysight locations (I cant speak for all because I dont know people at them all).
If they are so ungodly expensive, why do they have so much work? 
Sorry for the long winded rant

[silv3rstr3]

I don't necessarily think it's the customer's fault all the time.  They get stupid auditor's that come in most of the time and tell them what they need to be in compliance and pass audits.  They push getting full data and uncertainties on them when for most of their applications they don't really need it.  Then third party cal techs get stuck doing full data on Fluke 8010A's and Simpson 260's!! 

[RFCAL]

OMG USMC  Get REAL!!

[BamaKid]

I think 'USMC kalibrater' states the case very well.

We in the bio-medical industry are heavily regulated by the FDA to ensure the highest quality of our products and we use various elements of Z540.3 and ISO 17025 to help us evulate and select calibration suppliers and also to improve our internal calibrations.

Measurement Uncertainty, guard-banding, and measurement decisions are all essential elements in producing or accepting quality measurement results. The more critical the instrument and measurement application the more important these elements become.

[NC-Cals]

BamaKid - If you check the FDA requirements for a calibration program, they are not that stringent. It says you must have a calibration program traceable to NIST. You define your program and they inspect to see that you follow it. There is nothing in there (yet) that says you provider has to be ISO accredited. There is nothing in there that requires you to guardband. It is a good practice and recommended, but not required. Many pharmas have acceptance criteria and adjustment criteria. If the measurement is in spec, then it passes calibration. This insanity of it passes calibration unless you take uncertainty into account and then maybe it failed, but probably not, but since it could be bad its bad.

[BamaKid]

The FDA does not define the detailed requirements on how a bio-medical calibration or testing laboratory is maintained. The organization determines what are the best methods to use to have a well maintained, quality lab. We select many methods (measurement uncertainty, guard-banding, measurement decision rules, etc.) that will help us produce quality measurement results that will support the manufacturer of quality implantable products.

From the FDA web site:
The CGMP requirements were established to be flexible in order to allow each manufacturer to decide individually how to best implement the necessary controls by using scientifically sound design, processing methods, and testing procedures. The flexibility in these regulations allows companies to use modern technologies and innovative approaches to achieve higher quality through continual improvement. Accordingly, the "C" in CGMP stands for "current," requiring companies to use technologies and systems that are up-to-date in order to comply with the regulations. Systems and equipment that may have been "top-of-the-line" to prevent contamination, mix-ups, and errors 10 or 20 years ago may be less than adequate by today's standards.

It is important to note that CGMPs are minimum requirements.

Additionally, the FDA is requiring their own internal testing labs to be accredited to ISO 17025.

[briansalomon]

These quality standards are flexible because they are intended to satisfy most, if not all commercial enterprises that want or need them.

We have all seen instances where someone needs a unique or unusual item certified that for whatever reason has no tolerance assigned yet and when asked what tolerance it needs to be calibrated to they say "as accurate as you can do it".

That is someone who is not thinking about what they are doing or why they are doing it.

The answer is to simply look at what the requirements are for what it is supposed to do.

It should not be certified to an unreasonable tolerance because then it is more costly to calibrate and more likely to be found out of tolerance when calibrated but more importantly - it serves no rational purpose to do that -

Our quality standards should be viewed the same way.

For at least some of their uncertainty estmates, Fluke uses a coverage factor of 3 instead of the usual 2 and this increases their cost of calibration immensely while giving them a 15% return on that cost.

What happens when I buy a meter is I don't even think about it. I buy a Fluke. That's part of the return on their investment.

I try very hard to be thorough and rational and I simply will not lie to my customers. They definitely should understand their quality standards and what they are doing and why they are doing it. They frequently do not.

A big part of this job is to politely help them do that.

[N79]

For at least some of their uncertainty estmates, Fluke uses a coverage factor of 3 instead of the usual 2 and this increases their cost of calibration immensely while giving them a 15% return on that cost.

I give Fluke credit for actually stating a coverage factor or confidence level (and confusing the two), where as other manufacturers usually don't specify what sort of distribution their specs are representing. But, I honestly don't understand if there is a real difference between reporting the specs at a coverage factor of 3 instead of 2. As far as I know they both represent the same distribution just represented in two different ways. I mean, they could report it at K = 4 or 5 and it wouldn't change the specs, just the numbers that are published.

[griff61]

A big part of this job is to politely help them do that.

That would be my point in a nutshell, thanks

[briansalomon]

N79, the relationship between the instrument tolerance and the coverage factor is a very good place to look if we want to better understand what uncertainty in measurement means.

I found that the white paper written by Keith Bennett and Howard Zion (both PMEL guys) really illuminated this for me and helped me to explain this to customers. It only takes about 1/2 hour to read.

http://www.transcat.com/media/pdf/TUR.pdf

The way they explain it is that the coverage factor represents a big part of the total uncertainty of their guarantee that the instrument was in fact in-tolerance when they calibrated it.

The main factor in that is the tolerance of the standards used but would be intended to include everything else that affected the calibration.

It obviously gets complicated but that's what were here for. Mr. Bennett and Mr. Zion got it right and I refer to their paper a lot.

[N79]

I read through the paper but I couldn't find anything about expanded uncertainty coverage factors/confidence levels except a brief mention of the industry-standard 95% C.L. in the introduction. As far as I know the evaluation of the TUR / applying guardbanding doesn't have anything to do with what CF/CL the expanded uncertainty is reported as.

If I recall correctly, Fluke lists 99% and 95% CL specs for their 5700 family but there is not a fixed factor relating the two (i.e. 2.58 / 1.96 = 1.316, the 99% specs should be 1.316 times the 95% specs) that one would expect if they were calculated from the same distributions. So I'm curious on what the difference is between the two evaluations.

[Hawaii596]

I believe it is because there is not a linear relationship between the two.  95% approximates 2 standard deviations, and 99% approximates 3 standard deviations.