Does a scope only apply to ISO 17025, the international standard, or does it also apply to Z540 the national standards?
For example; the scope states " XYZ Laboratory has been assessed by ANAB and meets the requirements of international standard ISO/IEC 17025:2005 and national standard ANSI/NCSL Z540-1-1994 & ANSI/NCSL Z540.3-2006" in addition it says Pressure 0-10k psig, and an CMC uncertainty of 0.01 %, I know I can't state a better uncertainty, or a pressure above the range, for an Accredited ISO 17025 calibration, but what about a Z540 calibration, provided I adhere to the other requirements of Z540.
Specifically: Can XYZ Laboratory issue a Z540.1 certificate for a 12k psig pressure gauge?
Thank you for your time.
You state the scope range is 0-10k psig. You cannot issue an accredited cert for a 12k psig range unless you mark that measurement as non-accredited. you can issue a Z540.1 cert provided you do not add an accredited logo.
Wasn't Z540.1 cancelled and replaced by Z540.3?
"ANSI/NCSL Z540.3-2006 replaces Part II of ANSI/NCSL Z540.1-1994(R2002), which was withdrawn in July 2007."
Yeah, but dot 3 has not been and never will be fully accepted. That's why most labs are referencing 17025 and/or Z540.1
Quote from: metrologygeek on 03-10-2016 -- 10:25:02
Yeah, but dot 3 has not been and never will be fully accepted. That's why most labs are referencing 17025 and/or Z540.1
Why is that?
Because it is up in the clouds and we are on the ground--most places cannot comply--Tektronix will NOT comply.
Pretty much the bottom line why it wont be accepted anytime soon is cost. As stated above, the requirements are just ridiculous in areas. Calculating PFA's throw in some guard banding and many changes in the standard from "Should" to "Shall" is just to much for the return on quality. Is going from 99.9% quality rating to 99.92% quality rating while doubling your quality budget reasonable...many companies say nope.
Thanks for the explanation.
In my recent experience with the new and improved Tektronix calibration labs, they outsource their own stuff to some outfit on the west coast...oddness
I dont agree with the statement that the requirements are "ridiculous". I think the push back is because there are a lot of companies that realize they just do not have enough (or any) people who understand the statistical portions of the standard, thus would not be able to support it.
I do however think it's silly to have two standards governing the same industry. 17025 and Z540. There are alot of good things in DOT3...PFAs and guardbanding being two of them.. in my opinion.... but it's only an American standard.
17025 is the international standard (accepted globally),I think it's sufficient (far from perfect) and I think there are a lot of holes in it that need to be tightened up.
The piece to pay attention to is that many Gov contracts have replaced 17025 requirements with Z540.3 requirements and they are not waiving it. So I dont see Z540.3 going away anytime soon.
The fact that Tek and some other companies are digging their heels in has zero bearing on the outcome. There are plenty of companies who have completed dot3 accreditation, the work will simply go to them .
I have to agree with metrologygeek, RFCAL and ck454ss. The marginal (and VERY marginal) gains that you achieve by dot 3 compliance are not worth the expense in my opinion. We have reached the point where we are down in the dirt at the 3rd, 4th or 5th decimal place trying to determine our uncertainties and have lost sight of the fact that it is still an ESTIMATION of uncertainty.
I do agree in many aspects they are very similar and besides the added statistical processes the differences are marginal. Hence my suggestion that the fear driving the no Z540.3 train is the stats portion.
I don't understand Ol'Dave's comment about uncertainties. Why would you squeeze all the fat out of your budget? There is no requirement in either standard to have the smallest uncertainty possible. It surely is not for sales and marketing, atleast I havent seen many "tightest uncertainty in the business" marketing campaigns. I dont think the largest majority of customers even consider each competitors uncertainties when the select vendors. Lastly you only report uncertainties at 3 significant digits (at the most)...
I agree though that 17025 makes far more sense than Z540.3 from a practicality standpoint.
My Goodness USMC!! Dot 3 sucks in all aspects. It is very confusing and not very $$ friendly to comply. The Govt is pushing this and it WILL die as more and more companies say F---U! I can understand the guardbanding, but the rest of it is pure garbage!
I liken Z540.3 to replacing the chains on the sidelines of a NFL football game with a precision measuring tape; yet the largest uncertainty component is where the referee places the ball...
At the end of the day knowing the uncertainties does virtually nothing to improve the quality of the calibration. Sure, you must know your uncertainties to be NIST traceable, but to calculate each uncertainty for each measurement does nothing but give the client more information than they need to determine if their equipment met their needs. This push to do statistical analysis may be driven on the highest levels to assure the quality of calibration across the industry, but it has become a money-making conspiracy. Small commercial labs generally don't have the man-power required to keep up with these ever increasing demands. Then you add the cost of accreditation, artifact testing, caculating uncertainties, QA overhead...the cost to the client goes up and up for what is essentially the same calibration they got before accreditation.
You are right, it doesn't improve the quality of the calibration, it does however improve the quantification of the measurement. In some cases and to more companies than you might think, that can be of critical import.
As far as customers over paying for service, you should do some homework. Calibration service dollars, when adjusted for inflation, have been becoming more and more powerful over the last 40 years. In other words calibration service is cheaper than ever.
The fact that a company will pay for service they don't need on equipment they don't need calibrated is the fault of the customer, not the metrology industry.
Quote from: USMC kalibrater on 03-23-2016 -- 10:23:03
The fact that a company will pay for service they don't need on equipment they don't need calibrated is the fault of the customer, not the metrology industry.
I would have to disagree with that statement. I do not want to pay for a service I dont need but because of a blanket requirement I have to pay for it due to my customers needs. Do I really give a crap, or my customer, that a Fluke 87 DMM my maintenance people use to measure 480 or 115VAC for LOTO or for any other measurement they take fixing a machine has an uncertainty/quardband of whatever...no I don't but to get my pretty piece of paper I have to get it to do business with my customer. The "metrology machine" as I like to call it has gone the way of college professors and there experiments instead of real life manufacturing with common sense. While I may get better quantification of my measurements does it really matter? In most cases no because if I have a robust quality system in place and proper spc being used I should know well before a measurement device reaches a point where the uncertainty or even guardbanding becomes an issue on my product.
Quote from: USMC kalibrater on 03-23-2016 -- 10:23:03
The fact that a company will pay for service they don't need on equipment they don't need calibrated is the fault of the customer, not the metrology industry.
I would say that it's the fault of a calibration industry that has abdicated its responsibility to properly inform and support its customers. We say that we are the experts, but then balk at informing or talking responsibility for a dysfunctional system. It seems that we're happy to just cash the check sometimes.
I would lean towards agreeing that there are far too many statisticians in calibration now. The returns seem to diminish towards zero.
I hear what you are both saying griff and ck but I will argue
1) The fact that customers pay for accredited service on equipment that does not need it is the fault of the customer. It is the customers responsibility to understand their own quality system and determine the best method to support their own equipment, much like it's on the customer to determine intervals. Should we, as metrologists provide guidance? Sure, but ask the ones that do how often their advice is followed, then ask the ones who respond positively how accurately the advice was followed. Companies (a large majority of them do it) overspend on Metrology services because they do not want the risk of an NC, they simply prefer not to "complicate" the process or, and to a lesser extent, just don't understand their own quality system. Remember customers requirements drive our requirements..you can blame 17025 on the auto industry and Z540 on Uncle Sam.
"In most cases no because if I have a robust quality system in place and proper spc being used I should know well before a measurement device reaches a point where the uncertainty or even guard banding becomes an issue on my product." ...because an engineer has already performed guard banding calculations for this process....remember they have a TUR to achieve in their test as well.
2) You argue that companies are wasting money calibrating items that don't need calibrated or do not need calibration with full uncertainties and/or guard banding because it is a waste of money (to which I largely agree but disagree with you about where in fact the blame falls). Then you call a fire mission on the "over use" of uncertainty and guard banding practices that properly define and quantify measurements . This seems more self-serving than practical considering your position over calibration. It would seem to me that these practices, using your model, should be more prevalent since we would only be calibrating instruments that were actually providing some manner of quantifiable measurement. In some cases, such as your lab standards, especially if you are in a thermo-dynamic lab or mechanical standards lab, uncertainty can become quite important. (we all know this just stating it to make a point) I would say that it is of great import for a technician to have the capacity to look at an uncertainty and determine where that number came from and to transfer that uncertainty to next lower level. To a large extent (a very large extent) technicians do not understand and there in lies the issue. This is where most companies (especially third party labs) take issue with the current standards. Metrology is largely statistics gents... you may not see it at your level or where you work.
3) "The returns seem to diminish towards Zero. I would argue that this has much more to do with how a business is being run than anything else. The third party industry has whore'd itself out so much that most companies charge less than my mechanic. If you look at prices now and discount them back about 40s years we are charging significantly less (almost 45%) less than we did. In some cases its closer to 70% less...meanwhile OEMS have held constant. We look at OEM pricing and laugh at how expensive it is, meanwhile I was out at Fluke in Everett not that long ago and their service center was bursting at the seems with work, Ive been told that you will find the same thing at R&S in Maryland and at several of the Keysight locations (I cant speak for all because I dont know people at them all).
If they are so ungodly expensive, why do they have so much work?
Sorry for the long winded rant
I don't necessarily think it's the customer's fault all the time. They get stupid auditor's that come in most of the time and tell them what they need to be in compliance and pass audits. They push getting full data and uncertainties on them when for most of their applications they don't really need it. Then third party cal techs get stuck doing full data on Fluke 8010A's and Simpson 260's!!
OMG USMC Get REAL!!
I think 'USMC kalibrater' states the case very well.
We in the bio-medical industry are heavily regulated by the FDA to ensure the highest quality of our products and we use various elements of Z540.3 and ISO 17025 to help us evulate and select calibration suppliers and also to improve our internal calibrations.
Measurement Uncertainty, guard-banding, and measurement decisions are all essential elements in producing or accepting quality measurement results. The more critical the instrument and measurement application the more important these elements become.
BamaKid - If you check the FDA requirements for a calibration program, they are not that stringent. It says you must have a calibration program traceable to NIST. You define your program and they inspect to see that you follow it. There is nothing in there (yet) that says you provider has to be ISO accredited. There is nothing in there that requires you to guardband. It is a good practice and recommended, but not required. Many pharmas have acceptance criteria and adjustment criteria. If the measurement is in spec, then it passes calibration. This insanity of it passes calibration unless you take uncertainty into account and then maybe it failed, but probably not, but since it could be bad its bad.
The FDA does not define the detailed requirements on how a bio-medical calibration or testing laboratory is maintained. The organization determines what are the best methods to use to have a well maintained, quality lab. We select many methods (measurement uncertainty, guard-banding, measurement decision rules, etc.) that will help us produce quality measurement results that will support the manufacturer of quality implantable products.
From the FDA web site:
The CGMP requirements were established to be flexible in order to allow each manufacturer to decide individually how to best implement the necessary controls by using scientifically sound design, processing methods, and testing procedures. The flexibility in these regulations allows companies to use modern technologies and innovative approaches to achieve higher quality through continual improvement. Accordingly, the "C" in CGMP stands for "current," requiring companies to use technologies and systems that are up-to-date in order to comply with the regulations. Systems and equipment that may have been "top-of-the-line" to prevent contamination, mix-ups, and errors 10 or 20 years ago may be less than adequate by today's standards.
It is important to note that CGMPs are minimum requirements.
Additionally, the FDA is requiring their own internal testing labs to be accredited to ISO 17025.
These quality standards are flexible because they are intended to satisfy most, if not all commercial enterprises that want or need them.
We have all seen instances where someone needs a unique or unusual item certified that for whatever reason has no tolerance assigned yet and when asked what tolerance it needs to be calibrated to they say "as accurate as you can do it".
That is someone who is not thinking about what they are doing or why they are doing it.
The answer is to simply look at what the requirements are for what it is supposed to do.
It should not be certified to an unreasonable tolerance because then it is more costly to calibrate and more likely to be found out of tolerance when calibrated but more importantly - it serves no rational purpose to do that -
Our quality standards should be viewed the same way.
For at least some of their uncertainty estmates, Fluke uses a coverage factor of 3 instead of the usual 2 and this increases their cost of calibration immensely while giving them a 15% return on that cost.
What happens when I buy a meter is I don't even think about it. I buy a Fluke. That's part of the return on their investment.
I try very hard to be thorough and rational and I simply will not lie to my customers. They definitely should understand their quality standards and what they are doing and why they are doing it. They frequently do not.
A big part of this job is to politely help them do that.
Quote from: briansalomon on 04-01-2016 -- 10:01:29
For at least some of their uncertainty estmates, Fluke uses a coverage factor of 3 instead of the usual 2 and this increases their cost of calibration immensely while giving them a 15% return on that cost.
I give Fluke credit for actually stating a coverage factor or confidence level (and confusing the two), where as other manufacturers usually don't specify what sort of distribution their specs are representing. But, I honestly don't understand if there is a real difference between reporting the specs at a coverage factor of 3 instead of 2. As far as I know they both represent the same distribution just represented in two different ways. I mean, they could report it at K = 4 or 5 and it wouldn't change the specs, just the numbers that are published.
Quote from: briansalomon on 04-01-2016 -- 10:01:29
A big part of this job is to politely help them do that.
That would be my point in a nutshell, thanks
N79, the relationship between the instrument tolerance and the coverage factor is a very good place to look if we want to better understand what uncertainty in measurement means.
I found that the white paper written by Keith Bennett and Howard Zion (both PMEL guys) really illuminated this for me and helped me to explain this to customers. It only takes about 1/2 hour to read.
http://www.transcat.com/media/pdf/TUR.pdf
The way they explain it is that the coverage factor represents a big part of the total uncertainty of their guarantee that the instrument was in fact in-tolerance when they calibrated it.
The main factor in that is the tolerance of the standards used but would be intended to include everything else that affected the calibration.
It obviously gets complicated but that's what were here for. Mr. Bennett and Mr. Zion got it right and I refer to their paper a lot.
I read through the paper but I couldn't find anything about expanded uncertainty coverage factors/confidence levels except a brief mention of the industry-standard 95% C.L. in the introduction. As far as I know the evaluation of the TUR / applying guardbanding doesn't have anything to do with what CF/CL the expanded uncertainty is reported as.
If I recall correctly, Fluke lists 99% and 95% CL specs for their 5700 family but there is not a fixed factor relating the two (i.e. 2.58 / 1.96 = 1.316, the 99% specs should be 1.316 times the 95% specs) that one would expect if they were calculated from the same distributions. So I'm curious on what the difference is between the two evaluations.
I believe it is because there is not a linear relationship between the two. 95% approximates 2 standard deviations, and 99% approximates 3 standard deviations.
Yes, I see the 99% confidence interval as the result of a 2.58 coverage factor and a 95% confidence interval the result of a 1.96 coverage factor. That's Important.
So if I'm saying my instrument is in tolerance 95% of the time (most labs) I would use K= 1.96 and if I'm saying it's 99% I'd use 2.58.
And you're right N79, they don't spell that out in the paper but they do make a good point about the expense curve as you increase the confidence level.
I'm not sure that's right. The 95% CL representing an expanded uncertainty usually only pertains to the measurement and does not predict future performance or value. The "95%" is basically a way of saying that you're 95% sure the (unknowable) true value of the measurement falls within the stated uncertainty with the most probable value being the reported value (the mean).
If I take 20 measurements of an item and report the mean of those measurements as my measured value, I can also report the combined uncertainty for that measurement as the standard deviation. These two parameters describe the distribution of values, assuming the samples and combined Type B sources of uncertainty fit a normal curve. You then can report an expanded uncertainty which is just multiplying this "std dev" by some factor (usually either 2 or 3 or 1.96 or 2.58). But all of these values represent the same distribution, so I don't really see the point in reporting this type of expanded uncertainty at all. Or maybe I'm missing something... which is probably the case! :-D
Here is a good way to understand the basic idea.
Take the tolerance of the instrument and square it.
Take 5 readings and find the average. Then find the standard deviation and square that.
(standard deviation is slightly different from the mean)
Add these two together and find the square root.
For now, look at this as "all of the things that affect the accuracy of your readings" (unexpanded uncertainty, not just the tolerance)
Now multiply the unexpanded uncertainty by 1.96 to find the 95%confidence level (expanded uncertainty)
Now multiply the unexpanded uncertainty by 2.59 to find the 99% confidence level (expanded uncertainty)
The two factors are applied to "all of the things that affect the accuracy of your readings" and it makes the uncertainty larger or smaller.
If the expanded uncertainty is larger, it makes the distribution wider it more likely the readings will fall within the distribution.
If the expanded uncertainty is smaller it's less likely the readings will fall within the distribution. If we're using the normal distribution this is the bell curve we're familiar with. The bell curve can get larger or smaller but it stays the same shape.
It sounds too simple but you're just multiplying your unexpanded uncertainty by a smaller or larger number to "expand" it to a 95% or 99% confidence level.
After you get comfortable with this, there are more things that affect the accuracy of your readings like temperature which have to be addressed.
Once someone explained it to me this way I got a better idea of what uncertainty represented.
Quote from: briansalomon on 04-05-2016 -- 14:24:31
Take the tolerance of the instrument and square it.
I think it should be "Take the accuracy of the standard and square it".
I.E. if you are calibrating a handheld DMM you would RSS the output accuracy of your calibrator (5720, etc) and your standard deviations, not the tolerances of the instrument and the standard deviations.
Yes, thank you. It's the uncertainty of the calibration we would be interested in.
But, that is also at the time of the calibration, it has nothing to do with future performance and/or guarantee that the DUT will remain within the stated tolerances for the duration of the calibration cycle.
The coverage factors only really come into play if you are guardbanding or if the manufacturer lists the instrument's accuracies in a specific coverage factor (Fluke loves to use K=1). If they don't list a coverage factor, I believe it is assumed to be 2, but I'm not an expert on uncertainties by any means.
briansaloman, my point is that reporting the expanded uncertainty (at any confidence level or coverage factor) does NOT change the shape of the distribution that the reported value and reported uncertainty represent.
A simple way of thinking about this is imagining a set of data with mean of 0 and standard deviation of 1. If you plot this distribution (x-axis is the values, y values the probability of each value) you'd get your typical normal curve that peaks at 0. If you shaded the area under the curve between -1 and 1, you'd would be shading k = 1, or ~68% of the curve. If you shaded between -2 and 2 (two std devs, or k = 2) you'd end up shading ~95% of the curve. If you shaded between -3 and 3 you would end up shading ~99.7% of the curve and so on and so forth. I don't think it's possible to shade 100% of the curve unless you had an infinitely long piece of paper to draw this on.
So, the shape of the distribution doesn't change regardless of what coverage factor you report, so really it doesn't matter, all coverage factors and confidence levels of a gaussian distribution represent the same distribution with the same mean and variance or standard deviation (which is equivalent to the combined uncertainty in our world of metrology),
Quote from: briansalomon on 04-04-2016 -- 15:22:18
So if I'm saying my instrument is in tolerance 95% of the time (most labs) I would use K= 1.96 and if I'm saying it's 99% I'd use 2.58.
...Wuuut?
Stating my uncertainty at 95% does not imply that 5% of the time it might be out of spec! Confidence levels describe the probability of a reading being within a certain number of standard deviations...
In other words, if I claim 95% confidence level then I am stating that 95% of my readings will be within two standard deviations.
Thank you for correcting that for me. I should have begun my post by saying that I am no expert when it comes to uncertainty.
I'll try to be more thorough.
The back and forth on this topic brings in to stark relief for me the insanity that we are dealing with. The statisticians have taken over and ruined a once great industry. It's less and less about the measurements, and more and more about the math. That's not my cup of tea and the vast majority of calibrations don't require such rigorous analysis. That's not to say that there isn't any place for that stuff - especially with equipment getting more accurate by the day - but when it starts affecting Fluke 77 cals, I'm out.
Quote from: Duckbutta on 04-06-2016 -- 19:15:52
The back and forth on this topic brings in to stark relief for me the insanity that we are dealing with. The statisticians have taken over and ruined a once great industry. It's less and less about the measurements, and more and more about the math. That's not my cup of tea and the vast majority of calibrations don't require such rigorous analysis. That's not to say that there isn't any place for that stuff - especially with equipment getting more accurate by the day - but when it starts affecting Fluke 77 cals, I'm out.
So much this ^
The problem with these standards is it places to many "Shalls" in the requirement and not enough "Shoulds". The specs leave no room for me to make a judgment on whether I really need all these uncertainties/Guard Banding/Etc. on my equipment. As I said before, my maintenance crew does not need a 17025 cal on their meters used to verify voltage for Lock Out/Tag Out but because we are certified I have to do it. Its just plain stupid and a waste of money imo.
I think it's over kill in most applications. On a funny note though a co-worker reminded me of a funny story from the third party cal lab we used to work in together. One of the shipping and receiving guys was attempting to remove old calibration stickers off pipettes with his teeth!! We explained to him what pipettes are typically used for and he almost threw up! Another shipping guy around the same time took 15kV to the stomach from an ESD gun that was shipped to us holding a charge. Still makes me laugh thinking about stuff like that!
I agree--This is nothing but a waste of time and $$.
In my last ISO17025 audit, the auditor even made the K=2/3 relative to 95%/99% question. I know and reasonably well understand the small difference between 95% and K=2 and between 99% and K=3. But thankfully this was a smart auditor who wasn't so much nit-picking as testing me to see if I knew what I was doing. But yes, this is one of those things that makes so little difference in anything. I don't have time or inclination to spend time on this, but it would interesting to see what difference if any, that fraction of a percent in confidence makes. It shifts the indeterminate threshold a teeny bit. It bumps the M.U. up (or down a tiny bit), etc. But in terms of actual measurand, what does it do? Maybe some pragmatist uncertainty/statistical expert can give an example of its importance.
Agreed, it's usually a waste of energy.
On a less serious note - I do recall one instrument sent in to the PMEL for cal that was an IFF Transponder Test Set that was sent in with the complaint "Works in the IFF position, does not work in the OFF position"....
I'd just like to observe that I am at least 99% certain he wasn't following the procedure.
Quote from: Hawaii596 on 04-07-2016 -- 09:57:11
In my last ISO17025 audit, the auditor even made the K=2/3 relative to 95%/99% question. I know and reasonably well understand the small difference between 95% and K=2 and between 99% and K=3. ... But in terms of actual measurand, what does it do? Maybe some pragmatist uncertainty/statistical expert can give an example of its importance.
The coverage factor does not impact the measurand, the CF just asserts our confidence in any measured point with respect to the sample mean (or measurand in this case). Think about when you apply the CF in the uncertainty equation, at the end.
So if I give an uncertainty a confidence level of 2Sigma (95%) then all I am saying is that 95% of all measurements fell within 2 standard deviations of the mean (or measurand), the point I "think" Im measuring. If I claim 99% then Im claiming 3 standard deviations. So 99% confidence holds more possible results than 95% when applied to the same Uc.
Neither can directly be linked to greater overall accuracy or precision, that can only be determined by a review of the Type A analysis and certain types of Type B contributors.
As an example 99% would be more likely used on an instrument,such as a lab standard, with greater accuracy and precision where the variance fairly constant. K=3 would allow for a more quantifiable representation of the data. Remember if you are using an uncertainty with a 99% confidence level in a 95%CF calculation you must convert the 99% to 95%
A CF of 3 encompasses .66667 more area under the curve (assuming normal distribution) than a CF of 2.
see http://study.com/cimages/multimages/16/Normal_distribution_and_scales.PNG
NISTS two cents :D
http://physics.nist.gov/cuu/Uncertainty/coverage.html
I do understand all of those details, as I'm the one who has to answer all the auditors questions during our ISO17025 audits; and I developed all of the tools we use and train the technicians on how to use them. So I'm not so much looking for a mathematical answer as a philosophical one. About 5 to 10% of our business is ISO17025 Accredited calibrations. Among those customers, I have had personal conversations with just about every one of them, and almost none of them use the uncertainties at all. I have one customer who told me if it were up to him, he would just get a basic certificate, as he has never even used the data points, let along the M.U. associated. So I find it albeit, mathematically correct that K=2 equates to 0.9545, and 0.9500 equates to K=1.96, that it amounts to useless minutia to be concerned about that mathematic difference. This is not a difference in reading, just in confidence limits expressed by the expanded uncertainty. My experience has been that the greatest pragmatic value in the understanding of this difference is no more than impressing the auditor that you know what you are doing for audit purposes.
It does not seem to me that there is significant pragmatic value beyond that. So the answer I was seeking was if someone had some anecdotal thoughts to support pragmatically the importance of this difference (examples of real life circumstances where the difference betweem K=1.96 and K=2 has impacted anything). I believe there may be none. In that case, then, this is a "make-the-auditor-feel-good" issue.
If you want to spend all day calculating the measurement uncertainties including lead and connector loss calibrating a Simpson 260 knock yourself out. I'm going home to watch some Netflix in the mean time!! If you're able to acheive (4:1) I don't see the point of making unnecessary calculations. The previous company I worked for told all their technicians to always list the best case uncertainties straight off the scope of accreditation. Even when most of the time they weren't using the 5720A or 3458A to make those measurements. I'm pretty sure you're supposed to list the measurement uncertainty for the actual standards you are using. Super happy we don't have to be A2LA certified at my current employer!!
That is a no no. What's on the Scope is best case, and doesn't normally reflect real measurement uncertainties. I even heard a story of a major brand name OEM lab (that shall remain nameless) where I was shocked to learn through a friend who interviewed there, that that was how they were doing it.
Quote from: USMC kalibrater on 04-08-2016 -- 04:41:16
Quote from: Hawaii596 on 04-07-2016 -- 09:57:11
In my last ISO17025 audit, the auditor even made the K=2/3 relative to 95%/99% question. I know and reasonably well understand the small difference between 95% and K=2 and between 99% and K=3. ... But in terms of actual measurand, what does it do? Maybe some pragmatist uncertainty/statistical expert can give an example of its importance.
The coverage factor does not impact the measurand, the CF just asserts our confidence in any measured point with respect to the sample mean (or measurand in this case). Think about when you apply the CF in the uncertainty equation, at the end.
So if I give an uncertainty a confidence level of 2Sigma (95%) then all I am saying is that 95% of all measurements fell within 2 standard deviations of the mean (or measurand), the point I "think" Im measuring. If I claim 99% then Im claiming 3 standard deviations. So 99% confidence holds more possible results than 95% when applied to the same Uc.
Neither can directly be linked to greater overall accuracy or precision, that can only be determined by a review of the Type A analysis and certain types of Type B contributors.
As an example 99% would be more likely used on an instrument,such as a lab standard, with greater accuracy and precision where the variance fairly constant. K=3 would allow for a more quantifiable representation of the data. Remember if you are using an uncertainty with a 99% confidence level in a 95%CF calculation you must convert the 99% to 95%
A CF of 3 encompasses .66667 more area under the curve (assuming normal distribution) than a CF of 2.
see http://study.com/cimages/multimages/16/Normal_distribution_and_scales.PNG
NISTS two cents :D
http://physics.nist.gov/cuu/Uncertainty/coverage.html
Sorry to keep picking nits, but this isn't quite right either. The 95% CL only means that there is a 95% chance that the actual value of the test unit (the value we would measure if we had a perfect 0 uncertainty measurement device) falls between the reported value +/- the reported expanded uncertainty.
For instance, if I report a measurement on a resistor at 1.000023 ohms, with a uncertainty of 1 ppm @ a 95% CL, I'm saying that there is a 95% chance the actual value of the resistor is within 1.000022 and 1.000024, with the most likely value being 1.000023.
You'll see that if you test out your version that you'll ALWAYS find that ~95% or your readings/measurements will fall within 2 standard deviations of the mean and that 99.7% of your readings will fall in 3 standard deviations of the mean. This is by definition. The math just works out that way. Of course, this only applies for normal distributions which all multiple-reading measurements should create.
But I tend to agree about some of the silliness that goes into an accurate uncertainty calculation as anyone who uses the Welch–Satterthwaite equation to determine effective degrees of freedom has realized. There is an actual COMPLETELY SUBJECTIVE coefficient involved in each term that describes your "confidence" in that term... So if I feel very confident about a manufacturer's specs, I'd use 1, and if I feel less confident I use some value less than 1, but it's completely up to me to pull this number from my ass. As far as I know there is no good objective method to determine this.
Quote from: silv3rstr3 on 04-08-2016 -- 11:25:22
If you want to spend all day calculating the measurement uncertainties including lead and connector loss calibrating a Simpson 260 knock yourself out. I'm going home to watch some Netflix in the mean time!! If you're able to acheive (4:1) I don't see the point of making unnecessary calculations. The previous company I worked for told all their technicians to always list the best case uncertainties straight off the scope of accreditation. Even when most of the time they weren't using the 5720A or 3458A to make those measurements. I'm pretty sure you're supposed to list the measurement uncertainty for the actual standards you are using. Super happy we don't have to be A2LA certified at my current employer!!
But you don't really know you're at a 4:1 TUR unless you perform the calculations, and you can't perform the calculations until you have your Type A data which is gathered during the measurement. I've had actual cases where the Type A uncertainty (in this case, the standard deviation of repeated measurements) completely swamped the other sources of uncertainty. If I had just compared the specs of the standard and test unit, it would have surpassed the 4:1 requirement, but once the measurement was performed and the Type A was included it couldn't even meet 1:1. So, at least in some cases, it is VERY important to have this calculated dynamically at the time of measurement... at least if you actually want integrity in your calibrations.
It's a shame there isn't
good software that does all this for you.
Quote from: Hawaii596 on 04-08-2016 -- 08:43:03
I do understand all of those details, as I'm the one who has to answer all the auditors questions during our ISO17025 audits; and I developed all of the tools we use and train the technicians on how to use them. So I'm not so much looking for a mathematical answer as a philosophical one. About 5 to 10% of our business is ISO17025 Accredited calibrations. Among those customers, I have had personal conversations with just about every one of them, and almost none of them use the uncertainties at all. I have one customer who told me if it were up to him, he would just get a basic certificate, as he has never even used the data points, let along the M.U. associated. So I find it albeit, mathematically correct that K=2 equates to 0.9545, and 0.9500 equates to K=1.96, that it amounts to useless minutia to be concerned about that mathematic difference. This is not a difference in reading, just in confidence limits expressed by the expanded uncertainty. My experience has been that the greatest pragmatic value in the understanding of this difference is no more than impressing the auditor that you know what you are doing for audit purposes.
It does not seem to me that there is significant pragmatic value beyond that. So the answer I was seeking was if someone had some anecdotal thoughts to support pragmatically the importance of this difference (examples of real life circumstances where the difference betweem K=1.96 and K=2 has impacted anything). I believe there may be none. In that case, then, this is a "make-the-auditor-feel-good" issue.
To me, there are completely separate reasons for reporting uncertainty and being 17025 accredited. The accreditation is really for your customers as an assurance that your practices have been audited by a third-party and you're not just pulling numbers out of your ass. That you have at least a quality system in place, you perform interlab comparisons, you are traceable to national standards, you attempt to calculate uncertainty, etc., etc. You could actually be the best lab in the world, but without accreditation it's hard to sell that fact. Your customers may not use the data you provide, but accreditation at least gives the impression that the data you provide, including the parts that they do actually use, is legit.
As far as uncertainty, as a metrologist you have to know that any reported measurement is meaningless without a properly calculated uncertainty. It's not easy or fun or cheap, but it is the only thing that gives any kind of value to your measurement/calibration.
Edited to add an actual answer to your question: the reported result of a measurement along with the uncertainty of that measurement always describes a statistical distribution. If the uncertainty is described as a coverage factor (K = 1, K = 2, etc.) or a confidence level (95%, 99%, etc.) that implies that the distribution fits a normal curve. There are only two parameters needed to create this distribution: the mean (the reported value) and the standard deviation (the normalized uncertainty). From these two values the customer has the entire distribution of the measurement result and can convert it to whatever form they like. If it were up to me, I'd probably have this distribution graphically represented on the test report, but having the two parameters represents the same thing. So, it is important to use the right coverage factor or confidence level when reporting your uncertainty because you want to describe the right distribution. As far as what difference it makes, probably not much, but why not try to be as accurate as possible.
The technicians that actually had integrity and respected our field of work brought up that argument openly to management. But unfortunately at that company (the one I'm still being nice and not naming) was only concerned with the bottom line revenue number at the end of each month! I care immensely about the quality of my work. I've seen first hand a massive investigation when a CH-53D's rear rotor folded in on itself taking off on MCAS Futenma and crashed into a Japanese school on a Sunday! What we do especially in the Defense and Space industry is crucial. As far as knowing if the standards you're using are 4:1 it's not as complicated as some people here are arguing and making it out to be. The DOD procedures tell you in table one if it doesn't meet 4:1. If I have to substitute a standard at all in a procedure I pull the specifications and do simple math to make sure it's good enough to use. I've only recently been trying to wrap my head around the guard banding concept and why it's necessary. Plus most quality systems I've worked under stated that if a standard doesn't meet 4:1 you have to specify where and when on the certificate and/or data if required. I attended a NCSLI event a year ago or so and they taught a class on uncertainty classes and importance. The guy speaking impressed me with all the data and analysis he was doing for a manufacturing company. He was able to figure out a defect in a product that no one could figure out because of the math alone. I respect the principle of it all so don't get me wrong here. However, there isn't a whole lot of room for the analysis in the 3rd Party Cal world or even now in an in house lab either. The equipment most people are using is so obsolete it's ridiculous. The majority of the auditors I've witnessed in each environment may actually know one or two things about Metrology. They stick to the few things they know and harp on them because that's all they cared to learn. I've heard A2LA has recently tightened up their audits in the last few years. If that's true, good for them. Before that it was a dog and pony show when they were there and like most things....it all boiled down to money in the end. I've heard some outlandish stories from people at shady companies and how they passed their A2LA audits throughout the 2000 - 2012 years. I plan to make this a career as I do like the complexity of our profession. I respect all of you that can do a lab level uncertainty budget cause that's no walk in the park!!
Quote from: Hawaii596 on 04-08-2016 -- 08:43:03
I do understand all of those details, as I'm the one who has to answer all the auditors questions during our ISO17025 audits; and I developed all of the tools we use and train the technicians on how to use them. So I'm not so much looking for a mathematical answer as a philosophical one. About 5 to 10% of our business is ISO17025 Accredited calibrations. Among those customers, I have had personal conversations with just about every one of them, and almost none of them use the uncertainties at all. I have one customer who told me if it were up to him, he would just get a basic certificate, as he has never even used the data points, let along the M.U. associated. So I find it albeit, mathematically correct that K=2 equates to 0.9545, and 0.9500 equates to K=1.96, that it amounts to useless minutia to be concerned about that mathematic difference. This is not a difference in reading, just in confidence limits expressed by the expanded uncertainty. My experience has been that the greatest pragmatic value in the understanding of this difference is no more than impressing the auditor that you know what you are doing for audit purposes.
It does not seem to me that there is significant pragmatic value beyond that. So the answer I was seeking was if someone had some anecdotal thoughts to support pragmatically the importance of this difference (examples of real life circumstances where the difference betweem K=1.96 and K=2 has impacted anything). I believe there may be none. In that case, then, this is a "make-the-auditor-feel-good" issue.
Ahhh got ya, so a little story
95% = K = 2 (instead of 1.96) this is what I've experienced first hand. It depends on the statistics course you take in college. So Ive been working on my BS Physics for a few years and recently switched to a degree in Management just to get a BS done so I can get promoted (I plan on finishing my PHY degree later). When I switched degrees I changed campuses to one close to my home. (Tampa traffic...sucks!). I needed to reestablish residency at the new campus because you need so many credit hours at the campus you intend to graduate from.
So after completing Calc1,2 and 3, Diff E, Computational Derivation and Engineering Stats 1 & 2 in my Physics curriculum I decided Business Calc and Business Stats would be a pretty low hurdle to jump yet still be interesting enough. I was right on assumption one, wrong on assumption two. These classes are really really really dumbed down, I mean really to the point they weren't even interesting.
I digress, In Business Stats 95% always = 2 and 99% always = 3, I asked the instructor why she taught it this way and her answer boiled down to rounding. Rarely in business stats are you much concerned with decimal places after two...a generic and cheap answer but never argue with the prof. In her defense the book was written the same way she taught the course.
I'll have to go back and look in some of my notes and recalculate some of the experiments we did and see if .9545 changes the experiment results with any level of significance. I think I have a few that it will, Im almost positive that any thing in the quantum arena it will. I cant really think of any thing in Newtonian because the numbers aren't any where near the magnitudes of large or small you see in quantum.
I subscribe to the same theory that you do, in that, we do a lot of things that are just to "make the auditor happy" or "to make the auditor feel smarter".
Quote from: silv3rstr3 on 04-08-2016 -- 11:25:22
If you want to spend all day calculating the measurement uncertainties including lead and connector loss calibrating a Simpson 260 knock yourself out. I'm going home to watch some Netflix in the mean time!! If you're able to acheive (4:1) I don't see the point of making unnecessary calculations. The previous company I worked for told all their technicians to always list the best case uncertainties straight off the scope of accreditation. Even when most of the time they weren't using the 5720A or 3458A to make those measurements. I'm pretty sure you're supposed to list the measurement uncertainty for the actual standards you are using. Super happy we don't have to be A2LA certified at my current employer!!
Have you started watching the new season of Daredevil yet? Ive been binge watching Sons of Anarchy...why I am still watching it is beyond me. It really went down hill fast after about season 3
"However, there isn't a whole lot of room for the analysis in the 3rd Party Cal world or even now in an in house lab either. The equipment most people are using is so obsolete it's ridiculous"
I think most companies today practice the better safe than sorry calibration model. I know we do where I am at as well. It feels far safer to keep calibrating even the most obviously waste full items. Like the plant electricians DMM and the old analog meters on power supplies when they are being monitored by 6.5 digit DMMs.
Many companies just go through the motions to cover whatever quality system they subscribe to in order to keep things simple. Like others have pointed out, they pay for accredited calibrations yet don't even look at the data or need it, they calibrate everything that could possibly need calibrated even when the application doesn't require it or need it.
Be lean my friends!
What takes the cake for me was when I was instructed to calibrate a clipboard at my previous place of employment. The clipboard had a built on ruler and calculator. I actually made a professional looking excel datasheet for it. Did 1" length measurements up to 12" using gage blocks. And just to be a smart @$$ over the principle of how stupid this was, I included a Pass/Fail calculator accuracy table of addition, subtraction, division, and multiplication!!
I think it had a timer too, didn't it?
Yeah, I believe it did! LOL. It would have been funnier if they wanted uncertainty measurements for this clipboard on top of it!!!