One question is this helping our hurting our job field?
My thoughts on this ( based on a real case) Customer has approximately 1000 pieces needing A2LA calibration they go to the web to search the A2LA website were they find several labs located in there area. They contact the labs and decided to do a sampling on several items that will allow them to make an informed decision. Based upon price, service and they fact that all are A2LA certified quality is perceived to me equal. They make what the feel is an informed decision and begin to use the lab of choice. Approximately 6-8 months later the problems start and they learn this lab has barely been able to keep accredidation and there whole cal program is now in jeopardy.
1. Should A2LA have listed this lab under a difference staus?
2. Should there be a rating system within A2LA ? Something similiar to restuarants, if you enter a restuarant and see a rating of 65 are you really going to eat there, there is also a report posted showing there defiencies.
3. SHOULD A2LA be held accountable?
I feel that the accrediting bodies give a false sense of security to the novice looking for calibration services.
Let me know what you think :roll:
You are correct.
what should be encouraged of the customer is for them to come to the lab and do an audit themselves so they can see the quality of work.
Step beyond A2LA and go with NVLAP, that is head and shoulders above.
I'ld like to also call your attention to some of the lesser known Accrediting Bodies such as ILAA (www.ilaaonline.org). I would like to quote from their own web site:
What does ILAA charge for its services?
The ILAA Basic Package will include a quality manual, one applicable uncertainty calculation, and a certificate of compliance at a cost of $2,750.00 USD.
Other services are available at an additional cost. See our services page for descriptions and costs.
What services comes with the ILAA Basic Package?
After completing our Basic Package online process, ILAA will deliver to your company:
A custom Quality Manual that mirrors and complies with the ISO 17025 Standard.
One uncertainty calculation specific to your business model.
The Accreditation Certificate.
A Scope of Accreditation.
Will you need to come on-site?
Except in rare cases, our innovative on-line application process should eliminate on-site auditing. ILAA strives to be as unobtrusive as possible to your daily operations.
Wow....sounds painless to me!
Kinda gives you a nice warm fuzzy feeling too doesn't it....
You have gotta be kidding.......................who does this Joe's calibration haircare and tire center??????????
I believe it is a company named E...o.
Am I right?
After being on both sides of the accredidation coin both as an auditor and as the auditee here are some additional thoughts. Getting certified means you meet the GUIDLINES as stated in the the standards whether its 17025/NAVLAP/RAB etc. Nowhere in these standards does it tell you how to meet these standards, its a set of guidlines, and its done for a reason. Sometimes what works for one lab/business doesnt work for another. It gives you guidlines to follow and its up to the Lab/Business to figure out how to do it. Ive done audits on labs in which the procedures to calibrate equipment may not have been what I would have done but they documented exactly how they were calibrating equipment so they are not in violation of the spec. They are writing what they are doing in a procedure and following it. The key is that the customer needs to research and really have some sort of clue in what they are looking at. I can usually tell if a lab is doing what I think is right by looking at the data that is provided on the certs. If I have further questions I will call and ask to see a copy of the calibration procedure. To rate labs wouldnt be fair and would probably drive costs to a point in which companies wouldnt use the service. Do I really need a top notch lab with a 2ppm uncertainty to calibrate my DMM with a 1V uncertainty and charge me $500? Can I have a so-so lab with a 50mV uncertainty calibrate my gage for only $35. The so-so lab may not be the best but it accomplishes what I need. If people were so concerned about the "Best" calibration everyone woud send there gages to NIST for cal.
Just some thoughts
I think calibration is still some-what of a wild-wild west industry. . . no real rules. Accreditation does little to say that your calibration methods are actually valid, it only says you calibrated the way you said you would. I've seen flowmeters calibrated using a bucket and a stop watch as standards!
We're a little better in the AF with tech orders. . . but really, no one validates those other than the one guy responsible for maintaining the technical content. Qualified or not, a little oversight might be in order.
There is no federally mandated certification of calibration labs. . . it's all voluntary. Sort of funny in an industry that by definition is all about standardization and traceability, there really isn't any.
You should really refer to it as ISO17025 certified. A2LA is just one ISO17025 certifiying organization.
I would disagree with the notion that calibration is a wild west industry. I think the field of calibration has moved forward by leaps and bounds over the last 10 to 15 years. We have moved forward from determining whether we have a correct TAR to a TUR. We are even further advancing our field by looking at things like guardbanding and performing MSA/Gage R&R Studies on our own equipment. This is a far cry from the good old days of is the TI good or bad. Many people get caught up in the "How we are checking things" versus what we are checking. Calibration by definition is the verification of an unknown to a known. As long as I have a known I can determine an unknown with some sort of uncertainty involved. Like the example used in a previous post, I may have a bucket of water with a known amount, a stopwatch with some accuracy, including the operators thumb, and some sort of calculated uncertainty for the measurement of flow using the above equipment mentioned. The method, by definition of the standard is valid. It isnt a pretty setup but it can be a valid method. The ISO/IEC 17025 goes into great depths into explaining that validation of any procedure is required. 5. 4. 5. 1 of ISO/IEC17025 states "Validation is the confirmation by examination and the provision of objective evidence that the particular requirements for a specific intended use are fullfilled". If you can show "objective evidence" by an Uncertainty Budget, Gage R&R or an MSA I have met the requirements.
I look forward to more replies to this,
ck454ss
I think you helped prove my point. . . that this set-up can be considered valid. Mathematically and theoretically, yes; but there are so many inconsistencies and variables in that method that I can't even begin.
My real point though is that, unless you are talking to a metrologist and someone in the calibration industry. . . calibration doesn't necessarily mean a lot. Customers know they need to have stuff calibrated, but don't really know what that's all about. Many companies use the term to describe self-checks or ops check. . . we even get new items into the AF industry that are claimed to be "self-calibrating" and a suprising number of SPOs and IMs buy off on that. Anyone can open a "calibration lab" in their garage and most customers assume one calibration is as good as any.
Even in a gray industry like colleges and degrees. . . there are certain widely recognized accrediting bodies, without those your degree is basically worthless.
We don't really have that, yes some are recognized better than others, but usually only to those who really know the business. We have no real govt. regulation of the industry or widely accepted commercial accreditation. When the average person applies for a school you know you want one that is regionally accreditted. . . when they ask for calibration, all they really seek is a sticker that says calibrated. Maybe we are getting better. . . but for now I still think we are in the wild west with a lot of snake oil available on the market.
I agree that there is "snake oil" in the market, but that is not the majority. A2LA or NAVLAP audits are much tougher that an AFMETCAL audit and labs that have these accreditations are usually up to par with their cals.. I do agree with you about the problem with accreditation not being standardized. It is moving in that direction and it will be standardized in our life time. A lot of the problem is the ignorance of the customer too. They look at the accreditation and assume that they can provide a accurate cal though the labs uncertianties are though the roof. There are problems that need to be addressed , but I just don't see it all that doom and gloom. Just my thoughts! :-)
Quote from: flew-da-coup link=topic=634. msg9639#msg9639 date=1177622733
A2LA or NAVLAP audits are much tougher that an AFMETCAL audit and labs that have these accreditations are usually up to par with their cals.
Coop I have to call BS on that one, in an AFMETCAL audit anyone can be audited and the individuals performing the audits have years of experience on the area in which they conduct the audit. This is not always the case with A2LA, in most labs the technicians are hand picked to perform the testing and in most cases the same techs are used time after time and the auditors say nothing of this practice. What this does is provide a lab with 2-3 strong technicians the ability to pass the audit and become accredited. In reality much of the work will be done by the remaining tech's that do not have the experience or knowledge to properly perform the work they are assigned. As our industry continues to go toward onsite calibration the most experienced technicians will be onsite the majority of time to ensure the onsite dollars dont go away. So answer this one for me should the accredidation be linked some how to the technician performing the actual assesments and not the lab? In the previous scenerio remove those 2-3 strong techs and you now have a lab with no experience or knowledge in areas in which they are accredited does this help the customer? This would also give the power back to the tech's.
ISO 17025 is being used as a filter by the majority of companies out there. Is the customer better served by an A2LA accredited lab that performs limited calibrations on all there gear or one that is not accredited and can perform the full calibration.
Coop lets be honest, YOU know that there are labs that have all the paper work in order that just apply stickers. Lets us not forget about undue pressure, how about sending a tech onsite and giving him 2 days to complete a job that you know takes a week or giving a tech an hour to cal an 8566B. Ignorance of the customer is no excuse we have the obligation to ensure that an uninformed customer is educated to understand the basic concepts of what we do and the time required to do it right.
Coop I want to say this is not an attack on you as I have read many of your post and agree fully with the majority of them and it is obvious to me that you are a very qualified tech, this just happens to be a topic that really gets me going. For the most part we are all trained by the military and would have never allowed this type of action to take place in our lab, now we just look the other way.
Gentlemen welcome to the wild west. :-o
I have been through AFMETCAL and a NAVLAP and I find the AFMET a breeze for a tech. Follow the KPro and your good. No real knowledge needed other than being able to read. It's pretty clear to me. However, I see some of your point.
I concur with Newbie. Stating that an ISO 17025 accreditation audit is in any way comparable to the AFMETCAL MLC audit is simply BS. The depth at which MLC looks is far deeper than an A2LA audit. An ISO 17025 audit is briefly looking at your PRIMARY "i.e. cherry picked" technician in that area being evaluated on the scope. While quality system document review may be fairly extensive on an A2LA audit, and there is a focus on ensuring uncertainty budgets are accurate, there is little stress placed on the primary technician to perform they way they have to perform for MLC. This is a virtual apples and oranges comparison though...an ISO audit is conducted by an organization that you are paying to come and evaluate you. It is in their best interest for their future income that you pass the eval so you can continue to give them money for follow up audits to maintain accreditation. MLC comes in to evaluate the entire lab to determine if they are competent enough to continue calibrating for the Air Force. Every technician is subject to audit, and they all better have their sh#t up tight. You have problem during an A2LA audit...fix the paperwork and the problems go away. You have problems during an MLC audit, and you are in a world of sh#t. Passing an ISO 17025 accreditation audit mearly means your quality system documentation is good, your uncertainty budgets accurately reflect your capability, and your lead primary technician briefly and satisfactorily performed a task for the auditor. I've been through NUPIC, ISO 9000, ISO 17025, and AFMETCAL MLC audits...nothing compares to the stress of an MLC audit.
I have seen AFMETCAL audits setup to be easy. Like staging all the bunnies 4 weeks before the audit so that the techs only have to have an over the shoulder with cherry equipment. I have never seen an A2LA audit cherry picked. The auditor looks at the tech training record and then asks him to cal something that he is marked off for caling. I dunno, If AFMETCAL was so "Great" they would have not instructed techs to clean the high pressure piston from a Ruska 2465 with acetone and then state " this should give you 7-8 good readings before piston needs cleaning again". If AFMETCAL would just look at the manufactures note that you need to clean the piston with the specified soap then you would get 200-300 good readings before cleaning the piston again. After seeing that I never thought much of AFMETCAL. How could someone miss something so simple. If they missed that for many years ( before I said something) how can you say their audit is better? These incorrect instructions were even posted on their website until I was moved to the K6 area from K4 and pointed it out. Yes they do dig a little deeper, but alot of the stuff they check is BS and you guys know it. Just my 2 cents. Never mind me anyway, I am not feeling well and have a big chip on my shoulder, so I may change my mind when I am well. Hahaha.... Anyway I am going back to bed.
First. . . just because one person from AFMETCAL said something one time. . . means about as much as me telling you someone from your company said something one time and therefore I no longer trust your company. AFMETCAL is an organizaton made up of people. . . and believe or not there as many disagreements among the team as there are in any other lab. Yes, they do try to come to agreement on some issues, but when it comes down to it, the individual auditor makes the call based on his/her own experience, which may very well be less than the person they are evaluating.
Same goes for any assessors under any system though. I think it's just silly to discount the whole process and organization cuz they made a bad call once. . . I bet I could find someone in your lab to make a bad call and discount your lab's credibility just as easily.
I don't know the last time you've been through AFMETCAL. . . but they now target critical lab processes and not just whatever is left on the shelf. . . they will call in items if need be. The process is very different now. They still do a fair share of bunnies to evaluate 3-levels and what-not.
They do check a lot of BS. . . but overall I think you have to be a fairly good lab to pass. The object isn't really to fail labs. . . it really should be to assess minimum capability. . . like a restaurant getting their health license. My two cents. . . I'm not necessarily a big fan of AFMETCAL either. . . but I don't think A2LA or 17025 are the end-all to beat all either.
Mike, by no means do I believe that ISO 17025 ( A2LA ) is where it needs to be, but I believe that they are going in the right direction. Your point is taken in your last post , but my point is that AFMETCAL are not Gods and your post stated the same. :-)
Absolutely. . . AFMETCAL auditors are no more qualified than the average SNCO in the field, which is fairly well qualified. . . but certainly not gods!!
Cheers!
Not to mention that I think a 3rd party audit of AF TOs and uncertaintly budgets is sorely needed!
Agreed. :-)
Ahhh.................group hug
Okay, you are out of the Navy now. NO more guys hugging on each other. :-D
KUMBAYA!!!!!!
OH GOD!!! Please do not let an AB add ratings.They can't figure most of this stuff out now!!