Part 2 of Cannabis Testing Lab Results Summary Series
You may be asking yourself why I might postulate that bacteria are ingesting solvents out there in the hard reality-based objective world of Washington’s State-legal Cannabis Lab-Testing Industry? Aside from the possibility that the little things probably also enjoy alcohol (at low levels), it is an impression that arose while looking at the lab testing result data on both Residual Solvents and Microbial Screening.
My previous post on Moisture testing and it’s surprisingly low rate of failing cured bud (see HI-Blog) detailed what I’m doing, but I’d like to repeat here that my series evaluating lab testing results corresponds to data on testing for the period spanning June, July, and August of this year. This period was chosen, as I had shared some of my early work with people in the Cannabis Testing Lab Industry to increase their awareness that external eyes were watching. These most recent data also cover a period during which the labs would have been expected to have “shaken the bugs” out of their internal protocols and procedures.
As such, the data chosen afford an opportunity to see the “best” that the labs have to offer using a reasonably large sample of test results.
OK … back to drunken bugs. My coverage of these two testing panels will be limited, as the truly interesting things here are things I’m seeing at the level of the individual labs. I’ll share the detail of that in the final posting of this series, but will share some blinded lab-level highlights below. I’m not going to include charts in this article (sorry to my picture-hungry readers … .I’ll make up for it in next week’s posting on Foreign Matter and Cannabinoid Potency Testing).
Residual Solvents (bug-food)
At the lab level, there are some disturbing patterns in these data.
Overall, of the 14 licensed Cannabis testing Labs in the State, only 8 conducted more than a trivial number of tests for residual solvents in extracts during the 3 months ending in August.
For the 900 tests conducted, 37 were recorded as having failed the solvent test (with values of residual solvents in excess of 500 parts per million). This represents an overall failure rate of about 4.1%.
Four of the labs testing for solvents did not fail a single sample. Of the four that failed samples, one failed less than ½ of a percent of it’s tested samples, two failed between 3-4%,and one failed almost 1 in 7 tested samples (14.2%). The failure rate across time is fairly constant (statewide), and the sample size is relatively small. However, having a single lab (with a large proportion of the reported tests) showing a failure rate almost 4 times larger than anyone else is a bit surprising.
It is of interest, and of some concern, that another 4 tests had levels in excess of the allowed threshold, but were recorded as having “passed” the residual solvent test. One of these samples was recorded as having a residual solvent level of over 78,000 (which suggests that the sample in question contained over 7.8% residual solvent, by weight). It is surprising that this product appears to have been passed, as it’s solvent levels are over 150 TIMES the allowable level. One wonders if dabbing this product it might not result in a little back-flash? Regardless, inhaling it (or otherwise ingesting it) is likely not going to be good for whoever ultimately bought it (I may trace this specific lot through the system to see where it was sold …. Yet another question to put on the to-do list). At best, this is sloppy work. At worst, it is feeding into a theme that is showing up as I continue to compile my lab-level testing report card — namely, that some labs seem to be failing less product than seems to be expected. What is particularly interesting is that the four “failures to report failures” are not randomly distributed across the labs. Stay tuned for details later this month – but there increasingly appears to be an issue at the level of the individual labs that the State has granted the privilege of testing Washington’s regulated Cannabis. The labs that are keeping recreational cannabis users safe. The labs that will soon be keeping medical cannabis patients in Washington safe.
Microbial Screening (bugs, germs & poop)
Statewide, during the 3-month period in question, over 10,000 screening tests were conducted for the following microbial thingies: (with failure rates in parentheses). About 80% of these tests were on Flower lots.
- aerobic bacteria (0.55% of tested samples failed)
- bile-tolerant thingies (2.1% of samples failed)
- coliforms (0.67% of samples contained bad shit)
- e-coli and salmonella (0.05% of samples were very bad shit)
- yeast & mold (2.6% of samples failed)
Yielding an overall failure rate of 4.1%.
So, about 1 out of every 25 microbial screening tests conducted in the last three months failed at least one microbial test and, hence, is considered unfit for consumption as is. That is big money on the line for our producers and affiliated processors (well into the millions of dollars already).
I don’t know what a “good” or “expected” failure rate should be, but the beauty of large datasets containing the results of purportedly PRAGUE tests is that one might reasonably expect the distribution of failures to follow some sort of rational form. If, for example, a lab is doing 5 times the test volume of another lab, one might expect to see more failure in the high-volume lab than in the low-volume lab. As we’ll see when I post my lab-level report card, this is not always the case. Suffice to say that two of the top 3 volume testing labs in the state did not fail a single sample for bugs, germs and/or poop during this period (and they represent almost 40% of the samples tested in the the state). Maybe they only deal with clean growers, and not the dirty little unhygienic ones being seen by the other labs.
Cynical Jim (and I have not completed my assessment on this, so it’s just an impression at this point) seems to see a pattern emerging wherein labs that tend not to fail ANYTHING and/or tend to fail things AT VERY LOW LEVELS have a tendency to have a higher share of the lab-testing market. Just a correlation at this point. No conspiracy or failing of human nature is being implied. I’m way too sunshiny a type of person for that.
However, the experimental psychologist inside of me is just salivating at the possibility of tracing some of the producer and processor relationships with the labs and their evolution across time (as a function of pass/fail experience). I’m guessing I could do another degree with this stuff (would that make me a “Master Doctor”?). Probably won’t make many friends. Might help to make Washington’s State-legal Cannabis a bit safer, though.
Anyway, in large part from my study of the “American Herbal Pharmacopoeia Cannabis Inflorescence Standards Manual (rev. 2014)”, I’ve come to the conclusion that testing for bugs & germs & poop (and heavy metals and pesticides and other things) is a complex process. It is a process that is, fundamentally, dependent upon representative sampling and objective, unbiased testing and reporting. It is dependent upon machine calibration. It is dependent upon sample preparation (or, as I like to envision … why is this sample 100% ash …. Oh!, perhaps because I just smoked it … I’d say it has 55% THC …. Great bud. Next Sample, please!). It is complex … and it should be PRAGUE.
Regardless, complexity should not be a shield behind which bad (or corrupt) business practices are allowed to hide. These are certified testing labs. They are supposed to do complex. They are supposed to be good at complex.
I’ll leave you today with another aspect of this testing that will enter into my lab-level report card – the size of samples required by the different labs.
With relatively standardized methodology, all of this anti-microbial screening takes a certain amount of product and a certain amount of time to do correctly (e.g., one has to “grow out cultures”). Taken to an extreme, if a lab allows a sample of one one-millionth-of-a-gram to incubate for 1 millisecond, no failure is likely to be seen. There is a REASON for a minimal (and, ideally, standardized) sample size for some of these tests. If samples are smaller, they will tend to fail less frequently (systematically). Similarly, there is a reason that some of these tests take days to do properly.
On that happy note, the average size of samples reported ON A PER-LAB BASIS (looking at Flower only) ranges from a low of 2 grams to a high of 5 grams. The overall average sample size of flower tested is just over 3 grams.
There is no reason that I can see for variability like this to be present. I’d love to hear differently from the experts in the industry …. Please use the comments capability on this blog to let me know how my thinking here can be improved.
I am sure it is not lost on the typical farmer that having to submit 3 grams less per sample to the labs decreases the effective cost of the test by about the number of dollars that the farmer could otherwise receive by selling that product. Those sawbucks add up over time. Might even be able to pay some taxes with them.
I appreciate everyone’s patience on my frequency of posting (I’ve been spending time working with a pro on an upcoming facelift to my professional website, and I wanted to limit the amount of content needing to be transferred over) … I’ll try to have my foreign matter and potency posting up this coming week … and the “Lab Report Card” post up before Christmas.
I’d love to hear what you think of this work (even without charts and pictures).
Jim
Jim-
Another great analysis. I would just add that your comments about certified labs are “supposed to be good at complex” is right on the money. Speaking of money, complex is also expensive. You can’t get complex for $60, especially if the laboratories are actually preforming the tests.
P.S. Does anyone know what “dry labbing” is?
Answer:
dry-labbing. Noun. The act of supplying fictional yet plausible results in lieu of performing an assigned experiment.
Thank-you, Jason.
Your “$60.00” comment is, I suspect, a bit absolute for it’s own good. It is possible that 2 different labs might have such different cost structures and “efficiencies of throughput” that one can, actually, do a competent PRAGUE set of tests for much less COST than another lab. I simply don’t know what a reasonable range for such costs might be.
However, I suspect that you know quite a bit more about lab testing (and associated costs) than I, so I’ll defer to your opinion on this one.
Jim
Hi Jim,
I think it would be interesting to compare the fees laboratories outside the industry would ask for similar services. I think that would provide a good indication of the costs associated with doing it right. The big guys (intertek, microbac ect) have such high throughput that it will be very hard to offer better pricing than them. I think you will be shocked at the bargain rates some of the laboratories are charging in our industry. $50 for a full QC release of cannabinoids, microbiological contamination and residual solvent is just not economically viable, but i have seen it offered.
Thank-you.
I agree with your suggestion. If anyone would like to pull together a reasonably representative sampling of lab costs both WITHIN THE 502 LABS and also within the more general “testing industry”, that would be wonderful. That information would be a wonderful complement to an article I plan to write for CASP summarizing the lab-level “report card” and some policy ramifications of that work. I am increasingly seeing patterns in the data that suggest a degree of correlation (not absolute —- just a degree) between the share of the testing market enjoyed by labs and how “favorable” to a wholesalers’ short-term needs the aggregate results they are producing now are (or were between June and August).
The 5-panel anti-microbial screen and the 3-measure cannabinoid profile are crucial (as is the residual solvent testing of some products). These tests likely cost a bit (both big up-front capital and “consumable” reagents and labor etc). There are also different ways to do some of these things that appear to be OK (even though some offer very different cannabinoid profile results – due to decarboxylation or some such thing). Lots of room for variability in costs …. but looking at the larger testing market would be a hugely useful referent/benchmark to have. When Health determines if/what/when/how often pesticide testing is needed, these types of tests are likely to be the best analogy for expected costs (although it is my understanding that some labs may have to purchase additional equipment in order to do pesticide testing.
(Question for the labs …. or people knowledgeable about this stuff …. Can most pesticides be detected using spikes on a “spectrum” test like the mass spec, or does it have to be “pesticide-specific-reagent” type testing?)
Moisture results are crucial for bud (and, likely, certain types of hash — and, no doubt other things depending on related health/safety risks) … I can’t see these as being particularly expensive (aside from labor and limited up-front capital).
Foreign matter (my early nominee for most useless test on the planet … based on the failure rates I will be sharing later this week) strikes me about being primarily about labor (and employee mental health). A good lighted scope up-front (or equivalent) would be the primary up-front cost for this (plus a good sound system, I’d hope).
Again … thanks for your suggestion. If anyone wants to take the time to compile this info, I’d be quite willing to consider sharing authorship credits for the CASP article (which I anticipate finishing later this month). I probably won’t share authorship credit with anyone from the labs at this point …. that would be too much of a conflict re: the perceived objectivity of my ultimate assessment. I do, however, very much appreciate your suggestion.
I am very interested in the data and would love to help out, but as the company i work for is operating in Wa i think it is better that i keep distance. Good luck with your research, it is great to see someone take a look at this with a critical mind.
Understood. I respect your position. I wish our State Auditor held the ethics you just displayed.
Thank-you once again for your suggestion.
Anyone interested in compiling non-cannabis testing costs info? Let me know. I have a friend that ran a hospital lab for many years, and will check with her …. but I’m guessing that the agriculture testing market would be the better analogy for what testing should/will/might cost as this industry matures and testing volumes become more predictable to those risking capital to enable the tests to be done.
I cannot help but think that “dry labbing” is derived from the expression “dry humping”, first popularized in the days when zippers were not commonly found on jeans.
MOAR GRAFS PLOX?!?!?
XMO – AGREE me What GRAFS more BE GOOD Thing.
PLOX and Pox. Tables and Scatters, dots and bars …. ALL GOOD
PIES NOT SO MUCH
But words are OK, too … I hope you did not mind my lack of graphs TOOOOO much.
Just look at reading my words as the price that you have to pay to see the pretty charts and pictures I occasionally post.
Jim
Is it not plausible that some of the labs have a vested interest in failing producers, instead of just passing thus accounting for anomalies? I’m not saying it’s happening but I can think of 3 good reasons why if could be. 1. You have a lab that sell ancillary services to producers that fail to help them pass in the future. 2. You have a lab that is teamed up with producers (though on this you do have to believe in a big conspiracy, lol), passing team members and failing competitors. 3. You have a lab charging to retest assuming the LCB will allow it. Again not accusing any lab, just a thought.
Mark
Anything is possible, Mark …. but I do not believe that any of the labs would be of the mindset that failing samples for the purposes of drumming up repeat business would be a viable long-term approach to their businesses. Indeed, it is my understanding that a couple of the labs have recently stopped servicing the testing needs of Washington’s State-legal Cannabis market. When I went back and checked the data for these labs, it was clear that one of them was failing samples at a rate that appeared higher than the “average” lab. The other recently-closed lab does not seem to have stood out in this way.
The “ancillary services” idea you raise is interesting. I can imagine a world where, “for an additional fee, our technicians can take your flower and apply our lab-magic drying routine to ensure that your sample (and future samples) will fall below the 15.000% moisture threshold”. Looking at the data, one might almost think we were living in that world today (or were from June through August of this year).
From other comments I’ve received, I am beginning to understand that the LCB DOES allow some re-testing.
However, the thing that I’m increasingly seeing signs of as I develop the Lab “Report Card” is that there are some labs out there that appear to be failing virtually NO (or VERY VERY few)) samples. They may just be testing the product of a subset of producers and processors that are “cleaner” than average in their processes. I don’t know … and I promise to try not to editorialize too much … I’ll just try to stick with the facts. The simple, dirty facts.
tremendous Jim!
Full panel pesticide testing for 200 analytes cost $1,000 at an ISO accredited lab. Utilizing GC-ECD, GC-NPD, LC/MS/MS, HPLC. The complexity of the pesticide list and their respective set quantitation levels can eat into how much $$ is needed on sophisticated instrumentation and sample preparation. WAY, WAY too many people believe in the magic of CSI: (insert city here) where samples are submitted, put into some machine and blam, results. Also, when this type of simplicity is applied in the lab (usually by means of specialized instrumentation or automation), the lab has to pay more for ease. That will trickle down to the consumer ultimately.
The labs, in this case, are willing to pay a PREMIUM in order to cut down on sample prep time or instrument run time. This concept of a PREMIUM is completely lost on these stoned out producers and processors. I want my results and I want them NOW, they scream and shout. But no one dares want to pay a dime extra to move their samples to the front of the line. That’s like going to a restaurant and being put on a waiting list and then slapping the host/hostess hand with air and then expecting to be seated right away.
James T: You have a point, so I’m letting your comment through, but the folks using Washington’s “Certified by RJ Lee” labs are likely better referred to as “our beloved customers”, rather than “these stoned out producers and processors”.
From what I’ve seen in the industry, the broad brush with which you paint the wholesalers of this industry is both incorrect and inappropriate.
Perhaps I am drawn to those demonstrating professionalism, courtesy and reason …. but that is what I’ve tended to see amongst those producing the products for this market.