People are looking for trusted guidance around medical cannabis and there are lots of other websites producing great news and other fun content around this amazing plant… but nobody was really taking you by the hand and walking you step-by-step through how YOU can learn in this ever-expanding and always changing industry.
Today we’re going to give you a “look under the hood” to see how we publish some of the most advanced medical education training available in the cannabis industry.
Yes, you could go to the conferences, but you only get bits and pieces of useful information or you can hire consultants and lawyers, but that was amazingly expensive. When we release our RIGOROUSLY develop online medical cannabis education training, it incredible to see 1,000’s of healthcare practitioners interest in the latest research conducted from around the world.
But what was even more exciting was all the rave reviews our Subject Matter Experts (SME’s) got about how their medical education has helped people to learn more about the medicinal use of cannabis, grow their credibility through knowledge, and develop an edge in this super-competitive field – all without ever leaving the comforts of their home or professional medical workplace!
We got story after story about how medical practitioners started new careers, built new businesses, made new investments, and got their chance to participate in this blossoming medicinal cannabis industry. We knew we were onto something BIG, so we outlined our framework to ensure our SME’s were unbiased and completed a systematic cannabis literature review when publishing training.
To do this we worked alongside the world’s top SME’s to develop online training for medical cannabis, cannabis patient care, dispensary education, and much, much more. Now today, we’re thrilled that 1,000’s of medical practitioners all over the world are utilizing these best-in-class online training programs to help them succeed within every aspect of the cannabis sector you’ll find today.
What started as a dream has become a reality, and our greatest desire is that what we’ve built will help you too. The cannabis industry is filled with “self-proclaimed experts” that spread misinformation based on false beliefs or faulty science… we believe in a “higher standard” (pun intended).
That’s why we have our team of SME’s for strictly vetting medical cannabis research studies and our Instructional Design (ID) team has developed a comprehensive industry training publishing process.
Our Instructional Design Process
To create our curriculum, we clearly define the needs, roles, and skills required in the cannabis industry by utilizing our vast network of Subject Matter Experts (SME’s), industry leaders, and top-researchers who work with our Instructional Design (ID) team to publish their knowledge:
- We source, vet, and secure the top SME’s to create the objectives and content outlines.
- Our experienced SME’s and ID team to work closely with each other to produce cannabis industry-leading medical education, multimedia training, and learning activities.
- For our curriculum, we utilize the Analysis, Design, Development Implementation Evaluation (or ADDIE) model to shape and construct the content according to best practices for learning and retention. This is an ID framework used to develop training and the name is an acronym for the five (5) phases it defines for building education and performance support.
- We create various learning modalities to support learners, including expert-led video lectures, researched and reviewed articles, interactive learning activities, and helpful aids.
- The training is delivered using a state-of-the-art Learning Management System (or LMS) that makes the learning process fun, enjoyable, and successful. We ensure learner success by providing consistent support, encouragement, and resources to track learner completion and retention levels, to ensure they are absorbing the training material being presented.
- We review the curriculum and content annually to ensure relevancy to ensure each training is constantly improved and refined based on the latest medicinal research and studies.
Monitoring Cannabis Use Health Concerns
Our Instructional Design (ID) team and Subject Matter Experts (SME’s) used a PRISMA framework to ensure an unbiased and complete systematic literature review.1 The SME’s and ID team followed these general steps when reviewing each cannabis study in preparation for training we publish:
- Conduct a broad search of current peer-reviewed publications (ex. Medline) quarterly. Relevant articles cited in reviews or other primary studies are also included.
- Review relevant full-text articles identified in the search of medical cannabis research.
- Rate Findings: Each finding in the articles is rated as a high-, medium-, or low-quality finding based on the strengths and limitations of the methods. Evaluation of the strengths and limitations was based on criteria in the “GRADE system“, a well-accepted method for evaluating the quality of scientific evidence found from cannabis research and studies.
- Group Related Findings: Each finding is categorized based on population, exposure, and outcome (health effect), to answer specific questions about the medicinal use of cannabis.
- Weigh Evidence: Draft evidence-based training that summarizes the quantity and quality of evidence answering a specific question about the medicinal use of cannabis.
- Translate Evidence: Draft cannabis industry training that translates the evidence into language the public can easily understand the cannabis research study and findings.
- Synthesize Evidence: Draft training recommendations (ex. for education or curriculum) based on important information that is identified through the review process of studies.
- Identify Research Gaps: Draft training to articulate the research gaps identified when reviewing the cannabis scientific literature and studies research found during the process.
- Present Subject Matter Experts: Findings, scientific evidence, cannabis industry training and recommendations, and research gaps are formally presented to the SME’s for review.
- Public Comment: During the open SME meetings, interested stakeholders and our ID team are invited to provide comments relevant to the cannabis topics presented.
- Reach Consensus: SME’s come to a consensus on findings, evidence statements, cannabis training content and recommendations, and identify industry research knowledge gaps.
- Adopt Summary Statements: Subject Matter Experts votes to officially accept findings, evidence statements, cannabis training and recommendations, and research gaps.
Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)
Searching Cannabis Scientific Literature
Literature review methods were approved by the Subject Matter Experts (SME’s). Medline was the priority research database used to obtain articles for the review, though the Embase biomedical database and gray literature were secondarily reviewed when references in included articles were not included in the initial Medline search.
Articles cited in research or other primary studies also were included. Studies of medical cannabis use in humans were the PRIMARY focus and a review of animal studies was reserved for topics with limited human research.
In general, highly specialized research, such as brain imaging studies not directly associated with measurable clinical outcomes, was not evaluated in-depth unless an appropriately experienced SME’s were available. Research databases other than Medline were searched primarily when time allowed, though little additional data were found via these additional searches for cannabis research.
All identified peer-reviewed literature on a given topic was reviewed, regardless of the positive or negative findings or quality of the methods. For Medline searches, the appropriate Medical Subject Heading (MeSH) terms were chosen for each topic and used for the search. To find newer articles relevant to the topic (those without MeSH yet applied), a list of specific terms was established.
For example, the general search string used for cannabis scientific research and studies was:
- Cannabis Smoking
- Cannabis Abuse
- Hash Oil
Findings were rated as high, medium, or low quality based on the strengths and limitations of the methods. Evaluation of the strengths and limitations was based on criteria in the “GRADE approach to evaluating the quality of evidence.”2
The GRADE system is a well-established method for systematic literature review used by the Cochrane Collaboration, British Medical Journal, American College of Physicians, the World Health Organization, and others.2
The official definition is: “We are very confident that the true effect lies close to that of the estimate of the effect outlined in the study.” High-quality findings originate from well-designed and well-controlled scientific research and studies with few limitations about the medicinal use of cannabis.
In the context of observational epidemiology studies, which was the MOST common study type in this systematic review, high quality does not necessarily imply causation. High quality implies that an observed association persists between exposure and effect in an appropriately sized study population after adjusting for appropriate confounders that are from cannabis research studies.
The official definition is: “We are moderately confident in the effect estimate outlined in the study. The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different.” Moderate quality findings originate from studies that may be well designed but have limitations that affect the interpretation of medical research and scientific study results.
In the context of observational epidemiology studies, moderate quality IMPLIES the finding of an observed association with an interpretation that may be limited by a small study population or insufficient adjustment for important confounders that are from cannabis research studies.
The official definition is: “Our confidence in the effect estimate outlined in the study is limited. The true effect may be substantially different from the estimate of the effect.” Low-quality findings originate from studies with significant methodological limitations that affect the result interpretation.
In the context of observational epidemiology studies, low quality implies the finding of an observed association with an interpretation that is significantly restricted by study limitations. When CRITICALLY reviewing the literature, all findings were initially considered medium quality and subsequently adjusted up/down in quality based on the methodology’s strengths and limitations.
Quality ratings were applied to individual outcomes; therefore, it was possible for a single study to have multiple findings of differing quality. Criteria for evaluating strengths and limitations for this scientific medical cannabis literature review included all of the following steps by our experts:
- Methods of Selecting Exposed and Comparison Groups
- The Relevance of the Study Population to the Population of Interest
- Method for Measuring Exposure (self-report or other methods)
- Method for Describing the Extent of Exposure or Cannabis Use (ex. ever vs. never, frequency measured by days used, measured by a number of times used, etc.)
- Adequacy of Exposure and Outcome Group Sizes
- Methods for Measurement of the Outcome(s) (ex. validated tools, blinded if subjective, etc.)
- Adequacy of Adjustment for Confounders (ex. tobacco smoking, other drug use, education level, etc.) for Both Positive Effects and Lack of Positive Effect
- Full Reporting vs. Selective Outcome Reporting
- Effect Size and Width of Confidence Intervals
- Temporal Relationship Between Exposure and Effect
- Completeness of Follow-Up
- Adequacy of Sample Size for Assessing a Lack of Positive Effect
Grouping Findings & Weighing Evidence
Findings from individual studies were grouped together to facilitate weighing the overall scientific evidence for specific topics. Findings were usually grouped based on outcome (ex. health effect). However, in some situations, findings could be further subdivided based on factors such as:
- The age range of the exposed population
- Special subject circumstances such as pregnancy or breastfeeding
- Level or method of cannabis use
- Time period since the last use of cannabis
Standardized definitions of the level of use and age groups were established to help facilitate the grouping of findings from the cannabis scientific research and medical studies that were reviewed:
Cannabis Use Levels
- Daily or Near-Daily Use: 5 – 7 Days / Week
- Weekly Use: 1 – 4 Days / Week
- Less-than-Weekly Use: Less Than 1 Day / Week
- Acute Use: Used Last Few Hours (ex. short-term effects or symptoms being experienced)
- Child (Less 9 Years Old)
- Adolescent (9 – 17 Years Old)
- Young Adult (18 – 24 Years Old)
- Adult (25 – 64 Years Old)
- Older Adult (65+ Years Old)
Once findings were appropriately grouped, evidence statements (ex. “We found substantial evidence that adolescents who use cannabis weekly or more frequently are less likely than non-users to graduate from high school.”) were drafted based on the following criteria, which were approved:
1. Robust scientific findings that support the outcome with no credible opposing scientific evidence from other cannabis industry research studies, this was defined as any of the following:
- At least one (1) high-quality positive finding, plus supporting findings at least one (1) of which is medium quality, with no opposing findings – include studies of at least two (2) cohorts.
- At least three (3) medium-quality positive findings from studies of at least two (2) cohorts, with no opposing findings.
- Many high- and medium- quality positive findings from studies of at least two (2) cohorts that heavily outweigh opposing findings.
- At least two (2) high-quality positive findings from systematic reviews or meta-analyses published within the past ten (10) years.
2. A robust body of scientific literature that has examined the outcome and failed to demonstrate a positive finding from other cannabis industry research studies, this was defined as any of the following:
- At least one (1) high-quality study lacking a positive finding, plus at least one (1) medium quality supporting study, and no opposing findings – include at least two (2) cohorts studies.
- At least three (3) medium-quality studies lacking a positive finding from studies of at least two (2) cohorts, and no opposing findings.
- Many high- and medium-quality studies lacking a positive finding that heavily outweigh opposing findings.
- At least two (2) high-quality systematic reviews or meta-analyses published within the past ten (10) years lacking positive findings.
1. Strong scientific findings that support the outcome, but these findings have some limitations from other cannabis industry research studies, this was defined as any of the following:
- A single (1) high-quality positive finding, with no opposing findings.
- At least one (1) medium-quality positive finding, plus supporting findings with no opposing findings; supporting findings can include animal studies.
- Many medium- and low-quality positive findings from studies of at least two (2) cohorts that heavily outweigh opposing findings.
- A single (1) high-quality positive finding from a systematic review or meta-analysis published within the past ten (10) years.
2. A strong body of scientific literature that has examined the outcome and failed to demonstrate a positive finding from other cannabis industry research studies, defined as any of the following:
- A single (1) high-quality study lacking a positive finding, and no opposing findings.
- At least one (1) medium-quality study lacking a positive finding, plus supporting findings, and no opposing findings.
- Many medium- and low-quality studies lacking positive findings from studies of at least two (2) cohorts that heavily outweigh opposing findings.
- A single (1) high-quality systematic review or meta-analysis published within the past ten (10) years lacking positive findings.
1. Modest scientific findings that support the outcome, but these findings have significant limitations, from other cannabis industry research studies, this was defined as any of the following:
- A single (1) medium-quality positive finding.
- Two (2) or more low-quality positive findings from studies of at least two (2) cohorts.
- Many low-quality positive findings from studies of at least two (2) cohorts that outweigh opposing findings.
2. Modest scientific findings that have examined the outcome and failed to demonstrate a positive finding, this was defined as any of the following:
- A single (1) medium-quality study lacking a positive finding.
- Two (2) or more low-quality studies lacking positive findings from studies of at least two (2) cohorts.
- Many low-quality studies lacking positive findings from studies of at least two (2) cohorts that outweigh opposing findings.
Mixed evidence indicates both supporting and opposing scientific findings for the outcome with neither direction dominating, it was defined as “mixed findings, with neither direction dominating.”
Insufficient evidence indicates that the outcome has not been sufficiently studied, it was defined as:
- A single (1) low-quality positive finding with no supporting findings.
- There are no studies examining the outcome or relevant parameters.
The above criteria from other cannabis industry research studies were translated into evidence statements using the following guidelines:
- Substantial positive evidence becomes: “We found substantial evidence…”
- Substantial lack of positive evidence becomes: “We found a substantial body of research that failed to show an association…”
- Moderate positive evidence becomes: “We found moderate evidence…”
- Moderate lack of positive evidence becomes: “We found a moderate body of research that failed to show an association…”
- Limited evidence becomes: “We found limited evidence…”
- Limited lack of positive evidence becomes: “We found a limited body of research that failed to show an association…”
- Mixed evidence becomes: “We found mixed evidence for whether or not…”
- Insufficient evidence becomes: “There is insufficient evidence to determine…”
Evidence statements are drafted by our Instructional Design (ID) team, revised based on Subject Matter Experts (SME’s) review and feedback from trust sources of information and our stakeholders.
Cannabis Health Statements
Evidence statements were translated into cannabis health statements using a standardized convention to ensure traceability back to the scientific literature. Cannabis health statements were designed to ACCURATE reflect the evidence using language the public can understand.
Our Subject Matter Experts (SME’s) sought to ensure the following cannabis industry health statements:
- Conveyed the volume and quality of research related to the outcome.
- Provided a generalized framework to allow consistent language for all findings regardless of the training topic.
- Allowed the statement to stand on its own without context. These statements are drafted by our team, revised based on comments from SME’s, technical advisors, and stakeholders.
The standardized convention used for the translation is shown below:
- Standardized Convention: <level of> cannabis use <by specific group> <strength of relationship> associated with <outcome>, <specific circumstances>.
- Specific Example: “Weekly or more frequent cannabis use by adolescents and young adults is associated with impaired learning, memory, math, and reading achievement, even 28 days after last use.”
The standard language was chosen for the “strength of the relationship,” corresponding to the level of evidence from the evidence statements:
- Substantial positive evidence becomes “is strongly associated.”
- Substantial research lacking positive evidence becomes “an association is unlikely.”
- Moderate positive evidence becomes “is associated.”
- Moderate research lacking positive evidence becomes “an association appears unlikely.”
- Limited evidence becomes “may be associated.”
- Limited research lacking positive evidence becomes “might not be associated.”
- Mixed evidence becomes “There is conflicting evidence for whether or not __________ is associated.”
The wording “associated with” was specifically chosen to represent epidemiologically (ex. statistical or quantitative) associations to the study’s findings, and NOT to imply any causality whatsoever.
Cannabis Study Recommendations
Based on the literature review, cannabis study recommendations are drafted, and our Subject Matter Experts (SME’s) recommendations were separated into data quality issues and education recommendations.
Data quality issues were defined as recommendations to improve current data collection deficiencies at the clinical or governmental level that prevent full analysis of cannabis studies’ outcomes related to medicinal cannabis use.
Cannabis study recommendations were based on improving capacity to detect an acute health danger (ex. real-time emergency department surveillance for detection of poisonings from contaminated products), the ability to characterize chronic personal health dangers to support policy and other intervention decisions (ex. surveillance of cannabis-related traffic fatalities or skiing injuries), or the ability to generate epidemiologic data (ex. BRFSS survey questions) to contribute to planning.
Education recommendations were included to ENSURE evidence-based information on potential immediate and long-term health effects of cannabis use is provided to the appropriate audiences.
Cannabis Research Gaps
In addition to cannabis study recommendations, important research GAPS related to the population-based health effects of cannabis use were identified during the literature review process of scientific studies and medical research.
These scientific studies and medical research gaps were based on common limitations of existing research (ex. not enough focus on less-than-weekly cannabis use, distinct from weekly or daily use), exposures not sufficiently studied (ex. dabbing or edibles), outcomes not sufficiently studied; or issues important to public education or policymaking (ex. driving impairment infrequent users).
These research gaps provide an important framework for prioritizing research related to cannabis use and personal health. Statements articulating the cannabis study recommendations and research gaps were initially drafted by our team, revised based on comments from the Subject Matter Experts (SME’s), technical advisors, and public stakeholders, and finally approved by a VOTE of the SME team.
Subject Matter Experts Consensus & Approval
Our Instructional Design team and Subject Matter Experts (SME’s) formally present findings, evidence statements, cannabis study statements and recommendations, and research gaps to everyone for review and revision. During our review, interested stakeholders and our team members, are invited to provide comments relevant to the topics presented during our quarterly review.
The SME’s facilitate a consensus process to ensure everyone could agree on the scientific evaluation and wording. Once consensus was achieved, then our team of SME’s voted to officially accept these statements and recommendations into our cannabis industry medical curriculum that we offer.
Reviewing & Updating Training Content
Our Instructional Design team and Subject Matter Experts (SME’s) will continue to meet quarterly. All approved evidence statements, cannabis study recommendations, and research gaps will be reviewed on a quarterly cycle and training updated as needed based on new evidence.
The SME’s will also expand the reviewed literature to include new training topics as new research becomes available or NEW cannabis health concerns arise.
- David Moher, Alessandro Liberati, Jennifer Tetzlaff, Douglas G. Altman, “Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement,” BMJ Publishing Group Ltd, 2009;339:b2535, published July 21, 2009, retrieved from: https://www.bmj.com/content/339/bmj.b2535.
- Gordon H Guyatt, Andrew D Oxman, Gunn E Vist, Regina Kunz, Yngve Falck-Ytter, Pablo Alonso-Coello, Holger J Schünemann, “GRADE: An Emerging Consensus on Rating Quality of Evidence and Strength of Recommendations,” GRADE Working Group, published April 28, 2008, retrieved from: https://www.bmj.com/content/336/7650/924.
Let us know what you think.