Expanding Health Care Access Through Education: Dissemination and Implementation of the ECHO Model

ABSTRACT Project ECHO (Extension for Community Healthcare Outcomes) is an evidence-based model that provides high-quality medical education for common and complex diseases through telementoring and comanagement of patients with primary care clinicians. In a one to many knowledge network, the ECHO model helps to bridge the gap between primary care clinicians and specialists by enhancing the knowledge, skills, confidence, and practice of primary care clinicians in their local communities. As a result, patients in rural and urban underserved areas are able to receive best practice care without long waits or having to travel long distances. The ECHO model has been replicated in 43 university hubs in the United States and five other countries. A new replication tool was developed by the Project ECHO Pain team and U.S. Army Medical Command to ensure a high-fidelity replication of the model. The adoption of the tool led to successful replication of ECHO in the Army Pain initiative. This replication tool has the potential to improve the fidelity of ECHO replication efforts around the world.

Joanna G. Katzman, MD, MSPH*; COL Kevin Galloway, AN USA (Ret.)[dagger]; Cynthia Olivas, MSN, RN[double dagger]; Kimberly McCoy-Stafford, MSPH, MCHES§; Daniel Duhigg, DO, MBA?; George Comerci, MD¶; Summers Kalishman, PhD**; COL Chester C. Buckenmaier III, MC USA (Ret.)[dagger][dagger]; MAJ Laura McGhee, MS USA[double dagger][double dagger]; Kristin Joltes, PA-C, CCRP[dagger]; Andrea Bradford, MSc[double dagger]; Brian Shelley, MD§§; Jessica Hernandez, BS[double dagger]; Sanjeev Arora, MD??

*UNM Health Sciences Center Project ECHO, Department of Neurosurgery, University of New Mexico School of Medicine, 1650 University Boulevard, Albuquerque, NM 87102.

?Defense and Veterans Center for Integrative Pain Medicine (DVCIPM)/ HJF, 11300 Rockville Pike, Suite 709, Rockville, MD 20852.

?UNM Health Sciences Center Project ECHO, University of New Mexico School of Medicine, 1650 University Boulevard, Albuquerque, NM 87102.

?Army Project ECHO, Rehabilitation & Reintegration Division (R2D), Office of the Surgeon General, Defense Health Headquarters, 7700 Arlington Boulevard, Room 3SW132B, Fall Church, VA 22042.

?Department of Psychiatry, University of New Mexico School of Medicine, MSC 09 5030, 1 University of New Mexico, Albuquerque, NM 87131.

?Department of Internal Medicine, University of New Mexico School of Medicine, MSC10 5550, 1 University of New Mexico, Albuquerque, NM 87131.

**Office of Program Evaluation, Education and Research, University of New Mexico School of Medicine, MSC08 4550, 1 University of New Mexico, Albuquerque, NM 87131.

??Defense and Veterans Center for Integrative Pain Management, Uniformed Services University, 1004 York Lane, Annapolis, MD 21403.

??U.S. Army Research of Environmental Medicine, 15 Kansas Street, Building 42, Natick, MA 01760.

??First Choice Community Healthcare, South Valley Family Health Commons, 2001 North Centro Familiar Southwest, Albuquerque, NM 87105.

??UNM Health Sciences Center Project ECHO, Department of Internal Medicine, University of New Mexico School of Medicine, 1650 University Boulevard, Albuquerque NM, 87102.

U.S. Army MEDCOM and Project ECHO at the University of New Mexico Health Sciences Center (UNMHSC) collaboration conducted under the auspices of Government Interagency Personnel Agreements between UNMHSC and MEDCOM.

doi: 10.7205/MILMED-D-15-00044

INTRODUCTION

The ECHO Model: Proof of Concept

Dr. Sanjeev Arora, founded Project ECHO (Extension for Community Healthcare Outcomes) in 2003, at the University of New Mexico Health Sciences Center (UNMHSC) as an innovative disruption to increase capacity and address the scarce specialty health care resources for patients living in New Mexico, a large state with many health care disparities and geographic challenges.1 The first teleECHO clinic was developed to address the growing hepatitis C epidemic. A prospective study examining the efficacy of the Hepatitis C TeleECHO Clinic showed that the cure rate with antiviral medications was as good if not superior to the model of direct care at the University of New Mexico (UNM) Hospital Hepatitis C Clinic.2 This was achieved by connecting primary care sites throughout New Mexico to Project ECHO specialists and utilizing the Four-Point ECHO Model: (1) use technology (multipoint videoconferencing and Internet) to leverage scarce health care resources, (2) disease management model focused on improving outcomes by reducing variation in processes of care and sharing ?best practices?, (3) case-based learning: comanagement of patients with UNM specialists (learning by doing), and (4) Health Insurance Portability and Accountability Act compliant web-based database to monitor outcomes.3

In addition to hepatitis C and chronic pain, Project ECHO currently conducts teleECHO clinics for other chronic yet complex conditions including integrated addictions and psychiatry, rheumatology, HIV/AIDS, dementia, complex care, palliative care, women's health/genomics as well as endocrinology. Each program fulfills the six characteristics recognized vital to the success of the ECHO model: (1) the condition is common, (2) requires complex management, (3) has critical impact to public health and cost to society, (4) that best practices treatments continue to evolve, (5) improved outcomes with condition management, and (6) high societal burden if there is no treatment.4 Multipoint videoconferencing is used to connect specialists at regional centers (hubs) to primary care clinicians working in rural and urban underserved sites (spokes). The integrated team of specialists provide didactics and case-based learning while monitoring outcomes.1

Challenge for Project ECHO: Replication of the ECHO Model with Fidelity

Soon after the success of ECHO's prospective study, the interest in replicating the ECHO model grew significantly and at a very fast pace.2 Dozens of academic medical centers and other large organizations including the Veterans Affairs Health Care System as well as many international groups approached Project ECHO with interest in replicating particular disease conditions impacting their patient populations. To date, Project ECHO has 43 hub replication partners globally, 36 sites in the United States with an additional eight programs operating in five countries. This huge increase in demand created a challenge for Project ECHO to create a replication and training tool that incorporates the principles of the model, allows for rapid dissemination, and maintains the fidelity necessary to optimize patient outcomes.

ECHO Pain: Development of a Robust Replication Tool with the U.S. Army's Department of Defense

The Chronic Pain and Headache Management TeleECHO Clinic (ECHO Pain) was first developed in 2008, when the crisis of unintentional opioid overdose deaths in New Mexico peaked. The team is led by Dr. Joanna Katzman, a neurologist, and her interdisciplinary team from neurology, internal medicine, addiction psychiatry, nursing, and complementary specialties. Using the ECHO model, the team offers consultative services and delivers brief didactics to participating clinicians. No patients join this network. It is a clinician-toclinician educational forum.1,4 The project first developed in response to a 6 month wait time at the UNM Pain Center, the state's only academic medical center and safety net hospital. The goal is to develop Primary Care Pain Champions in rural sites throughout New Mexico allowing more patients access to best practices pain care and safe opioid prescribing through education of their primary care clinician. Telementoring occurs through ongoing consultations via case presentations during the videoconferences. ECHO Pain provides postgraduate pain education for primary care clinicians that is commonly lacking in medical and nursing schools.5,6 The program improves pain assessment skills, confidence, and attitudes in a community of practice while providing no-cost continuing medical education.7-10

ARMY PAIN ECHO

In 2009, the Department of Defense chartered the Pain Management Task Force in recognition of similar phenomena. The Pain Management Task Force Report (2010) included 109 recommendations to implement a strategy that was holistic, multimodal, and multidisciplinary. This strategy included complementary integrative medicine and holistic models for pain management.11-14

The U.S. Army Medical Command (MEDCOM) immediately developed a comprehensive pain management campaign plan to implement the task force recommendations across Army Medicine. Interdisciplinary Pain Management Centers were established in each of the five MEDCOM Regional Medical Command regions to serve as the pain consultation centers for the medical treatment facilities in their region. One of the biggest challenges involved synchronizing the care of this large network of specialty and primary care clinicians.

In 2012, the Army determined that Project ECHO would be a key mentor in this process. In collaboration with the ECHO Pain team, MEDCOM developed a phased Army Pain ECHO roll-out plan covering all regions in the hub and spokes model (Fig. 1).

Given the magnitude of this global replication of ECHO, the intent of the Army's collaboration with the ECHO Pain team was to design a deliberate replication model with sufficient fidelity and standardization to withstand geographic, information technology, and synchronization challenges.

The Hub Readiness Replication Model was created to

(1) Ensure fidelity of the Four-Point ECHO Model.3

(2) Create consistency in the replication process.

(3) Be adaptable for use by all ECHO disease-specific clinical and nonclinical ECHO replication programs.

METHODS

Echo Replication: Hub Readiness Replication Model

Four specific sequential phases were designed with a unique set of objectives. The learning acquired from each phase of training builds upon the next in this 5-month process (Table I).

Phase I (Preparatory Phase) allots adequate time for video technology installation and testing at the hub and spoke sites and establishes adequate administrative and staffing requirements. These processes are essential to the effective roll-out of any teleECHO clinic given the technology and administrative capabilities needed to implement such a program. This phase typically lasts for 4 weeks.

Phase II (Preliminary Phase) educates hub and spoke teams via a 1-month observation phase of Project ECHO. Teams review pertinent Project ECHO literature and videos, as well as participate in an ECHO Pain Boot Camp. The ECHO Pain Boot Camp is an intensive 2-day training intended to give participants a preliminary introduction and overview of the ECHO model which is similar in idea to a traditional boot camp which trains new military personnel. A modified ECHO Pain Boot Camp is provided to Army hub and spoke clinicians, administrative and informatics teams from Southern Regional Medical Command (SRMC) and Northern Regional Medical Command (NRMC) as an on-site 2-day training session. A modified ECHO Pain Boot Camp was delivered virtually (via videoconferencing) over a 2-day period to the Pacific Regional Medical Command (PRMC) and European Regional Medical Command (ERMC) because of personnel and geographic requirements. The lead clinical facilitators for the PRMC and ERMC hubs traveled to New Mexico to provide an on-site representative during the training. The 2-day ECHO Pain Boot Camp introduces key concepts of ECHO, teaches critical facilitation and presentation skills via Mock ECHO simulation training, incorporates traditional lectures, and details critical administrative and information technology tasks necessary to launch an independent ECHO.

Phase III (Training Phase) was developed to unite the hub team (specialty care) and spoke teams (primary care pain champion) from the same region. All clinicians (hubs and spokes) participated as spokes to the ECHO Pain team for 6 weeks of clinical immersion. Clinicians presented cases to the ECHO Pain team as guided practice. The hub clinicians benefit from modeling effective, noncritical feedback and interpersonal skills while they participate as spoke clinicians for this training period. Teaching strategies include case-based learning and evidence-based didactics.

Phase IV (Hub Roll-Out Phase) allows the hub team of pain specialists to launch an independent Army Pain ECHO although still maintaining partial connection to ECHO Pain. The ECHO Pain team synchronously observes via video to provide written constructive feedback within 48 hours to the Army Pain ECHO hub team, providing ongoing telementoring. Technological and administrative assistance is also provided.

RESULTS

A mixed methods study used both quantitative and qualitative analyses to determine the success of the Army Pain ECHO Hub Readiness Replication Model. Successful completion of each phase was imperative to establish a functional and independent ECHO hub. Continuing medical education credits were quantified, ECHO Pain Boot Camp surveys were evaluated, and a focus group for hub clinicians was also performed. Institutional review board approval was obtained from UNMHSC.

Two-Day ECHO Pain Boot Camp Survey

Key findings from the ECHO Pain Boot Camp surveys demonstrated that both the on-site and virtual participants had an increase in knowledge of pain management, as well as an increased motivation to learn new skills and teach others. Additionally, the majority of clinicians rated their training of the Mock ECHO experience as highly effective. Both the on-site boot camp and the virtual boot camp were rated as effective training methods (Table II).

Continuing Medical Education Credits

The Army Pain ECHO participants obtained continuing medical education and continuing education units from the UNM Office of Continuing Medical Education beginning with Phase I of the Hub Readiness Replication Model. Once each of the regional Army hubs launched their own Army Pain ECHO (Phase IV), the continuing medical education credits became the responsibility of the Army Pain ECHO Program Office (Table III).

The diverse group of interprofessionals participating in the ECHO Pain Boot Camp training is reflected in Table IV. Almost 25% of the participants in the Army Pain ECHO replication project were physicians, 18% were mid-level providers (nurse practitioners, physician assistants, pharmacists, or psychologists), 10% were licensed to provide rehabilitation services, and 12% were nurses or nursing assistants. There was a large category for other specialists ranging from complementary integrative medicine providers to health technicians.

Hub Clinician Focus Group Results

To obtain insights and feedback from this deliberate approach to hub education, training, and support, we invited hub leaders from the four MEDCOM regions with active ECHO initiatives (SRMC, NRMC, PRMC, and ERMC) to participate in a 90 minute, multipoint videoconference focus group (May 2014). We focused on their feedback related to the ECHO Hub Readiness Replication Model and their perceptions about the benefits and challenges of the 2-year Army Pain ECHO and ECHO Pain collaboration.

A range of one to four members from each of the Army hubs participated in the focus group as well as three members from the Army'sOffice of the Surgeon General.

There were four broad focus group questions. They included (1) What is the impact of Army Pain ECHO in education of colleagues and primary care clinicians?, (2) What is the influenceofArmyPainECHOtraining(HubReadiness Phases) on facilitation of an independent ECHO?, (3) How has your involvement with Army Pain ECHO influenced you in training others?, and (4) What are the benefits and challenges of Army Pain ECHO? The focus group was facilitated by evaluation team members from Project ECHO, and the focus group was recorded and transcribed. The focus group transcript was analyzed by the evaluation team members for general themes. The analysis and report were reviewed by the ECHO Pain team and later forwarded to the participants.

THE IMPACT OF ARMY PAIN ECHO

What Is the Impact of Army Pain ECHO in Education of Colleagues and Primary Care Clinicians?

Participation in the ECHO Pain training changed several teams' approach to education. One region reported transformation in their approach to case discussions from prescriptive conferences to a focus on education.

"Now we have open communication with these various providers who not only communicate with us through ECHO, but they also communicate with us on the more difficult cases through email and sometimes over the phone." "It has given us a tool for educating the primary care clinicians."

"It's impacted not just our clinical practice but the practices ...in other clinics."

"It's a way to treat complicated patients in a comprehensive approach."

"Primary care providers were initially very overwhelmed by how to wean people off opioids."

"Through Army Pain ECHO trainings the spoke clinicians learned how valuable behavioral health is and how important it is to have multidisciplinary meetings, and support case managers."

Videoconferencing also helped further support a longterm regional pain program among the participating hubs and spokes. The ECHO model helps to address primary care clinicians' feelings of isolation in care of patients by fostering on-going virtual face-to-face interactions.

What Is the Influence of Army Pain ECHO Training (Hub Readiness Phases) on Facilitation of an Independent ECHO?

Team members learned about preparing and preplanning for the teleECHO clinics including recruiting participants and the importance of strong interpersonal skills. Most said they could not have launched Army Pain ECHO in their region without the help of Project ECHO, especially related to developing the didactic presentations and prepared curriculum.

"Spending a lot of quality time with our acupuncturists, our chiropractors, and our medical massage therapists was a really good team building process for our clinic. The boot camp really helped, as they participated with it, to see where we were going."

"The boot camp and the preliminary phase were important to be able to get out there and see how ECHO actually works from hub site. That helped us be able to establish it here ...."

"It was valuable to have to present cases to UNM. We realized how time consuming it is to ...prepare the case and how ...uncomfortable you are when you don't know the answer."

"You're more careful and kind when your (spoke) participant doesn't know the answer, and you're more considerate to them when they have issues and they're kind of fumbling around."

"It gave us time to work on how we wanted to proceed ... and made us a much better working team..."

How Has Your Involvement With Army Pain ECHO Influenced You in Training Others?

One hub team indicated that they reframed their thinking about how they interacted with the primary care clinicians; instead of offering advice specific to one patient, they generalized the information about recommendations and suggested that these were things to think about and include in approaches to patients like this.

"From our end, making the spokes comfortable to ask questions and not feel like they're going to be berated. They tend to interact much more."

"What I've learned from ECHO is a lot of those soft skills ...encouraging clinicians who are busy. They haven't presented in years. They've been out of medical school for a long time, and it's daunting sometimes. You do not want to scare them away."

Several hub participants emphasized their greater appreciation for interdisciplinary team care and the knowledge and skill offered by others, as well as the expanding roles of different professionals on the team.

"This forum expands the role of the clinical pharmacist. It expands the role of the behavioral health specialist in the pain practice as well."

What Are the Benefits and Challenges of Army Pain ECHO?

The focus group participants also discussed the importance of dedicated time and leadership support to sustain their successful Army Pain ECHO replication efforts. Participants identified the limitations of dedicated time and multiple competing priorities as constraints that their primary care colleagues reported.

"We are struggling with getting sites to value the time. Just getting time set aside to participate ...."

"Sometimes there is poor (teleECHO clinic) attendance due to time constraints."

Leadership can play a larger role in supporting the Army Pain ECHO clinic. Several participants suggested that there are "competing missions from a primary care level, and it can be difficult to participate with busy panels" as constraints for leadership to consider and address. The frequency of concepts discussed is illustrated in Figure 2.

DISCUSSION

Hub Readiness Phases

The goal of Project ECHO is to increase the knowledge, confidence, and improve the skills of the primary care spoke clinicians who are managing patients in rural and underserved communities. The Hub Readiness Replication Model educates both the primary care and the specialty team in the essential components of the ECHO model. This unique model reduces the gap between primary and specialty care by demonopolizing medical knowledge, which is traditionally confined to academic medical centers and specialists located in major cities.1 By demonopolizing medical knowledge and with ongoing telementoring, primary care clinicians can effectively manage many of their patients who may have previously been referred to a specialist, or even a super-specialist.15 Training clinicians through deliberate guided practice is the essential component of the Hub Readiness Replication Model. The Army Pain ECHO roll-out has been successful for all MEDCOM regions that completed each of the four phases (SRMC, NRMC, PRMC, ERMC). Western Regional Medical Command has not been able to fully launch secondary to staffing requirements.

The ECHO Pain Boot Camp trainings during Phase II of the Hub Readiness Replication Model are highlighted by the Mock ECHO simulation trainings. Real, deidentified cases are practiced among the hub and spoke clinicians outside of an official teleECHO clinic, offering the clinician a safe environment to practice and learn through the many trainings offered. The clinicians' skills, confidence, interpersonal communication, and interest increase at each session of this guided and deliberate practice.16-19

The Phase III (Training Phase) allowed both the primary and specialty teams to present cases to ECHO Pain. The focus group results suggest that this was critical for the specialists to understand how it felt to "wear the shoes of the primary care clinician presenting the case." The primary focus of the Hub Readiness Replication Model prepares the hub specialty team to effectively facilitate a teleECHO clinic. It is assumed that the hub specialist does not need improvement in knowledge and skills within their specialty, but that they have the interpersonal skills and professionalism necessary to educate primary care clinicians in front of a large videoconferencing network. These interpersonal skills are an important aspect of social cognitive theory. They are rarely learned in medical school, but are critical to the success of a teleECHO clinic. Effective mentorship and role modeling have a positive effect on the hub clinicians' ability to facilitate a successful teleECHO clinic.20,21

The hub specialists benefit from the Hub Readiness Replication Model in many critical and diverse ways. The Army Pain ECHO focus group highlighted the increased confidence that the pain specialists have developed in their primary care colleagues, the improved care coordination between primary and specialty care services, the improved work satisfaction noted by specialists because of enjoyment related to teaching others, and the interprofessional team environment.

Modeling of the ECHO Pain training facilitators' behavior also occurs during all 4 phases of the Hub Readiness Replication Model, beginning with observation of the teleECHO clinics, intensifying during the boot camp Mock ECHO trainings, and continuing throughout the training phase.3,22 This modeling helps to maintain the fidelity and the integrity of the ECHO model necessary for behavioral change.23

Two-Day ECHO Pain Boot Camp Survey

Phase II of the Hub Readiness Replication Model included the 2-day ECHO Pain Boot Camp training. Findings from the participant surveys revealed that both on-site and virtual ECHO Pain Boot Camp trainings improved clinician motivation to learn new skills and teach others. The on-site trainings were significantly more effective in providing Mock ECHO trainings, however. The in-person Mock ECHO simulation trainings both increased skills and improved social connectivity among hub and spoke team members. The in person training is easier to facilitate a dynamic exercise such as the Mock ECHO, although, even the virtual Mock ECHO training was rated highly. Each participant from the virtual boot camp trainings was able to participate on-site with the ECHO Pain team. This could have explained the high ratings and social connectivity of this experience.

Continuing Medical Education Credits

Tremendous opportunity exists for no-cost continuing medical education credits for clinicians in all disciplines utilizing the ECHO Hub Readiness Replication Model. Project ECHO is particularly beneficial for chronic and complex diseases in which access to care is limited and primary care specialists need assistance with management of a large population of patients in rural and underserved areas. During the Army Pain ECHO roll-out, participants were eligible to receive over 9,689 no-cost continuing medical education hours. This averages to 969 no-cost continuing medical education hours eligible per regional medical command per year. Direct health care system benefits include clinician satisfaction, maintenance of license, maintenance of clinician at practice site rather than traveling to medical conference with resulting loss of patient care days, and social connectivity during the teleECHO clinic. Indirect health care system benefits may include decreased referrals to specialty care out of network, decreased travel costs, better control of chronic conditions if pain treated earlier, decreased use of expensive interventions and medications if unnecessary, and many other possible returns on investment.

Diversity of Participants

In a specialty such as chronic pain, interprofessional and interdisciplinary engagement is now considered best practices both in prelicensure education and in the clinical setting.5,6,24 As the shortage of primary care clinicians continues to increase in the United States, it becomes even more vital to engage every member of the health care team to the highest level of their licensure.25

CONCLUSION

In addition to collaboration with the Army MEDCOM's Pain Program, ECHO Pain is now engaged with the U.S. Navy and U.S. Air Force to assist their clinicians with the continued roll-out of the Hub Readiness Replication Model for pain management. ECHO Pain has helped to replicate the ECHO programs for many other academic medical centers (University of Washington's TelePain, University of California Davis ECHO Pain), the Veterans Affairs (VA SCAN-ECHO, with seven pain hubs), the Indian Health Service (IHS ECHO Pain and Addiction), and ECHO Ontario Pain and Opioid Stewardship (with approval from the Canadian Health Ministry).26-29 The ECHO Hub Readiness Replication Model was specifically developed for the Army's Pain Management ECHO collaboration and proved to be a successful method to replicate ECHO across a large system.

The four phases of the Hub Readiness Replication Model are deliberate and add fidelity to the original Four-Point ECHO Model. This ECHO replication tool can easily be adopted for all chronic and complex diseases and conditions across a large enterprise. It provides the flexibility necessary to account for personnel and geographic variation. Even non-health care related entities, such as the University of Wyoming ECHO for Assistive Technologies may benefit from this model. The University of Wyoming institute for Disabilities, for instance, has developed an ECHO platform to reach out to clinicians and clients in rural Wyoming (www.uwyo.edu).

The ECHO model may be considered for other conditions within the Military Health System, the Veterans Affairs Health Care System, large academic medical centers, private organizations, and other large health care systems outside the United States. It is a novel and robust tool that offers fidelity to the ECHO Model which can help to disseminate much needed medical knowledge and education throughout the United States and beyond its borders while simultaneously helping expand access to health care.

ACKNOWLEDGMENTS

We thank the following individuals for helping to develop and implement the Hub Readiness Replication Model: Jeannie Boyle, MSN, RN; COL Paul Barras, CRNA, MSN, MSSR; Rebecca Monette; Lainey Flatow-Trujillo, BS; Melissa Britt, BS; Eric Castillo, MPA; Lester Gresham, BA; Ronselle Harrison, BA; Sarah Zalud-Cerrato, MPH; William Szaroletta, PE; Benson Daitz, MD; Rachel Foster, FACHE; and Steve Steffensen, MD.

REFERENCES

1. Arora S, Kalishman S, Dion D, et al: Quality profi le: partnering urban academic medical centers and rural primary care clinicians to provide complex chronic disease care. Health Aff 2011; 30(6): 1176 - 84.

2. Arora S, Thornton K, Murata G, et al: Outcomes of treatment of hepatitis C virus infection by primary care providers. N Engl J Med 2011; 364(23): 2199 - 07.

3. Arora S, Kalishman S, Thornton K, et al: Expanding access to hepatitis virus treatment-Extension for Community Healthcare Outcomes (ECHO) project: disruptive innovation in specialty care. Hepatology 2010; 52(3): 1124-33.

4. Arora S, Geppert CM, Kalishman S, et al: Academic health center management of chronic diseases through knowledge networks: project ECHO. Acad Med 2007; 82: 154 - 60.

5. Doorenbos AZ, Gordon D, Tauben D, et al: A blueprint of pain curriculum across prelicensure health sciences programs: one NIH pain consortium Center of Excellence in Pain Education (CoEPE) experience. J Pain 2013; 14(12): 1533 - 8.

6. Fishman SM, Young HM, Lucas Arwood E, et al: Core competencies for pain management: results of an interprofessional consensus summit. Pain Med 2013; 14(7): 971 - 81.

7. Katzman J: Making connections: using telehealth to improve the diagnosis and treatment of complex regional pain syndrome, an underrecognized neuroinflammatory disorder. J Neuroimmune Pharmacol 2013; 8(3): 489- 93.

8. Katzman JG, Comerci G, Boyle JF, et al: Innovative telementoring for pain management Project ECHO Pain. J Cont Ed Health Prof 2014; 34(1): 68- 74.

9. Katzman J, Comerci G, Landen M. Rules and values: a coordinated regulatory approach to the public health crises of chronic pain and addiction. Am J Pub Health 2014; 104(8): 1356 - 62.

10. Barnett S, Jones SC, Bennett S, Iverson D, Bonney A. General practice training and virtual communities of practice: a review of the literature. BMC Fam Prac 2012; 13: 87.

11. Buckenmaier CC III, Rupprecht C, McKnight G, et al: Pain following battlefield injury and evacuation: a survey of 110 casualties from the wars in Iraq and Afghanistan. Pain Med 2009; 10(8): 1487 - 96.

12. Kerns R, Otis J, Rosenberg R, Reid MC. Veterans' reports of pain and associations with ratings of health, health-risk behaviors, affective distress, and use of the health care system. JRRD 2003; 40(5): 371- 80.

13. U.S. Department of Defense: 2008 Survey of health related behaviors among active duty military personnel Q & A. Washington, DC, Military Health System, updated October 2009. Available at http://www.tricare .mil/2008SurveyQ&As.pdf; accessed 8 September, 2014.

14. U.S. Army: Army health promotion risk reduction suicide prevention report 2010; updated July 29 2010. Available at http://www.army.mil/news/2010/07/28/42934-army-health-promotion-risk-reduction-and-suicideprevention-report/index.html; accessed September 8, 2014.

15. Arora S, Thornton K, Komaromy M, Kalishman S, Katzman J, Duhigg D: Demonopolizing medical knowledge. Acad Med 2014; 89(1): 30 - 2.

16. Bandura A: Self-Efficacy: The Exercise of Control. New York, W. H. Freeman and Co, 1997.

17. Lateef F: Simulation-based learning: just like the real thing. J Emerg Trauma Shock 2010; 3(4): 348-52.

18. Kaufman DM: ABC of learning and teaching in medicine: applying educational theory in practice. BMJ 2003; 326: 213 - 6.

19. Lave J, Wenger E: Situated Learning: Legitimate Peripheral Participation. New York, Cambridge University Press, 1991.

20. Bastable SB: Applying learning theories to healthcare practice. In: Nurse as Educator: Principles of Teaching and Learning for Nursing Practice, Ed 4, pp 63 - 110. Edited by Braungart MM, Braungart RG, Gramet PR. Burlington, MA, Jones and Bartlett Learning, 2014.

21. Arora S, Kalishman S, Dion D, et al: Knowledge networks for treating complex diseases in remote, rural, and underserved communities. In: Learning Trajectories, Innovation and Identity for Professional Development Innovation and Change in Professional Education, pp 47 - 70. Edited by McKee A, Eraut M. London, England, Springer Science & Business Media, 2012.

22. Vygorsky L: Mind in Society: The Development of Higher Psychological Processes. Cambridge, MA, Harvard University Press, 1978.

23. Mars T, Ellard D, Carnes D, et al: Fidelity in complex behaviour change interventions: a standardized approach to evaluate intervention integrity. BMJ 2013; 3:e003555.

24. Institute of Medicine of the National Academies Committee on Advancing Pain Research, Care and Education: Relieving Pain in America: A Blueprint for Transforming Prevention, Care, Education and Research. Washington, DC, National Academic Press (US), 2011.

25. Martinez-Gonzalez NA, Djalali S, Tandjung R, et al: Substitution of physicians by nurses in primary care: a systematic review and metaanalysis. BMC Health Serv Res 2014; 14: 214.

26. Eaton LH, Gordon DB, Wyant S, et al: Development and implementation of a telehealth-enhanced intervention for pain and symptom management. Contemp Clin Trials 2014; 38(2): 213-20.

27. Scott JD, Unruh KT, Catlin MC, et al: Project ECHO: a model for complex, chronic care in the Pacific Northwest region of the United States. J Telemed Telecare 2012; 18(8): 481 - 4.

28. Mchaourab AS, Ober S, Gaudino CP: Innovation in specialty care delivery to rural areas. Poster no. 152 presented at 2013 American Academy of Pain Medicine Conference 2013. Available at http://www .painmed.org/2013posters/abstract-152/; accessed February 9, 2015.

29. Mendonca KB: SCAN-ECHO: an effective way to provide patientcentered care. U.S. Department of Veterans Affairs, 21 September, 2012. Available at http://www.sanfrancisco.va.gov/features/SCAN_ECHO.asp; accessed November 15, 2014.