A Novel Approach to Objective Validation of YouTube™ Educational Videos for the Instruction of Regional Anesthesia Nerve Blocks


 Background

YouTube™ (“YouTube”) is often used as an educational tool to instruct anesthesia providers on regional anesthesia nerve blocks. However, there is no current objective standard to assess the educational quality of these user-uploaded videos. A new approach was used to objectively validate these videos by comparing them to high quality educational sources for the seven most commonly used nerve blocks.
Objective

We sought to evaluate the educational quality of user-uploaded videos when compared to the highest quality anesthesia society websites (NYSORA, ACEP, USRA)
Methods

We first catalogued the characteristics of the instructional material on three of the highest quality anesthesia society websites (NYSORA, ACEP, USRA) into 18 different items. Next, we surveyed the five most popular anesthesia block videos on YouTube for each of the seven peripheral nerve blocks in question. The presence or absence of each category was recorded.
Results

Although there were varying degrees of correlation between the high quality sources and the videos, rarely did YouTube videos contain as high a percentage of these educational characteristics as the well-established sources. Some videos contained very few of these important educational characteristics.
Conclusion

Although YouTube has been used an educational tool, we recommend that only high quality sources be used to teach or illustrate regional anesthesia nerve blocks.


Abstract Background
YouTube™ ("YouTube") is often used as an educational tool to instruct anesthesia providers on regional anesthesia nerve blocks. However, there is no current objective standard to assess the educational quality of these user-uploaded videos. A new approach was used to objectively validate these videos by comparing them to high quality educational sources for the seven most commonly used nerve blocks.

Objective
We sought to evaluate the educational quality of user-uploaded videos when compared to the highest quality anesthesia society websites (NYSORA, ACEP, USRA)

Methods
We first catalogued the characteristics of the instructional material on three of the highest quality anesthesia society websites (NYSORA, ACEP, USRA) into 18 different items. Next, we surveyed the five most popular anesthesia block videos on YouTube for each of the seven peripheral nerve blocks in question. The presence or absence of each category was recorded.

Results
Although there were varying degrees of correlation between the high quality sources and the videos, rarely did YouTube videos contain as high a percentage of these educational characteristics as the well-established sources. Some videos contained very few of these important educational characteristics.

Conclusion
Although YouTube has been used an educational tool, we recommend that only high quality sources be used to teach or illustrate regional anesthesia nerve blocks.

Background
As one of the most popular open source video content websites available today, YouTube has expanded from being a primarily entertainment website to an instrument for education and information sharing. [1][2][3][4] Numerous medical procedures and clinical skills instructional videos can be found quickly, changing the landscape of medical education. Students and providers alike are turning to social media platforms to augment classroom learning and refresh clinical skills. 1-2 While access to these videos is nearly instantaneous, every piece of content must be used with the awareness that it is often neither peer reviewed nor certified for accuracy. However, non-peer reviewed periodicals, websites, conferences and social media have been noted to be sources some healthcare professionals turn to prior to seeking out peer-reviewed material. 3 Nonetheless, many YouTube videos providing misinformation have been reported, leading to potential dangers to patients, and compromising providers' credibility if used as the sole source of information. [4][5][6] Previous studies have analyzed instructional content for various specialties, the majority of which used subject matter experts. These are experts in "content analysis", to grade the videos' content based on predefined criteria. [7][8][9][10][11] Most concluded that, while convenient, YouTube content lacked the quality required to safely guide a provider through a specific procedure. Few studies have evaluated the value of regional anesthesia procedures posted to YouTube. [9][10][11] While others have evaluated large amounts of content posted to YouTube, we asserted that users are most apt to use the first few videos they come across, mostly corresponding to the "top hits," or most viewed. Therefore, our aim was to design a system to objectively evaluate the 5 most viewed instructional videos for some of the most commonly performed regional anesthesia nerve block procedures.

Methods
YouTube (www.youtube.com) searches were conducted for each nerve block ("interscalene nerve block", "supraclavicular nerve block", "infraclavicular nerve block", "axillary nerve block", "femoral nerve block" "popliteal nerve block" and "transversus abdominis plane block"). Search returns were organized using "filter", then "sort by" and "view count." The top five most viewed videos for each block were evaluated. Inclusion criteria included content in the English language and ultrasound guided blocks (vs nerve stimulation only). The content of each video was reviewed by four boardcertified anesthesiologists, each with a minimum of four years of post-residency regional anesthesia experience.
Eighteen key characteristics were established for analysis by viewing the online nerve block content 4 of three well-vetted societies for regional anesthesia: The New York School of Regional Anesthesia (NYSORA) 12 , American College of Emergency Physicians (ACEP) 13 , and Ultrasound for Regional Anesthesia (USRA) 14 (Table 1). Each YouTube video was analyzed for the presence or absence of each of these 18 characteristics, and this was recorded for further analysis. Characteristics were recorded as present only if the information contained was correct as determined by the attending anesthesiologist

Results
The content of the three reference websites (NYSORA, ACEP and USRA) were analyzed for completeness using the categories outlined in Table 1. NYSORA had the most complete instructional content, recording 100% of the categories for each of the seven blocks. ACEP and USRA achieved means of 93.5% and 95.2%, respectively, of categories discussed. The TAP block was omitted for statistical calculations for ACEP, as they did not provide any instructional content for this particular block. "Indications" was the category missed most by USRA, with five out of seven blocks missing this content, while ACEP did not discuss "catheter placement" in any of its instructional videos.
Searches for each of the seven nerve blocks yielded multiple pages of related videos. The five top hits that met inclusion criteria for the study, after sorting based on view count, were analyzed and evaluated.
The least included category in the YouTube videos was "references". Only 6 of the 35 (17.1%) videos screened referenced primary sources. "Guide for catheter placement" (40%) and "Atlas picture of anatomy" (48.6%) were the second and third least referenced categories over all YouTube videos. Table 2 lists the frequency each category was referenced in all 35 YouTube videos evaluated.
The regional anesthesia societies' websites offered very thorough instruction for each nerve block. Finally, the overall completeness of content published by regional anesthesia societies versus YouTube videos was analyzed ( Table 3). The regional anesthesia societies had significantly more complete content, discussing greater than 94% of the categories for each of the seven nerve blocks in 5 question (Note: ACEP TAP block was omitted for statistical calculations, as they did not provide any instructional content for this particular block). The most thorough nerve block instruction from the YouTube content was for interscalene blocks, with the five most viewed videos combined covering 90% of the categories in question (see Fig. 2). The least comprehensive nerve block on YouTube was the popliteal block, with only 68.9% of the topics covered.

Discussion
As the second most popular website in the world 15 , with over one billion hours of video watched per day 16 , YouTube is an extremely influential resource in entertainment and information gathering for millions of people. Not surprisingly, the site has become a frequently relied upon source of patient medical information gathering, especially by younger physicians, residents and students 17 . Given its popularity and global reach, YouTube has the potential to be an important educational tool for patients, conveying essential health information that could help decrease morbidity and economic burden of preventable diseases 18 . The challenge remains that, while abundant in quantity, the information available is often published by a non-vetted, non-peer reviewed source, and lacks the quality patients require. Multiple studies evaluating the content of YouTube videos as a source of patient information across different specialties have found them to be of subpar quality and have questioned their usefulness in finding reliable and accurate information. [19][20][21][22] Just as YouTube has become a quickly turned to source of medical information for patients, medical students and providers are turning to its content more frequently. 1-2 With younger generations of medical professionals having access to an exponentially increasing amount of content on YouTube and other social media platforms to augment their learning, caution must be taken prior to utilizing these sources. Their content and publishers should be thoroughly vetted to ensure the information presented is complete and accurate. Other studies have evaluated YouTube content for physical exam skills, procedures, and surgeries and have found, similar to health informational videos accessed by patients, that their content is often questionable in quality and completeness. [23][24][25][26] Visual aids and hands on simulations can be invaluable tools in learning proper techniques in regional 6 anesthesia. Recognizing the important role videos can play in augmenting regional anesthesia education, we set out to evaluate the content of the five most viewed videos for each of 7 ultrasound guided nerve blocks. While a few of the videos were very thorough, touching on nearly 90% of the 18 categories we designated as evaluation criteria, many were lacking a significant amount of information. The overall completeness of the YouTube videos was compared to the content published on the websites of three well respected authorities in regional anesthesia (NYSORA, ACEP and USRA) and noted to be significantly lacking. Figure 2 summarizes these findings.
As previous studies on the utility of YouTube content have noted, we found that even among the five most viewed instructional videos per nerve block that there was a significant variation in completeness. Among all categories evaluated, "references" was missing from nearly 86% of YouTube videos. This is concerning given the fact that the site is an open source platform, allowing anyone to upload content without oversight. In contrast, the regional anesthesia society websites cited multiple references for each nerve block in question 100% of the time.
Whereas YouTube has the potential to be an invaluable tool for patients and providers, offering a plethora of extremely easily accessible content, it must be used in the proper manner in order to avoid receiving incomplete, and possibly inaccurate information. As students and young physicians, consulting the content published by the established authorities on regional anesthesia to learn foundational knowledge for each nerve block would be most advisable. However, there may be a role in consulting further video content after the provider has knowledge and experience with the procedure to evaluate different techniques or refresh their skills.

Conclusion
YouTube has become an extremely influential educational website, offering a huge amount of medical video content. Regional anesthesia providers are prime candidates that could benefit from visual aids, as all procedures require an intimate knowledge of anatomical landmarks and proper technique.
When compared to regional anesthesia societies' website content, the content of the five most viewed instructional videos for each nerve block in question lacked in multiple categories. YouTube videos should likely not be used as primary sources when initially learning a nerve block; however, 7 they may be of value to augment already established knowledge for the experienced provider wishing to refresh on skills or explore different techniques.     Percent of educational categories referenced in educational material by nerve block