Breast cancer is the leading cause of cancer death in Hispanic women in the USA. According to the American Cancer Society, as many as one in nine Hispanic women will develop invasive breast cancer during their lifetime and one in 49 Hispanic women will die from invasive breast cancer.¹ Female breast cancer is also the most common and leading cause of cancer death in Latin American and Caribbean countries, with an estimated 114,900 new diagnoses and 37,000 deaths each year.^2,3

Patients who receive a cancer diagnosis are often faced with difficult decisions regarding their treatment and prognosis and thus they rely on multiple sources of information to make an informed decision. The internet is an increasingly important source of health-related information for patients in the USA and worldwide. According to a National Health Interview Study survey from 32,139 adults, approximately 43% reported searching the internet for health information.⁴ Fox and colleagues found that 83% of Hispanic adults living in the USA obtained health information from media sources, with 79% of those acting on the health-related information obtained from these sources.⁵

YouTube, a global repository of videos and social networking platforms, is a popular way for people to access health-related information, with over a billion hours viewed every day.⁶ According to a study conducted by the Pew Research Center, 72% of adults online use some form of video-sharing platform, of whom 63% use YouTube as a source of health-related information.⁷ A variety of online outlets – including advertisers, government agencies, news agencies, patient health portals, patients and health professionals – produce health-related videos and upload them to YouTube. However, several studies have questioned the quality and reliability of health-related information on YouTube.^8–10 A study published in 2018 evaluating breast self-examination videos on YouTube found 54 out of 87 videos misleading.¹¹

Objectives

While previous studies have evaluated the quality and reliability of online information in English, no studies have assessed the quality and reliability of YouTube videos in Spanish containing health-related information pertaining to breast cancer.^11–16 For Spanish-speaking-only patients, who face language barriers in receiving health information, high-quality, reliable online health-related information that is available in Spanish is essential.^17,18 Our main objective was to determine the quality and reliability of online videos in Spanish regarding breast cancer.

Methods

We conducted a descriptive analysis of Youtube videos using the search terms “cancer de mama” (translation: “breast cancer”) to investigate what patients would find when they are seeking information via YouTube. We chose these words over other options because this is the most broadly used set of words to describe breast cancer in Spanish. Search history, cookies and caches were cleared from the personal computers of the investigators to avoid any possible influence from prior browsing history. We created a new YouTube account in Harlingen, Texas. The first 200 video results from uniform resource locators of the search engine results page were transferred onto a playlist to avoid daily variances in search results using the default search filters. All inactive videos, duplicates, non-Spanish duplicates, or those with audio only, or without accompanying audio were excluded. Out of the initial 200 videos, a total of 173 were included in the study for further analysis. Twenty-seven videos (15%) were excluded because they had no audio (n=5), no relevant information (n=8), were in a non-Spanish language (n=10), in multiple parts (n=3) or were duplicates (n=1).

All videos were viewed in their entirety, with information collected from each video, including the number of ‘likes’, ‘dislikes’, total number of views, country of origin, video length and video upload date. Videos were categorized into one of five typology groups according to the source of upload: ‘commercial’ (for-profit companies), ‘government/news agencies’, ‘health portal’ (health information website), ‘personal’ (patient or family), and ‘Professional’ (university, hospital, physician groups or non-profit physician) (Table 1). The content of each video was assessed for the following: signs and symptoms, screening and detection, risk factors, diagnosis, research, treatment, reconstruction, survivorship/advocacy, and male breast cancer.

Video quality and reliability were assessed using three validated tools. The Journal of the American Medical Association (JAMA) Benchmark Criteria utilizes four core standards to evaluate online health information and identify the essential components of transparency and reliability: authorship, attribution of references, currency and disclosure of any potential conflict of interest. A point is given for the presence of each criterion, with a minimum score of 0 and a maximum score of 4.⁴ The Global Quality Score (GQS) is a subjective tool graded on a 5-point Likert scale, with a score of 1 representing poor quality and not being beneficial for patients, and a score of 5 representing excellent quality and very useful for patients.¹⁹ The modified 5-point DISCERN score is a questionnaire used to evaluate the quality of information with a maximum potential score of 5 points indicating high quality and a minimum of 0 points indicating poor quality (Table 2). In summary, the JAMA score evaluates information reliability, while the GQS and DISCERN scores assess information quality. Combining the scores provides a broader portrait of the quality and reliability of the video, as a whole. Scores were generated by two internal medicine resident physicians who were fluent in English and Spanish (IL and DC) independently at different times, who also selected the videos from the original search. The corresponding Cohen’s kappa coefficients for GQS and DISCERN scores were 0.923 and 0.960, respectively. A third physician (FD) resolved any disagreements on scores.

The ordinal scales were used to measure the video quality and reliability scores. Since these measurements are in an ordinal scale, we summarize them using median and interquartile ranges (IQR). All numerical measures are summarized as median ± IQR and all qualitative variables were summarized as counts (percentages), as the best summary measures for categorical variables are percentages. The qualitative variables were recorded as percentages (year video uploaded, length of video, country of origin and typology). The differences in the quality and reliability measures were evaluated by typology using the Kruskal–Wallis (KW) non-parametric test.²⁰ This is mainly due to having ordinal type measures (quality and reliability scores) for small sample sizes within certain typology categories, and hence not being able to satisfy the normality assumption to use the one-way analysis of variance.²¹ It should be noted that, in the KW test, we deal with either the median (when the data distribution has the same shapes) or mean rank (when the data distribution do not have similar shapes). In our case, the shape of the data distribution for JAMA, GQS and DISCERN scores, within typologies are not the same and hence we report the mean ranks. Bonferroni adjusted pairwise comparisons with Dunn test were used to identify the significant differences in the three scores, between each pair of typology.²² To detect a significant association among qualitative variables within each typology group, either the Pearson’s chi-square or Fisher’s exact tests were utilized.

Linear regressions were used to evaluate the impact of variables (number of likes, dislikes, duration of videos and number of views) on the three quality and reliability scores after adjusting for the typology and the contents of the videos. If the assumptions of linear regression models were not satisfied, a corresponding generalized additive model (GAM) was used to evaluate the impact of each variable. GAM provide a general framework for extending a standard linear model by allowing non-linear functions of each of the variables. All tests were two-tailed, and a 0.05 significance level was used to assess the statistical significance. Data analysis was performed using R (V.4.1.2).

Results

Approximately 4% of videos were uploaded before 2010, 31% uploaded 2010-2014, and 63.5% uploaded 2015-2019. The median length of the videos was approximately 5 minutes, with an IQR of 9.30. Approximately 48% of videos lasted less than 5 minutes, 19% were between 5 and 10 minutes, 20% were between 10 and 20 minutes, and 12% lasted longer than 20 minutes. The videos in our sample had a median number of views of 23,539. The median number of ‘likes’ per video was 159, with a range of 0–47,000 (IQR: 468). The median number of ‘dislikes’ was 10, with a range of 0–721 (IQR: 38). Most videos were published in Spain (36.4%), followed by Mexico (17.9%) and the USA (10.4%). Of the remaining videos, 26.6% were uploaded from 12 other Latin American countries and, in 8.7%, the country of origin could not be identified (Table 3)

With regards to video typology, 63 videos (36.4%) were uploaded by patients or their family or friends (‘personal’), 43 videos (24.9%) by a government or news agency (‘government/news agency’), 40 videos (23.1%) by medical professionals (‘professional’), 14 videos (8.1%) by a stand-alone health portal (‘health portal’), and 13 videos (7.5%) by commercial companies (‘commercial’) (Table 3).

We next evaluated the distribution of video content based on their typology. Of the ‘professional’ videos, 50% focused on treatment modalities, while very few referenced breast reconstruction, survivorship and advocacy, or breast cancer in men. Videos that discussed signs and symptoms of breast cancer were mainly uploaded by ‘commercial’ typologies (54%), videos discussing screening and detection were uploaded primarily by ‘government/news agency’ (44%), and videos discussing risk factors were uploaded mostly by standalone ‘health portals (21%) (Figure 1).

The details of video characteristics were stratified by the source of upload as shown in Table 4. ‘Personal’ videos had the longest duration on average (8 minutes) while ‘health portal’ videos had the shortest average (2.8 minutes). No significant differences were found in the median time since upload among the different typology types (p-value: 0.053).

The overall median JAMA score of all videos was 2, with a range of 0 to 3 (IQR: 0.00), in which 82.7% scored between 2 and 4 possible points. Only 3.5% of videos achieved 3 points, and no videos reached the maximum score of 4 points. The overall median GQS score was 2.00, with a range of 1 to 4 (IQR: 1.00). Approximately 21% of videos scored 1 point, 37% scored 2 points, 38% scored 3 points, 4% achieved 4 points, and no videos obtained all 5 possible points. The overall median DISCERN score was 2 with a range of 0 to 5 points (IQR: 2.00). Approximately 8% scored 0 points, 15% scored 1 point, 39% scored 2 points, 14.5% scored 3 points, 2.9% scored 4 points and only one video (0.6%) scored all 5 points (Table 5).

The correlation among the three reliability scores and the length of videos, likes, dislikes and views were calculated overall, and by typology (Figure 2). Overall, the length of the videos had shown a significant positive correlation with the GQS (r=0.28) and DISCERN (r=0.23) scores. Similar positive correlations for GQS and DISCERN scores were observed for length of videos within all typology types except for commercial videos (r=−0.56). Note that the JAMA score correlation was not calculated, as the scores were the same for all commercial videos.

The mean rank and median scores by the source of upload (typology) were analysed and are shown in Table 6. Notably, for the JAMA score, the mean rank for ‘professional’ videos was highest among typologies at 99.5 (standard deviation [SD]: 24.3) compared with 73.6 (SD: 37.3) for ‘personal’ and 71.0 (SD: 50.3), for ‘health portal’ the lowest. Concerning the GQS score, the mean rank for ‘professional’ was at 102.9 (SD: 40.7), 98.5 (SD: 44.9) for ‘health portal’ (second highest), and 72.3 (SD: 50.8) for ‘personal’, the lowest. Concerning DISCERN score, ‘health portal’ had the highest mean rank at 110.1 (SD: 48.9) followed by 101.1 (SD: 46.5) for ‘professional’ while ‘personal’ was lowest at 68.9 (SD: 48.2). Table 6 depicts the significant mean rank differences for JAMA, GQS, and DISCERN scores as identified by pairwise comparisons. We observe that the mean ranks of three quality and reliability scores between Professional and Personal to be significantly different with p-values: JAMA:0.001; GQS:0.013; and DISCERN: 0.009, respectively. Additionally, we found that the mean rank for JAMA between ‘government/news agency’ and ‘personal’ (p-value: 0.002) and the mean rank for DISCERN between ‘health portal’ and ‘personal’ (p-value: 0.002) was significantly different.

As per GAM analyses, video length was the only variable that had shown a significant impact (p-value: 0.0002) on the DISCERN score, after adjusting for typology and the contents of the videos. None of the variables were associated with either the JAMA or GQS scores (Tables 7 and 8).

Discussion

Hispanic people are the largest minority group in the USA, and nearly one-third identify as foreign-born.²³ According to estimates from the United States Census Bureau, approximately 58 million Americans, or nearly 19% of the total population, identified as Hispanic or Latino in 2016, and this is expected to double over the next four decades.²⁴ Additionally, data from the North American Central Cancer Registries (NACCR) showed that Hispanic females were less likely than non-Hispanic whites to be diagnosed with localized stage breast cancer (57% versus 65%, respectively).²⁵ Another study also found that Hispanic women were less likely than non-Hispanic white women to present with stage I disease.²⁶ This difference has been hypothesized to be related to the fact that Hispanic people, on average, belong to communities of lower socioeconomic status, which tend to have inadequate access to care.²⁷ Individuals in lower socioeconomic groups are also less likely to be insured. One study, using data from the National Cancer Database, found that uninsured and Medicaid-insured patients had higher proportions of distant stage cancer at presentation than those with private insurance, or Medicare plus supplemental insurance.²⁸ Data from the NACCR also show that the incidence of breast cancer among Hispanic women increased from 2006 to 2015, at an average rate of 0.4% per year. Other studies have also suggested that Hispanic women are less likely than non-Hispanic white women to receive appropriate and timely breast cancer treatment and are more likely to be diagnosed at an advanced stage.^29,30 One multi-institutional analysis of breast cancer found, in addition to confirming the aforementioned differences in stage at presentation compared with non-Hispanic populations, that Hispanic women were more likely to present with triple-negative breast cancer (an aggressive subtype of breast cancer) and were less likely to receive breast-conserving surgery than non-Hispanic women.³¹

Interestingly, foreign-born Hispanic women experience lower rates of breast cancer compared with USA-born matched controls. This is thought to be related to reproductive factors such as younger age at first birth, higher parity, higher rates of breast cancer, and less use of hormone therapy for contraception and menopause, which confer a reduced risk of breast cancer.³² However, lower screening rates may also explain the decreased prevalence in Hispanic women in the USA compared with non-Hispanic white women in the USA and the reduced prevalence in foreign-born Hispanic women compared with USA-born Hispanic women.^33,34

Screening for breast cancer occurs less frequently in Hispanic women than in non-Hispanic white women.³⁵ Data from the National Health Interview Survey show that the prevalence of screening mammography was 61% in Hispanic women, compared with 65% in non-Hispanic white women.

With over 2 billion users in over 100 countries, YouTube has become an increasingly important source of health information for patients and physicians.⁶ As the platform grows in popularity, so does the number of videos published on medical topics. Previous studies have assessed the quality of videos concerning breast self-examination, mammography and mastectomy in English, using the same tools we utilized.^11,15,16 Other studies have also assessed the quality and reliability of information found in videos on various medical topics, from lumbar spinal fusion to food allergies.³⁶ As a company, YouTube employs community guidelines enforced by human reviewers and machine learning technology, in an attempt to “make YouTube a safer community” while still giving creators the freedom to share a broad range of experiences and perspectives.³⁷ YouTube’s community guidelines outline which type of content is kept off the platform and covers a broad range of topics, including spam and deceptive practices, sensitive content, violent or dangerous content, regulated goods, and misinformation.³⁷ Notably, YouTube’s community guidelines are fluid and continually changing in the midst of recent political trends and misinformation in the setting of the COVID-19 pandemic.

To our knowledge, this is the first study that analyses the quality, reliability and content of online videos relating to breast cancer exclusively in Spanish. On average, the videos examined in this study were viewed more than 128,000 times, demonstrating the popularity of the videos among the Spanish-speaking community. However, the unregulated nature of videos posted on this platform raises concerns about the quality and reliability of material available to the patient population. Our study demonstrates low JAMA, GQS and DISCERN scores for videos uploaded in Spanish. Notably, no video achieved the maximum possible scores for JAMA or GQS. When the scores of the videos were stratified according to typology, we found differences amongst upload sources to be statistically significant, with those uploaded by professional organizations having the highest mean rank scores. Professional videos had the highest mean rank for JAMA score and GQS, while the highest mean rank for DISCERN scores was recorded for ‘health portal’ videos. When stratifying the content of videos by typology, we found that roughly half of the videos uploaded by a ‘professional’ typology discussed treatment modalities and one-quarter of the videos spoke about the signs and symptoms found in indiviuals with breast cancer. The remaining content categories, including screening, risk factors, diagnosis, advocacy, breast cancer in men, disease pathophysiology, and types and stages, were addressed in only a small percentage of videos. However, this topic was found in higher frequency in other video typologies of lower quality. We also found that only approximately 10% of videos were uploaded from a source in the USA, with Spain and Mexico generating most of the videos.

Several limitations of our study exist. Firstly, our search strategy (“cancer de mama”) may not capture what patients search for on YouTube for information. Secondly, we only examined 200 YouTube videos, despite thousands of videos resulting when our search strategy was employed. This likely excluded mostly videos classified as ‘personal’. Other limitations of our study include the lack of data collection pertaining to the average number of views per video and the specific source of content provided. Lastly, two reviewers reviewed the videos. While there was only a minimal discrepancy in scoring between the two viewers, including additional viewers may have added to the validity of the scores given to each video.

Conclusion

Our study aimed to evaluate the quality and reliability of breast cancer videos produced in Spanish and found on YouTube, the most popular social media platform, with over 2 billion active users. We found that the overall quality and reliability of videos in Spanish were poor, and those of higher quality did not reliably cover some of the subjects. This is particularly important to federal agencies such as the National Institutes of Health and the National Cancer Institute, and academic institutions, where efforts have been made to recruit Hispanic patients with cancer and reduce disparities. The importance of social media platforms in the Hispanic community should not go unrecognized. Providing high-quality and reliable health information through videos targeting Hispanic people can be a helpful modality in strengthening the recruitment of these individuals to educate them on their illness so that they can ultimately make more informed decisions with their physician. Considering the growing Hispanic demographic in the USA and worldwide, hospitals and specialists should consider producing educational videos to increase the number of high-quality videos found online and dilute those promoting scientifically unsubstantiated information. A mechanism to achieve this may be via a peer-review process, which is employed in the publishing of patient information by Cancer.Net, the American Society of Clinical Oncology’s patient information website.³⁸

The lack of high-quality videos covering these topics from a reputable source is concerning. We believe false or misleading information in all subjects, especially those not broadly covered, contributes to public misperception and may result in ill-informed and potentially harmful decision-making. The importance of accurate and reliable information becomes especially relevant in socioeconomically disadvantaged groups with inadequate health literacy and who may rely heavily on video media for their medical information. For videos published by non-professional sources, the HONcode is a certification system that exists and could be utilized by YouTube. The HONcode is a non-governmental organization with a code of ethics that guides website managers on how best to disseminate reliable and high-quality health information and provides yearly assessments for websites seeking its certification.³⁹

Moving forward, major academic institutions and national organizations should conduct a comprehensive needs assessment to ensure accurate information is thoroughly presented for patients and their families.