Critical Assessment of Internet Medical Information (Tech Topic #10)
Tech Topic #10:
Critical Assessment of Internet Medical Information
Purpose
This Tech Topic reviews various initiatives that address the quality of health information found on the Internet. This material can be useful both for evaluating different health-related Internet resources and for updating your own Web sites to ensure that they meet modern quality criteria.
Contents:
- Introduction
- 1. Health on the Net Foundation
- 2. Internet Healthcare Coalition/ eHealth Code of Ethics
- 3. Criteria for Accessing the Quality of Health Information on the Internet
- 4. Guidelines for Medical and Health Information Sites on the Internet. Principles governing AMA Web sites
- 5. EU - Quality Criteria for Health Related Web-sites
- 6. MedCERTAIN
- 7. Intute
- 8. DISCERN
- Additional resources
- References
- Checklist
Many physicians report that Internet health information can affect their clinical decisions. Access to health information on the Internet has also revolutionized how medical patients learn about their illnesses. Valuable information can be found online; however, many health Web sites contain inaccurate or misleading information. Here are a few examples from recent research:
- HIV-AIDS patients' evaluation of health information on the Internet: the digital divide and vulnerability to fraudulent claims. Participants were less critical of health information found online than medical professionals and made smaller distinctions between high-quality and low-quality information. Assigning credibility to low-quality information was predicted by lower incomes and educational attainment, poorer reading comprehension, lower literacy levels, and irrational health beliefs (1)
- Quality of Internet Geriatric Health Information: The Geriatric Web Project. A lot of Internet geriatric health information (IGHI) is not controlled for quality, and there is no available process to assess the quality of IGHI used in clinical practice or in medical education. Much publicly available IGHI is of suboptimal quality, particularly with regard to credibility and educational domains. Healthcare professionals need to be aware of how to locate high-quality IGHI and how to assess the quality of available IGHI. (2)
- Portals to Wonderland: health portals lead to confusing information about the effects of health care. Information accessible through health portals is unlikely to be based on systematic reviews and is often unclear, incomplete and misleading. Portals are only as good as the Web-sites they lead to. (3)
- Reporting on post-menopausal hormone therapy: an analysis of gynecologists' web pages. The study was designed to analyze Web pages of German gynecologists with regard to postmenopausal hormone therapy (HT). There is a growing body of evidence that the overall health risks of HT exceed the benefits. Making one's own informed choice has become a central concern for menopausal women. The study focused on the analysis of basic criteria such as the "last modified" date and quality of HT information content. Evidence-based information resulting from the Women Health Initiative trial was insufficiently represented on gynecologists' Web pages. Web pages of gynecologists do not appear to be recommendable for women because they do not provide recent evidence-based findings about HT. (4)
As a solution for these problems different organizations have tried to establish Codes of conduct that address quality of health information. These are based on principles of ethical behavior and sets of quality criteria. Codes of conduct rely on self-certification by participating Web sites. Another approach is third-party certification - "this requires recurrent validation of compliance with a set of standards. These standards may or may not be based on some of the codes of conduct and ethics. In all cases, third-party certification requires payment of fees to the certifying company" (5). The third approach is a "tool-based evaluation. This is mostly based on a predefined questionnaire that would yield a certain "quality score" for the content under evaluation. Tool-based evaluation is primarily intended for use by citizens, who would invoke the particular tool to assess the quality of a given site" (5).
1. Health on the Net Foundation
URL: http://www.hon.ch/HONcode/
The Health on the Net (HON) Foundation in Geneva Switzerland is a not-for-profit organization established in 1995, funded primarily by the State of Geneva and the Geneva Ministry of Health. The HON Code is one of the first health-related quality initiatives on the Internet.
The HON Code sets 8 principles for basic ethical standards for the health Internet (available in 27 languages):
1. Authority - Any medical or health advice provided and hosted on this site will only be given by medically trained and qualified professionals unless a clear statement is made that a piece of advice offered is from a non-medically qualified individual or organization;
2. Complementarity - The information provided on this site is designed to support, not replace, the relationship that exists between a patient/site visitor and his/her existing physician;
3. Confidentiality - Confidentiality of data relating to individual patients and visitors to a medical/health Web site, including their identity, is respected by this Web site. The Web site owners undertake to honor or exceed the legal requirements of medical/health information privacy that apply in the country and state where the Web site and mirror sites are located;
4. Attribution - Where appropriate, information contained on this site will be supported by clear references to source data and, where possible, have specific HTML links to that data. The date when a clinical page was last modified will be clearly displayed (e.g. at the bottom of the page);
5. Justifiability - Any claims relating to the benefits/performance of a specific treatment, commercial product or service will be supported by appropriate, balanced evidence in the manner outlined above in Principle 4;
6. Transparency of authorship - The designers of this Web site will seek to provide information in the clearest possible manner and provide contact addresses for visitors that seek further information or support. The Webmaster will display his/her E-mail address clearly throughout the Web site;
7. Transparency of sponsorship - Support for this Web site will be clearly identified, including the identities of commercial and non-commercial organizations that have contributed funding, services or material for the site;
8. Honesty in advertising & editorial policy - If advertising is a source of funding it will be clearly stated. A brief description of the advertising policy adopted by the Web site owners will be displayed on the site. Advertising and other promotional material will be presented to viewers in a manner and context that facilitates differentiation between it and the original material created by the institution operating the site;
Sites that conform to those 8 principles are allowed to display the active HON Code logo on their pages. The HON Code logo can be found on more than 3,000 health-related Web sites. HON does try to police the use and abuse of its logo through the following mechanisms:
- An alert of breach is sent to the provider
- A warning is issued to the offending site
- Removal of the live link between the HON logo on the provider site and the HON site
HON also provides guidelines that explain the applications of each of its principles.
The HONcode is not an award system, nor does it intend to rate the quality of the information provided by a Web site. It only defines a set of rules to:
- Hold Web site developers to basic ethical standards in the presentation of information;
- Help make sure readers always know the source and the purpose of the data they are reading
2. Internet Healthcare Coalition/ eHealth Code of Ethics
URL: http://www.ihealthcoalItion.org/
The mission of the Internet Healthcare Coalition is Quality Healthcare Resources on the Internet. In pursuit of this mission, the Internet Healthcare Coalition strives for the following:
- Well-informed Internet healthcare consumers, professionals, educators, marketers, and both healthcare and mainstream media, as well as public policymakers with regard to the full range of uses of the Internet - current and potential - to deliver high-quality healthcare information and services.
- A self-regulated Internet in which voluntary guidelines provide effective means for the legitimate dissemination of accurate healthcare information.
- A nurturing online community that promotes ethical, innovative, and high-quality resources of healthcare information and services.
On its Web site you can find information on the Coalition's initiatives, meetings, conferences, articles and presentations, and tips for consumers "Finding Quality Health Information on the Internet" (URL: http://www.ihealthcoalition.org/content/tips.html)
The goal of the eHealth Code of Ethics (http://www.ihealthcoalition.org/ethics/ehealthcode0524.html) is to ensure that people worldwide can confidently and with full understanding of known risks realize the potential of the Internet in managing their own health and the health of those in their care. Anyone who uses the Internet for health-related reasons has a right to expect that organisations and individuals who provide health information, products or services online will uphold the following guiding principles:
Candor - Disclose information that if known by consumers would likely affect consumers' understanding or use of the site or purchase or use of a product or service.
Honesty - Be truthful and not deceptive
Quality - Provide health information that is accurate, easy to understand, and up to date. Provide the information users need to make their own judgments about the health information, products, or services provided by the site.
Informed Consent - Respect users' right to determine whether or how their personal data may be collected, used, or shared.
Privacy - Respect the obligation to protect users' privacy.
Professionalism in Online Health Care - Respect fundamental ethical obligations to patients and clients. Inform and educate patients and clients about the limitations of online health care.
Responsible Partnering - Ensure that organizations and sites with which they affiliate are trustworthy.
Accountability - Provide meaningful opportunity for users to give feedback to the site.
3. Criteria for Accessing the Quality of Health Information on the Internet (Health Information Technology Institute of Mitretek Systems, Inc.)
URL: http://hitiweb.mitretek.org/docs/criteria.html
URL: http://hitiweb.mitretek.org/docs/policy.html
Originally funded by Mitretek Systems Inc., this IQ Tool is one of the earliest tools-based scoring methods for assessing the quality of health Web sites. (Last updated - May 1999). The ultimate aim of this project was to improve the quality of health information on the Internet. The goals were to develop criteria to assess the quality of health information on the Internet and to educate the consumer, the content provider, and the policymaker on these criteria.
Identified criteria fall into seven broad categories:
Credibility: includes the source, currency, relevance/utility, and editorial review process for the information.
Content: must be accurate and complete, and an appropriate disclaimer provided.
Disclosure: includes informing the user of the purpose of the site, as well as any profiling or collection of information associated with using the site.
Links: evaluated according to selection, architecture, content, and back linkages.
Design: encompasses accessibility, logical organization (navigability), and internal search capability.
Interactivity: includes feedback mechanisms and means for exchange of information among users.
Caveats: clarification of whether site function is to market products and services or is a primary information content provider.
4. Guidelines for Medical and Health Information Sites on the Internet. Principles governing AMA Web sites
URL: http://www.ama-assn.org/ama/pub/category/1905.html
To provide guidance for all Web sites of the American Medical Association, the Guidelines for the AMA Web Sites address the creation and updating of content, acquisition and posting of advertising, the preservation of privacy and confidentiality, and the provision of reliable and efficient e-commerce. Principles to govern the presentation and functionality of the 4 major areas for which quality standards were needed: content, advertising and sponsorship, privacy and confidentiality, and e-commerce.
Principles for content include: Site Ownership; Site Viewing; Viewer Access, Payment, and Privacy; Funding and Sponsorship; Quality of Editorial Content; Review (of content before posting); Date of Posting, Revising, and Updating and Timeliness of Editorial Content; Sources of Editorial Content; Linking; Intersite Navigation; Downloading Files; Navigation of Content.
Principles for advertising and sponsorship state that "Digital advertisements must be readily distinguishable from editorial content. If the distinction is unclear, the word "advertisement" should be added. All financial or material support for electronic collections of articles, Web site content, and other types of online products (such as condition-specific Web sites, online databases, or material on CD-ROM) will be acknowledged and clearly indicated on the home screen or via a link from the home screen".
Principles for Web-site privacy and confidentiality: "All site visitors should have the opportunity to opt in or out of allowing personal information to be tracked. The AMA takes extensive measures to ensure the safety and security of its Web site servers and to guard against divulging private information. The AMA believes that Web site visitors should know who (eg, the site organization or third party) is tracking personal information and the types of personal information that are tracked and should have the right to opt out of such information being collected at any time".
Principles for e-commerse are intended to ensure that users and purchasers of information, products, and services on the site will have access to secure, efficient transactions for online and remote customer fulfillment.
5. European Commission - Quality Criteria for Health Related Web-sites
URL: http://europa.eu.int/information_society/eeurope/ehealth/quality/draft_guidelines/index_en.htm
The objective of quality criteria for health related Web-sites are "to boost consumer confidence in use of such sites and foster best practice in the development of sites. The primary objective of the eEurope initiative in this field is to provide a common basis from which Member States can develop quality assurance tools at national or regional level."
Draft Guidelines include the following criteria:
- Transparency and Honesty
- Authority
- Privacy
- Currency
- Accountability
- Accessibility
URL: http://www.medcircle.org/
MedCIRCLE consists of a consortium of three European health portals in Spain, France, and Germany, which are already active in the field of rating/evaluating/accrediting health websites. MedCIRCLE builds on, expands and continues work on rating health information on the Internet piloted within the MedCERTAIN project (2000 - 2002). Both projects - MedCIRCLE and MedCERTAIN - are complementary semantic web projects with the overall objective to develop and promote technologies able to guide consumers to trustworthy health information on the Internet, to establish a global web of trust for networked health information, and to empower consumers to "filter" or positively select high quality health information on the Web.
MedCERTAIN defined 4 levels for the award of a trust mark: Level I: Transparency Mark (self-certification); Level II: Verification of Level I claims and formal assessment of the Web site by professional volunteers based on the quality criteria; Level III: Third-party assessment and rating of content; Level IV: Outcome evaluation
The MedCERTAIN approach combined aspects of consumer education, helping and stimulating information providers to implement best practice guidelines, encouraging self-labeling by information providers, external evaluations and monitoring. To obtain a "level-1" MedCERTAIN transparency mark, the information provider has to disclose certain information in a standardised, computer-readable format using the "Health Information Disclosure, Description and Evaluation Language" (HIDDEL, formerly known as medPICS) expressed in XML/RDF. The same language can be used by gateways to express opinions (annotations) of evaluators about other sites.
The MedCIRCLE project expands the self description with a metadata concept and a third-party rating concept by actively involving existing subject gateways (portal sites/rating services) across Europe, stimulating the implementation of the meta-data vocabulary, demonstrating how interoperability and data exchange between heterogeneous subject gateways/rating services can be achieved, and how decentralised evaluations can be performed and harvested.
MedCIRCLE aims to encourage other health gateways, portal sites, and rating services to join the collaboration by implementing the metadata vocabulary "HIDDEL (Health Information Disclosure, Description and Evaluation Language)". It is hoped that MedCIRCLE provides a stimulus for other gateways and accreditation bodies to join the collaboration by implementing HIDDEL, thereby weaving a global, collaborative, open semantic web of trust for health information.
7. Intute: Health and Life Sciences
URL: http://www.intute.ac.uk/healthandlifesciences/
A free catalogue of hand-selected and evaluated Internet resources in Health and Medicine. Its objective is to provide access to evaluated, quality Internet resources in the health and life sciences, aimed at students, researchers, academics, and practitioners. This service was formerly known as OMNI. All of the resources have been evaluated according to the BIOME Evaluation Guidelines:
URL: http://www.intute.ac.uk/healthandlifesciences/BIOME_Evaluation_Guidelines.doc
URL: http://www.discern.org.uk/
DISCERN is funded by UK National Health Service Executive Research and Development Programme. The DISCERN Project is based in the University of Oxford, Division of Public Health and Primary Health Care, at the Institute of Health Sciences.
The DISCERN tool is a brief questionnaire which provides users with a valid and reliable way of assessing the quality of written information on treatment choices for a health problem. DISCERN can also be used by authors and publishers of information on treatment choices as a guide to the standard which users are entitled to expect. Its two main objectives are: to enable consumers to judge the quality of written information on treatment choices; and to facilitate the production of high quality evidence-based patient information
The DISCERN Web site is an experimental site consisting of an online version of the DISCERN instrument and handbook. The handbook is designed to help you understand and use DISCERN effectively. It has been written from the perspective of a health consumer (or patient), but can be used by anyone interested in information about treatment choices. DISCERN was initially developed for use with consumer health information in conventional print media. However, DISCERN may also be a useful tool for assessing a broad range of consumer health information on the Internet.
DISCERN consists of 15 key questions plus an overall quality rating. Each of the 15 key questions represents a separate quality criterion - an essential feature or standard that is an important part of good quality information on treatment choices.
The questions on DISCERN are organized in three sections as follows:
Questions 1 - 8 address the reliability of the publication and should help you consider whether it can be trusted as a source of information about treatment choices
Questions 9 - 15 focus on specific details of the information about treatment choices. Questions 9 to 11 are concerned with the ‘active' treatments described in the publication and can include self-care. ‘No treatment' options are dealt with separately in Question 12
Question 16 is the overall quality rating at the end of the instrument. Your answer to this question should be based on your judgement of the quality of the publication as a source of information about treatment choices after rating each of the 15 preceding questions. However, you should only rate a publication as good quality if it rated well on the majority of questions.
How to evaluate the quality of health related websites. Gattoni F, Sicola C. Radiol Med (Torino). 2005 Mar;109(3):280-7. (In English and Italian)
URL: http://www.minervamedica.it/index2.t?show=R24Y2005N03A0280
Authors describe main criteria for quality of Web-sites that are related to its content and usability. They make the conclusion "that it is unnecessary to apply strict rules to medical Web site developers. We want to stress the importance of guidelines and recommendations to be modified with the development of web technology and the cultural evolution of patient and physicians. In the near future the presence on the Internet of websites certified by national or international medical web authorities will lead users to trust and give their preference to such sites, leading to the self-regulation of website developers and users."
Review of Internet Health Information Quality Initiatives. Risk A., Dzenowagis J. Journal of Medical Internet Research. Vol. 3, Issue 4, Article e28
URL: http://www.jmir.org/2001/4/e28/
A review of the major self-regulatory initiatives in the English-speaking world to develop quality and ethical standards for health information on the Internet. The review compares and analyzes the approaches taken by the different initiatives and clarifies the issues around the development and enforcement of standards. The initiatives were compared in 3 ways: (1) Analysis and comparison of: key concepts, mechanism, or approach. Analysis of: the obligations that a provider has to meet to comply with the given initiative, the intended beneficiaries of that initiative, and the burdens imposed on different actors. These burdens are described in terms of their effect on the long-term sustainability and maintenance of the initiative by its developers. Analysis of the enforcement mechanisms. (2) Analysis and comparison by type of sponsoring organization, the reach of the initiative, and the sources of funding of the initiative or the sponsoring organization. (3) How the various initiatives fall under 1 of 3 key mechanisms and comparison of the advantages and disadvantages of these key mechanisms
(1) Benotsch EG, Kalichman S, Weinhardt LS. HIV-AIDS patients' evaluation of health information on the Internet: the digital divide and vulnerability to fraudulent claims. J Consult Clin Psychol. 2004 Dec;72(6):1004-11.
URL: http://content.apa.org/journals/ccp/72/6/1004
(2) Hajjar I, Gable SA, Jenkinson VP, Kane LT, Riley RA. Quality of Internet Geriatric Health Information: The GeriatricWeb Project. J Am Geriatr Soc. 2005 May;53(5):885-90
URL:http://www.blackwell-synergy.com/openurl?genre=article&sid=nlm:pubmed
&issn=0002-8614&date=2005&volume=53&issue=5&spage=885
(3) Glenton C, Paulsen EJ, Oxman AD. Portals to Wonderland: health portals lead to confusing information about the effects of health care. BMC Med Inform Decis Mak. 2005 Mar 15;5(1):7.
URL: http://www.pubmedcentral.nih.gov/
articlerender.fcgi?tool=pubmed&pubmedid=15769291
(4) Bucksch J, Kolip P, Deitermann B. Reporting on post-menopausal hormone therapy: an analysis of gynaecologists' web pages. Med Inform Internet Med. 2004 Sep-Dec;29(3-4):211-20
URL: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi
?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=15742988&query_hl=1
(5) Risk A., Dzenowagis J. Review of Internet Health Information Quality Initiatives. Journal of Medical Internet Research. Vol. 3, Issue 4, Article e28
URL: http://www.jmir.org/2001/4/e28/
L. Larsson. Quality of Information On and Off the Web - A Checklist
http://courses.washington.edu/~hs590a/modules/33/quality33.html
| Characteristics | 1 | 2 | 3 | 4 |
|
ESSENTIAL ELEMENTS |
||||
| A URL is on each page (usually on bottom of page) |
||||
| A link to a HOME page or document is on the site | ||||
| Feedback mailto: link to Webmaster on each page for comments on the site (usually on bottom of page) AND/OR mailto: link to authors for comments on the content, if authors are not the Webmaster |
||||
| TITLE of document is accessible (look in upper left hand corner of Netscape window) |
||||
| DATE of last revision AND/OR Site provides DATE of site/document creation |
||||
| LOGO or other identifying information (institution name, company, etc.) | ||||
| CONTACT INFORMATION name of organization mailing address phone/fax/contact email | ||||
| Site specific/internal SEARCH ENGINE available if the site is large | ||||
| SITE MAP available | ||||
|
CONTENT ELEMENTS |
||||
Type of Site
|
||||
| Accuracy The information is correct/accurate AND its accuracy can be verified. |
||||
| If there are limitations to the data or content, such as scope, currency, statistical methodology, etc., they are listed.
There are NO spelling errors, grammatical errors, or miscellaneous typos. |
||||
| It is clear WHO is responsible for the accuracy of the content. | ||||
| Accessibility Is the reading level appropriate for the audience? |
||||
| The graphics used are appropriate and load quickly. | ||||
| Graphics are labeled with alternative text. | ||||
| If the page audience speaks a language other than English, is that language being used? | ||||
| Hierarchy of Evidence Does the document include a bibliography? |
||||
| Does document rely on other sources that are NOT in the bibliography? | ||||
| Review Process Is there any indication of review of content by: subject peers or experts (academic peer review) librarians/other experts in evaluating content panel awarding "best of type" awards? |
||||
|
DISCLOSURE |
||||
|
Purpose |
||||
|
Site Purpose: educational informational/explanatory promotional advocacy other |
||||
| Does the information on the site REFLECT the MISSION or PURPOSE? | ||||
| Profiling / Intelligence gathering Does the site COLLECT INFORMATION from you in the form of cookies, surveys, chat rooms, how you use the information on the site, or other intelligence gathering methods? AND Can you REFUSE to provide this information and still get access to the content on the site? |
||||
| IN GENERAL, the information on this site is: interesting, but not immediately useful interesting and useful worth bookmarking worth adding to my personal list of "Great Sites to Revisit" Wow, can I ever use this, right now! of NO interest whatsoever what a dog! | ||||
|
OVERALL GUT LEVEL RATING: |
Compiled by:
Irina Ibraghimova
Coordinator, Medical Information Resources
American International Health Alliance
ibra@zadar.net
Updated: October 25, 2006
