Reporting Items for Updated Clinical Guidelines: Checklist for the Reporting of Updated Guidelines (CheckUp)

Pablo Alonso-Coello and colleagues describe a checklist to guide the reporting of clinical guideline updates.

Published in the journal: . PLoS Med 14(1): e32767. doi:10.1371/journal.pmed.1002207
Category: Guidelines and Guidance
doi: https://doi.org/10.1371/journal.pmed.1002207

Summary

Pablo Alonso-Coello and colleagues describe a checklist to guide the reporting of clinical guideline updates.

Background

Trustworthy clinical guidelines aim to assist decision making by providing recommendations that are informed by the best available evidence and include an assessment of the benefits and harms of alternative care options [1,2]. Because of the continuous emergence of new research evidence (i.e., changes in available interventions, effects, or cost) [3], appropriate updating to maintain the trustworthiness of clinical guidelines is challenging since it requires regular surveillance and reviewing of the new evidence [4,5].

Updating clinical guidelines is a process that includes different stages: (1) prioritisation of candidate guidelines or recommendations to update [6], (2) identification of new scientific evidence [3,6–8], (3) assessment of the need to update [3,6,9], (4) updating the recommendations [6,10–12], and (5) publication of the updated guideline [6,13]. However, there is no consensus about what is the optimal methodology to operationalise each of these steps or how to report on the process [5,14,15,16]; the available guidance from guideline institutions is suboptimal [17,18].

Trustworthiness standards for guidelines have been published by both the Institute of Medicine (IOM) and the Guidelines International Network (G-I-N) [1,2]. Additionally, instruments are available for assessing the quality of clinical guidelines, such as the Appraisal of Guidelines for Research and Evaluation (AGREE) II Instrument [19], while others, such as the GIN-McMaster Guideline Development Checklist [20], support developing and implementing trustworthy clinical guidelines. However, guideline updating requires some different methodological considerations and unique communication procedures. Currently, none of the existing tools address these issues.

To address this gap, in a partnership of the Iberoamerican Cochrane Centre (www.cochrane.org), the AGREE Collaboration (www.agreetrust.org), and the G-I-N Updating Guidelines Working Group (www.g-i-n.net/working-groups/updating-guidelines), we have developed the Checklist for the Reporting of Updated Guidelines (CheckUp). This article about CheckUp is targeted at guideline developers and users of guidelines. In the article, we present the methodology of the development process and the final checklist. In a supplementary file, we present explanations and examples for each item (S1 Appendix).

Methodology

For reporting the development process of CheckUp, we followed Enhancing the QUAlity and Transparency Of health Research(EQUATOR) and Moher’s criteria [21,22]. The development of CheckUp consisted of four phases: (1) panel selection, (2) generation of the initial checklist, (3) optimisation of the checklist, and (4) approval of the final checklist (Fig 1).

**Fig. 1. Checklist development process.**

Panel Selection

To advise on the development of the CheckUp, a panel comprising individuals with relevant experience in clinical guideline development and updating and/or in systematic reviews/guidelines research methodology was convened. Invited panel participants were identified based on a review of the main authors in the field, as well as the AGREE Trust (www.agreetrust.org) and the G-I-N (www.g-i-n.net) members. The purpose of the panel was to provide expert advice during the development process and to participate in the Delphi survey. A core group (RWMV, PAC, MB, and LMG) was established to design the protocol and provide more time-sensitive and operational advice.

Generation of the Initial Checklist

The core group first developed an initial list of items—including explanation and examples—through brainstorming and discussion, taking into account (1) research evidence in the field [15,16,18], (2) the AGREE II Instrument [19], and (3) the panel experience. We used three core updating publications as a starting point from which the initial version of the checklist was generated [15,16,18]. These studies include an overview of the available guidance from guideline methodological handbooks [18], a systematic review of the published methodological research [16], and an international survey about the experiences of the main guideline developers [15].

Optimisation of the Checklist

We optimised the initial checklist through a multistep process that included an assessment of existing updated clinical guidelines, semistructured interviews, a Delphi consensus survey, and an external review with clinical guideline methodologists and users (Fig 1).

Assessment of updated clinical guidelines

We piloted the initial checklist among a convenience sample of updated clinical guidelines to assess the used terminology, to identify missing items, and as a first step to explore its face validity. We included updated clinical guidelines that were (1) developed by G-I-N members, (2) published in English or Spanish, and (3) published between 2011 and 2013. We searched the G-I-N library (www.g-i-n.net) and the National Guideline Clearinghouse (www.guidelines.gov). Two reviewers (RWMV and SPS) applied the checklist, solving disagreements by consensus. The core group discussed the results and refined the initial list of items.

Semistructured interviews

To refine the checklist and to identify missing items, we conducted semistructured interviews with clinical guideline experts. We chose a convenience sample of participants, outside the CheckUp panel, with (1) experience in updating clinical guidelines, defined as having participated in the updating process of at least one clinical guideline over the past year, and (2) fluency in English. We identified the participants by contacting professionals associated with G-I-N or researchers in the field. When someone did not respond or could not participate, a new person was recruited. We continued to recruit participants and collect data until data saturation was achieved.

In each interview, participants were asked about their experiences and challenges in updating clinical guidelines. Subsequently, the participant was prompted by the interviewer (RWMV) to reflect on the strengths, weaknesses, missing concepts, and redundancies in the checklist. The interviews were audiotaped, and key themes were identified. The core group discussed the results and refined the list of items.

Delphi consensus survey

We reviewed the refined list of items through a Delphi consensus survey [23], with all members of the CheckUp panel. The Delphi participants assessed the inclusion, comprehensiveness, clarity, and coverage of each item and tried to identify potentially additional items for the checklist.

Using a seven-point Likert scale (one meaning strongly disagree and seven meaning strongly agree) [24], we asked participants to rate whether the item should be included in the checklist. For each item, participants were asked whether their perceptions of (1) the completeness, (2) the usability, and (3) the quality of a clinical guideline would be influenced if the item was reported. We included a free text box for suggestions to modify the items, the explanation, or the examples. We used online software to design the survey and to collect the responses (www.surveymonkey.com).

We calculated the median score for inclusion, completeness, usability, and quality for each item and classified them into (1) items with a median score of 0 to 3 points, which were excluded; (2) items with a median score of 4 to 5 points or with substantial comments that needed important revision, which were retained, modified, and further tested; and (3) items with a median score of 6 to 7 points and without substantial comments, which were included and not evaluated further in the following rounds.

One reviewer (RWMV) analysed the quantitative and qualitative results and suggested potential solutions. The core group discussed the results and potential solutions and refined the list of items accordingly. We continued with additional rounds until consensus for inclusion or exclusion was reached and no more relevant comments were provided.

External review with clinical guideline methodologists

To evaluate the usability of the checklist, we conducted a survey with clinical guideline methodologists who had experience in updating clinical guidelines, as measured by having participated in the updating process of at least one clinical guideline over the past year. We also invited all of the G-I-N institutional member contacts to participate in the external review. If the contact person was not able to participate, we asked them to provide contact details of another expert working at the same institution.

Using a seven-point scale (one meaning strongly disagree and seven meaning strongly agree), we asked participants to rate the usability of each item and its influence on the confidence in an updated clinical guideline if the item was reported. A free text option was included for suggestions to modify the items, the explanation, or the examples. We used online software to design the survey and to collect the responses (www.surveymonkey.com).

We calculated the median score for usability and confidence for each item. One reviewer (RWMV) analysed the quantitative and qualitative results and suggested potential solutions. The core group discussed the results and potential solutions and refined the list of items accordingly.

External review with clinical guideline users

We conducted semistructured interviews with clinical guideline users to evaluate the usability of the checklist. We engaged individuals who were (1) health care professionals who used clinical guidelines in clinical practice and (2) located in Canada, Spain, or the Netherlands. We identified the participants with the help of the panel members. When someone did not respond or could not participate, a new person was recruited. We continued to recruit participants and collect data until the information was repeated and no new information emerged (data saturation).

For each interview, participants were asked whether reporting of the item in an updated clinical guideline would increase their confidence in the guideline. The participant and interviewer (RWMV) reviewed the checklist, and the participants were prompted to consider missing concepts, redundancy, and the usability of the checklist. The interviews were audiotaped, and key themes were identified. The core group discussed the results and refined the list of items.

Approval of the final checklist

The checklist was presented and discussed in a workshop at the 2015 G-I-N Conference in Amsterdam. In this workshop, we asked the participants whether the checklist was deemed adequate for assessment of the updated clinical guidelines [25]. We also asked the participants to give an overall impression of the checklist. The core group discussed the results and agreed on the final list of items.

Results

CheckUp Panel

Fifty-six potential individuals were invited to be part of the CheckUp Panel. In total, 33 professionals, 20 males and 13 females, (response rate: 58.9%, 33/56) confirmed their participation (17 from Europe, 9 from South America, 5 from North America, and 2 from Oceania). The primary role of the panellists was health care researcher (60.6%; 20/33), guideline developer (30.3%; 10/33), and clinical guideline user (9.1%; 3/33).

Generation of the Initial Checklist

The initial checklist included 13 items within the following domains: updating rationale, scope and purpose of the updated clinical guideline, participants in the updating panel, conflicts of interest, updating methodology, differentiating original and new information, and reasons for the changes in the recommendations.

Optimisation of the Checklist

Assessment of updated clinical guidelines

Initially, we assessed a convenience sample of ten updated clinical guidelines from the G-I-N library with the initial checklist [26–35]. The items more frequently reported in the included clinical guidelines were related to the literature search strategy (60%), the composition of the panel (50%), and the external review (40%). The other checklist items (e.g., assessment for the need of updating, evidence selection, rationale for updating, and rationale for changes) were reported in less than 20% of the included clinical guidelines. No additional concepts or items emerged (Table 1).

**Tab. 1. CheckUp: Stages of the optimisation process (objective, sample, and results by optimisation processes).**

Semistructured interviews

We conducted semistructured interviews with clinical guideline developers (5 from Europe, 5 from North America, and 3 from South America). In total, we interviewed 13 participants, at which point saturation was reached. As a result, we modified five items and added four new ones. The modifications were related to (a) differences in the objectives, purpose, or aim between the original and updated version; (b) the identification of new evidence; (c) the rationale for changing recommendations; and (d) the funding. The new items were related to (a) the scope of the update (partial or complete), (b) the target audience, (c) the changes in the recommendations, and (d) the plans and methodology reported to update the clinical guideline (Table 1).

Delphi consensus survey

All the members of the CheckUp panel (n = 33) were invited to participate in the Delphi consensus survey. Twenty-seven (82%) members participated in the first Delphi round, thirty-one members (93.9%) in the second Delphi round, and all (100%) members in the third and final round.

In the first round, the participants provided substantial feedback on various items, their explanation, and the accompanying examples. This feedback triggered modification in the order of the items, phrasing of the items, explanations, and examples (Table 1). All items met the inclusion criteria, and no major comments were reported. The median score for whether the participants believed that an updated clinical guideline would be more complete, usable, and of higher quality whenever the item was reported was six for all questions.

In the second round (n = 17 items), the amount of feedback was substantially smaller than in the first round (Table 1). We merged some items, and the checklist was reduced to 16 items. Again, all items met the inclusion criteria, and no major comments were reported. The median score for whether the participants believed the clinical guideline would be more complete, usable, and of higher quality was 7.0, 6.0, and 6.0, respectively.

In the third and last round, a general consensus was reached for all items, explanations, and examples. The median score for item inclusion, completeness, usability, and quality was ≥6 in all items (except for two items with a median score of 5.5 in the usability and quality question) (Table 2).

**Tab. 2. Results of the Delphi survey (third round).**

External review

External review with clinical guideline methodologists. We conducted a survey with 53 clinical guideline methodologists (53/59, response rate 90%). The median scores of usability and confidence for each item were ≥6 (Table 3). Participants provided comments that improved the writing style of the items, explanations, and examples (Table 1).

External review with clinical guideline users. We had conducted semistructured interviews with 10 clinical guideline users (3 from Spain, 2 from the Netherlands, and 5 from Canada) when saturation was reached. All participants acknowledged that all items were useful to evaluate the reporting of the updating process in updated clinical guidelines. Neither new items nor modifications were proposed (Table 1).

Final Checklist

The checklist includes 16 items that can be broadly categorised into three themes: (1) presentation (e.g., clinical guideline sections and recommendations), (2) editorial independence (e.g., the working group and funding), and (3) the methodology used (e.g., search strategy and evidence synthesis) (Table 4). Those attending the presentation of the checklist workshop at the G-I-N 2015 conference reviewed and agreed with the final version of the checklist.

Discussion

We developed CheckUp through a comprehensive development process, including the use of systematic reviews, assessment of updated clinical guidelines, and engagement of the international guideline community through semistructured interviews, a Delphi consensus survey, and an external review.

Main Findings

Across the different processes, an alignment and consensus of opinion emerged between what was documented in the literature and the expectations of clinical guideline developers, users, and researchers in regards to what information ought to be reported in an updated clinical guideline. CheckUp includes 16 items regarding the presentation of the updated clinical guideline, editorial independence, and the methodology used in the clinical guideline updating process.

CheckUp was primarily developed to evaluate the completeness of reporting in updated guidelines. Additionally, the tool can inform guideline developers about strategies for updating clinical guidelines and their reporting requirements. An explanation and elaboration article for the CheckUp is published as a supporting information article (S1 Appendix). CheckUp can be used in several ways. The checklist can provide guidance to developers who update clinical guidelines, by providing methodological principles that should be incorporated into the updating process, as well as strategies for reporting this information. The checklist can be applied by users or appraisers of clinical guidelines to assess whether updated clinical guidelines align with the CheckUp items. We suggest that a minimum of two reviewers assess the reporting of the guideline updating process independently, with the help of a third reviewer if there is a need of reaching consensus.

Our Results in the Context of Previous Research

Updating is a crucial part of maintaining the trustworthiness of clinical guidelines [1,2]. Since clinical guidelines have a limited lifespan, updating clinical guidelines is crucial to maintain the validity of the recommendations [3,9,36,37]. Although the importance of regular updating has been recognised and clinical guidelines may have an “expiration date,” little research has been conducted in the field so far [13,15–18]. Published standards for trustworthy guidelines require the description of updating plans [1,2]; however, these standards do not provide specific guidance about the detailed reporting of the updating process of guidelines.

Strengths and Limitations

Our CheckUp proposal has several strengths. For the development process, we systematically reviewed the evidence and followed EQUATOR and Moher’s criteria [21,22]. Also, by applying a formal consensus method (Delphi survey) and collecting experts’ opinions (semistructured interviews and external reviews), we reached a fair understanding of clinical guideline methodologists’ and users’ perceptions about the updating of clinical guidelines. Finally, there was fairly strong overall consensus during the development of CheckUp.

Our study has some limitations. We used consensus methods and convenience samples of clinical guideline stakeholders. However, across the different processes, an alignment and consensus of opinion emerged on what clinical guideline developers, users, and researchers expect to see reported in updated clinical guidelines. Another potential limitation is that CheckUp includes some items that may partially overlap with some items that are present in other instruments [19,20]; however, we think this is a minor limitation as CheckUp differs for the most part and has a very specific and differentiated goal. Finally, we did not collect potential conflicts of interest in our panel.

Implications for Practice and Research

CheckUp can be used for multiple purposes. Firstly, guideline developers can use it both for the reporting of their guidelines and to plan their updating processes. Guideline users can assess the reporting of updated guidelines. Editors may request its completion from guideline authors. CheckUp provides an overall picture of how complete the updating process is reported in updated clinical guidelines. Being a reporting checklist, CheckUp does not evaluate the quality of the updating processes, as there are no gold standards for this process. Currently, the G-I-N Updating Guidelines Working Group (http://www.g-i-n.net/working-groups/updating-guidelines) is undertaking an analysis of current guideline updating methods worldwide. From this work, strategies or advice might come on how we might assess guideline updating quality.

There are currently no gold standards for guideline updating methodology. Nonetheless, updating is key to ensuring trustworthy, implementable, and clinically relevant recommendations. Current guideline evaluation tools or guideline method resources (e.g., AGREE II, Grading of Recommendations Assessment, Development, and Evaluation (GRADE), IOM Standards, and the like) are not simply transferable to the conceptual requirements of an updated guideline. CheckUp addresses the gap: it has been supported by our study participants and is a resource that complements (rather than competes with) the other high-quality tools available in the guideline enterprise.

Further rigorous research in updating clinical guidelines is warranted, and we invite users to comment on the items and the usability of CheckUp. It would be important to assess the impact of CheckUp in the updating clinical guideline field over the next few years [16]. When dynamic or living guidelines become a reality, [38] some adaptation of CheckUp could potentially be necessary. Finally, the G-I-N Updating Guidelines Working Group will continue to play a key role in this work and in moving forward the updating agenda in the clinical guideline enterprise.

Supporting Information

Attachment 1

Zdroje

1. Laine C, Taichman DB, Mulrow C. Trustworthy clinical guidelines. Ann Intern Med. 2011;154(11): 774–775. doi: 10.7326/0003-4819-154-11-201106070-00011 21646561

2. Qaseem A, Forland F, Macbeth F, Ollenschläger G, Phillips S, van der Wees P, et al. Guidelines International Network: toward international standards for clinical practice guidelines. Ann Intern Med. 2012;156(7):525–31. doi: 10.7326/0003-4819-156-7-201204030-00009 22473437

3. Shekelle PG, Ortiz E, Rhodes S, Morton SC, Eccles MP, Grimshaw JM, et al. Validity of the Agency for Healthcare Research and Quality clinical practice guidelines: how quickly do guidelines become outdated? JAMA. 2001;286(12): 1461–1467. 11572738

4. Grupo de trabajo sobre actualización de GPC. Actualización de Guías de Práctica Clínica en el Sistema Nacional de Salud. Manual Metodológico. Plan de Calidad para el Sistema Nacional de Salud del Ministerio de Sanidad y Política Social. Instituto Aragonés dológico. de Ciencias de la Salud-I+CS; 2009. Guías de Práctica Clínica en el SNS: I+CS N° 2007/02-01

5. Shekelle PG. Updating practice guidelines. JAMA. 2014;311(20): 2072–2073. doi: 10.1001/jama.2014.4950 24867011

6. Agbassi C, Messersmith H, McNair S, Brouwers M. Priority-based initiative for updating existing evidence-based clinical practice guidelines: the results of two iterations. J Clin Epidemiol. 2014;67(12): 1335–1342. doi: 10.1016/j.jclinepi.2014.06.013 25216900

7. Gartlehner G, West SL, Lohr KN, Kahwati L, Johnson JG, Harris RP, et al. Assessing the need to update prevention guidelines: a comparison of two methods. Int J Qual Health Care. 2004;16(5): 399–406. doi: 10.1093/intqhc/mzh081 15375101

8. Martínez García L, Sanabria AJ, Araya I, Lawson J, Solà I, Vernooij RW, et al. Efficiency of pragmatic search strategies to update clinical guidelines recommendations. BMC Med Res Methodol. 2015;15: 57. doi: 10.1186/s12874-015-0058-2 26227021

9. Martínez García L, Sanabria AJ, García Alvarez E, Trujillo-Martín MM, Etxeandia-Ikobaltzeta I, Kotzeva A, et al. The validity of recommendations from clinical guidelines: a survival analysis. CMAJ. 2014;186(16): 1211–1219. doi: 10.1503/cmaj.140547 25200758

10. Eccles M, Rousseau N, Freemantle N. Updating evidence-based clinical guidelines. J Health Serv Res Policy. 2002;7(2): 98–103. 11934374

11. Johnston ME, Brouwers MC, Browman GP. Keeping cancer guidelines current: results of a comprehensive prospective literature monitoring strategy for twenty clinical practice guidelines. Int J Technol Assess Health Care. 2003;19(4): 646–655. 15095770

12. Parmelli E, Papini D, Moja L, Bandieri E, Belfiglio M, Ciccone G, et al. Updating clinical recommendations for breast, colorectal and lung cancer treatments: an opportunity to improve methodology and clinical relevance. Ann Oncol. 2011;22(1): 188–194. doi: 10.1093/annonc/mdq324 20605933

13. Martínez García L, McFarlane E, Barnes S, Sanabria AJ, Alonso-Coello P, Alderson P. Updated recommendations: an assessment of NICE clinical guidelines. Implement Sci. 2014;9: 72. doi: 10.1186/1748-5908-9-72 24919856

14. Burgers JS, Cluzeau FA, Hanna SE, Hunt C, Grol R. Characteristics of high-quality guidelines: evaluation of 86 clinical guidelines developed in ten European countries and Canada. Int J Technol Assess Health Care. 2003;19(1): 148–157. 12701947

15. Alonso-Coello P, Martínez García L, Carrasco Gimeno JM, Solà I, Qureshi S, Burgers JS, et al. The updating of clinical practice guidelines: insights from an international survey. Implement Sci. 2011;6: 107. doi: 10.1186/1748-5908-6-107 21914177

16. Martínez García L, Arévalo-Rodríguez I, Solà I, Haynes RB, Vandvik PO, Alonso-Coello P, et al. Strategies for monitoring and updating clinical practice guidelines: a systematic review. Implement Sci. 2012;7(1): 109.

17. Becker M, Neugebauer EA, Eikermann M. Partial updating of clinical practice guidelines often makes more sense than full updating: a systematic review on methods and the development of an updating procedure. J Clin Epidemiol. 2014;67(1): 33–45. doi: 10.1016/j.jclinepi.2013.06.021 24125894

18. Vernooij RW, Sanabria AJ, Solà I, Alonso-Coello P, Martínez García L. Guidance for updating clinical practice guidelines: a systematic review of methodological handbooks. Implement Sci. 2014;9: 3. doi: 10.1186/1748-5908-9-3 24383701

19. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ. 2010;182(18): E839–842. doi: 10.1503/cmaj.090449 20603348

20. Schünemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ. 2014;186(3): E123–142. doi: 10.1503/cmaj.131237 24344144

21. Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7(2): e1000217. doi: 10.1371/journal.pmed.1000217 20169112

22. Moher D, Weeks L, Ocampo M, Seely D, Sampson M, Altman DG, et al. Describing reporting guidelines for health research: a systematic review. J Clin Epidemiol. 2011;64(7): 718–742. doi: 10.1016/j.jclinepi.2010.09.013 21216130

23. Hsu C, Sandford B. The delphi technique: making sense of consensus. Pract Assessment Res Eval. 2007;12: 1–8.

24. Likert R. A technique for the measurement of attitudes. Archives of psychology. 1932.

25. Burgers J, Alonso-Coello P, Vernooij R. Staying alive: methods to keeping guideline programs up to date. In: 12th G-I-N Conference; 2015. http://www.g-i-n.net/conference/12th-conference/2015-scientific-programme.

26. Rooke TW, Hirsch AT, Misra S, Sidawy AN, Beckman JA, Findeiss L, et al. Management of patients with peripheral artery disease (compilation of 2005 and 2011 ACCF/AHA Guideline Recommendations): a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2013;61(14): 1555–1570. doi: 10.1016/j.jacc.2013.01.004 23473760

27. Qaseem A, Wilt TJ, Weinberger SE, Hanania NA, Criner G, van der Molen T, et al. Diagnosis and management of stable chronic obstructive pulmonary disease: a clinical practice guideline update from the American College of Physicians, American College of Chest Physicians, American Thoracic Society, and European Respiratory Society. Ann Intern Med. 2011;155(3): 179–191. doi: 10.7326/0003-4819-155-3-201108020-00008 21810710

28. Khatcheressian JL, Hurley P, Bantug E, Esserman LJ, Grunfeld E, Halberg F, et al. Breast cancer follow-up and management after primary treatment: American Society of Clinical Oncology clinical practice guideline update. J Clin Oncol. 2013;31(7): 961–965. doi: 10.1200/JCO.2012.45.9859 23129741

29. Haas FJLM, de Vries RRP. CBO Richtlijn Bloedtransfusie. 2011.

30. Lougheed MD, Lemiere C, Ducharme FM, Licskai C, Dell SD, Rowe BH, et al. Canadian Thoracic Society 2012 guideline update: diagnosis and management of asthma in preschoolers, children and adults. Can Respir J. 2012;19(2): 127–164. 22536582

31. Grupo de trabajo de la guía de práctica clínica sobre dispepsia. Manejo del paciente con dispepsia. Guía de práctica clínica. Barcelona: Asociación Española de Gastroenterología, Sociedad Española de Medicina de Familia y Comunitaria y Centro Cochrane Iberoamericano; Actualización 2011. Programa de Elaboración de Guías de Práctica Clínica en Enfermedades Digestivas, desde la Atención Primaria a la Especializada: 3.

32. National Health and Medical Research Council. Clinical Practice Guideline for the Management of Borderline Personality Disorder. Melbourne: National Health and Medical Research Council; 2012.

33. National Collaborating Centre for Mental Health (UK), Social Care Institute for Excellence (UK). Antisocial Behaviour and Conduct Disorders in Children and Young People: Recognition, Intervention and Management. Leicester (UK): British Psychological Society; 2013.

34. New Zealand Guidelines Group. (2012). Cancer control strategy guidance completion: update of evidence for prostate-specific antigen (PSA) testing in asymptomatic men. Wellington: Ministry of Health.

35. Scottish Intercollegiate Guidelines Network (SIGN). Acute coronary syndromes. Edinburgh: SIGN; 2013. (SIGN publication no. 93). [February 2013]. www.sign.ac.uk

36. Lyratzopoulos G, Barnes S, Stegenga H, Peden S, Campbell B. Updating clinical practice recommendations: is it worthwhile and when? Int J Technol Assess Health Care. 2012;28(1): 29–35. doi: 10.1017/S0266462311000675 22236808

37. Alderson LJ, Alderson P, Tan T. Median life span of a cohort of National Institute for Health and Care Excellence clinical guidelines was about 60 months. J Clin Epidemiol. 2014;67(1): 52–55. doi: 10.1016/j.jclinepi.2013.07.012 24139089

38. Vandvik PO, Brandt L, Alonso-Coello P, Treweek S, Akl EA, Kristiansen A, et al. Creating clinical practice guidelines we can trust, use, and share: a new era is imminent. Chest. 2013;144(2): 381–389. doi: 10.1378/chest.13-0746 23918106