Assessing and Strengthening African Universities' Capacity for Doctoral Programmes

Summary

article has not abstract

Relationship between Indigenous Doctoral Programmes in Africa and Better Health Outcomes

The generation of local research, and the ability to innovate and to use research results, are essential for good policy making and ultimately for better health outcomes [1]. Research for policy should be led by a country's own scientists [2],[3], but very few universities in low-income African countries are able to “home grow” sufficient world class researchers [4]. Traditionally, doctoral students from low-income countries have been trained overseas, where they often learn skills they cannot use when they return home. Consequently there has been a recent shift in funding emphasis towards supporting students to remain in their home institution.

Implicit in this new approach is the need to strengthen African universities' capacity to deliver doctoral programmes, and to focus efforts not only on the training itself but also on creating an enabling environment for research, in particular, leadership, career development, infrastructure, and access to information [4]. Currently, the international academic community has insufficient understanding of the policies and processes required to develop research capacity in African universities, and this makes it difficult to target resources strategically towards priority capacity gaps [5].

Lack of Methodologies to Evaluate Gaps in Universities' Policies and Processes for Doctoral Programmes

To enhance the capacity of African universities to run doctoral programmes, the Malaria Capacity Development Consortium (MCDC) [6] has funded 19 African researchers to undertake doctoral training in universities in Ghana, Malawi, Senegal, Tanzania, and Uganda. The doctoral programmes in each of the African universities were at different stages of maturity at the start of the programme, with the number of registered doctoral students varying from none to 147. The doctoral programme coordinators in each institution recognised that having a cohort of new doctoral students provided an opportunity for their institutions to strengthen their systems for doctoral programmes or, for those universities just starting doctoral programmes, to develop the necessary structures and processes. The African project coordinators therefore asked the MCDC secretariat for support in identifying ways in which their doctoral programmes could be improved. The project was advertised, and following a selection process, a contract to evaluate the African doctoral programmes was awarded to a team of researchers from the United Kingdom and Africa (IB, RP, RM-P, and SP).

A process was needed to identify gaps in the African institutions' existing doctoral programmes, so discussions between representatives from the universities and our research team defined the critieria that should be met by such a process (Box 1). However, published information about evaluating capacity development is very scarce [7], and a search of the literature failed to identify any single process that could be used to evaluate all the policies and processes needed to run successful doctoral programmes. The purpose of our study was therefore to develop and test an evidence-based process that could be used to evaluate all the components of doctoral programmes and to standardise the process so that it is transferable across universities.

Development of the Process for Evaluating Policies and Systems for Doctoral Programmes in Africa

To ensure that the final evaluation process was robust, it was derived from published evidence. We scanned published literature, Web sites and documents from universities, educational agencies, and regulatory bodies to identify and synthesize existing methods for evaluating any aspect of doctoral programmes (Box 2). Through this process we produced a list of all the policies, processes, and facilities needed to run doctoral programmes (summarised in Figure 1). We amalgamated all the information obtained into a draft evaluation process, which consisted of a list of stakeholders to be interviewed, an interview guide for each of the different cadres of stakeholders, a list of documents to be reviewed, and a list of facilities to be visited.

Testing and Finalising the Evaluation Process

The evaluation took place during site visits of 2–3 days to each of the five African universities that are partners in MCDC. These visits were preceded by pro-active engagement of key individuals in the universities who would facilitate the evaluations. The research team conducting the evaluation was independent of the MCDC managers, and the team's four members had expertise in health care delivery, research and doctoral supervision, academic and health care systems, and educational development. Interview bias was reduced by using different combinations of research team members to conduct the interviews.

Initially no assumptions were made about which questions could be answered by which interviewee, and all interviewees were asked every question. Interviewees were enthusiastic about being interviewed, but they were able to provide reliable information only about the aspects of doctoral programmes in which they were directly involved, and so there were many instances where inaccurate or incomplete information was provided. We were therefore meticulous about double-checking all the information we were given by asking the same question of more than one individual. Responses from the interviewees were corroborated by referring to institutional documents and directly observing facilities as appropriate. Any discrepancies were resolved through discussions between the researchers and the doctoral programme coordinators in each institution.

As the evaluations progressed we were able to match questions to specific types of interviewees, thereby improving the efficiency of the interviews. By honing the questions, removing duplications and adding new interviewees or observations to the process, as suggested by interviewees, we refined and streamlined the process and reduced the interview times from over 60 minutes to around 20 minutes. No new interviews or observations were added to the process after the third site visit, indicating the we had reached saturation for the interview guide. In total across the five medical schools we interviewed 83 individuals involved in all aspects of doctoral programmes, reviewed 40 documents, and visited laboratories, libraries, computer centres, and field sites.

The final evaluation process consisted of interviews using the developed guide (i.e., a grid with questions mapped against specific interviewees) (see Text S1), and a review of documents (e.g., policies, regulations, course handbooks) and facilities (e.g., laboratories, libraries, computer centres) to corroborate information from the interviews. The list of interviewees comprised university policy makers (e.g., principals, provosts, deans), researchers (e.g., doctoral students, post-doctoral scientists, research supervisors, research centre staff), and support staff (e.g., ethicists, administrators, accountants, librarians, laboratory scientists). The questions in the final interview guide were compared to the initial list of doctoral programme components we had extracted from the literature to ensure that the iterative adaptations of the process had not resulted in any major omissions.

Outcomes of the Evaluation Process

At the end of the site visits the doctoral programme coordinator at each university was provided with a confidential report about their own institution and also an anonymised overview report that amalgamated and summarised the key findings from the individual institutional reports. The institutional reports provided a narrative account of corroborated interviewee responses, a list of gaps in doctoral programme provision, and potential remedies proposed by the interviewees. It was possible to identify gaps in provision because our evaluation process was based on an extensive literature review and was therefore a “benchmark” that contained all the elements needed to run a successful doctoral programme.

Following the evaluation process the institutions used the evaluations to develop and implement their own plans to remedy the gaps in capacity and to derive indicators that could be used to monitor progress. Evidence suggests that these indicators will need to be revised regularly as the universities' doctoral programmes mature and become more sophisticated [8]. The capacity gaps that occurred in more than one institution, and potential remedies to address these gaps, have been amalgamated in Table 1. Interestingly, over half of these gaps could potentially be addressed by the universities themselves without any additional external resources (e.g., see Box 3). To address other capacity gaps, such as lack of access to suitable electronic resources and the inexperience of many of the supervisors, additional external inputs would be required (e.g., see Box 4). Using the recommendations in the report, each university coordinator prioritised the areas identified as needing support, identified the steps that would be needed to fill these gaps, and estimated the costs of doing so. The MCDC programme will be able to finance some of these action plans. By applying this standardised evaluation process to several institutions it was also possible to identify common capacity gaps (Box 5), which could become the focus of cross-institutional efforts by external agencies.

Limitations of the Evaluation Process

The quantitative indicators we identified for monitoring progress in strengthening institutional capacity (e.g., number of students; time to complete course; number of publications, grants, or presentations) can be measured relatively easily, but they do not adequately capture factors that contribute to developing an enabling environment for research [9]. Our evaluation process therefore also included some qualitative indicators (e.g., student satisfaction, quality of learning spaces), although we recognise that these may be more difficult to measure than quantitative indicators. This strategy of combining qualitative and quantitative indicators is similar to the approach taken in the few other published studies that have evaluated research capacity development [7].

The evaluation process was developed with and for health faculties in universities in developing African countries, and it has not been evaluated beyond this context. Nevertheless, because the key components outlined in the overview (Figure 1) were derived from the global literature, the process is likely to be applicable to doctoral programmes in faculties and universities outside Africa. However, the specific types of capacity gaps may vary between countries with different levels of socio-economic development (e.g., slow Internet access and lack of doctoral research supervisors were common gaps in our five African institutions, but this may not be the case in other regions).

Conclusion

We have developed a comprehensive evidence-based process for evaluating all the policies and systems required for doctoral programmes. Unlike previously published methods for evaluating doctoral programmes, our process incorporated the perspectives of students, staff, the local research community, and the universities' policy makers and was applicable across different countries and programmes of differing maturity. Our standardised evaluation process not only enabled the universities to develop and monitor strategies to address their own capacity gaps but also provided them with a mechanism for justifying, planning, commissioning, and monitoring inputs by external funders while retaining leadership of the process. Unlike the traditional, externally controlled accountability imposed by international donors' agendas, there is evidence that this “endogenous accountability” type of monitoring is likely to promote better ownership and performance [10].

Supporting Information

Attachment 1