Building the Field of Health Policy and Systems Research: Social Science Matters

Summary

article has not abstract

Introduction

The first paper in this series on building the field of Health Policy and Systems Research (HPSR) in low- and middle-income countries (LMICs) [1] outlined the scope and questions of the field and highlighted the key challenges and opportunities it is currently facing. This paper examines more closely one key challenge, the risk of disciplinary capture—the imposition of a particular knowledge frame on the field, privileging some questions and methodologies above others. In HPSR the risk of disciplinary capture can be seen in the current methodological critique of the field, with consequences for its status and development (especially when expressed by research leaders).

The main criticisms are reported to be: that the context specificity of the research makes generalisation from its findings difficult; lack of sufficiently clear conclusions for policy makers; and questionable quality and rigour [2]. Some critique is certainly warranted and has come from HPS researchers themselves. However, this critique also reflects a clash of knowledge paradigms, between some of those with clinical, biomedical, and epidemiological backgrounds and those with social science backgrounds. Yet, as HPSR is defined by the topics and questions it considers rather than a particular disciplinary approach, it requires engagement across disciplines; indeed, understanding the complexity of health policy and systems demands multi- and inter-disciplinary inquiry [3].

To develop the science of HPSR it is, therefore, important to start by recognising the diversity of disciplinary perspectives, as well as shared concerns. Richer methodologies for addressing these concerns must then be developed. And, as health policies and systems are themselves social and political constructions, it is important to acknowledge the particular value of social science perspectives in the field. Each of these issues is addressed in the following sections, and they are considered further in paper three of the series [4].

Knowledge Paradigms

Figure 1 characterises key areas of difference between the dominant knowledge paradigms that underpin the disciplines applied within HPSR. The figure deliberately polarises the paradigms to spark debate. Some disciplines are dominated by a particular paradigm and some are spread across paradigms.

The positivist worldview is reflected in much clinical, biomedical, and epidemiological, and some social science, research. This view starts from the same position as the natural and physical sciences. The phenomena being investigated comprise a set of facts, a single reality that can be observed and measured by the researcher without disturbing them. The central aim of research is to detect causal mechanisms through the deductive process of testing hypotheses derived from theory and past experience against empirical facts. At their simplest, such mechanisms represent the prediction that “x will cause y” in any other setting. Simple HPSR hypotheses might include, for example, “limited financial incentives cause low motivation” or “a lack of health facilities undermines access to health services.” Sometimes such hypotheses are tested through statistical analysis of secondary data [5]; sometimes studies are designed to allow hypotheses to be tested [6]. Indeed, the positivist perspective underpins the recent rise of experimental methodology in impact evaluation. As the emphasis in such studies is on measuring the magnitude of an intervention's impact, and ensuring that this estimate is unbiased, careful attention is paid to selecting an appropriate control group (randomized or otherwise) and controlling the influence of possible confounding factors. Much less emphasis is placed on understanding how the intervention works and which contextual or other factors mediate its impact.

Much social science work that is qualitative is located at the relativist end of the spectrum. Such research is essentially based on the understanding that the world around us is subject to human interpretation. Health policies and systems are, therefore, understood to be constructed and brought alive by social actors through the meaning they attach to (their interpretations of) their experiences. Whereas positivist researchers focus on facts and regularities (that is, causes and effects), relativist researchers see interpretations as the primary subject of inquiry, proposing that different interpretations of the same experience represent multiple realities. In this tradition, researchers study human behaviour in everyday or natural settings, generating qualitative data that are primarily analysed inductively to generate categories and explanations of experience. Such analysis also involves interpretation by the researcher, in interaction with respondents. It may be guided by, and/or generate, what is called middle range theory, i.e., ideas about how the world works, comprising categories and concepts derived from analysis, and suggestions about how they are linked together. Middle range theory may be tested against evidence through the process of analysis or highlights questions and ideas to be considered in future studies.

Relativist HPSR studies focus, for example, on how health system actors understand and experience particular services or policies [7], and what social and political processes, including power relations, influence them [8],[9]. The development and testing of middle range theory is also supported by studies that adopt a critical realist position. This knowledge paradigm falls somewhere in the spectrum between positivism and relativism, and is of growing interest in HPSR [10] (see FEMhealth, http://www.abdn.ac.uk/femhealth/). However, these sorts of questions are still only quite rarely addressed in the wider HPSR literature [1].

Shared Concerns and the Value of Multiple Perspectives

Although HPS researchers from different disciplinary traditions have some difficulty understanding each other's perspectives, they also have some shared starting points: a common focus, health policies and systems, and a concern about how to strengthen health systems to benefit those being served by them. The complexity of the phenomena being investigated may also generate a willingness to think creatively about how to investigate issues. Therefore, HPS researchers tend not to fall at the extreme ends of the spectrum outlined in Figure 1—and this makes multi- and inter-disciplinary work more possible.

Review of existing HPSR work demonstrates, moreover, that bringing together research from different traditions generates broader and deeper understanding on the issues of focus. Box 1, for example, shows the breadth of questions that have been addressed around one critical HPS issue for LMICs, user fees; and the different papers examining the household level impacts of out of pocket payments together provide deeper and richer insights on these experiences than would come from one perspective alone.

Learning from Relativist Social Science Perspectives

Health policies and systems are fundamentally shaped by political decision-making, whilst the routines of health systems are brought alive through the relationships among the actors involved in managing, delivering, and accessing health care, and engaged in wider action to promote health, including researchers [11]. In essence, therefore, health policies and systems are constructed through human behaviour and interpretation, rather than existing independently of them. As relativist social science perspectives see all phenomena as at least partially constructed in this way, they have particular value in building the methodological foundations of HPSR. Three contributions are discussed here: generalising from rich contextual understanding; supporting policy learning; and approaches to ensuring research rigour.

Taking Account of Context in Drawing out Generalisations

Multiple contextual factors influence the working of health systems. Health worker motivation, for example, reflects a range of personal, organisational, and societal factors, including relationships with others, and itself influences many aspects of the provision of health care. Similarly, patients' decisions to use services, or adhere to treatment advice, are responses to many contextual factors: their own understandings of illness, and how best to treat it; advice received from friends and family; past experience of health providers; the availability of cash to cover costs; and the gender dynamics influencing household decision-making. There are also multiple interpretations of the same experience as different people bring different contexts to bear on its interpretation. Health workers, for example, respond differently to the same financial incentive, and patients vary in their response to treatment advice. The causal mechanisms underpinning the changes brought about by new health policies or health system interventions are, thus, complex.

As a result, investigation of HPS issues demands research that seeks to understand and explain experiences by reference to the many layers of their context, whilst acknowledging the often quite different interpretations of experience across people. Reducing relevant contextual factors to a set of simple quantifiable measures for statistical analysis is, simply, difficult. On the other hand, case study research, widely used in organisational and political science work, supports the “thick descriptions” of particular experiences situated within their context that allow understanding and explanations of the phenomena of focus by reference to that context [12]. For example, a study of Brazilian health system decentralisation, involving anthropological work in three case study areas, investigated the factors shaping the extent of local decision-making actually achieved, with consequences for quality of care improvement possibilities. A range of contextual factors were influential, including political relationships among layers of government, the potential of generating tax revenue at the local level, differences between rural and urban areas in the opportunities for community participation in decision-making, and existing patterns of political patronage; and these also combined with individual management styles and health worker commitment to the local area [13].

In studies with multiple cases, systematic and deliberate cross-case comparison supports, moreover, analytic generalisation (Box 2). The aim in such analysis is not to draw conclusions that can be statistically generalised to a wider study population, or that will hold across time and place. Instead, analytic generalisation entails the development of general conclusions that, although derived from a limited number of particular experiences, provide theoretical insights that can be put forward for consideration, and testing, in other, similar situations. This includes middle range theory, as outlined earlier, and theory that offers ideas about the causal mechanisms likely to underpin interventions that achieve their goals.

Active Support for Policy Learning

Health research has traditionally seen knowledge generation as essentially a process of adding to the existing stock of facts and predictions, with researchers acting largely as disinterested scientists feeding evidence into the decision-making process [14]. Learning from that knowledge then entails the simple transfer of knowledge from one setting to another [15]. Even current HPSR debates about the importance of getting research into policy and practice and knowledge translation sometimes see this process as quite linear [16].

However, for a relativist, researchers contribute to the process of learning as active participants, using both formal and tacit knowledge in active debate with policy makers [15]. Thus, some social scientists argue that in addressing problems that matter in their own communities, researchers should pay particular attention to the ways in which values and power shape those problems and responses to them [17], assisting policy actors to negotiate mutually acceptable solutions to problems, and ensuring that underrepresented groups are heard [18]. For others, building the possibility of such action into research design is an ethical requirement and key hallmark of good quality research [19].

Social science perspectives, therefore, challenge the HPSR community to think more deeply about how to support policy and system change through their research, including how to address the thorny issue of the boundary between researcher and advocate. For example, what sorts of participatory and action research with citizens, health managers, and health workers can support the reflective enquiry that generates positive change in current practices? And should and can we initiate processes that stimulate public debate about research findings—such as active media engagement, debates on public platforms, or engagement with civil society organisations?

Ensuring Research Rigour

For some traditions of health research, validity and reliability are the hallmarks of rigorous research, and are ensured through careful study design, appropriate tool development and data collection, and correct approaches to statistical analysis. In contrast, relativist (qualitative) social science research is premised on the understanding that there are multiple realities, reflecting actors' different understandings of common experiences (Figure 1). These understandings are either seen to have significant influence over the issues of focus or to be the focus of inquiry. Researchers from this tradition, moreover, aim not just to identify and report such understandings, but instead, through analysis and engagement, to produce their own interpretations of them, explaining why and how actors behave and think as they do. For relativist research, the “trustworthiness of researchers'” interpretations is the key hallmark of research rigour, implying that the interpretation is widely recognised to have value beyond the particular examples considered. Such trustworthiness is, in essence, negotiated between researchers and research users on the basis of transparent information on study design and the processes of data collection, analysis, and interpretation. Table 1 summarises the critical steps researchers must take to ensure that their analysis is both based on rich insight into the experience examined and has been subject to challenge, and to offer a transparent account of their research process to the user.

At a minimum, improving the quality of HPSR requires paying due attention to the particular approaches to research rigour relevant to the specific paradigm of knowledge underpinning any study. However, because of the complexity of the issues investigated, social science perspectives on rigour offer valuable insights for all empirical HPSR. As HPSR is often more investigation than observation, all stages of research must always be conducted with caution. Rigorous investigation involves the following [19]–[21]:

• an active process of questioning and checking in inquiry—asking how and why things happened and not only what happened, checking answers to questions to identify further issues that need to be followed up to deepen understanding of the experience;

• a constant process of conceptualising and reconceptualising—using ideas and theory to develop an initial understanding of the problem or situation of focus to guide data collection, but using the data collected to challenge those ideas and assumptions and when necessary, to revise your ideas in response to the evidence;

• crafted, interpretative judgements—based on enough evidence, particularly about context, to justify the conclusions drawn, as well as deliberate consideration of contradictory evidence (negative case analysis) and review of initial interpretations by respondents (member checking);

• researcher reflexivity—being explicit about how your own assumptions may influence your interpretation, and testing them in analysis.

Finally, although currently rarely conducted in HPSR, mixed-method research in which qualitative and quantitative analyses are undertaken sequentially, with one stage of work deliberately feeding into the next [22], offer important opportunities for the triangulation across methods and knowledge paradigms that can broaden and deepen investigation of health policy and systems issues [23].

Conclusions

The current interest in HPSR provides exciting opportunities for the field, but also brings the threat of “disciplinary capture” by the clinical, biomedical, and epidemiological disciplinary perspectives dominant in wider health research. Yet, social science perspectives are vital to HPSR. Health policies and systems are complex social and political phenomena, constructed by human action rather than naturally occurring. Advancing the science of HPSR, thus, demands we take steps to build understanding across disciplinary boundaries, for example, by ensuring that we can speak each other's languages around generalisability and knowledge generation; sharing experience of supporting policy learning; and clarifying expectations of each other's disciplinary culture. Valuing social science perspectives and building interdisciplinary understanding both represents the cutting edge of HPSR and demonstrates that the field is at a scientific cutting edge.