Shining a light on the dark side of coproduction

Advocacy of co-produced research

Many, if not most applied research projects, are undertaken with some degree of collaboration between researchers and ‘stakeholders’, who include research funders, policymakers or practitioners, members of the public and civil society, or other actors such as patients in health studies. This collaboration takes various forms (e.g. consulting on research topics, or working iteratively throughout the research process), and in general, people feel positively about it. Funders often require it, or at least support it, and researchers have argued that it is the most effective way to ensure that evidence is used or translated into practice. Today, there are many forms of collaborative research practice, including co-production, co-design, co-creation, stakeholder and public engagement, participation/involvement and integrated knowledge translation which sit under this umbrella. They reflect a very diverse set of motivations, activities, identities and discourses about how research interacts with the rest of society. Here we use ‘co-production’ as shorthand to discuss these practices, but acknowledge the diversity and differences in the above.

People argue that collaborative and co-productive research

• Helps with better-quality research, through improving our understanding of issues, mechanisms, and, of course, the research questions of most importance
• Makes it more likely that this research will be acted on
• Is the right way to do research (especially if publicly funded), being more inclusive and less elitist
• Makes potential users feel more empowered, trusted and more persuaded of the importance and veracity of the research findings.

Facing up to the challenges and costs of co-production

Undoubtedly, making research more relevant and used is the aim for many applied researchers, and their attempts to do so and document this are laudable. There is a need to help uncover if, when, and how collaboration is the best way to achieve this (normative) aim.

Firstly, there is very limited empirical evidence about whether collaborative research does lead to improved uptake of findings – even if we could agree what that looks like in reality.

Secondly, collaborative research brings significant challenges. Collaborative research may be more uncertain, slower, or less innovative than non-collaborative research. These challenges and related costs may be experienced throughout the research process, as the lists below indicates.

Challenges of coproduction

Developing mixed research teams

• Stakeholders not homogeneous, and can disagree
• ‘Usual suspects’ can take over, where coproductive discussions are dominated by certain individuals

Framing research questions

• Stakeholders and researchers may have different priorities and values
• Useful research can lack originality
• Research can be co-opted by partners, for example, to justify status quo or historical decisions

Collecting data

• Researchers may pressure stakeholders to allow their organisational resources to be used to facilitate data collection – e.g. using staff time or applying pressure for site access

Analysing and interpreting data

• Stakeholders may want to know which participant agreed to participate or what they contributed to the dataset
• Stakeholders may want to help analyse the data

Formulating recommendations

• May be little agreement about the importance of research
• Researchers may be pressed to frame findings in particular ways

Disseminating research

• Researchers or stakeholders may be prevented from sharing unwanted findings
• Stakeholders may want to share findings before researchers are ready

Implementing change

• Tension between advocating for research, or advocating for policy/practice changes
• Researchers show little interest in providing assistance with implementation efforts

Costs of coproduction

Developing mixed research teams

• The research process may take more time compared to a traditional research process
• Shared decision-making is threatened when process dominated by certain voices or interests

Framing research questions

• Damage to interpersonal or organisational relationships
• Damage to research careers
• Damage to researcher independence and credibility

Collecting data

• Damage to interpersonal or organisational relationships, particularly with more powerful stakeholders

Analysing and interpreting data

• Violation of research ethics obligations
• Researcher needs to train stakeholders and format data in an appropriate way to conform with research ethics obligations

Formulating recommendations

• Findings are misrepresented
• Damage to researcher independence and credibility

Disseminating research

• Damage to researcher independence and credibility
• Damage to the credibility of the research process

Implementing change

• Can damage relationship with practice or policy colleagues
• Implementation of research findings fail

Finally, there are other very significant potential costs of co-production, which, in our experience are often unequally borne by junior, untenured, female members of staff.

• Practical and administrative resources such as the time required to arrange meetings with busy people for whom research is not their primary activity. Managing group dynamics in such meetings can require interpersonal skills with which researchers may not be endowed.
• Stressful interpersonal interactions: these can be dismissed as mere relational difficulties. We must take the potential for disagreement, conflict, reputational risk and power imbalances seriously as part of the research process. The consequences of mismanaging these are severe.
• Individual researchers already have to balance their teaching, research and administrative responsibilities. Developing another set of professional skills and networks to create collaborative research projects with real world impact is simply too great a leap for some. The implication of this is that some or all of the co-production activities related to a research project could be led by a specialist in knowledge transfer and exchange rather than by members of the research team. This approach has been advocated and implemented by some.
• Some feel that taking part in applied, highly collaborative research can lead to researchers becoming, or seeming partisan and biased, or as academic “lightweights” producing little of substance
• The research outputs themselves may be co-opted to serve the political agendas of others. Co-produced research findings (possibly no different from any research) can be appropriated and used to serve the self-interest of more powerful groups. Some groups lack the skills to engage in the use and promotion of research findings so lose out to more skilled and better connected groups. There is also the risk that co-produced research is more likely than other forms of research to produce findings biased in favour of prevailing norms of what is ‘correct’. This last type of research leads to repetitious, ‘safe’ research

What do we need to do differently?

We may be able to avoid some of these costs, or they may be an intrinsic part of collaborative processes, in which case we need to work out the best balance between costs and benefits.

Armed with a better understanding of the costs and benefits of co-production, those planning new projects should be much better placed consistently to ask themselves:

1. What is everyone bringing to the table? For example, policy-makers and funders bring money, knowledge of the political context, pressure for answers; researchers bring topic and methodological expertise; the public and patients bring their experiences.
2. Under which circumstances are these resources needed, for what purpose, and at which stages of the research process? For example, when is it better to have patient representatives articulate the user perspective rather than derive understanding from a systematic review of patient experiences?
3. What are the costs, and how will they be borne and defrayed by those involved?
4. How will decisions about the direction of the research be taken, and how will responsibility and accountability for decisions be shared? Will group dynamics, market forces, formal authority or some other basis be used? In turn, how will this be governed?

In parallel, research organisations and funders also need to consider:

1. How to create (co-create) and support the infrastructure and leadership for co-production
2. How to provide training in co-production, and help interested researchers and funders take this seriously as a necessary skill
3. How to reward good practice, and to recognise the work co-production may take even if it does not lead to research impact
4. How to evaluate the potential impact(s) of co-production
5. How to ensure that co-production supports diversity and quality in research and policy

Conclusion

Co-production is an exciting approach to research that can, with care, generate truly novel, unexpected findings and impacts. Yet it takes time and investment, and there is still little evidence about how co-production changes research, policy or practice, or how it compares to alternatives. We think more reflection about how, why and when we do co-production would be helpful, as would more discussion about how co-production influences the process of research, and the roles and responsibilities of everyone involved in collaborations. Ultimately, we need to be clear about how and why exactly we work with each other, to transform evidence for society.

This blog grows from a paper by the authors published in Health Research Policy and Systems 2019: 17: 33 https://doi.org/10.1186/s12961-019-0432-3

Kathryn Oliver is Associate Professor of Sociology and Public Health, Department of Public Health, Environments and Society, London School of Hygiene and Tropical Medicine, UK

Anita Kothari is Associate Professor, School of Health Studies, University of Western Ontario, Canada

Nicholas Mays is Director of the Policy Innovation and Evaluation Research Unit (PIRU) and Professor of Health Policy, Department of Health Services Research & Policy, London School of Hygiene and Tropical Medicine, UK