A tool to evaluate the impact of lived experience involvement in research: the Brain and Genomics Hub Impact Log literature review and protocol
Division of Psychiatry, University College London, UK
Bipolar UK, London, UK
Centre for Neuropsychiatric Genetics and Genomics, Division of Psychological Medicine and Clinical Neurosciences, School of Medicine, Cardiff University, Wales, UK
School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Canada
Brain and Genomics Hub, Specialist Lived Experience Advisory Panel, UK
School of Health and Wellbeing, University of Glasgow, UK
Faculty of Psychology, Universitas Gadjah Mada, Indonesia
University of Manchester, School of Health Sciences, Division of Nursing, Midwifery and Social Work, UK
EPPI Centre, Social Research Institute, University College London, UK
Independent Researcher, UK
Caprich International inc., Canada
Department of Psychiatry, University of Oxford
Adferiad, Colwyn Bay, Wales, UK
DATAMIND, Swansea University Medical School, Swansea UK
National Centre for Suicide and Self-harm Research, Wales, UK
*Corresponding author: tania.gergel@ucl.ac.ukAbstract
Background
Despite widespread recognition of the value of lived experience (LE) involvement in healthcare research and increased LE involvement activity, we lack established implementation methods and instruments for reporting and evaluating impact. We present a protocol for an innovative LE-led Impact Log tool and co-production framework, which may help to address some fundamental barriers to co-production. The Impact Log will be implemented within a five-year multidisciplinary transdiagnostic research project on severe mental illness, the Brain and Genomics Hub of the UKRI Mental Health Platform, and is also designed for wider adaptation and use. Part I presents a short narrative review of literature pertaining to defining, evaluating, and enhancing the impact of co-production, to provide in-depth background and aid future development. Part II presents the Impact Log protocol.
Methods
The Impact Log framework is designed to integrate inclusive and impactful co-production throughout all research stages, and to record and evaluate its impact across three domains using an accessible short form. The three research domains are: design and delivery; interpersonal and environmental aspects; systems and processes. Impact Log design and implementation is led by LE study leads and a specialist advisory panel, who are integrated fully within the wider research team, and all have combined research experience and LE of bipolar or psychotic disorders. All Hub research participants will be offered accessible opportunities for remunerated lived experience input, and there will be outreach to ensure diverse representation, aided by the Hub’s charity partners. Data collection and analysis will be LE led and will include iterative analysis to inform continuing development. Diverse formal and informal dissemination throughout the project will maximise wider stakeholder engagement.
Discussion
The potential value of this research is to implement a novel tool and framework for facilitating, recording and evaluating co-production in complex mental health research, which can be adapted for wider use. Strengths in design are LE leadership and cross-cutting LE research integration, incorporation of multiple domains, and a focus on facilitating diversity and inclusion within co-production. Potential limitations for this project and wider adaptation may include limited resources, risk of bias and health challenges.
Lay Summary
We have provided a brief lay summary to help people without a research background understand our project.
This article explains our plan to develop and test a new way of understanding how research changes when people with personal experience of a mental health condition are part of the research team. We are a team of mental health researchers and many of us have direct experience of bipolar and psychosis. We work alongside other researchers, including people who might also have worked in mental health services or in charities that provide support. Our research project aims to better understand what is happening in the brain, body, lives and experiences of people who have bipolar and psychosis. Many people believe that research is better when it includes the views of people who have direct experience of the health condition being studied. This is called “lived experience”. We have developed a structured approach to make sure that people with lived experience are meaningfully involved in our research team. We have also created a simple tool, called the Impact Log, to record when lived experience members contribute and to help us understand how their involvement influences the research. Finally, we wanted to better understand what other researchers have said about lived experience involvement. We reviewed many published academic studies and reports and brought their findings together in what is called a “narrative review”. This review summarises what is already known about the difference lived experience involvement can make in research.
Article notes
Competing Interest Statement
TGs work is supported by the following awards: NIHR167361 (Home-based transcranial direct current stimulation in bipolar depression: a randomised, double-blind, placebo-controlled trial); NIHR132773 (Aripiprazole/ sertraline combination: a non-inferiority design in comparison with quetiapine in bipolar depression. An open label randomised controlled trial). OD receives consulting fees from McGill University (DIALOG) and the Feinstein Institutes (VITAL); TK has received consultancy fees from Bristol Myers Squibb and Boehringer Ingelheim; has received research grants from Takeda Pharmaceuticals and has received research grants and acted as a consultant for Akrivia Health for work unrelated to the current manuscript.
Funding Statement
The work of all authors on this manuscript is supported by funding from the Medical Research Council (MRC) to the Brain and Genomics Hub of the Mental Health Platform (Grant No. MR/Z503745/1). TG and TK also received funding for the initial development of the Impact Log from the Wellcome Trust, UK (grant number 203376/Z/16/Z), as part of the Mental Health and Justice project. The Brain and Genomics Hub is also supported by The National Centre for Mental Health, a collaboration between Cardiff, Swansea, and Bangor universities funded by the Welsh Government through Health and Care Research Wales. AJ receives funding from the MRC for DATAMIND (MR/Z504816/1) and the National Centre for Suicide Prevention and Self-Harm Research (Health and Care Research Wales); JHa is supported by the Hodge Foundation and The Waterloo Foundation; TG receives and TW received salary from Bipolar UK (until 30th August 2025). CH and JHe receives salary from Adferiad.
Background
There is growing international recognition of the importance of Lived Experience (LE) involvement, and it is increasingly incorporated into design and delivery of health research and service development. LE involvement can also be referred to as “service user” or “consumer” involvement, “PPI” (public and patient involvement) or “participatory” methods. There is ongoing debate concerning the extent and nature of family and caregiver involvement or “indirect” LE involvement. While caregiver perspectives are recognised as valuable, there is concern that this indirect viewpoint could supplant direct LE voices, particularly in mental health contexts where substitute decision-making and structural disempowerment are common.(1–3)
LE involvement was initially viewed as an alternative approach, challenging dominant methodological and epistemological paradigms. However, it is progressively becoming absorbed within mainstream medical research, with mental health often leading on both structural adaptation and scale of inclusion. There is clear evidence of systemic change and increasing LE involvement within research,(1, 4, 5) and LE involvement is increasingly identified as a research requirement in research funding, policy, and publishing (2, 6–9). Co-production is a model of LE involvement underpinned by deeper collaboration and equity. The term “co-production” generally refers to transformative inclusion of people with LE of the relevant health condition/s as collaborative partners, sharing leadership and decision-making, although it can sometimes include people with indirect LE.
Initially, LE involvement was driven by ethical and disability-rights motivations, with the right to inclusion and representation in medical and policy decisions exemplified by a principle of “nothing about us without us”.(10) More recently, there is growing recognition of the epistemological value of LE involvement, with “expertise by experience” understood as constituting a distinct form of knowledge, inaccessible to those without direct personal LE (3, 4, 8, 11). This is accompanied by recognition of co-production’s transformative potential for systemic and relational change Such change can include increased empowerment and structural inclusion of people with LE, together with attitudinal shifts, such as stigma reduction amongst researchers, clinicians, and the wider community.(1)
While increased emphasis on co-production is welcomed, co-production remains a relatively new branch of research. Significant gaps relating to methodology, systems, and culture must be addressed to facilitate complex, integrated, and impactful co-production.(12) A key challenge is developing standardised and replicable processes and tools, while ensuring that co-production retains the authenticity and diversity of LE voices: creating integrative frameworks which reflect established academic models, while simultaneously providing flexible and open spaces to articulate and respect voices and perspectives external to these established models.(11, 13–18)
A major barrier to overcome is to ensure that LE involvement is not tokenistic or ‘box-ticking’.(2, 6, 8, 9, 19–22) This will require recognition of distinct LE roles, skills, and contributions, alongside structural adaptation to create opportunities for LE leadership, decision-making, adequate resources, and cross-cutting LE involvement.(4, 5, 7, 19, 20, 22–26) Greater inclusivity and representativeness is needed, particularly from groups underrepresented both demographically and in relation to particular mental health conditions and their severity.(4, 5, 7, 11, 14, 15, 17–19, 27) We need, for example, to involve more people living with severe conditions such as bipolar and psychotic disorders. It is also important to ensure that LE involvement opportunities are created within biomedical and multidisciplinary research domains, and not restricted to psychosocial and qualitative research.
Systemic integration of co-production requires the development of robust and reproducible methods for recording and evaluating its scope and impact, while also acknowledging that co-production remains a developing field, incorporating diverse approaches and perspectives. Methods for recording and evaluation must be designed to capture breadth and complexity of impact and to enhance understanding, while also being adaptable for diverse study types and environments. Currently, to our knowledge, there is no widely used tool for recording and evaluating the impact of co-production which meets these criteria.(5, 17, 28–31)
The primary aim of this protocol paper is to present a novel co-produced tool, the Impact Log, for evaluating the impact of co-production (Part II). This is an exploratory project, implementing a co-production impact evaluation tool and laying down a framework for future development and adaptation. The research setting is a complex multidisciplinary mental health research project, the Brain and Genomics Hub (BG Hub) of the UKRI Mental Health Platform. To provide a more detailed overview of the background context for developing the Impact Log, we also provide a brief narrative review of literature relating to defining, evaluating, and increasing the impact of co-production (Part I).
Finally, development of the Impact Log and manuscript preparation has been led and largely conducted by people living with severe mental illness. Our co-production strategy integrates researchers with and without direct LE within an equitable research environment recognising all forms of expertise. We therefore hope to provide an example demonstrating the potential of fully co-produced research, led by researchers with LE, despite the health challenges that we face.
Part I of this paper presents a short narrative review on the impact of co-production. Part II presents the Impact Log protocol.
[For a table showing LE and other relevant experience and expertise amongst the authorial please see Table 2 in Authors’ Information below]
[For a table reporting details of integration of co-production into all stages of this project, see the GRIPP 2 form in Supplementary Materials 4]
Part 1: Defining, evaluating, and increasing the impact of co-production: a narrative literature review
1.1Narrative review aims and methods
This short narrative review aims to bring together literature relating to defining, evaluating, and increasing the impact of LE involvement, with “impact” understood here as a positive change on people, systems, or process. This review provides an in-depth overview of core ideas which have contributed to developing the Impact Log and its research framework. Previous reviews have identified evidence of impact, while also noting that this impact is often restricted to early stages of research and limited in terms of demographic and health diversity. They have also highlighted the need for increased reporting, evaluation, and consistency, which can capture different contexts and domains of impact, including the interpersonal.(4, 5, 20, 21, 24, 32–36)
This literature review was conducted by the BG Hub specialist LEAP (Lived Experience Advisory Panel), with a search and screening strategy designed to provide an overview, rather than systematic and comprehensive evidence synthesis. This included expert review; non-systematic literature search of peer-reviewed journal articles and grey literature; additional reference searches. TG conducted a PubMed search of title and abstracts on 24th July 2025, using terms combining core concepts of co-production. Results were screened for potential relevance and exported into a shared drive in Excel after deduplication (for more details on search strategy see Supplementary Materials 5.1). An iterative process of screening for relevance was carried out by EV, LG, HG, and TG during the data extraction phase, with any uncertainties discussed between at least two team members. A grey literature search based on core concepts identified above was conducted by CE using Chat GPT 4.0 in October 2025 and then checked to identify relevant publicly available materials from sources other than peer reviewed journals. References were exported into Excel by HG, re-checked and screened for relevance by HG, JHe, and TG. All LE team members were asked to review results and add papers or resources of potential relevance, and this LE expert review and reference search process continued throughout manuscript preparation, with additional papers integrated into analysis as it progressed.
A data extraction table created in Excel by LG was shared with EV and TG for discussion and approval, and adapted for grey literature by HG. This included study date, setting, and participants (for non-review articles); focus and key findings; how impact is defined and measured and main suggestions for increasing impact. Data extraction of peer-reviewed literature was shared between LG and EV, and grey literature between HG and JHe.
Combining guidelines for narrative synthesis in systematic reviews and thematic analysis,(37–39) we used an iterative deductive methodology to identify themes and subthemes relating to defining, evaluating, and improving co-production impact within the extracted data. This process was informal, insofar as a single researcher (TG) created and populated codes and themes, checking and adapting with narrative review team members at various stages.
1.2Narrative review results
We included 91 peer-reviewed journal articles including reviews, studies, and methodological papers, and five grey literature items, including two sets of guidelines, two comment pieces, and a workshop report. Most items originated in Europe, North America, and Australasia, particularly the United Kingdom, Australia, and Canada. However, we also included individual studies from Brazil, Ethiopia, Nepal, and Israel.
[See Supplementary Materials 5.2 and 5.3 for a full table of themes, subthemes, articles, and references for included articles].
We identified two themes relating to defining and understanding the impact of co-production: “Change to research / service processes and outcomes” and “Interpersonal and cultural change”. For processes and outcomes, impact was understood as an increase in quality, relevance, and impact of research and interventions; systemic changes to related institutional structures, processes, and policies, manifesting in areas such as leadership and resources; and changes occurring throughout all stages in the design and delivery of research or services.
Interpersonal and cultural change focused on manifestation of impact through changed attitudes, such as stigma reduction and increased acceptance amongst researchers and clinicians towards mental illness and co-production.
Four themes were identified relating to evaluating impact. Two themes, “Process indicators” and “Assessing impact of co-production on outcomes and dissemination”, suggested that impact could be measured through assessing tangible changes, whether systemic or in quality of outcomes and dissemination. The most dominant theme related to methodologies for assessing impact, within which the most dominant subtheme was “qualitative reflections”. This emphasised the need to gather primarily reflective and qualitative data, often without clarity about whether these reflections should come from LE or non-LE researchers. Other methodological subthemes were less common and more specific, including using specific methods for data collection and analysis or tools designed to assess co-production, and the need to include input from non-LE researchers and clinicians. Finally, another theme with diverse articulations was “The need for alternative ways of understanding impact when it is being assessed”, which referred to measuring impact from outside conventional psychiatric evaluative frameworks or through examining evidence of critique and change to conventional models and reduced stigma.
We identified two broad themes, with multiple and diverse subthemes, relating to increasing impact from co-production. These centred on the need for systemic and environmental change and for recognising the value and diversity of LE involvement. A common suggestion was environmental adaptation to facilitate support, power-sharing, building trust and flexibility. This included systemic changes to research structures, such as LE leadership, co-authorship, establishing academic LE posts, changes to language, offering payments for involvement work, and sustainability. Other subthemes were the need to provide training for both LE and non-LE researchers and to report and evaluate co-production, to increase transparency, adoption, and recognition. The most common subtheme focused on boosting diversity and recognition of value was the need for true integration of LE researchers, moving away from tokenism and stigma, with clarity, transparency, security, and clear communications surrounding roles and expectations. Another recurrent subtheme was the need for co-production to extend through all stages of research or service design and delivery. Two other subthemes related to making co-production more representative, through including diverse demographics and health conditions and through creating safe and valued spaces for LE contributors to maintain authenticity and facilitate critique.
1.3Narrative review discussion and implications
The importance of facilitating, reporting, and evaluating co-production is widely recognised. Nevertheless, it is widely acknowledged major systemic change will be needed to achieve this. These changes range from facilitating feasible and equitable LE employment, through specifics such as reimbursement, leadership, and authorship, to broader concepts such as support and flexibility. Finally, there is clear recognition that conventional research practices are insufficient to facilitate truly representative, diverse, and authentic LE involvement and capture its full impact. There is a need to recognise the importance of interpersonal and cultural shifts resulting from co-production, such as changes in researcher attitudes, power structures, inclusivity, and research models. A key theme relating to this need was the importance of integrating qualitative experiential reflections, and some also explicitly suggested that impact should be evaluated through assessing evidence of critique and conceptual change.
In conclusion, we can group findings into three well-established core domains relevant to all aspects of impact: change in design, delivery, and outcomes of research; systemic change in research processes and environment; and interpersonal and cultural change within the research community.(34) This clearly suggests that any impact evaluation tool and co-production strategy designed to capture the true breadth and complexity of impact must capture all three domains. However, our review also highlights a well-recognised challenge and conceptual divide facing co-production. Despite widespread recognition that we need more standardised and replicable ways to implement, record, and evaluate co-production, we risk compromising the authenticity and value of co-production by adhering too closely to conventional research models. It may not be possible to reach a perfect solution. However, work on co-production must be cognizant of this challenge and transparent about any resulting limitations.
Part 2: The Brain and Genomics Hub Impact Log protocol
2.1Methods /Design
2.1.1Aims
The BG Hub Impact Log project aims to implement a novel and widely adaptable tool to collect data related to the impact of co-production on research design, delivery, processes and on interpersonal and environmental aspects of research. Impact Log development is LE-led and co-produced, and we hope that analysis of Impact Log data will help expand our understanding of what constitutes the ‘impact’ of LE involvement in complex mental health research and how this can be evaluated. The Impact Log tool will be implemented in the BG Hub, a 5-year multidisciplinary and transdiagnostic study of bipolar disorder, schizophrenia, and schizoaffective disorder. The Impact Log is embedded in a broader BG Hub co-production framework, with a core objective to increase diversity and inclusion within LE involvement, including some training and career development. Another important objective is to provide opportunities for including meaningful LE involvement throughout all research stages.
Our broader aim is to produce a replicable and adaptable tool and framework for use within diverse mental health research contexts. In particular, we hope that this framework will facilitate increased co-production in complex and multidisciplinary mental health research, including biomedically-orientated components and research on severe mental illness.
2.1.2Study setting: The Brain and Genomics Hub
The BG Hub is a transdiagnostic and co-produced investigation of clinical, social, developmental, biomedical (neuroscientific/genomic), and LE factors relating to severe mental illness. It is one of six new research Hubs established in 2024 by the Medical Research Council and UKRI to form the Mental Health Platform and aims to transform understanding, diagnosis and management of severe mental illness. The BG Hub unites an interdisciplinary community of researchers and clinicians from South Wales and the Southwest of England, led by Cardiff University in partnership with Swansea, Exeter, Bath, and Bristol Universities, the charities Adferiad and Bipolar UK, and other LE advisors. The BG Hub has a complex and innovative co-production strategy (described below).
Psychotic and bipolar disorders are associated with a high burden of disease, severely reduced quality of life and premature mortality of between 10-20 years compared to the general population.(40–42) Despite this immense burden, our understanding of these conditions remains relatively limited. Unlike other areas of medicine, diagnosis in psychiatry is based exclusively on groups of symptoms and is not yet informed by aetiology or supported by biological markers. Symptoms often overlap across diagnostic categories and diagnoses are often changed, while individuals with the same diagnosis may present with markedly different clinical profiles, treatment responses, and outcomes. This diagnostic heterogeneity, instability, and overlap continue to hinder mental health research and the delivery of effective, personalized care.(43–46) To improve outcomes, it is essential to deepen our transdiagnostic understanding of the aetiology of schizophrenia, schizoaffective disorder, and bipolar disorder, identify early risk markers, and develop better predictive tools.
The BG Hub aims to investigate ways to improve diagnosis and understanding of these conditions to identify new targets for treatment and ultimately improve clinical care for people living with these conditions. There are two major research programmes: B-SPRINT will recruit a large, deeply characterised cohort of individuals with schizophrenia, bipolar disorder, and schizoaffective disorder (target N = 600). Data collection will include self-report questionnaires, detailed lifetime symptom interviews, cognitive tasks, biological samples for genomics and biomarkers, MEG and MRI brain imaging, and consent for linkage to routine health and social records for retrospective and longitudinal analyses. Once collected, this data will be made available in a secure and anonymous way to other researchers. B-SPRINT has received ethical approval from Wales REC 7 (reference: 25/WA/0027).
- The Bipolar, Schizophrenia, Psychosis Research Initiative (B-SPRINT) — a recruited cohort study.
- The Brain and Genomics Hub e-Cohort — a population-level data study using anonymised linked records.
The BG Hub e-cohort will use de-identified data from the SAIL Databank via the DATAMIND Hub, which securely links health, education, and social care records for people living in Wales. The study aims to identify early signs and risk factors that are linked to later diagnosis of schizophrenia, schizoaffective disorder and bipolar disorder.
2.1.3Study design: the BG Hub Impact Log
2.1.3.1BG Hub Co-production
The Impact Log has been co-produced as part of a broader framework of co-production aims and activities within the BG Hub.
[See Figure 1: BG Hub Co-production and Impact Log Structure]
Opportunities for LE involvement in research are often restricted to a few individuals, with input limited by time, budgets, and by assumptions from researchers about where LE involvement may not be feasible and valuable. A barrier to co-production is restricted demographic diversity across age, sex, and ethnicity, and limited involvement opportunities for people living with severe mental illness. Definitions of LE are often broad or vague, and studies may include people without direct experience of a relevant condition or only indirectly experienced through supporting others. The BG Hub is implementing a complex model to try to move beyond these restrictions.
First, the BG Hub is working with institutional, LE, and charity partners to help recruit diverse participants who are broadly representative of regional demographics. We aim to build a “community” amongst our participants, with opportunities for LE advisory work. The B-SPRINT consent process includes an option to give consent to being contacted about future research. However, it also includes a second question, giving participants the option to consent to be contacted in relation to opportunities for LE involvement: “I agree to let the study team contact me about opportunities to take part in both paid and unpaid lived experience advisory work. This includes opportunities where my lived experience of bipolar, psychosis, schizophrenia or schizoaffective disorder may be helpful in terms of giving advice to the research team in relation to the design, analysis and dissemination of this and future studies”.
We will develop varied and accessible opportunities for LE advisory work, aiming to involve a minimum of 120 (20%) study participants in remunerated LE advisory work, working to ensure demographic diversity. We have included the question about unpaid opportunities to allow, for example, for future surveys where budgetary and logistical constraints may make payment impractical. However, any substantive LE advisory work will be remunerated in accordance with NIHR guidelines.(47) For LE members of the core research team, remuneration will be line with current institutional salaries.
Given the complexity of the BG Hub and co-production framework, we also include what we have termed “dual expertise”. Dual expertise refers to people who combine expertise through LE (often known as “expertise by experience”) with research and/or clinical expertise.(48) Our aim is that our dual expertise researchers help bridge the divide between the research and wider community, using their combined expertise to help non-LE researchers navigate the co-production process, and helping develop ways to maximise inclusivity and accessibility for LE advisors without prior research experience. The BG Hub includes a minimum of six and maximum of eight dual expertise researchers with LE of bipolar and psychosis within its leadership team and within a Specialist LEAP (Lived Experience Advisory Panel). There may be occasional membership fluctuations due to changing commitments amongst LEAP members. All Specialist LEAP members have research experience and play a central role within the BG Hub (for details of LEAP demographics see Author Information). Career development opportunities and training will be offered to people with LE, with the Specialist LEAP offered opportunities to develop existing research experience, and other LE advisors offered an introduction to research work through initial paid LE involvement opportunities.
2.1.3.2Charity partners
The BG Hub partners with two charities, Adferiad and Bipolar UK. Both charities are represented at Hub leadership and include co-investigators and other members who will be involved throughout the research process. Bipolar UK is a charity involved in peer support, education, campaigning and research related to bipolar. Many staff members, including the research team, have LE of bipolar. Adferiad is a member-led charity providing support for people with mental ill health, addiction, and co-occurring complex needs across Wales, with established LE networks. In this partnership, Adferiad supports the co-production of recruitment and involvement processes with people with LE to enhance inclusivity, representativeness, and meaningful participation in mental health research.
2.1.4Impact Log development
2.1.4.1Origins of the Impact Log: the Mental Health and Justice Project
The BG Hub Impact Log is based on a version designed and used in a multidisciplinary five-year project, Mental Health and Justice, funded by the Wellcome Trust. As dual expertise researchers and Co-Leads of the project Service User Advisory Group (SUAG, a group of eight people with LE of mental health conditions), TG and TK created a simple Word document designed to record how service user input impacted on the research process (see Supplementary Materials 7). This form was not part of initial project design and was not included in the formal study protocol.
Impact Log forms were circulated to members of the research team after attending SUAG meetings. Researchers were asked to specify the questions they brought to the SUAG, SUAG input, how SUAG input impacted on their research, with options for additional details, and a final question requesting a brief indication of their general experience working with the SUAG. Impact Log data was qualitative, non-anonymised, and was not collected from SUAG members. The Impact Log data was shared with SUAG members who were offered the opportunity to participate in analysis. TG, who has expertise in qualitative methodologies, provided basic training in thematic analysis to SUAG members involved in the analysis process, and the initial coding was completed by 6 SUAG members.
Due to time and financial constraints, final analysis and dissemination of the Impact Log results and methods was not completed. However, this initial version of the Impact Log showed that both LE and non-LE members of the research community were keen to participate in providing data on impact and the data analysis process. Initial analysis strongly suggested SUAG consultation had great value for the researchers and had positive impacts including increased rigour, relevance and feasibility within various research stages, often exceeding the researchers’ initial expectations about the depth and scope of potential LE impact.
2.1.4.2Initial adaptation of the Impact Log for the BG Hub
Based on the success and feasibility of the initial iteration, the BG Hub team planned to use an adapted version of the pilot Impact Log to gather data on the impact of co-production. The Impact Log idea and original form was shared with the LEAP and wider BG Hub team, and the following initial adaptations were decided.
- To ensure that a broad understanding of impact is reached, the impact of co-production is understood as impact on study design and delivery, on all people involved in co-producing research, and on the research environment.
- To ensure that a full range of perspectives on LE impact are reflected, Impact Log data will be collected from everyone involved in any form of co-production activity, including non-LE research team members, the BG Hub LEAP, and LE advisors taking part in individual research activities.
- To ensure that people feel able to respond openly and honestly about their views, forms will be anonymous. All identifying information will be removed before analysis so that data is anonymised and non-identifiable.
- To ensure data security, Impact Log data will be collected and stored in accordance with the BG Hub wider data protection protocols.
- To maximise ease and accessibility, data will be collected via a simple electronic form, with paper versions made available if requested, and any data collected on paper transferred into secure data storage by a study team member.
- To maximise rigour in governance, the usability of data, and its potential impact, the BG Impact Log forms and process will be included in the BG Hub study protocol and Ethics Committee assessment.
- To ensure that a mixture of reflective and standardised data is collected, Impact Log forms will include a mixture of quantitative (closed) and qualitative (open) questions, amenable to mixed-methods analysis.
2.1.4.3Form development
Following initial consultation, an initial version of the BG Hub Impact form was co-designed and trialled by members of the LEAP and wider research team. Three distinct forms were created and approved following initial consultation and piloting (Copies of the Impact Log forms are available in Supplementary Materials 1-3): LEAP members and research team members engaged with BG Hub co-production activities on an ongoing basis were assigned a unique identifier number and given the option to include this on their Impact Log forms in order to allow longitudinal analysis of Impact Log data.
- The “Specialist LEAP” impact Log Form, for members of the LEAP who have combined expertise by experience and research expertise, and will contribute regularly throughout the BG Hub, as research team members.
- The “Lived Experience Advisory Work” Impact Log Form, created for PwLE who take part in individual advisory work for the BG Hub, with no expectation of specialist research knowledge or experience.
- The “Researcher” Impact Log Form, created for the wider research team employed by the Brain and Genomics Hub, who are not working on the BG Hub in a LE capacity and (unless undisclosed) lack direct LE of severe mental illness.
The “Lived Experience Advisory Work” form was also initially piloted through the BG Hub Participant Journey Day, a day-long visit to the B-SPRINT study site, led by the BG Hub research team, during the B-SPRINT project design phase. This involved eight people with LE, who would fit inclusion criteria for the study, but with no prior knowledge of the BG Hub. Participants were remunerated for their time in line with NIHR guidance and reimbursed for travel. The aim was to lead them through the experience of the planned study process and receive feedback. The day concluded with participants being asked to fill in a paper version of the pilot Impact Log form and provide feedback on layout, content, and usability. Answers and feedback showed that participants had a positive engagement with the Impact Log process and were comfortably able to complete the form within five minutes. Participants also highlighted the importance of providing a digital version of the form.
Following these stages of development the final version of the form includes a section on consent and a section asking for details of date, topic, and people involved in the co-production activity. People are then asked three pairs of questions about opportunities to contribute, the adequacy of explanations given, and the extent to which they felt that their opinion was listened to. Each pair includes a quantitative question using a Likert scale to gauge extent and a free text question asking for an explanation of this rating. Finally, there is one open question asking for any input relating to the individual’s experience. The researchers are given an additional question asking how LE involvement impacted on the specific aspect of the research process which they brought to LE advisors.
2.1.5Data management and analysis
2.1.5.1Data collection and storage
Digital versions of the Impact Log Forms were created using REDCap software and all Impact Log data is stored securely on Cardiff University servers, following data security protocols and ethics permissions for the B-SPRINT study. The data is only accessible to members of the BG Hub research team with permissions and an appropriate institutional login. Any members of the specialist LEAP or Charity partner representatives who choose to be involved in analysis of the Impact Log data will be given training in data governance and provided access to the data via secure Cardiff University servers.
Impact Log forms will be distributed to anyone involved in a BG Hub co-production activity on completion of the activity. This is usually done via a secure link or QR code. Initial consultation has shown that advisors generally prefer a digital form. However, we will make it clear that a written form is available if preferred. Impact Logs will be distributed and their completion tracked by TG, the BG Hub LE lead, who will not themselves complete the Impact Log forms. This is to mitigate the risk of bias, given that TG will be viewing and tracking all forms and leading on Impact Log development and analysis. For researchers, we will also circulate a follow-up form two months after the initial activity, to facilitate reflection about any changes which they may have now implemented or be planning to implement following from the co-production activity.
2.1.5.2Data analysis
Two types of data analysis are planned. All Impact Log data collected through the BG Hub will be analysed in the final year of the project. This will include longitudinal analysis of “Specialist LEAP” and “Researcher” forms, if digital identifiers have been included. There will also be iterative informal and formal analysis of the Impact Log data at various stages during the BG Hub. The results of this early analysis will be used to facilitate research team reflections on how the BG Hub co-production process is working and evolving, and whether any adaptations to the co-production or Impact Log should be considered. Finally, a key part of analysis will also be evaluation of the Impact Log tool itself, examining how well it is working, recording dates and contexts for circulating Impact Log forms, monitoring response rates and ease of use, recording any modifications made during the course of the project, and surveying users of the Impact Log at various stages.
Given that the Impact Log tool is a novel tool whose design has not included any established outcome measures, the analysis of Impact Log data will be an exploratory process. Analysis will focus on assessing impact across the three domains identified in our aims: impact on design, delivery, and outcomes of research; systemic impact on research processes and environment; and interpersonal and cultural impact within the research community. To achieve this, the Impact Log data will be considered in relation to the wider research context of the co-production activities being evaluated. Analysis will be led by the BG Hub LE lead, with assistance from members of the specialist LEAP and other BG Hub research team members. To ensure greater inclusivity and representativeness, we also aim to involve LE advisors from the wider study community in the analysis and dissemination process, following a framework for involvement to be developed by the specialist LEAP.
Data will be extracted into Excel for cleaning and data analysis, given the suitability of Excel for mixed-methods analysis of non-complex data. Using Excel also allows for maximal accessibility for LEAP involvement in data analysis. Finally, a qualitative analysis tool, such as NVivo, may be used for qualitative analysis. If NVivo is used, the Hub will ensure that the whole research team, including LEAP members, have access to NVivo software.
As the forms combine quantitative and qualitative questions, it is anticipated that we will analyse quantitative data descriptively using standard statistical methods, and qualitative data using thematic analysis. Quantitative and qualitative data will be analysed separately and then brought together to contextualise and explore patterns observed in the numerical responses. Where appropriate we will use regression models to analyse longitudinal changes in ratings, using statistical software decided in collaboration with the LEAP, and narrative synthesis to describe how themes or experiences evolve.
2.1.6Dissemination
There are two key dissemination objectives for the BG Hub Impact Log: to contribute to the ongoing development and dissemination of co-production activities and their impact within the BG Hub; and to inform design and delivery of other mental health research, and the training of mental health researchers and clinicians. Our hope is that the Impact Log and surrounding co-production framework can work as an adaptable blueprint for complex and impactful LE involvement in research.
Impact Log materials and evolving analysis will be shared internally within the BG Hub to support ongoing use and development of consistent co-production across workstreams. Impact Log materials, forms and guidance will also be made publicly available, enabling replication and adaptation elsewhere. Along with Impact Log data, study documentation (e.g., protocols, consent templates, and other participant-facing materials) will be hosted on DATAMIND, the designated data resource sharing platform of the UKRI Mental Health Platform. This data will be accessible via a data access committee to bona fide researchers. The availability of the data on DATAMIND promotes transparency and reproducibility and may be particularly useful for early career researchers seeking to conduct similar research.
Impact Log findings will be communicated in multiple formats and tailored to different audiences. Formal academic dissemination will include peer-reviewed publications and conference presentations. For applied settings, findings will be integrated into CPD activities such as internal training sessions or knowledge-exchange events for clinicians, researchers and LE experts.
Informal and public-facing dissemination will focus on accessibility and engagement. Key findings will be summarised through infographics, plain-language summaries, social media posts, and short animated or recorded videos designed for lay audiences. These formats will be developed collaboratively with LEAP members to ensure alignment with LE communication preferences and priorities.
Overall, the methods and data generated by the Impact Log initiative has a broad application scope and significant potential to be useful for clinicians, researchers, experts by experience, lay audiences, and ethics committees. Our dissemination strategy maximises the reach and utility of the Impact Log methodology and findings. By sharing materials openly and in multiple formats, we aim to support public understanding, enhance research and clinical capacity, and contribute to the evolving definition of meaningful impact in co-produced mental health research.
2.1.7Lived experience perspectives on the Impact Log design and dissemination process
To understand more about LE experiences of participating in this project, one of our Specialist LEAP members, JK, created and conducted analysis of a short anonymous survey, asking Specialist LEAP members about their experiences of contributing to the Impact Log design and dissemination process and their reflections on the importance of the Impact Log. Four main themes emerged: the importance of collaboration; the value of integration within the research team; the Impact Log’s potential as a methodological resource; the value of the Impact Log for increasing transparency in research and demonstrating the value of co-production (for themes and selected illustrative quotes see Table 1. For full results see Supplementary Materials 6).
LEAP members emphasised the importance of collaboration, both within the LEAP and the wider research team. Key themes were the potential of LE voices to enrich the research process, the power of including a diverse range of voices, and the learning that takes place during the co-production process. Another key theme was the value of LE integration into the wider study team and how this might help to mitigate the potential for tokenistic involvement and power imbalance in co-production, by facilitating the recognition and incorporation of their contributions to the research.
Another theme was the Impact Log’s potential as a methodological resource. LEAP members emphasised its potential utility across different aspects of the research process and noted its potential to capture and operationalise the inherently subjective and nuanced process of co-production. They also highlighted the Impact Log’s potential utility beyond the BG Hub and how the formal recording and evaluation of LE involvement in research might help embed co-production as a key methodological component within mental health and wider medical research. This utility might extend to stakeholders beyond the immediate research team and could include, for example, funding bodies.
Finally, LEAP members emphasised the Impact Log’s potential to increase transparency and accountability within research and to demonstrate the value of co-production. They suggested that the Impact Log tool might help researchers move beyond tokenistic involvement and identify areas where LE input has the greatest impact and areas where LE involvement could be improved and strengthened. The Impact Log was seen as something offering mutual benefit, both to the research project, but also, crucially, to the LE researchers by enabling them to see the impact of their involvement and thereby making the research process more beneficial and meaningful for them, in turn supporting their own development.
2.1.7.1Implications of the LE perspective survey responses
- The Impact Log tool has potential value for all research stakeholders and could help to establish co-production as a key research component, presenting a framework which can also be used, for example, by institutions associated with funding and policy.
- LEAP responses suggest that working explicitly to increase the integration and significance of LE roles within a research project, rather than framing LE contributions as supplementary, will increase the meaningfulness and impact of co-production for everyone involved.
- Given that the input of individuals with LE can be tokenistic and receives insufficient attention, it is critical to promote transparency and accountability in co-production. Having a tool which can evaluate impact and identify any problems in the co-production process would allow teams to take steps to address areas which are not working well and reevaluate the process.
2.3Discussion
In this protocol paper, we have outlined the development and plans for implementing the BG Hub Impact Log tool and the co-production framework and research context within which it is embedded. This tool will allow us to record and evaluate the impact of co-production across three research domains: design and delivery; systems and process; interpersonal and environmental. The Impact Log and co-production framework will also provide a replicable tool which can be easily adapted and used within other healthcare research contexts. It is important to emphasise that this study constitutes an implementation of a novel tool and that both design and analysis will involve an iterative process of development, as we observe how the tool and framework are performing within the BG Hub research context.
2.3.1Strengths and limitations
The key strengths of the BG Hub Impact Log and co-production framework lie in their explicit design to address and counter widely recognised challenges in co-production implementation, particularly tokenism and lack of structural integration, diversity, and acknowledgement of the complex and multifaceted potential of co-production. Co-production is integrated throughout the BG Hub, with robust plans to ensure that LE perspectives inform all research stages. People with LE of the conditions being researched are part of the core study team, including senior leadership, and the strength and value of this collaborative integration is already evidenced in our survey of LE perspectives on the Impact Log design and dissemination process.
LE leads, specialist LEAP members and other LE advisors have already had significant impact on major decisions about study design, and involvement in early dissemination including the BG Hub Launch, and structures are in place to ensure these contributions continue. Systemic adaptation and structural integration are further achieved by ensuring that LE contributions are swiftly and adequately remunerated, and that training, support, and resources are provided when needed. The BG hub is underpinned by a dedication to inclusivity and demographic diversity across demographics and health conditions. Our specialist LEAP exemplifies this approach, with members from minoritised ethnicities, varied ages, gender and sexual minorities, and disabilities (see Author Information below for further details on LEAP demographics). This has produced nuances within discussion which would otherwise be neglected. Moreover, the broader co-production framework has been designed to dedicate resources to ensuring that this diversity and inclusivity is embedded in wider LE involvement amongst our participant community.
To our knowledge the Impact Log tool and co-production framework are highly innovative in many regards. A key strength the high potential for adaptation and implementation within other healthcare research, making a valuable contribution towards methodological development in co-production and LE involvement.
There are some potential limitations, however, both in relation to both the BG Hub context and to wider implementation. The BG Hub Impact Log depends on resource availability, including funding, people, and time, which may be challenging elsewhere, particularly if resource allocation for LE involvement is limited during individual stages or throughout a research project. Successful implementation of the Impact Log will require sustained commitment to co-production and evaluation throughout research delivery. This may be difficult, even within the BG Hub, with the research team’s deep commitment to co-production, particularly when time is limited due to competing demands. Such challenges would be exacerbated by any power imbalances which leave LE team members with less power to press for prioritisation of co-production with other team members.
Ideally, implementation also requires people with LE and some degree of research experience embedded in the study leadership team. This might not always be possible. TG and TW had salaried positions supporting some amount of work in leading and facilitating co-production in the development of the BG Hub application, enabling the Impact Log and co-production framework to be built into study design from the initial stages. In general, however, a lack of funding supporting LE involvement in initial project development and funding applications is a well-known major barrier to successful co-production, and this could affect integration of an Impact Log framework.
Ongoing health challenges amongst the study team might also present difficulties, particularly in projects focusing on severe and chronic health conditions. The Impact Log framework is built around continuing input of a LE lead or leads, and other LE advisory research team members. Continuity may be disrupted by illness. It will be important to factor in contingency plans to ensure that co-production and impact evaluation can continue during any research team absences or reduced work capacity. Nevertheless, such contingency measures can be difficult to resource and it is recognised that LE researchers may feel uncomfortable disclosing that their health is impacting on work due to concerns about this creating negative judgements about their role as researchers amongst the wider study team.
Many factors might also contribute to a risk of bias in Impact Log responses submitted by LE and non-LE researchers, especially if there are concerns about anonymity. Although the Impact Log forms will be analysed anonymously, there is a high chance that the identity of respondents may be identifiable to the study team member/s responsible for collation and analysis. The potential for identification may be particularly high in smaller studies, with a smaller LEAP or number of LE advisors, and in longer studies where LE and non-LE team members will be submitting multiple Impact Log forms, especially if these are linked by an Impact Log ID number, known to the LE lead or other team members. Concerns about anonymity may constrain honesty in Impact Log form responses, particularly for negative opinions, if people have concerns about causing offence amongst fellow study-team members or impacting negatively on the research project. Conversely, negative views, for example, about the research model of a particular project, could potentially lead to negative bias in Impact Log responses. The risk may be mitigated to some degree, by emphasising the importance of honest disclosure, balance, and the fact that forms will be analysed collectively with identifying details removed. Moreover, if people feel confident in their integration and status within the study team, this may increase their confidence in providing honest answers, even if critical. Nevertheless, the potential for bias must be recognised and considered throughout.
2.3.2Future directions
The Impact Log will be used to collect data for the duration of the BG Hub research project and collected data will be used to inform ongoing development of the tool. It is hoped that this protocol paper will provide sufficient information and materials for wide dissemination, adaptation, and timely use of the Impact Log tool. There are already plans in place for BG Hub team members to begin to integrate the tool into other research projects.
The Impact Log tool and surrounding co-production framework involve many aspects which relate to ongoing discussions and methodological development in co-production. Alongside our work on developing the Impact Log within the BG Hub and disseminating this tool, there is much work to be done, both by the BG Hub study team and others, to explore wider development of co-production methodologies and to create future research projects enabling methodological innovation to be implemented and assessed. Finally, is it is hoped that the Impact Log project can support advocacy work relating to the promotion and integration of impactful, representative, and equitable co-production in healthcare research and practice.
3.2.1Creating an Impact Log and co-production community – please keep in touch
As part of our Impact Log development the BG Hub team would like to know more about people’s views and possible adoption of the Impact Log, to build a network and more collective understanding of how the tool might be used in different contexts. If you are interested in using the Impact Log tool, we would be very grateful if you might consider informing the corresponding author, TG, by email. Currently, we have not developed a formal feedback or communication process, but would be pleased to start building a network to allow for these future possibilities.
Supporting information
Abbreviations
- LE
- Lived Experience
- BG Hub
- Brain and Genomics Hub
- LEAP
- Lived Experience Advisory Panel
Declarations
Authors’ information
BG Hub LE researcher / specialist LEAP demographic details
All eight BG Hub LE researchers provided some basic demographic information, with the agreement that this would be collated and described to ensure anonymity and options to choose not to answer particular questions. Four LE researchers (50%) are aged between 25-34; with three between 35-44 and one researcher aged 45-54. Gender of LE researchers included male, female, trans-female, and non-binary. Disclosed sexuality was almost evenly split between researchers identifying as heterosexual/straight and bisexual. Half the team identified as white, with three other ethnic groups represented. Three researchers identified as having no religion, while five distinct religions/beliefs were named by the remaining researchers. Finally, under ‘additional information’ one researcher identified LE of mood disorder not captured in the Author Information table.
Data availability
Datasets generated and analysed in preparing this article are available in Supplementary Materials. These comprise the full set of responses for the survey “Lived experience perspectives on the Impact Log design and dissemination process” (Supplementary Materials 6) and a table of themes and reference list for the narrative review (Supplementary Materials 5). Given its length, the data extraction table for the narrative review has not been included in the Supplementary Materials. However, it will be made available to researchers via the corresponding author if contacted.
The Impact Log data collected during the BG Hub will be disseminated in accordance with the plans described above under “Dissemination”. Given that the Impact Log forms will gather potentially detailed qualitative data, full publication of the Impact Log forms dataset will not be possible. However, within the dissemination process, there will be consultation within the BG Hub research team and LE advisors from the wider Hub community about the publication of sets of collated responses to particular questions, with identifying details removed.
Ethics approval and consent to participate
B-SPRINT, including the Impact Log has received ethical approval from Wales REC 7 (reference: 25/WA/0027).
Competing interests
TG’s work is supported by the following awards: NIHR167361 (Home-based transcranial direct current stimulation in bipolar depression: a randomised, double-blind, placebo-controlled trial); NIHR132773 (Aripiprazole/ sertraline combination: a non-inferiority design in comparison with quetiapine in bipolar depression. An open label randomised controlled trial). OD receives consulting fees from McGill University (DIALOG) and the Feinstein Institutes (VITAL); TK has received consultancy fees from Bristol Myers Squibb and Boehringer Ingelheim; has received research grants from Takeda Pharmaceuticals and has received research grants and acted as a consultant for Akrivia Health for work unrelated to the current manuscript.
Funding
The work of all authors on this manuscript is supported by funding from the Medical Research Council (MRC) to the Brain and Genomics Hub of the Mental Health Platform (Grant No. MR/Z503745/1). TG and TK also received funding for the initial development of the Impact Log from the Wellcome Trust, UK (grant number 203376/Z/16/Z), as part of the Mental Health and Justice project. The Brain and Genomics Hub is also supported by The National Centre for Mental Health, a collaboration between Cardiff, Swansea, and Bangor universities funded by the Welsh Government through Health and Care Research Wales.
AJ receives funding from the MRC for DATAMIND (MR/Z504816/1) and the National Centre for Suicide Prevention and Self-Harm Research (Health and Care Research Wales); JHa is supported by the Hodge Foundation and The Waterloo Foundation; TG receives and TW received salary from Bipolar UK (until 30th August 2025). CH and JHe receives salary from Adferiad.
Consent for publication
Not applicable.