Patient Focused Medicines Development: The Book of Good Practices
Evolution of Methodology

Chi Pakarinen
PFMD/The Synergist
Paula Wray
NIHR INVOLVE, UK
Nicholas Brooke
PFMD/The Synergist
Katherine Deane
School of Health Sciences
University of East Anglia, UK
Simon Stones
Patient Advocate, Consultant, and Researcher
Mary Alice Dwyer
Medical Innovation and Insights
Thierry Escudier
Pierre Fabre Medicament
Anne-Marie Hamoir
PFMD/ The Synergist
Daniel De Schryver
Janssen
P

atient Focused Medicines Development (PFMD), today a global coalition of 32 members, began its Framework Building workstream in 2016 in order to respond to a need for a practical and actionable framework to define how stakeholders can start their patient engagement (PE) journey and continue to do more effective and meaningful PE. Starting with a landscape analysis of existing PE frameworks, one of the first outputs—the Patient Engagement Quality Guidance (PEQG)—was a co-creation effort of multiple Working Groups that involved more than 100 experts globally (representing more than 51 organisations). The PEQG went through four feedback and validation rounds, from internal reviews to a public consultation, before its release in May 2018.

Another identified need was the need for consensus among stakeholders of what “good PE” looks like, as well as an opportunity to satisfy a high demand to see practical examples demonstrating the quality and outcomes of PE. In response, PFMD created the Book of Good Practices (BOGP) to accompany the practical guidance provided by the PEQG. It is a set of real-life PE cases from different stakeholders illustrating how they have achieved the PE Quality Criteria presented in the PEQG (Figure 1).

This article explains the methodology used to select and assess initiatives to become a part of the BOGP, and it provides examples of the chosen initiatives that help illustrate what a “good practice” in PE could look like.

The first edition of the BOGP was released together with the PE Quality Guidance in May 2018 with eight case studies from industry, research, and patient organisations. The second edition of the BOGP, released at the end of October (2019), enriches and complements the initial set of examples and represents the evolving PE ecosystem.

Patient Engagement Quality Criteria
Figure 1. Seven PE Quality Criteria.

Maintaining the Initial Selection Process While Improving Scoring Methodology

In order to be included in the BOGP, initiatives had to go through a two-phased review procedure. In the first phase, there was an initial curation conducted by PFMD based on existing cases in the Synapse database and cases proposed to PFMD by any organisation during a submission period. In the second phase, there was a two-round anonymous review conducted by the external review group which consisted of six contributors (patient advocates, public sector and pharmaceutical industry representatives, as well as academic and independent experts), who participated in the co-creation of the PEQG and were actively involved in the PFMD Working Groups. Only cases that scored the highest in two or more PE Quality Criteria were included in the second edition of the BOGP published on October 31, 2019.

Initial Selection Process by PFMD
The selection process for the second edition was conducted using the same process as in the first edition of the BOGP (Figure 2):

  1. Identification: The first step was the identification of initiatives submitted to PFMD via the Synapse database (n=40). This initial selection excluded cases that were already reviewed in the first edition of the BOGP (n=197) and thus showed a smaller selection to begin with.
  2. Screening: First-pass screening and shortlisting was carried out based on completeness of information on the initiative and on the involvement of multiple stakeholders, one of which needed to be a patient or patient organisation.
  3. Eligibility: PFMD carried out the first review based on the information from the submissions and requested additional details from initiative owners.
  4. Inclusion: The remaining initiatives that entered the last stage went through an enhanced scoring process by the external review group to identify and reach consensus on cases for inclusion in the BOGP, as explained below.
photo of Initial selection process for PE initiatives
Figure 2. Initial selection process for PE initiatives to be included in the second edition of the Book of Good Practices review process.
Initiative Evaluation and Selection Process by the External Review Group
In the second phase, the selected initiatives from the first phase were anonymised and reviewed by the external reviewers in a two-round process. The first round consisted of an overall evaluation of the initiative and how well it addressed or demonstrated each of the seven PE Quality Criteria—with a possibility for reviewers to ask initiative owners for further details in order to obtain a more complete picture of the initiative (Figure 3).
image of The Book of Good Practices review process by the external review group
Figure 3. The Book of Good Practices review process by the external review group.
The second review round evolved from a simplified assessment of whether or not the initiative exemplified each PE Quality Criterion (Yes/No) to a more granular scoring system. To avoid “reinventing the wheel,” PFMD proposed utilising the four levels of engagement (as used in Synapse) as the basis of the improved scoring system—a proposal the external reviewer group approved. The scoring was set up as a four-point Likert scale (Table 1), and only initiatives that received majority scores of “Advanced” or “Outstanding” in two or more Quality Criteria were included in the second edition of the BOGP.
Level of patient engagement (in Synapse) Level of exemplification of PE Quality Criteria
Inform

  • No involvement or engagement. Main goal is to keep the patient or public informed about the project
 None to low
Does not exemplify criteria (Likert: 1)
Consult

  • Obtaining unput, preferences, and feedback on project, analyses, decisions, etc.
 Fair/Somewhat
Somewhat exemplifies or addresses criteria (Likert: 2)
Involve (including collaboration)

  • Working or partnering with the patient community directly and throughout the process; preferences, cocerns, and expectations are taken into consideration; and patients know how their input influenced the decision
 Advanced (Good PE)
Good example of the criteria (Likert: 3)
Co-Design (where higher level is empowerment)

  • Patients are involved in innovating, designing, developing, and formulating solutions and alternatives, and in each aspect of the decision-making process. Recommendations are incorporated into decisions to the maximum extent possible
  • Higher level: patients have decision-making power
 Outstanding (Exceptionally PE)
Exceptionally high level of PE in the criteria (Likert: 4)
Initiatives Selected for the Book of Good Practices
The five initiatives or “good practices” that passed the review process to be included in the BOGP represent real-life cases where patient engagement has been thought through from the beginning of the project, together with partners. All five initiatives exemplified three or more Quality Criteria, and three of these initiatives exemplified all seven. Although not an exhaustive list, the tables below provide a summary highlighting some of the highly scored Quality Criteria, positive outcomes, and lessons learned by these five initiatives. To explore the five initiatives further, please download the BOGP from PFMD website here.
Supplemental Resources

PE Quality Guidance: A practical tool to facilitate planning, developing, and assessing the quality of PE activities and projects throughout the research, development, and lifecycle of medicines.

Read about the first edition of the Book of Good Practices in the PFMD blog.

Synapse: A patient engagement landscape mapping and networking tool for stakeholders to share knowledge and resources as well as access co-created PE tools.

Conclusion

The BOGP was created out of a need and contributes to raising awareness of what good PE could look like. The aim of the selection, evaluation, and scoring process was to provide a more systematic mechanism to assess diverse PE initiatives and, in the future, a way to standardize considerations of the PE quality. The scoring process as a whole will be improved based on reviewers’ feedback. It will continue to evolve together with (and as a reflection of) the fast-moving PE landscape, as demonstrated by the increase of eligible initiatives from 4.1 percent in the first edition to 12.5 percent in the second edition.

To summarise the overall reflections of the external reviewers, the BOGP provides stakeholders with a set of graded PE examples, allowing them to compare their practice with others and to identify methods they might use to further enhance their PE. “It not only establishes fundamental criteria for meaningful PE, but also provides ideas and examples of how PE can be done,” and it can be used as a reference tool both by those who are experienced with PE and by those who are just beginning their PE journey.

Tables 2-6. Short summaries of examples in the Book of Good Practices, second edition.
Initiative Pregnancy and Parenting with Arthritis
PE Quality Criteria Highlights 1 – Shared purpose
A project proposal was drafted and shared with the [involved stakeholders] engaged in the pregnancy and parenting project. The shared purpose was the subject of regular updates at monthly CAPA* conference calls and email interactions with stakeholders.
5 – Capacity and capability for engagement
Ongoing communication with stakeholders was central to the implementation of this project. Because of regular check-ins with stakeholders, there were no issues or concerns with the ability of all participants to contribute effectively and meaningfully. No concerns were expressed by stakeholders at any time during the project.
6 – Documentation and transparency in communication
A project page was created on the website to post all relevant project materials for members and other stakeholders. In addition, a resource page was created along with important information highlighting collaborations which can increase patient knowledge relating to pregnancy and parenting with arthritis. Project updates were communicated via the organization’s newsletter and thorough collaboration with arthritis bloggers and other patient groups.
Positive Outcome
  • The survey identified several unmet medical needs, such as gaps in counselling provided to people with arthritis as well as regulatory gaps with medications used during pregnancy and breastfeeding.
  • As a result of the project, new relationships developed between CAPA* and others in the arthritis community.
Lessons Learned Given that the project was led by a volunteer patient organization, the project was undertaken over a long period of time. There were difficulties managing resources and time commitments due to the volunteer nature of our work and other life commitments, including managing a serious, complex disease like inflammatory arthritis.
* CAPA: Canadian Arthritis Patient Association
Initiative HeadUp Collar: Co-creation of a new cervical orthosis for patients with Motor Neuron Disease
PE Quality Criteria Highlights

5 – Capacity and capability for engagement

  • Members of Research advisory group were invited to training days and educational activities, such as the International Clinical Trials Day and a wide range of public outreach events, a free online training course, and access to University of Sheffield library services.
  • The Public and Patient Involvement and Engagement focus groups were given information about the project, and care was taken to ensure that they felt that they could speak freely on both positive and negative aspects of different neck collar prototype designs.

6 – Documentation and transparency in communication
The group was updated with all aspects of the developing design process and the deliverables of the collar’s clinical assessment, including recruitment rates for participation in user testing. Group members were involved in disseminating the research and featured in press releases and a short film (available on YouTube).
7 – Continuity and sustainability
Forward-looking budgeting for the group beyond the duration of the HeadUp project was applied for, providing a model for funding newly set up groups in other neurological disease areas.
Positive Outcome
  • The results of the HeadUp Study 100 collars project showed that 80 percent of patients preferred the HeadUp collar (a patient-prioritized and -codesigned solution to patient-identified unmet need) over existing designs.
  • Continued patient involvement in press-related activities following the launch of the product has helped raise awareness and drive demand for the collar, with a spike in healthcare providers and private customers making enquiries for the product after these activities, extending the reach of the new device.
  • New IP generation through patient-driven research prioritisation and better investments in research and development.
Lessons Learned
  • The format of the Research Advisory Group of Patients and Careers facilitated long-term co-development of research and succeeded in taking a patient-identified priority from an idea to a clinically adopted and commercially available product.
  • The value of patient engagement from the early stages of research can sometimes be overlooked when overstretched investigators need to arrange activities ad hoc or find the financial means for any incurred service charges. A relatively small amount of public or collective funding to run a group that researchers can easily access can have a large impact. In this case, the total cost to run such a group was no more than £1,500 per year, including administrative costs. The total global market potential for cervical orthoses is $27M. Applied over many different research areas, the value of patient engagement as an investment should not be underestimated.
Initiative A sickle cell community built by the community for the community
PE Quality Criteria Highlights

3 – Representativeness of stakeholders

  • The Working Groups have a wide variety of experiences and perspectives that are valuable to the sickle cell ecosystem, including those of patients, advocates, caregivers, healthcare professionals, researchers, and pharmaceutical manufacturers.
  • By inviting leaders with various types of expertise, a matrix of specific expertise categories was created to ensure that the various aspects of the disease and lifestyle will be addressed.

4 – Roles and responsibilities
Written contracts provided the formal relationship defining the roles and responsibilities of each Working Group member, including a schedule of meetings, time commitments, and a description of the work to be performed.
7 – Continuity and sustainability
At the outset, the oneSCDvoice.com team provided long-term contracts to each Working Group member using simple language to ensure that everyone understood that the commitment was not just to build an online community and launch it, but to nourish and grow it over time. The commitments were clearly delineated during both the pre-launch phase and the post-launch maintenance phase.
Positive Outcome The use metrics of the knowledge library identified unmet educational needs of the community, including deeper education about clinical trials and overcoming the barriers to diversity in clinical trial participation.
Lessons Learned A consultation with community leadership at the creation stage of oneSCDvoice.com avoided costly missteps and provided a patient-focused service that truly resonated with the end user as an authentic home on the internet.
Initiative Patient involvement in preparing peer-reviewed clinical research publications or results summaries: A systematic review
PE Quality Criteria Highlights

1 – Shared purpose

The shared purpose of this project was agreed upon (1) verbally during author candidate calls and the author kick-off meeting, and (2) in writing with every author signing an official Authorship Agreement that outlined the shared purpose of the project.

2 – Respect and accessibility

To help ensure respect and accessibility in this publication project, we:

  • prepared a plain language Authorship Agreement and a plain language summary of the Good Publication Practice 3 guidelines,
  • clarified payment considerations in the Authorship Agreement,
  • provided practical ways to meet as authors and gain feedback from authors,
  • provided electronic copies of documents to all authors, and
  • ensured all authors were aware of the rules of conduct when working together as co-authors on a publication.

4 – Roles and responsibilities
We also had regular checkpoints during the project so that responsibilities could be clarified and reinforced. Communication was encouraged during the calls and at any time between calls if any author required further information or explanation.
Positive Outcome Positive effects for patients from this project included:
Providing patients with access to robust evidence, proving that patients can be involved as authors on peer-reviewed publications.
Lessons Learned Practical tools and plain language documents can and should be prepared to facilitate ethical and effective involvement of patients in publications. Developing these tools and documents takes time, effort, and resources, but they can be re-used and shared to the benefit of others.
Initiative Consulting a patient and career panel on the design and delivery of a proof of concept drug repurposing trial in Parkinson’s Disease
PE Quality Criteria Highlights 1- Shared purpose
The overarching purpose of the research interest group was agreed on by members upon joining and by researchers in advance of engaging with the group. The specifics of the activity were agreed on by email and in person during the first meeting chaired by a Parkinson’s UK Public and Patient Involvement coordinator.
3 – Representativeness of stakeholders
There was a balanced mixture of representatives in terms of male-to-female ratio, lifestyle, years since diagnosis, and experience with research participation among both patients and family members in the research interest group. This balanced distribution was not specifically selected but arose by chance via the self-selected regular patient group memberships.
6 – Documentation and transparency in communication

  • Before meeting face-to-face, the following documentations were shared between all stakeholders: a drafted summary of the study, patient facing documents, questions to consider in relation to these documents, and the contact details of the research team.
  • To ensure that timelines were still reasonable and workable for research interest group members, they were agreed on in advance and confirmed in person at the face-to-face meeting. The excellent email responsiveness from the research interest group members suggests that all communication and documentation was useful and well-implemented.
Positive Outcome Co-developing modifications to the study protocol and including the patient voice to documents that were submitted for ethical and regulatory review likely decreased the time to study registration.
Lessons Learned The introduction segment of the session made sure that patient volunteers understood all aspects of the proposed study and felt comfortable asking questions throughout the duration of the project. Their full comprehension was key to co-developing the protocol and emphasized the importance of building relationships with patients to facilitate a frank and open discussion of the ongoing research. Continued involvement will facilitate relationship building and provide an opportunity for patient input at all stages of the research cycle.
Back to Issue