Anna Mierzynska¹, Natalia Sabolova², Katarzyna Piotrowicz³, Eric Eisenstein⁴, Monika Hricova², Ljuba Bacharova⁵

¹Gruca Orthopedic and Trauma Teaching Hospital, Department of Rehabilitation, Otwock, Poland

² Pavol Jozef Safarik University, Kosice, Slovakia

³ Military Institute of Medicine National Research Institute, Warsaw, Poland

⁴ Duke University, Durham, NC, USA

⁵ International Laser Center CVTI, Bratislava, Slovakia

Abstract

The International Research Interdisciplinary School (IRIS) program is a training for young researchers with a biomedical background who are interested in acquiring the methodological knowledge and experience in preparing a study protocol for a project. The IRIS program is an outcome-oriented problem-solving workshop designed to promote team collaboration. The paper describes the process of moving from the training project proposal into a real-life research project. It compares the initial proposal and its assumptions with the reality of writing a research protocol and management of the study. It also reflects on the obstacles met at each stage of the project (protocol preparation / team recruitment / data collection and analysis / manuscript writing) and strategies to overcome difficulties regarding conducting the study inspired by the training project proposal.

Key words: research study protocol preparation, research management, academic writing, education, action crisis

Introduction

The research protocol is a document that describes “what a clinical study will do and how it will be done (a).” The development of a well-conceived research protocol is seen as an essential prerequisite for successfully managing and completing a clinical research project. The program of the International Research Interdisciplinary School (IRIS) is focused on training young researchers to prepare a study protocol for a project (1), in an outcome-oriented problem-solving manner.

The IRIS program consists of four workshops. Participants are divided into small groups that are international and interdisciplinary. All workshops have a similar structure: (1) participants work in groups, (2) each group’s results are presented for the plenary discussion, and (3) the group receives comments/suggestions from other participants and the international faculty. In the first three workshops, group and plenary discussions are focused on: (1) selecting a research topic of common interest defining the research topic’s significance and developing a study hypothesis; (2) developing the optimal study design to test the hypothesis, and (3) defining the study’s variables and data collection methods.  

The IRIS program’s fourth workshop is focused on the study’s project management and administration. Discussions are centered around creating a project timetable that is as realistic as possible, considering possible obstacles such as national conditions, rules or laws and personnel requirements.

However, even a detailed discussion of a “training project” does not necessarily cover all facets of a “real-life” situation. Therefore, we were interested in how a project timetable prepared during the IRIS training compared with the actual timetable for the same project. In this paper, we compare the “training timetable” of a project developed during an IRIS program with the timetable of its actual project as completed.

Brief characteristics of the real project are as follows:

The title: Action Crisis in Orthopaedic Rehabilitation Process (ActCORP);

• The principal investigators: Natalia Sabolova (a former IRIS participant), Slovakia, and Anna Mierzynska, Poland (a former IRIS participant, now IRIS faculty);

•Project site: Poland;

• International Research Interdisciplinary School mentors: Ljuba Bacharova, Slovakia, Eric Eisenstein, USA, Katarzyna Piotrowicz, Poland (international IRIS faculty);

•Monika Hricova (the official mentor of Natalia Sabolova);

•Communication: on-line, using MS Teams.

Comparison of timing and duration of individual activities: the plan versus reality.

Table 1 compares the timing and duration of individual activities in the training project and in the actual project, showing the agreements and disagreements between planning and reality. The description of each timetable section and reasons for differences between the training and the real project are discussed below. Figure 1 represents the timeline and subsequent tasks and challenges that the research team had to face, which had an impact on the team engagement and enthusiasm towards the project, compared to the hype cycle model (5).

Preparation of protocol

Preparation of protocol is the most challenging part of a research project since it shapes all the next steps. In this particular project, the impetus for the study was a project protocol developed during an IRIS workshop. For this reason, the initial idea regarding the scope of the study, primary variables and assessment tools was already available for the actual project’s research team. However, debating over the final shape of the project protocol required frequent meetings and various adjustments to the initial research design. These changes derived from the different approaches and experiences of new research team members, as well as national/local rules and conditions, and needed to be discussed during research team meetings.

The training project protocol was designed to assess action crisis in diabetic patients.

The term “action crisis” refers to a phase in goal processing (e.g., commitment to treatment or rehabilitation), when obstacles can occur. Experiencing an action crisis may lead to experiencing higher level of negative emotions (frustration, anger, sadness) and results in goal disengagement (e.g., nonadherence to treatment or disengagement from rehabilitation). A higher level of action crisis can be also associated with a slower or compromised recovery after an injury (6). Therefore, it would be valuable to assess the action crisis in patients whose treatment requires a commitment to effective collaboration with a medical team, such as diabetic patients, or those during the rehabilitation process.

However, the initial project needed to be adapted to a real-life setting. Since the actual project team member responsible for data acquisition worked in the cardiology and rehabilitation fields, these patients’ populations were considered as possible study groups. The final choice of assessing action crisis in orthopedic patients was made based on the setting being more predictable for data gathering (fixed time of hospital stay, more homogenous patient group in terms of demographic and clinical characteristics). Patients undergoing post-trauma rehabilitation might experience an action crisis that leads to goal disengagement. Moreover, there is a lack of knowledge of whether orthopedic trauma patients experience action crisis related to their personal goals obstructed by their diagnoses and how comprehensive orthopedic rehabilitation affects progress in the personal goal attainability and desirability.

Therefore, the research team decided that the aim of the study will be to examine the level of action crisis in personal goals obstructed by patients' diagnosis in orthopedic trauma patients and the association between the action crisis level and the goal progress before and after rehabilitation. Furthermore, the health-related quality of life (HR-QoL) and illness perception can be associated with action crisis. For that reason, the additional aim of our study was to assess the relationship between HR-QoL, illness perception and action crisis in post-trauma patients.

Taking together, the preparation duration for the real project was nine months.

Figure 1. Timeline and the level of engagement and enthusiasm towards the project. The course of the enthusiasm of the team members was surprisingly comparable with the Gartner Hype Cycle, which has five phases (1) technology trigger, (2) peak of inflated expectations, (3) trough of disillusionment, (4) slope of enlightenment, and (5) plateau of productivity (5)

Funding documents

Research projects are often financially demanding due to the costs of the materials, administration of the study, and publishing the study’s results. Being an effective funding applicant is therefore considered an additional research skill, and in many cases is as useful as writing the project protocol. For this reason, many institutions hire funding administration specialists, who provide researchers with crucial support in applying for and managing research funds. In many countries, the only options available are projects performed with the use of the clinic’s own funding (statutory funding) and national research grants. Applying these funding options requires knowledge of funding application calendars as missing a deadline can lead to delays in research projects.

In preparing our project, we considered several funding possibilities, both in Poland and Slovakia, and finally decided to apply for funding from the National Centre of Science in Poland’s section for pilot studies. Collecting the required documents, writing the proposal, and providing additional required information, e.g., the approval of the department head, took around 12 weeks. This delay was due to administrative process in the main site and the research team’s meeting schedule. The time to receive the proposal review and learn the funding decision can take up to 6 months. We were informed that despite the proposal’s good reviews and the researchers’ experience, it did not achieve a high enough overall score to qualify for funding. Thus, while in the training project the preparing the funding documents and application was planned for one month, in the real project, it took more time as the schedule was shifted to meet local deadlines.

Team recruitment

In IRIS workshops, participants are assigned to project teams by the faculty. This means that the main team recruitment concern is to define a study scope that will engage all team members and utilize their areas of expertise. In this way, IRIS training allows participants to explore and compare their knowledge and research interest of participants. The IRIS experience encourages participants to collaborate with specialists, who typically would not be considered as research team members, and it helps to creatively utilize the expertise of every team member. In the training project it was assumed that team recruitment would take six months.

In a real-world setting, a narrower approach is usually preferred. This means that the study aim largely drives the search for appropriate team members. In the present project, there was an opportunity to balance these two approaches. The first draft study protocol was developed during an IRIS workshop, which sought to involve all group members. When moving to a real-world setting, the initial team members recognized the need to collaborate with researchers who could improve the initial design and would have relevant experience in collecting and analyzing clinical research data. This meant that they needed to include clinical researchers who were able to recruit patients and collect the appropriate data in a standardized manner. The time for team recruitment in the actual project was 7 months.

Ethic documents

Ethical evaluation and approval is a crucial research element, which ensures that participants’ well-being and their rights are respected. Bioethical committee approval occurs during or just after applying for funding. Some funding institutions require that the ethical approval is included in the project proposal. Similarly, funding institution may approve funding but requires assurance that an ethical evaluation will occur before the first participants are enrolled.

In the present study, we assumed that acquiring ethical approval would be simple. This was because the study design was observational, there was no interference with treatment, and participants would not be at any risk for their well-being. Contrary to our assumptions, obtaining ethics approval was time-consuming. While there were no issues with the study design, various administrative factors did cause delays. The hospital where participants were to be recruited was a part of a broader medical teaching institution that usually requires a full ethical evaluation prior to the study data collection. However, this requirement, according to the institution’s regulations, was only applicable to experimental studies. In contrast, observational studies did not require full institutional ethical evaluation, and their ethics approval could be made solely with a hospital’s approval. Obtaining appropriate feedback from both the parent institution and the hospital was a time-consuming process that delayed patient recruitment.

The most important lesson for the research team was that they needed a thorough understanding of the person or body responsible for ethical evaluation appropriate for a specific type of study.

Data collection

Data collection can be the most time-consuming project phase and typically requires that informed consent and raw study data are obtained from the participants. In research projects that use psychological variables requiring participants’ self-evaluation, study subjects are expected to answer multiple questions regarding their well-being, psychological characteristics, or other variables included in the research design. According to the principles of conducting psychological studies, the research team member who is responsible for data collection should accompany the patient and be prepared to explain the study’s aim or help the patient to understand questions or items which are unclear to them.

In our study, recruitment and data collection were incorporated into the hospital’s routine psychological screening. This meant that participants were asked to give their consent to participate in the study and complete additional questionnaires after they had completed the questionnaires used to evaluate patients’ well-being during initial psychological assessments. Patients also were informed that they would be asked to complete the same set of questionnaires at the end of the rehabilitation process so that their self-evaluations before and after the treatment could be compared. Since the additional time spent on data collection was not burdensome for both parties (patient/researcher), patients usually agreed to take part in the study. Patients often discussed questions with the researcher, referring to their experience with the treatment and the impact of the disability on their lives. This information, even if not meant to be analyzed in the quantitative study, provided additional insight into their adjustment and expectations towards the treatment, and it was a valuable input into the collaboration with the clinical team.

The two main differences between the planned and actual data collection derived from (1) the need to make sure that the Polish translation of the questionnaires was understandable for participants and (2) the unexpected termination of the recruitment.

Addressing the first issue required asking patients for their feedback about the questions and altering a few phrases to make the questionnaires clearer and more concise, while still being an accurate translation of the original Slovak version. The data obtained during the time the translation was being corrected (first two months) was not included in the results analysis. 

The second issue related to an unforeseen change in the form of research team collaboration with the site. These changes forced the team to end patient recruitment before the scheduled date and shorten it from the planned 12 to 7 months (Table 1).

Data analysis

The analysis of study data requires both knowledge of statistical methods and the study variables. Conducting the appropriate analyses involves not only mathematical skills but also an understanding of the data collected during the results interpretation. Since teaching statistical analysis is not always included in the core curriculum of medical professions, research teams frequently have to either recruit data scientists/statisticians or outsource these tasks to trusted individuals with experience in analyzing biomedical and/or psychosocial data. The IRIS workshop includes a session during which participants identify the types of variables and basic statistical tests that will be used to conduct appropriate analyses of their study data.

In the present study, team members had previous experience with the variables and types of data analysis described in the protocol. This meant that the study team could analyze the data collected in the study without outsourcing this task. The study’s results were interpreted by two research team members who had experience in analyzing psychosocial variables, and their findings were discussed with all team members to formulate the results discussion for the manuscript.

Developing the study manuscript requires efficient collaboration between all team members. It is reasonable to discuss and agree upon the order of authors and their writing assignments beforehand. This agreement will divide the work between team members and facilitate collaboration in completing the final manuscript. Since journals typically require information regarding authors’ individual input, having an agreement will help to avoid confusion over authors’ rights. It is also wise to reach an agreement between authors on the deadlines for different stages of manuscript preparation (e.g., outline, first draft, final version). This will avoid unnecessary delays that may be caused by various professional responsibilities and other situations that may influence the pace of drafting the paper. Most journals have instructions for authors that describe their preferences and expectations regarding the manuscript’s formal characteristics, e.g., length or style, therefore it is necessary to identify the journal where the manuscript will be submitted before writing the draft.

For the present project, the team members decided upon the authorship of future research papers at the time the research team was formed. This agreement defined each team member’s scope of responsibilities, allowing work to be shared based on each researcher’s experience and responsibilities during project implementation. Since the study investigated psychological concepts, research team members with a psychological background identified journals that might be interested in publishing the study’s primary manuscript. This manuscript’s introduction, study aim, and methods were written at an early stage (the research protocol preparation); however, they needed to be reformatted to meet the journal’s guidelines. During the manuscript writing period, the team met frequently, which sustained their motivation to advance the project and obtain support and guidance from each other.

Conclusion

Experienced researchers are well aware of the discrepancy between planning and execution. The experience gained in this project showed the importance of paying attention to the critical elements affecting the efficiency of the process and the time to completion. This experience emphasizes the importance of the research protocol, which will create the road map for implementing a successful scientific project.

Lessons to learn:

•                               A need for balancing the scientific interest and feasibility

•                               Importance of talking and listening

•                               Make the study as simple/clear/explicit as possible

•                               Importance of the support from the official mentor / the head of the department

•                               Management

•                               Starting with a smaller project before moving to huge international projects

Peer-review: External and  Internal

Conflict of interest: None to declare

Authorship: A.M., N.S., K.P., E.E., M.H., and L.B. contributed equally to the study and preparation of manuscript

Acknowledgement and funding: None to declare