Outcome evaluation by patient reported outcome measures in stroke clinical practice (EPOS) protocol for a prospective observation and implementation study
Neurological Research and Practice volume 1, Article number: 28 (2019)
The impact of stroke-related impairment on activities of daily living may vary between patients, and can only be estimated by applying patient-reported outcome measures. The International Consortium for Health Outcome Measurement has developed a standard set of instruments that combine clinical and longitudinal patient-reported outcome measures for stroke. The present study was designed (1) to implement and evaluate the feasibility of the use of it as a consistent outcome measure in clinical routine at the stroke center of a German university hospital, (2) to characterize impairment in everyday life caused by stroke, and (3) to identify predictive factors associated with patient-relevant outcomes.
We plan to enroll 1040 consecutive patients with the diagnosis of acute ischemic stroke, transient ischemic attack, or intracerebral hemorrhage in a prospective observational study. Demographics, cardiovascular risk factors, and living situation are assessed at inpatient surveillance. At 90 days and 12 months after inclusion, follow-up assessments take place including the Patient-reported Outcomes Measurement Information System 10-Question Short Form (PROMIS-10), the Patient- Health Questionnaire-4, and the simplified modified Ranking Scale questionnaire. The acceptance and feasibility (1) will be assessed by a process evaluation through qualitative semi-structured interviews with clinical staff and patients and quantitative analyses of the data quality evaluating practicability, acceptance, adoption, and fidelity to protocol. The primary outcome of objective 2 and 3 is health-related quality of life measured with the PROMIS-10. Additional outcomes are depressive and anxiety symptoms and patient participation in their social roles. Patient-reported outcomes will be assessed in their longitudinal course using (generalized) mixed regressions. Exploratory descriptive and inference statistical analyses will be used to find patterns of patient characteristics and predictive factors of the outcome domains.
The results will describe and further establish the evaluation of stroke patients of a stroke center by standardized PROMs in everyday life.
Stroke is a major cause of death and disability worldwide. Although the mortality of stroke has decreased through improved treatment by new interventions and more efficient management strategies, the overall societal burden of stroke has further grown during the last decades [8, 17]. Three months after stroke, about 40% of stroke patients are either dead, living in a nursing home, or depending on continuous care from others . Direct deficits resulting from stroke, e.g. paresis or aphasia, are easily spotted, and well characterized by clinical rating scales, which are used in clinical practice. There are, however, indirect consequences of stroke which are more difficult to detect unless they are looked for by purpose. Stroke patients are known to suffer from problems with impaired cognition, anxiety, and depression amongst others, which in turn, affect functional status, and satisfaction in daily living, and which often only become manifest in the long term in everyday life as outpatients [1, 16, 23]. These highly relevant consequences of stroke are often missed in standard assessments by physicians or in clinical scales applied by stroke researchers.
The past years have brought a movement towards a more patient-centered evaluation of quality and value in health care, summarized by the term of `value-based healthcare´ [20, 21]. In this context, the evaluation of quality of care and treatment success by patient-reported outcome measures (PROMs) play a central role [10, 24]. Retrospective studies of patients after mild ischemic, and hemorrhagic strokes have reported altered quality of life using PROMs of the National Institute of Health’s Patient-reported Outcome Measurement Information System (PROMIS), showing changes in physical, social, and mental domains . To overcome the problem of the great variability of PROMs used for assessment, the International Consortium for Health Outcomes Measurements (ICHOM) has coordinated the development of a consensus Standard Set of PROMs for Stroke based on PROMIS in order to enable comparable assessment of healthcare value in stroke management across different settings [13, 14, 22]. There is, however, a lack of data on patient-reported outcomes of stroke management from routine clinical practice. Studies reporting process evaluation of implementation of PROMs in the clinical practice are also scarce, although in form of process evaluations, facilitators and barriers of such an implementation can be discovered, and the implementation process can be adjusted for a successful realization of the desired intervention or enquiry.
Aim of the trial
The objectives of the current study are to implement the ICHOM-Standard Set for Stroke (ICHOM-SSS) in routine stroke care, and appraise this implementation, i.e. the acceptance and feasibility of its use in clinical practice by means of a process evaluation, to systematically characterize the long-term functional status and health-related quality of life after stroke in a longitudinal setting, and to identify factors influencing outcome and course of outcome after stroke. The addressed research questions are to which extend can the ICHOM-SSS be implemented into the routine health care for stroke patients and how are acceptance, benefit, and feasibility appraised by patients and clinical staff? How is the outcome of stroke and stroke treatment in a consecutive sample and which medical and patient-related factors influence the treatment outcome?
Study description and study design
We perform a prospective exploratory observational and implementation study with a longitudinal design. The sample consists of a consecutive cohort of patients referred to the stroke unit at the University Medical Centre Hamburg-Eppendorf for acute ischemic or hemorrhagic stroke within a 15 months period.
Inclusion criteria are:
treatment at the stroke unit of our hospital with acute ischemic stroke (AIS) [ICD10 I63], transient ischemic attack (TIA) [ICD10 G45], or intracerebral hemorrhage (ICH) [ICD10 I61]
informed consent by the patient or legal guardian
Exclusion criteria are:
severe disturbances of the ability to communicate, i.e. due to severe aphasia, dementia;
insufficient knowledge of the German or English language.
Sample size estimation
In the past years, approximately 1300 patients with a relevant diagnosis were admitted at the stroke unit of our hospital annually. Of these patients, 60% had an ischemic stroke, 32% had a transient ischemic attack, and 8% were admitted with intracerebral hemorrhage. With an expected dropout of approximately 25%, we aim to include n = 1040 patients (Fig. 1). This number allows for exploratory descriptive and inferential analyses with high precision and power, including subgroup analyses and the use of multivariate and longitudinal models. While the planned analyses are of exploratory nature, sample size calculations, done with GPower  show that in order to identify a small effect (H0 ρ2 = 0.01 and H1 ρ2 = 0.05 in a two-sided test with a power of 0.90, a regression analysis with 25 predictors the analysis of data of 846 cases are necessary, which is below the expected sample size.
Inpatients of the stroke unit will be screened for participation via the electronic health record (EHR) by study nurses after admission. They will not be involved in treatment of the patients and use set ups of the EHR to filter diagnoses and to obtain an overview of included and not included patients for minimizing losses.
Arms and intervention
In order to implement the ICHOM-SSS into the clinical routine, it will be integrated into the medical center’s EHR to ease identification, inclusion, and storage of data. A process evaluation will be conducted, measuring the degree of feasibility in this study, acceptance of patients, clinical and administrative staff, practicability, realization, and adoption of the implementation process and execution of this assessment, reach, and fidelity to protocol (Table 1). These outcomes are chosen by means of the current standards of feasibility studies [6, 11, 18]. According to a mixed method approach, the evaluation is subdivided into quantitative and qualitative analyses. The quantitative part consists of two data quality and plausibility checks. The first analysis is conducted shortly after start of data collection, and the second is conducted after the data collection is completed.
The qualitative data are collected in semi-structured interviews with patients as well as clinical and study staff. Aim of the interviews is to capture opinion and experience of the intervieews regarding the extended health evaluation with the focus on acceptance, adequacy, and purpose as well as facilitators and barriers of the implementation process. It is planned to conduct 15 patient interviews, matched for severity of stroke impact, measured with the simplified modified Ranking Scale questionnaire (smRSq). While the patient interviews will be held by phone after the 12 months follow up is completed, the staff interviews (n = 5) will be conducted twice; once shortly after the introduction of the ICHOM-SSS to the clinical routine and once after the termination of the enquiry phase. The first data check and staff interview serve not only as an inspection of feasibility but allow for potentially necessary adjustments to the process. The second data check and expert interview repeat and finalize the inspection of feasibility, respectively. Interviewing professionals of different contexts and patients shall increase validity of the results by capturing various perspectives on the enquiry.
If meeting the inclusion criteria, patients will be interviewed and their clinical records analyzed during inpatient stay concerning demographics, risk factors, and patient characteristics. The interview takes place twice; once as soon as possible after intake and the second at discharge. If attending the interviews or answering the questionnaire is not possible due to disability, e.g. aphasia, the patient’s proxy will be contacted and asked to provide the information.
Assessment and parameters are collected according to the ICHOM-SSS(International Consortium for Health Outcomes Measurement (ICHOM), 2017), consisting of living situation, pre- and post-stroke functional status, cardiovascular risk factors including atrial fibrillation, the National Institute of Health Stroke Scale (NIHSS), and administrative data i.e. length of stay, survival, planned rehabilitation procedures, and discharge destination (Fig. 2). Follow up assessments take place 3 and 12 months after admission respectively. The assessment strategy is twofold. Firstly, a paper-and-pencil questionnaire is sent to the patients, consisting of a repetition of questions listed above, additional to the PROMs described below. Secondly, a short telephone interview takes place to assess the smRSq as well as record PROMs if patients were unable to fill in and send back the questionnaire themselves. If contact cannot be established postal or via three phone calls within 2 weeks patients will be excluded from further analysis.
The primary outcome of this project is the change in patient-reported global health after stroke over the course of 1 year. This is measured three and 12 months after stroke using the Patient-reported Outcomes Measurement Information System 10-Question Short Form (PROMIS-10) questionnaire. The PROMIS-10 has been shown to be feasible and valuable in the assessment of patients after stroke . Relevant domains are physical and mental health, for both used a t-score calculated from a sum-score with standardized t-values, with M ± SD = 50 ± 10, as fit in healthy American subjects. Lower values will reflect a poorer outcome. Additional information may be gained from single questions concerning participation in social activity and general health. The direction of scoring is the same. Depressive and anxiety symptoms are assessed using the Patient Health Questionnaire-4 (PHQ-4). It is a four-item score and has been established as a self-reported written questionnaire in a broad German and American cohort containing different threshold values . Its scale reaches from one to six for each subdomain, with higher scores indicating worse status, and a score above 3 considered positive for a potential presence of anxiety or depression. Lastly, according to the ICHOM-SSS, we will assess motor function by the smRSq. This is done by phone interview. The smRSq has been shown to reflect motor disability with a comparable value as the traditional mRS . Its scale does not differ from the latter, and reaches from zero for no symptoms, to six for death.
Data analysis and management
Research question 1: A process evaluation is performed to answer the first research question. This means process data on the implementation and enquiry process will be collected. Two structured analyses of process- and outcome data will be performed on congruency and completeness in order to detect potential discrepancies between conception and realization. The results of the evaluation will be examined via descriptive statistics. The interviews will be recorded, transcribed and analyzed using a realist thematic analysis approach , specifically a framework content analysis . The thematic analysis approach is a method by which qualitative data is coded into themes. We will use a mainly deductive approach, as our feasibility outcomes are already predefined (see Table 1). Coding schemes are developed beforehand and discussed regularly. Nevertheless, we are open to the possibility of inductive theme generation, if data suggests. The results will be reported using consolidated criteria for reporting qualitative research .
Research questions 2 and 3: The data will be analysed by descriptive and inferential statistics, Depending on the outcome, (generalised) multilevel linear (patient-reported global health) and logistic (mental and motor health) regression analyses will be conducted. Factors that are believed to possibly influence the outcomes will be integrated in form of covariates. These will comprise demographics, pre-existing deficits and prior strokes, and deficits at admission (NIHSS) since recent studies have shown a negative influence of impaired motor function on different PROMIS domains(Katzan, Thompson, et al., 2018). Additionally, survival analyses (mortality) will be performed. Exploratory structural equation models are fitted to identify differential characteristic patterns of the outcomes. Further, exploratory subgroup analyses are planned, using regression models that fit the respective research questions. Results with p < .05 will be considered as statistically significant. As this study is of explorative nature, no adjustments for multiple testing will be made. However, the elevated risk of an occurrence of type-I errors will be regarded at the interpretation of the results. Missing values will be accounted for by using mixed modelling techniques. Sensitivity analyses will be performed when no mixed models are used. The influence of the NIHSS on the outcomes will be analysed separately using median and nonparametric tests.
The electronic health record will be used for data collection and storage. For preparation and most quantitative data analyses, it is anticipated to use the software R (https://www.r-project.org) and IBM SPSS Statistics (https://www.ibm.com/analytics/spss-statistics-software). Lastly, the software MAXQDA (https://www.maxqda.com) will be used for qualitative data analyses.
EPOS is conducted as a single-center study by the University Medical Center Hamburg-Eppendorf in collaboration of the Department of Neurology, Department of Medical Psychology, and the Office for Quality Management and Clinical Process Management. EPOS is funded by the Innovation Fund of the German Federal Joint Committee.
Our study protocol represents the first prospective cohort study using the ICHOM Standard Set for Stroke to assess PROMs in stroke patients. Through open inclusion criteria, we attempt to gather a broad spectrum of stroke patients that is representative for the population of acute stroke patients treated on stroke unit in daily routine care in a German metropolitan regions. Information gathered from the process evaluation will help to understand the facilitators and barriers of the standard use of PROMs –also in the long term- in the routine stroke care. The analysis will show, what might have to be adjusted for a smooth and successful implementation and realization of an extended and long-term treatment assessment. Should the analysis show that such an implementation is feasible and accepted amongst affected persons, measures will be taken that the implemented enquiry remains in use within the routine health care at the stroke unit of the University Medical Centre Hamburg-Eppendorf after the end of the trial. Furthermore, the results of the process evaluation will help to accomplish and accelerate the implementation of such enquiries in other settings. Information analyzed from follow up assessments should describe health-related difficulties in everyday life and might show predictive patient or therapy-related factors concerning different groups of patients. Our analysis will help in identifying subgroups of stroke patients with increased risk of impaired quality of life after stroke that may be targeted by specific interventions. Overall, the information collected in our study will help to bring the patients’ perspective into the focus of stroke care and may inform future strategies for improving stroke treatment at different levels.
Availability of data and materials
The data will be deposited in a protected server of the University Medical Centre Hamburg-Eppendorf. Access is strongly regulated even for study personnel. Owing to the difficulty of de-identification (routine care, qualitative data, etc.), individual participant data will not be shared publicly. Upon reasonable request that includes a methodologically sound proposal for the usage of data that is also approved by the responsible review committee data may be shared.
Acute ischemic stroke
Electronic health record
International Statistical Classification of Diseases and Related Health Problems 10th revision
The International Consortium for Health Outcomes Measurements
The International Consortium for Health Outcomes Measurements -Standard Set for Stroke
Patient Health Questionnaire-4
Patient-reported Outcomes Measurement Information System 10-Question Short Form
Patient-reported Outcome measures
simplified modified Ranking Scale questionnaire
Transient ischemic attack
Barker-Collo, S., Feigin, V., Lawes, C., Senior, H., & Parag, V. (2010). Natural history of attention deficits and their influence on functional recovery from acute stages to 6 months after stroke. Neuroepidemiology, 35(4), 255–262. https://doi.org/10.1159/000319894.
Booth, A., Hannes, K., Harden, A., Noyes, J., Harris, J., Tong, A., et al. (2014). COREQ (consolidated criteria for reporting qualitative studies). In Guidelines for reporting Health Research: A User’s manual (Vol. 21, pp. 214–226).
Braun, V., & Clarke, V. (2006). Using thmatic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.
Bruno, A., Akinwuntan, A. E., Lin, C., Close, B., Davis, K., Baute, V., et al. (2011). Simplified modified Rankin scale questionnaire. Stroke, 42(8), 2276–2279. https://doi.org/10.1161/STROKEAHA.111.613273.
Chan, A., Tetzlaff, J. M., Gøtzsche, P. C., Altman, D. G., Mann, H., Berlin, J. A., et al. (2013). RESEARCH METHODS AND REPORTING SPIRIT 2013 explanation and elaboration : Guidance for protocols of clinical trials. BMJ \ Research Methods and Reporting, 346, e7586. https://doi.org/10.1136/bmj.e7586.
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., Petticrew, M., & Medical Research Council Guidance. (2008). Developing and evaluating complex interventions: The new Medical Research Council guidance. BMJ (Clinical Research Ed.), 337(September), a1655. https://doi.org/10.1136/bmj.a1655.
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences FRANZ. Behavior Research Methods, 39(2), 175–191.
Feigin, V., Krishnamurthi, R., Parmar, P., Norrving, B., Mensah, G., Bennett, D., et al. (2013). S. P. E. (2015). Update on the global burden of disease of Ischaemic and Haemorrhagic stroke in 1990-2013: The GBD 2013 study. Neuroepidemiology, 45(3), 161–176. https://doi.org/10.1159/000441085.UPDATE.
Gale, N. K., Heath, G., Cameron, E., Rashid, S., & Redwood, S. (2013). Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Medical Research Methodology, 13(1), 117. https://doi.org/10.1186/1471-2288-13-117.
Glasgow, R. E., Kaplan, R. M., Ockene, J. K., Fisher, E. B., & Emmons, K. M. (2012). Patient-reported measures of psychosocial issues and health behavior should be added to electronic health records. Health Affairs, 31(3), 497–504. https://doi.org/10.1377/hlthaff.2010.1295.
Glasgow, R. E., McKay, H. G., Piette, J. D., & Reynolds, K. D. (2001). The RE-AIM framework for evaluating interventions: What can it tell us about approaches to chronic illness management? Patient Education and Counseling, 44(2), 119–127. https://doi.org/10.1016/S0738-3991(00)00186-5.
Heuschmann, P. U., Wiedmann, S., Wellwood, I., Rudd, A., Di Carlo, A., Bejot, Y., et al. (2010). Three-month stroke outcome: The European registers of stroke (EROS) investigators. Neurology, 76(2), 159–165. https://doi.org/10.1212/wnl.0b013e318206ca1e.
International Consortium for Health Outcomes Measurement (ICHOM). (2017). Measuring results that matter Ability to communicate Stroke. Data Collection Reference Guide. http://www.Ichom.Org/Medical-Conditions/Stroke, 32.
Katzan, I. L., & Lapin, B. (2018). PROMIS GH (patient-reported outcomes measurement information system global health) scale in stroke a validation study. Stroke, 49(1), 147–154. https://doi.org/10.1161/STROKEAHA.117.018766.
Katzan, I. L., Schuster, A., Newey, C., Uchino, K., & Lapin, B. (2018a). Patient-reported outcomes across cerebrovascular event types. Neurology, 91(23), e2182–e2191. https://doi.org/10.1212/wnl.0000000000006626.
Katzan, I. L., Thompson, N. R., Uchino, K., & Lapin, B. (2018b). The most affected health domains after ischemic stroke. Neurology, 90, e1364–e1371. https://doi.org/10.1212/wnl.0000000000005327.
Krishnamurthi, R. V, Feigin, V. L., Forouzanfar, M. H., Mensah, G. A., Connor, M., Bennett, D. A., … Murray, C. (2013). Global and regional burden of first-ever ischaemic and haemorrhagic stroke during 1990–2010)70089-5. Global and regional burden of first-ever ischaemic and haemorrhagic stroke during 1990–2010. The Lancet. Global Health, 1(5), e259–e281. https://doi.org/10.1016/S2214-109X(13)70089-5.Global
Lewis, C. C., Fischer, S., Weiner, B. J., Stanick, C., Kim, M., & Martinez, R. G. (2015). Outcomes for implementation science: An enhanced systematic review of instruments using evidence-based rating criteria. Implementation Science, 10(1), 1–17. https://doi.org/10.1186/s13012-015-0342-x.
Löwe, B., Wahl, I., Rose, M., Spitzer, C., Glaesmer, H., Wingenfeld, K., et al. (2010). A 4-item measure of depression and anxiety : Validation and standardization of the patient health Questionnaire-4 ( PHQ-4 ) in the general population. Journal of Affective Disorders, 122(1–2), 86–95. https://doi.org/10.1016/j.jad.2009.06.019.
Porter, M. E. (2010). What is value in health care? New England Journal of Medicine, 363(26), 2477–2481. https://doi.org/10.1056/NEJMp1011024.
Porter, M. E., & Teisberg, E. O. (2006). Redefining Health Care : Creating Value-Based Competition on Results Issues in Health Care Reform Health. Boston, Mass.: Harvard Business School, (June 2004).
Salinas, J., Sprinkhuizen, S. M., Ackerson, T., Bernhardt, J., Davie, C., George, M. G., et al. (2016). An international standard set of patient-centered outcome measures after stroke. Stroke, 47(1), 180–186. https://doi.org/10.1161/STROKEAHA.115.010898.
Van Mierlo, M. L., Van Heugten, C. M., Post, M. W. M., Hajós, T. R. S., Kappelle, L. J., & Visser-Meily, J. M. A. (2016). Quality of life during the first two years post stroke: The Restore4Stroke cohort study. Cerebrovascular Diseases, 41(1–2), 19–26. https://doi.org/10.1159/000441197.
Willke, R. J., Burke, L. B., & Erickson, P. (2004). Measuring treatment impact: A review of patient-reported outcomes and other efficacy endpoints in approved product labels. Controlled Clinical Trials, 25(6), 535–552. https://doi.org/10.1016/j.cct.2004.09.003.
LR contributed to the acquisition of data and wrote the manuscript; LL contributed to the planning of the study and revised the manuscript critically for important intellectual content; MF conceptualized the study, drafted the study design, and revised the manuscript critically for important intellectual content; HA contributed to the acquisition of data and revised the manuscript critically for important intellectual content; HJB conceptualized the study and revised the manuscript critically for important intellectual content; LK conceptualized the study, drafted the study design, and revised the manuscript critically for important intellectual content; CG contributed to the planning of the study and revised the manuscript critically for important intellectual content; MH conceptualized the study, drafted the study design, and revised the manuscript critically for important intellectual content; GT conceptualized the study, drafted the study design, and revised the manuscript critically for important intellectual content. All authors approved the final version and take responsibility for the accuracy and integrity of the study.
This work is funded by the Innovation Fund of the German Federal Joint Committee.
(Gemeinsamer Bundesausschuss, https://www.g-ba.de/sys/kontakt/), with the grant number 01VSF16023. The German Federal Joint Committee reviewed and approved the study design during the grant application process. It had no role in the conduct of the study and the publication process.
Ethics approval and consent to participate
The study is carried out following the Helsinki Declaration of the World Medical Association and according to the principles of good clinical and good scientific practice. Study participation is voluntary and can be withdrawn at any time. Written and audio-recorded informed consent will be obtained prior to participation, if applicable. Patients will be fully informed about aims, procedures, data collection and the use of collected data in this study. Rejecting participation has no negative consequences for patients. No foreseeable risk results from participating. No compassionate use is carried out. No invasive intervention is conducted. This study is registered at clinicaltrial.gov (NCT03795948). Principles of data protection will be kept. Approval of the local ethics committee (Ethik-Kommission der Ärztekammer Hamburg) has been obtained (PV54565, 14.03.2017). The results of this study will be disseminated via peer-reviewed journals. Deviations from protocol, i.e. inclusion, recruitment, procedure, and statistical data analysis will be discussed. The plot of our description is oriented on the SPIRIT guidance for protocols of clinical trials .
Consent for publication
LR, LL, MF, HA, HJB, LK, and MH have no conflict of interest. CG reports personal fees from Amgen, Bayer Vital, Bristol-Myers Squibb, Boehringer Ingelheim, Sanofi Aventis, Abbott, and Prediction Biosciences outside the submitted work. GT reports receiving consulting fees from Acandis, grant support, and lecture fees from Bayer, lecture fees from Boehringer Ingelheim, Bristol-Myers Squibb/Pfizer, and Daiichi Sankyo, and consulting fees and lecture fees from Stryker outside the submitted work.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Rimmele, D.L., Lebherz, L., Frese, M. et al. Outcome evaluation by patient reported outcome measures in stroke clinical practice (EPOS) protocol for a prospective observation and implementation study. Neurol. Res. Pract. 1, 28 (2019). https://doi.org/10.1186/s42466-019-0034-0