The aim of this study was translating and exploring psychometric properties of Serbian Pittsburgh Sleep Quality Index (PSQI) in a sample of "good" and "bad" sleepers suffering from depression or obstructive sleep apnea (OSA). Formal translation and validation were performed on a sample of healthy controls, patients with untreated OSA, and with diagnosed major depressive disorder with evaluation of internal consistency, test–retest reliability, and construct and criterion validity. Controls and OSA subgroups were recruited from a larger sample of commercial drivers. One hundred and forty subjects, 84.3% male, 22–67 years old, were included. OSA subgroup had 59 subjects and depression subgroup had 40 subjects (22 females). Mean ± SD total PSQI was 3.5 ± 2.2 in controls, 4.9 ± 3.6 in OSA subjects, and 9.0 ± 4.9 in patients with depression. Cronbach’s α for total PSQI was 0.791. Subscale scores were significantly correlated to global PSQI in all subgroups. Intraclass correlation coefficient for global PSQI was 0.997 (p < .001). Epworth Sleepiness Scale score was significantly correlated to global PSQI ( = 0.333, p < .001). Three subgroups differed significantly in total PSQI and PSQI ≥ 5, even after adjustments for age and gender (p < .001). OSA patients had higher mean PSQI than controls but not significantly (p = .272). PSQI-reported sleep latency did not correlate with PSG-measured sleep latency (r = .130, p = .204). Total PSQI was significantly correlated to OSA severity ( = 0.261, p < .05). Serbian PSQI showed good internal consistency, test–retest reproducibility, and adequate construct and criterion validity, which supports further exploration of its use as a sleep quality screening tool in different target populations.
This study examined dimensionality and item-level psychometric properties of an item bank measuring activities of daily living (ADL) across inpatient rehabilitation facilities and community living centers. Common person equating method was used in the retrospective veterans data set. This study examined dimensionality, model fit, local independence, and monotonicity using factor analyses and fit statistics, principal component analysis (PCA), and differential item functioning (DIF) using Rasch analysis. Following the elimination of invalid data, 371 veterans who completed both the Functional Independence Measure (FIM) and minimum data set (MDS) within 6 days were retained. The FIM-MDS item bank demonstrated good internal consistency (Cronbach’s α = .98) and met three rating scale diagnostic criteria and three of the four model fit statistics (comparative fit index/Tucker–Lewis index = 0.98, root mean square error of approximation = 0.14, and standardized root mean residual = 0.07). PCA of Rasch residuals showed the item bank explained 94.2% variance. The item bank covered the range of from –1.50 to 1.26 (item), –3.57 to 4.21 (person) with person strata of 6.3. The findings indicated the ADL physical function item bank constructed from FIM and MDS measured a single latent trait with overall acceptable item-level psychometric properties, suggesting that it is an appropriate source for developing efficient test forms such as short forms and computerized adaptive tests.
Regularly reported patient surveys are an important dimension of hospital quality management. This study investigates whether providing hospital staff with interim feedback on patient survey results following a best practices workshop can help hospitals improve patient centeredness. Standardized surveys with consecutive patient samples were administered in accredited breast cancer center (BCC) hospitals in one German state (18 million inhabitants), over a 6-month period, in 2012. Two studies were conducted by applying a combination of regression point displacement (RPD) and interrupted time series (ITS) designs. In Study 1, 2 of the 27 hospitals that had previously participated in a best practices workshop to discuss patient-centeredness issues were randomly chosen and were provided interim feedback of patient survey results and workshop minutes. In Study 2, 4 randomly chosen hospitals of 32 that had not participated in the workshop also received interim feedback but no workshop minutes. Control hospitals in both studies neither received feedback nor workshop minutes. The impact of interim feedback was evaluated by applying graphical assessments and multiple regression analyses. Both graphical assessments (locally weighted scatterplot smoothing (LOESS) lines, RPD plots) suggested an effect of interim feedback. Multiple regression results did not unambiguously support these findings. The suggested design approach may prove particularly useful to assess effects in pilot studies, when resources are not available to conduct a randomized study or when its conduct is contingent on initial, positive evidence.
We describe challenges in the 6-year longitudinal cluster randomized controlled trial (CRCT) of Positive Action (PA), a social–emotional and character development (SECD) program, conducted in 14 low-income, urban Chicago Public Schools. Challenges pertained to logistics of study planning (school recruitment, retention of schools during the trial, consent rates, assessment of student outcomes, and confidentiality), study design (randomization of a small number of schools), fidelity (implementation of PA and control condition activities), and evaluation (restricted range of outcomes, measurement invariance, statistical power, student mobility, and moderators of program effects). Strategies used to address the challenges within each of these areas are discussed. Incorporation of lessons learned from this study may help to improve future evaluations of longitudinal CRCTs, especially those that involve evaluation of school-based interventions for minority populations and urban areas.
Recovery is understood as living a life with hope, purpose, autonomy, productivity, and community engagement despite a mental illness. The aim of this study was to provide further information on the psychometric properties of the Person-in-Recovery and Provider versions of the Revised Recovery Self-Assessment (RSA-R), a widely used measure of recovery orientation. Data from 654 individuals were analyzed, 519 of whom were treatment providers (63.6% female), while 135 were inpatients (10.4% female) of a Canadian tertiary-level psychiatric hospital. Confirmatory and exploratory techniques were used to investigate the factor structure of both versions of the instrument. Results of the confirmatory factor analyses showed that none of the four theoretically plausible models fit the data well. Principal component analyses could not replicate the structure obtained by the scale developers either and instead resulted in a five-component solution for the Provider and a four-component solution for the Person-in-Recovery version. When considering the results of a parallel analysis, the number of components to retain dropped to two for the Provider version and one for the Person-in-Recovery version. We can conclude that the RSA-R requires further revision to become a psychometrically sound instrument for assessing recovery-oriented practices in an inpatient mental health-care setting.
This study evaluated the extent to which medical students with limited English-language experience are differentially impacted by the additional reading load of test items consisting of long clinical vignettes. Participants included 25,012 examinees who completed Step 2 of the U.S. Medical Licensing Examination®. Test items were categorized into five levels based on the number of words per item, and examinee scores at each level were evaluated as a function of English-language experience (English as a second language [ESL] status and scores on a test of English-speaking proficiency). The longest items were more difficult than the shortest items across all examinee groups, and examinees with more English-language experience scored higher than those with less experience across all five levels of word count. The effect of primary interest—the interaction of word count with English-language experience—was statistically significant, indicating that score declines for longer items were larger for examinees with less English-language experience; however, the magnitude of this interaction effect was barely detectable (2 = .0004, p < .001). Additional analyses supported the conclusion that the differential effect for examinees with less English-language experience was small but worthy of continued monitoring.
The success of integrated care interventions is highly dependent on the internal and collective capabilities of the organizations in which they are implemented. Yet, organizational capabilities are rarely described, understood, or measured with sufficient depth and breadth in empirical studies or in practice. Assessing these capabilities can contribute to understanding why some integrated care interventions are more effective than others. We identified, organized, and assessed survey instruments that measure the internal and collective organizational capabilities required for integrated care delivery. We conducted an expert consultation and searched Medline and Google Scholar databases for survey instruments measuring factors outlined in the Context and Capabilities for Integrating Care Framework. A total of 58 instruments were included in the review and assessed based on their psychometric properties, practical considerations, and applicability to integrated care efforts. This study provides a bank of psychometrically sound instruments for describing and comparing organizational capabilities. Greater use of these instruments across integrated care interventions and studies can enhance standardized comparative analyses and inform change management. Further research is needed to build an evidence base for these instruments and to explore the associations between organizational capabilities and integrated care processes and outcomes.
Research service cores at academic health centers are important in driving translational advancements. Specifically, biostatistics and research design units provide services and training in data analytics, biostatistics, and study design. However, the increasing demand and complexity of assigning appropriate personnel to time-sensitive projects strains existing resources, potentially decreasing productivity and increasing costs. Improving processes for project initiation, assigning appropriate personnel, and tracking time-sensitive projects can eliminate bottlenecks and utilize resources more efficiently. In this case study, we describe our application of lean six sigma principles to our biostatistics unit to establish a systematic continual process improvement cycle for intake, allocation, and tracking of research design and data analysis projects. The define, measure, analyze, improve, and control methodology was used to guide the process improvement. Our goal was to assess and improve the efficiency and effectiveness of operations by objectively measuring outcomes, automating processes, and reducing bottlenecks. As a result, we developed a web-based dashboard application to capture, track, categorize, streamline, and automate project flow. Our workflow system resulted in improved transparency, efficiency, and workload allocation. Using the dashboard application, we reduced the average study intake time from 18 to 6 days, a 66.7% reduction over 12 months (January to December 2015).
This study examined the psychometric properties of the Chinese version of the Personal Diabetes Questionnaire (C-PDQ). The PDQ was translated into Chinese using a forward and backward translation approach. After being reviewed by an expert panel, the C-PDQ was administered to a convenience sample of 346 adults with Type 2 diabetes. The Chinese version of the Summary of Diabetes Self-Care Activities (C-SDSCA) was also administered. The results of the exploratory factor analysis revealed a one-factor structure for the Diet Knowledge, Decision-Making, and Eating Problems subscales and a two-factor structure for the barriers-related subscales. The criterion and convergent validity were supported by significant correlations of the subscales of the C-PDQ with the glycated hemoglobin values and the parallel subscales in the C-SDSCA, respectively. The C-PDQ subscales also showed acceptable internal consistency (α = .61–.89) and excellent test–retest reliability (intraclass correlation coefficients: .73–.96). The results provide preliminary support for the reliability and validity of the C-PDQ. This comprehensive, patient-centered instrument could be useful to identify the needs, concerns, and priorities of Chinese patients with type 2 diabetes.
The principal aim of this study was to investigate the psychosocial well-being of emerging adults using psychological states associated with this transitional phase and classic measures of emerging adulthood. We expected psychological states to be more closely associated with psychological well-being than classic markers of achieved adulthood. Data were collected in the Cohort Study on Substance Use Risk Factors from 4,991 Swiss men aged 18–25 years. The assessment included the Short Form of the Inventory of Dimensions of Emerging Adulthood (IDEA-8), classic markers of achieved adulthood (e.g., financial independence, stable relationship), and psychosocial well-being. Structural equation models (SEMs) were conducted to test the association between measures of emerging adulthood and psychosocial well-being. Overall, the results highlighted contrasting associations of measures of emerging adulthood and psychosocial well-being. Youths facing negative psychological states (dimension "negativity") and exploring life without knowing how to define themselves (dimension "identity exploration") had a decreased psychosocial well-being. On the contrary, youths exploring many opportunities with an optimistic perspective (dimension "experimentation") had an increased psychosocial well-being. By contrast, classic markers of adulthood were less related to psychosocial well-being. The IDEA-8 Scale appeared to be a useful screening tool for identifying vulnerable youths, and emerging adulthood should be measured with a focus on the psychological states associated with this period. This information may be valuable for mental health systems that have not yet adapted to emerging adults’ needs.
Identification of patients’ and health professionals’ quality improvement preferences is an essential first step in collaborative improvement models. This includes experience-based codesign (EBCD), where service change is strategically introduced following stakeholder consultation. This study compared the number and types of improvement initiatives selected by outpatients and health professionals. Using electronic surveys designed to inform EBCD studies, 541 outpatients (71.1% consent) and 124 professionals (47.1% response) selected up to 23 general initiatives. On average, outpatients selected 2.4 (median = 1, interquartile range = 1–3) initiatives and professionals selected 10.7 (median = 10; interquartile range = 6–15) initiatives. Outpatients demonstrated a strong preference for improvements to clinic organization, such as appointment scheduling and clinic contact. Outpatients selected relatively fewer initiatives potentially reducing the complexity of service change and resources required to address preferences. Comparatively, professionals indicated a greater degree of change is needed and selected initiatives related to communication with patients and other professionals, including coordinating multidisciplinary care. Improvements to information provision were commonly selected by both groups and offered a strategic opportunity to address patients’ and professionals’ preferences. By quantifying the ways in which preferences differed, this study emphasizes the need for collaborative approaches to health service change and may be used to initiate an informed discussion on patients’ and professionals’ quality improvement preferences in tertiary care.
Transition-to-adulthood themes, or thoughts and feelings about emerging adulthood, have been measured by the Inventory of the Dimensions of Emerging Adulthood (IDEA) and found to be associated with substance use among emerging adults. It has been suggested, however, that the IDEA is lengthy and may not include the most unique and theoretically relevant constructs of emerging adulthood. The Revised Inventory of the Dimensions of Emerging Adulthood (IDEA-R) was developed as an alternative instrument, but research has yet to determine the relationship between the IDEA-R and substance use among emerging adults (ages 18–25 years). College students completed surveys indicating their identification with transition-to-adulthood themes and substance use. Logistic regression models examined the associations between transition-to-adulthood themes and marijuana use and binge drinking, respectively. Participants who felt emerging adulthood was a time of identity exploration were less likely to report marijuana use, while feelings of experimentation/possibility were positively associated with marijuana use and binge drinking. The IDEA-R may be useful for identifying correlates of substance use among emerging adults. Future research should evaluate the IDEA-R among representative samples of emerging adults to confirm the findings of this study. Health professionals working in substance use prevention may consider targeting the themes of identity exploration and experimentation/possibility in programs intended for emerging adults.
Motivational interviewing (MI) is a popular evidence-based method to support health behavior change. We examined evaluations from 10 years of interprofessional workshops on MI to identify trends in trainees’ MI-related knowledge, attitude, and behavior. From 2006 to 2015, 394 trainees participated in continuing education MI workshops with our team and completed a validated posttraining questionnaire. Participants were 90% female and 66% White, with M = 12 years in practice. They worked in pediatric and adult care; urban and rural locations; and inpatient, outpatient, and nonhealth settings. The largest groups were nurses (20%), allied health professionals (20%), and health educators or case managers (15%). Trainees’ professional diversity increased over time, their average age and years in practice decreased, and the percentage with prior MI training increased. Practitioners in telehealth and nonhealth settings had lower scores overall. Outcomes varied significantly by professional discipline: Mental health professionals, case managers, health educators, and nurses had higher scores on some outcome variables than nonhealth professionals. Years of clinical experience predicted MI-consistent attitude, but prior training, other demographic variables, and training process variations had no consistent effects. Although many trainees had already received MI training, outcome measures showed room for improvement. MI presents continued opportunities for interprofessional education.
This article presents a practitioner-based approach to identify key combinations of contextual factors (C) and mechanisms (M) that trigger outcomes (O) in Dutch community-based health-enhancing physical activity (CBHEPA) programs targeting socially vulnerable groups. Data were collected in six programs using semi-structured interviews and focus groups using a timeline technique. Sessions were recorded, anonymized, and transcribed. A realist synthesis protocol was used for data-driven and thematic analysis of CMO configurations. CMO configurations related to community outreach, program sustainability, intersectoral collaboration, and enhancing participants’ active lifestyles. We have refined the CBHEPA program theory by showing that actors’ passion for, and past experiences with, physical activity programs trigger outcomes, alongside their commitment to socially vulnerable target groups. Project discontinuity, limited access to resources, and a trainer’s stand-alone position were negative configurations. The authors conclude that local governance structures appear often to lack adaptive capacity to accommodate multilevel processes to sustain programs.
The aim is to evaluate the effects of the Competencias para adolescentes con una sexualidad saludable (COMPAS) program and compare them with an evidence-based program (¡Cuídate!) and a control group (CG). Eighteen public high schools were randomly assigned to one of the three experimental conditions. Initially, 1,563 Spanish adolescents between 14 and 16 years of age participated, and 24 months after their implementation, 635 of them completed a survey. Self-report measures collected data on sexual behavior, knowledge, attitudes, intention, sexual risk perception, and perceived norm. Compared to the CG, COMPAS increased the level of knowledge about sexually transmitted infections and improved the attitudes toward people living with human immunodeficiency virus at the 2-year follow-up. Neither intervention had a long-term impact on behavioral variables. Results suggest that COMPAS has a comparable impact to the other intervention on the variables predicting consistent condom use. Reinforcing the messages and skills that have the greatest impact on condom use and adding booster sessions following program completion as strategies to maintain long-term effects are necessary.
As international research studies become more commonplace, the importance of developing multilingual research instruments continues to increase and with it that of translated materials. It is therefore not unexpected that assessing the quality of translated materials (e.g., research instruments, questionnaires, etc.) has become essential to cross-cultural research, given that the reliability and validity of the research findings crucially depend on the translated instruments. In some fields (e.g., public health and medicine), the quality of translated instruments can also impact the effectiveness and success of interventions and public campaigns. Back-translation (BT) is a commonly used quality assessment tool in cross-cultural research. This quality assurance technique consists of (a) translation (target text [TT1]) of the source text (ST), (b) translation (TT2) of TT1 back into the source language, and (c) comparison of TT2 with ST to make sure there are no discrepancies. The accuracy of the BT with respect to the source is supposed to reflect equivalence/accuracy of the TT. This article shows how the use of BT as a translation quality assessment method can have a detrimental effect on a research study and proposes alternatives to BT. One alternative is illustrated on the basis of the translation and quality assessment methods used in a research study on hearing loss carried out in a border community in the southwest of the United States.
Grading instruments are an important part of evidence-based medicine and are used to inform health policy and the development of clinical practice guidelines. They are extensively used in the development of clinical guidelines and the assessment of research publications, having particular impact on health care and policy sectors. The positive effects of using grading instruments are, however, potentially undermined by their misuse and a number of shortcomings. This review found eight key concerns about grading instruments: (1) lack of information on validity and reliability, (2) poor concurrent validity, (3) may not account for external validity, (4) may not be inherently logical, (5) susceptibility to subjectivity, (6) complex systems with inadequate instructions, (7) may be biased toward randomized controlled trial (RCT) studies, and (8) may not adequately address the variety of non-RCTs. This narrative review concludes that there is a need to take into account these criticisms and domain-specific limitations, to enable the use and development of the most appropriate grading instruments. Grading systems need to be matched to both the research question being asked and the type of evidence being used.
The protective effects of social support on health have been documented in a variety of groups. For HIV-infected persons released from correctional settings, strong social support may be particularly important for obtaining effective postrelease medical treatment and supportive services. Researchers and program evaluators seeking to improve access and adherence to postrelease HIV medical care in this population need accurate measures for the level and type of social support, but current measures have not been fully validated for incarcerated individuals with HIV infection. We used the Rasch model to test the Medical Outcomes Study (MOS) social support survey. Data for the analysis were collected as part of the EnhanceLink project in the five urban jails where the MOS was administered. Findings indicate that the MOS survey items may not capture the entire variability of person abilities. Respondents showed problems in discriminating among response options, indicating potential systematic bias. In addition, while there was no significant gender difference, overall levels of social support differed by gender. Further research is warranted to develop more effective social support measurement tools that can better guide interventions for persons transitioning from jail and prison to the community.
A self-report instrument assessing work-related factors among people with psychiatric disabilities would be useful when trying to match possible employment or prevocational opportunities with people’s desires and capacities. The aim of this study was to explore the factor structure, internal consistency, and construct and criterion validity of the Worker Role Self-Assessment (WRS) in this group as well as possible floor and ceiling effects. The participants were 283 clients from day centers for people with psychiatric disabilities and from outpatient units for people with psychosis. They completed the WRS and instruments selected to assess construct validity in terms of convergent (motivation for work and current activity level as reference variables) and discriminant validity (quality of life and self-rated health as reference variables). Two factors were identified, one tapping beliefs in a future worker role and one reflecting current capacities and routines. The internal consistency for the scale as a whole was good at 0.84. The factor reflecting a future worker role correlated as expected with the reference variables used to assess convergent and discriminant validity, whereas current capacities and routines showed a moderate association with quality of life and self-rated health, assumed to indicate discriminant validity. Criterion validity was shown in that those who had recent work experiences scored higher than the others on WRS. No floor or ceiling effects were identified. The findings indicate acceptable psychometric properties of the WRS. Further development is still warranted, however; the factor solution needs to be replicated and the construct validity should be further established.
Burnout has been identified as an occupational hazard in the helping professions for many years and is often overlooked, as health-care systems strive to improve cost and quality. The Maslach Burnout Inventory (MBI) and the Areas of Worklife Survey (AWS) are tools for assessing burnout prevalence and its associated factors. We describe how we used them in outpatient clinics to assess burnout for multiple job types. Traditional statistical techniques and seemingly unrelated regression were used to describe the sample and evaluate the association between work life domains and burnout. Of 838 eligible participants, 467 (55.7%) were included for analysis. Burnout prevalence varied across three job categories: providers (37.5%), clinical assistants (24.6%), and other staff (28.0%). It was not related to age, gender, or years of tenure but was lower in part-time workers (24.6%) than in full-time workers (33.9%). Analysis of the AWS subscales identified organizational correlates of burnout. Accurately identifying and defining the operative system factors associated with burnout will make it possible to create successful interventions. Using the MBI and the AWS together can highlight the relationship between system work experiences and burnout.
We postulated that associations between two specific provider characteristics, class (nurse practitioner relative to physician) and primary care providers who are proficient and interested in women’s health (designated women’s provider relative to nondesignated) and overall satisfaction with provider, were mediated through women veterans’ perception of enough time spent with the provider. A national patient experience survey was administered to 7,620 women veterans. Multivariable models of overall patient satisfaction with provider were compared with and without the proposed mediator. A structural equation model (SEM) of the mediation of the two provider characteristics was also evaluated. Without the mediator, associations of provider class and designation with overall patient satisfaction were significant. With the proposed mediator, these associations became nonsignificant. An SEM showed that the majority (>80%) of the positive associations between provider class and designation and the outcome were exerted through patient perception of enough time spent with provider. Higher ratings of overall satisfaction with provider exhibited by nurse practitioners and designated women’s health providers were exerted through patient perception of enough time spent with provider. Future research should examine what elements of provider training can be developed to improve provider–patient communication and patient satisfaction with their health care.
Assessing the practical or clinical significance (CS) of an intervention program’s outcomes is useful in determining its effectiveness. The CS approach gives information beyond traditional analyses by quantifying the proportions of people who meaningfully improve and deteriorate. We link latent transition analyses (LTA) to the CS literature and use a case study to contrast it with the long-standing Jacobson and Truax (JT) approach. Data came from 2,717 individuals convicted of a substance-related offense who participated in an indicated prevention program Prime For Life® (PFL). We selected outcomes describing drinking beliefs and behavior. Both CS approaches categorized a majority of participants as improved (i.e., transitioning from baseline subgroups with risky behaviors and cognitions into posttest subgroups showing lower risk). Results demonstrate how the JT approach allows the assessment of improvements on individual outcomes, while the LTA provides more nuanced information about risk groupings. Selecting a CS approach depends on research goals, availability of normative data, and data considerations. JT is an appropriate method when evaluating single outcomes. In contrast, LTA is better when a multivariate description is desired, advanced missing data handling methods are needed, or outcomes are not normally distributed. Although infrequently done, evaluating CS provides useful information about program effectiveness.
In the present study, we assessed the psychometric proprieties of the Positive and Negative Affect Scale–Child Version (PANAS-C) in a large sample of Palestinian children (N = 1,376) of different age ranges living in refugee camps. In particular, we used standard confirmatory factor analysis to test competing factor structures for the PANAS-C, with a view to developing a stable version of the instrument, suitable for speedy administration in applied and research settings in the contexts of military violence. Four alternative models of the PANAS-C were evaluated: unidimensional; two-dimensional with independent PA and NA scales and covariance of item-level errors unallowed; two-dimensional with dependent PA and NA scales and covariance of item-level errors unallowed; and two-dimensional with dependent PA and NA scales and covariance of item-level errors. The results of the statistical analysis supported a 20-item measurement model comprising the PANAS-C20 Arabic version for children. The items in this best fitting model loaded on two different and negatively correlated factors. These findings encourage full adoption of the PANAS-C20 as a tool for assessing both PA and NA in Palestinian children living in contexts of warfare.
Increasingly, health care is being delivered in large, complex organizations, and physicians must learn to function effectively in them. As a result, several medical and business schools have developed joint programs to train physician leaders who receive both medical degree (MD) and master of business administration (MBA) degrees. We examined several themes in relation to these programs, revolving around concerns about who is attracted to them and whether exposure to the differing cultures of medicine and business have an impact on the professional identities of their graduates as manifested in their motivations, aspirations, and careers. We addressed these issues by studying students in the joint MD/MBA program at Harvard Medical School (HMS) and Harvard Business School (HBS). Our data came from several internal sources and a survey of all students enrolled in the joint program in spring 2013. We found relatively few differences between joint program students and equivalent cohorts of HMS students in terms of personal characteristics, preadmission performance, and performance at HMS and HBS. Contrary to the concerns that such programs may draw students away from medicine, the vast majority embraced careers involving extensive postgraduate medical training, with long-term plans that leveraged their new perspectives and skills to improve health care delivery.
Surveys of health professionals typically have low response rates, and these rates have been decreasing in the recent years. We report on the methods used in a successful survey of dentist members of the National Dental Practice–Based Research Network. The objectives were to quantify the (1) increase in response rate associated with successive survey methods, (2) time to completion with each successive step, (3) contribution from the final method and personal contact, and (4) differences in response rate and mode of response by practice/practitioner characteristics. Dentist members of the network were mailed an invitation describing the study. Subsequently, up to six recruitment steps were followed: initial e-mail, two e-mail reminders at 2-week intervals, a third e-mail reminder with postal mailing a paper questionnaire, a second postal mailing of paper questionnaire, and staff follow-up. Of the 1,876 invited, 160 were deemed ineligible and 1,488 (87% of 1,716 eligible) completed the survey. Completion by step: initial e-mail, 35%; second e-mail, 15%; third e-mail, 7%; fourth e-mail/first paper, 11%; second paper, 15%; and staff follow-up, 16%. Overall, 76% completed the survey online and 24% on paper. Completion rates increased in absolute numbers and proportionally with later methods of recruitment. Participation rates varied little by practice/practitioner characteristics. Completion on paper was more likely by older dentists. Multiple methods of recruitment resulted in a high participation rate: Each step and method produced incremental increases with the final step producing the largest increase.
The purpose of this study was to compare several different measures of physician–patient communication. We compared data derived from different measures of three communication behaviors, patient participation, physician information giving, and physician participatory decision-making (PDM) style, from 83 outpatient visits to oncology or thoracic surgery clinics for pulmonary nodules or lung cancer. Communication was measured with rating scales completed by patients and physicians after the consultation and by two different groups of external observers who used rating scales or coded the frequency of communication behaviors, respectively, after listening to an audio recording of the consultation. Measures were compared using Pearson’s correlations. Correlations of patients’ and physicians’ ratings of patient participation (r = .04) and physician PDM style (r = .03) were low and not significant (p > .0083, Bonferroni-adjusted). Correlations of observers’ ratings with patients’ or physicians’ ratings for patient participation and physician PDM style were moderate or low (r = .15, .27, .07, and .01, respectively) but were not statistically significant (p > .0083, Bonferroni-adjusted). Correlations between observers’ ratings and frequency measures were .31, .52, and .63 and were statistically significant with p values .005, <.0001, and <.0001, respectively, for PDM style, information giving, and patient participation. Our findings highlight the potential for using observers’ ratings as an alternate measure of communication to more labor intensive frequency measures.
Employment is fundamental to mental health recovery. The aim of this study is to construct a parsimonious profile indicating employment potential of people with major depressive disorder (MDD) to facilitate clinical assessment on employment participation. Data were drawn from Waves 1 (2001–2002) and 2 (2004–2005) of the National Epidemiologic Survey on Alcohol and Related Conditions. We included participants who had MDD at Wave 1 and were interviewed in both waves (N = 2,864). We conducted Classification and Regression Tree (CART) analysis to identify key characterizing factors of Wave 2 employment among 32 Wave 1 risk and protective factors. The results show that 82.1% of those employed at Wave 1 were likely to be employed at Wave 2. Among those unemployed at Wave 1, 51% of those motivated to work, measured by work-seeking behavior in the prior year, were likely to be employed at Wave 2. Among those unemployed and motivated to work, better functional mental health was associated with employment (>25.3 vs. ≤25.3). Results highlight the importance of motivation to work, shown in active work seeking, in facilitating employment despite clinical conditions.
The case for a more diverse health-care workforce has never been stronger given the rapidly changing demographics of the United States and the continued underrepresentation of certain racial and ethnic groups across the health professions. To date, progress toward diversifying the health-care workforce has been and continues to be deterred by a mix of factors at the societal, institutional, and individual levels. Since the 1970s, the Federal government has invested resources in initiatives that support the training and development of the existing workforce as well increase the supply of new health professionals—particularly those from underrepresented minority groups and/or from disadvantaged backgrounds. However, limited studies have been published detailing the processes, outputs and, where available, outcomes of such investments across multiple years. This article describes how the Health Resources and Services Administration’s Bureau of Health Workforce used retrospective case study methodology to evaluate processes and outputs associated with the Scholarships for Disadvantaged Students program—an over US$40 million annual Federal investment aimed at offsetting tuition costs for health professions students from disadvantaged backgrounds—over a 5-year period. Lessons learned and recommendations for strengthening the program’s design and requirements are provided.
Studies that have examined gender differences in smoking cessation have produced mixed results. The purpose of the study was to examine whether there are gender differences in long-term smoking abstinence rates in smokers treated with nicotine patches at a smoking cessation clinic in Taiwan, where 39% of men and 5% of women smoke. This study included 1,065 smokers, comprising of 940 men and 125 women. Smokers were invited to attend the clinic every 1–2 weeks for a maximum of eight visits over 90 days, where they received prescriptions for nicotine patches, counseling, and educational materials. Participants were contacted by telephone at 1 and 3 years after the first visit and were asked whether they had smoked at all over the past 7 days. The results showed that women were significantly less likely than men to be abstinent at 1 year (adjusted odds ratio [aOR] = 0.64; 95% CI [confidence interval] = [0.41, 0.99]; p = .044) and 3 years (aOR = 0.44; 95% CI = [0.27, 0.74]; p = .02). More effective ways are needed to help female smokers quit in societies where smoking in women is rare and may be associated with social stigma.
Alcohol-related problems have traditionally been conceptualized and measured by an effect indicator model. That is, it is generally assumed that observed indicators of alcohol problems are caused by a latent variable. However, there are reasons to think that this construct is more accurately conceptualized as including at least some causal indicators, in which observed indicators cause the latent variable. The present study examined the measurement model of a well-known alcohol consequences questionnaire, the Rutgers Alcohol Problem Index. Participants were 703 students from a large public university in the Northeast mandated to an alcohol intervention. We conducted a zero tetrad test to examine a measurement model consisting solely of effect indicators and a model with both causal and effect indicators. Overall, the results suggested the hybrid model fit the data better than a model with only effect indicators. These findings have implications regarding the theoretical underpinnings of alcohol-related consequences.
Clinical Practice Guidelines for Treating Tobacco Use and Dependence advocate for using counseling targeted at tobacco users’ motivation to quit during each office visit. We evaluate tobacco use screening and counseling interventions delivered during routine periodic health examinations by 44 adult primary care physicians practicing in 22 clinics of a large health system in southeast Michigan. 484 office visits were audio-recorded and transcribed. For this study, current tobacco users (N = 91) were identified using pre-visit surveys and audio-recordings. Transcripts were coded for the delivery of tobacco-related counseling interventions. The extent to which counseling interventions were used and/or targeted to the patients’ readiness to quit was the main outcome measure. The majority of tobacco users (n = 77) had their tobacco use status assessed, and most received some sort of tobacco-related counseling (n = 74). However, only 15% received the recommended counseling targeted to their readiness to quit. On the other hand, 19% received less counseling than recommended given their readiness to quit, 7% received only nonindicated counseling, and 59% received nonindicated counseling in addition to indicated counseling. Results illustrate physicians’ commitment to cessation counseling and also identify potential opportunities to improve the efficiency of tobacco-related counseling in primary care.
The patient-reported outcome measurement information system (PROMIS®) initiative has developed and evaluated a set of publicly available, efficient, and flexible measures of patient-reported outcomes in different health domains, including mental health. The objective of this study was to translate the PROMIS Depression item bank into German and evaluate the psychometric properties of the translated items. Items were translated using forward and backward translation and cognitive interviews. Distribution characteristics, unidimensionality, Rasch model fit, reliability, construct validity, and internal responsiveness were investigated in a sample of 234 patients in in-patient psychosomatic rehabilitation centers in Germany. The translated items showed good psychometric properties, the distribution characteristics were satisfactory, and the sufficient unidimensionality was obtained by fitting a bifactor model. The construct validity was demonstrated, and it was reliable and was shown to detect clinically significant changes. The translated items can be recommended for the assessment of depression. Future studies should examine the generalizability of the results.
"Eigenständig werden 5+6" (Becoming independent 5+6) is a German school-based smoking prevention program that draws on social competence and social influence approaches. It was investigated whether the program’s effect on smoking onset is mediated by substance-specific skills and cognitions such as knowledge, attitudes/risk perception, normative expectations, resistance skills, and refusal self-efficacy. Multiple mediation analyses revealed a statistically significant total indirect effect that accounted for 30.8% of the total effect. When considered separately, significant indirect effects could be found for normative expectations concerning peer smoking and the resistance skill of saying ‘no’. Between these two mediators, the percentage of total effect mediated varied between 9.8% and 10.3%. Results of the current study emphasize the importance of substance-specific skills and cognitions in the effectiveness of school-based programs combining social competence and social influence curricula in preventing adolescent smoking onset.
At least three factors may be driving the evolution of the vape shop industry, a rapidly growing market sector that specializes in the sales of electronic cigarettes: (1) the tobacco industry, (2) the public health sector and its diverse stakeholders, and (3) consumer demand. These influences and the responses of the vape shop sector have resulted in a rapidly changing landscape. This commentary briefly discusses these three factors and the implications for the health professions, as they address the vape shop industry and its consequences for public health.
In efforts to decrease practice variation, clinical practice guidelines for neck pain have been published. The purpose of this study was to determine the effect of receiving guideline adherent physical therapy (PT) on clinical outcomes, health care utilization, and charges for health care services in patients with neck pain. A retrospective review of 298 patients with neck pain receiving PT from 2008 to 2011 was performed. Clinical outcomes, utilization, and charges were compared between patients who received guideline adherent care and nonadherent care. Patients in the adherent care group experienced a lower percentage improvement in pain score compared to nonadherent care group (p = .01), but groups did not significantly differ on percentage improvement in disability (p = .32). However, patients receiving adherent care had an average 3.6 fewer PT visits (p < .001) and less charges for PT (p < .001). Additionally, patients receiving adherent care had 7.3 fewer visits to other health care providers (p < .001), one less prescription medication (p = .02) and 43% fewer diagnostic images (p = .02) but did not differ in their charges to other health care providers (p = .68) during the calendar year of undergoing PT. Although receiving guideline adherent care demonstrated positive effects on health care utilization and financial outcomes, there appears to be a trade-off with clinical outcomes.
Recent studies have estimated the prevalence of depression during pregnancy to be between 10% and 30%, which is higher than that in the postpartum period. Pharmacological treatment during pregnancy is difficult because of the possible side effects of antidepressants on the mother and the fetus. The aim of this study was to examine whether a supervised exercise program (EP) reduces depressive symptoms in pregnant women. A randomized controlled trial was designed. One hundred eighty four healthy pregnant women from Fuenlabrada Hospital were included (31.37 ± 3.62 years). Women from the exercise group (EG) participated in a supervised EP consisting of three, 55- to 60-min sessions per week throughout pregnancy. The main outcome measure was the patients’ depression level assessed by means of the Center for Epidemiologic Studies Depression Scale (CES-D). A total of 167 pregnant women were analyzed; 90 were allocated to the EG and 77 to the control group (CG). Significant differences were found between groups at the end of the study in CES-D scores (EG: 7.67 ± 6.30 vs. CG: 11.34 ± 9.74, p = .005) and in percentages of pregnant women depressed (EG: n = 11/12.2% vs. CG: n = 19/24.7%, p = .04). Our results show that supervised physical exercise during pregnancy reduces the level of depression and its incidence in pregnant women.
More than half of all children in the United States aged 3 to 6 years are enrolled in child care centers. Maine received funds from the U.S. Department of Health and Human Services’ Communities Putting Prevention to Work to promote the adoption of Nutrition and Physical Activity Self-Assessment for Child Care (NAPSACC), an evidence-based program for the child care setting. We evaluated the rollout and adoption of NAP SACC in Maine using multiple methods. Our findings suggest that the NAP SACC program has been successfully adopted in Maine. Nutrition and physical activity policies and offerings have improved, especially with regard to purchasing healthier options in the child care setting.
The purpose of this study was to develop a short Hebrew version of the Zimbardo Time Perspective Inventory that can be easily administered by health professionals in research, therapy, and counseling. First, the empirical links of time perspective (TP) to subjective well-being and health protective and health risk behaviors are reviewed. Then, a brief account of the instrument’s previous modifications is provided. Results of confirmatory factor analysis (N = 572) verified the five-factor structure of the short version and yielded acceptable internal consistency reliability for each factor. The correlation coefficients between the five subscales of the short (20 items) and the original (56 items) instruments were all above .79, indicating the suitability of the short version for assessing the five TP factors. Support for the discriminant and concurrent validity was also achieved, largely in agreement with previous findings. Finally, limitations and future directions are addressed, and potential applications in therapy and counseling are offered.
Narrative reviews conclude that behavioral therapies (BTs) produce better outcomes than control conditions for cannabis use disorders (CUDs). However, the strength and consistency of this effect has not been directly empirically examined. The present meta-analysis combined multiple well-controlled studies to help clarify the overall impact of behavioral interventions in the treatment of CUDs. A comprehensive literature search produced 10 randomized controlled trials (RCTs; n = 2,027) that were included in the final analyses. Analyses indicated an effect of BTs (including contingency management, relapse prevention, and motivational interviewing, and combinations of these strategies with cognitive behavioral therapy) over control conditions (including waitlist [WL], psychological placebo, and treatment as usual) across pooled outcomes and time points (Hedges’ g = 0.44). These results suggest that the average patient receiving a behavioral intervention fared better than 66% of those in the control conditions. BT also outperformed control conditions when examining primary outcomes alone (frequency and severity of use) and secondary outcomes alone (psychosocial functioning). Effect sizes were not moderated by inclusion of a diagnosis (RCTs including treatment-seeking cannabis users who were not assessed for abuse or dependence vs. RCTs including individuals diagnosed as dependent), dose (number of treatment sessions), treatment format (either group vs. individual treatment or in-person vs. non-in-person treatment), sample size, or publication year. Effect sizes were significantly larger for studies that included a WL control comparison versus those including active control comparisons, such that BT significantly outperformed WL controls but not active control comparisons.
The declining number of physician scientists is an alarming issue. A systematic review of all existing programs described in the literature was performed, so as to highlight which programs may serve as the best models for the training of successful physician scientists. Multiple databases were searched, and 1,294 articles related to physician scientist training were identified. Preference was given to studies that looked at number of confirmed publications and/or research grants as primary outcomes. Thirteen programs were identified in nine studies. Eighty-three percent of Medical Scientist Training Program (MSTP) graduates, 77% of Clinician Investigator Training Program (CI) graduates, and only 16% of Medical Fellows Program graduates entered a career in academics. Seventy-eight percent of MSTP graduates succeeded in obtaining National Institute of Health (NIH) grants, while only 15% of Mayo Clinic National Research Service Award-T32 graduates obtained NIH grants. MSTP physician scientists who graduated in 1990 had 13.5 ± 12.5 publications, while MSTP physician scientists who graduated in 1975 had 51.2 ± 38.3 publications. Additionally, graduates from the Mayo Clinic’s MD-PhD Program, the CI Program, and the NSRA Program had 18.2 ± 20.1, 26.5 ± 24.5, and 17.9 ± 26.3 publications, respectively. MSTP is a successful model for the training of physician scientists in the United States, but training at the postgraduate level also shows promising outcomes. An increase in the number of positions available for training at the postgraduate level should be considered.
The purpose of this study was to determine the measurement properties of the Low Back Activity Confidence Scale (LoBACS) in individuals with post-acute low back pain (LBP) receiving nonsurgical intervention, including construct validity, factorial validity, and internal consistency reliability. Data were analyzed from an existing randomized clinical trial involving 112 patients with LBP. Evidence for convergent validity was observed through significant correlations between LoBACS subscale scores and other function, pain, and psychobehavioral measures. LoBACS subscales accounted for 36% of the unique variance in dependent variable measurements, suggesting a satisfactory level of statistical divergence between the LoBACS and other psychobehavioral measurements in this study. Cronbach’s α ranged from .88 to .92 for LoBACS subscales, and item-total correlations exceeded .6, indicating high internal consistency reliability. Principal axis factoring confirmed the hypothesized three-subscale structure by correctly classifying 14 of the 15 items. These findings indicate the LoBACS is valid and internally consistent to measure domain-specific self-efficacy beliefs.
This study aimed to translate into Chinese the Diabetes Care Profile (DCP), a measure of psychosocial factors and diabetes treatment, and to test the reliability and validity of the instrument within a Chinese population. The English version of the DCP was translated into Chinese following the standard translation methodology with consideration to cultural adaptation. The questionnaire was administered to 313 people with type 2 diabetes in an urban community in Beijing, China. Cronbach’s α coefficient was used to calculate reliabilities, which ranged from .55 to .86 on DCP subscales. Mean values on the DCP differed by diabetes treatment as expected and supports the construct validity of the DCP. The overall score on the DCP correlated well both with blood glucose levels and with a validated measure of a Chinese version of the Diabetes Specific Quality of Life scale, thus supporting the DCP’s criterion validity. The DCP is an acceptable measure of the psychosocial factors related to diabetes and its treatment in a Chinese population with type 2 diabetes.
Quality improvement and cost containment initiatives in health care increasingly involve interdisciplinary teams of providers. To understand organizational functioning, information is often needed from multiple members of a leadership team since no one person may have sufficient knowledge of all aspects of the organization. To minimize survey burden, it is ideal to ask unique questions of each member of the leadership team in areas of their expertise. However, this risks substantial missing data if all eligible members of the organization do not respond to the survey. Nursing home administrators (NHA) and directors of nursing (DoN) play important roles in the leadership of long-term care facilities. Surveys were administered to NHAs and DoNs from a random, nationally representative sample of U.S. nursing homes about the impact of state policies, market forces, and organizational factors that impact provider performance and residents’ outcomes. Responses were obtained from a total of 2,686 facilities (response rate [RR] = 66.6%) in which at least one individual completed the questionnaire and 1,693 facilities (RR = 42.0%) in which both providers participated. No evidence of nonresponse bias was detected. A high-quality representative sample of two providers in a long-term care facility can be obtained. It is possible to optimize data collection by obtaining unique information about the organization from each provider while minimizing the number of items asked of each individual. However, sufficient resources must be available for follow-up to nonresponders with particular attention paid to lower resourced, lower quality facilities caring for higher acuity residents in highly competitive nursing home markets.
Improving market orientation and patient safety have become the key concerns of nursing management. For nurses, establishing a patient safety climate is the key to enhancing nursing quality. This study explores how market orientation affects the climate of patient safety among hospital nurses. We proposed adopting a cross-sectional research design and using questionnaires to collect responses from nurses working in two Taiwanese hospitals. Three-hundred and forty-three valid samples were obtained. Multiple regression and path analyses were conducted to test the study. Market orientation was defined as the combination of customer orientation, competitor orientation, and interfunctional coordination. Customer orientation directly affects the climate of patient safety. Although the findings only supported Hypothesis 1, competitor orientation and interfunctional coordination positively affected the patient safety climate through the mediating effects of hospital support for staff. Health care managers could encourage nurses to adopt customer-oriented perspectives to enhance their nursing care. In addition, to enhance competitor orientation, interfunctional coordination, and the patient safety climate, hospital managers could strengthen their support for staff members.
This study examined the psychometric properties of the Greek version of the Smoking Efficacy Scale (SES) for adolescents in a sample of 536 high school students. The factorial structure of the SES was examined by means of a series of exploratory factor analyses. The structural validity, the internal consistency, the temporal stability, and the concurrent validity of the SES were assessed. Exploratory and confirmatory factor analyses revealed a clear three-factor (emotion, opportunity, and friends) structure. Furthermore, SES predicted students’ smoking behavior. Overall, the multidimensionality of the SES was supported by our findings, suggesting that the Greek version of the SES appears to be a psychometrically sound instrument that can be used for the evaluation of smoking prevention and smoking cessation programs for high school students.
Blinded assessments of technical skills using video-recordings may offer more objective assessments than direct observations. This study seeks to compare these two modalities. Two trained assessors independently assessed 18 central venous catheterization performances by direct observation and video-recorded assessments using two tools. Although sound quality was deemed adequate in all videos, portions of the video for wire handling and drape handling were frequently out of view (n = 13, 72% for wire-handling; n = 17, 94% for drape-handling). There were no differences in summary global rating scores, checklist scores, or pass/fail decisions for either modality (p > 0.05). Inter-rater reliability was acceptable for both modalities. Of the 26 discrepancies identified between direct observation and video-recorded assessments, three discrepancies (12%) were due to inattention during video review, while one (4%) discrepancy was due to inattention during direct observation. In conclusion, although scores did not differ between the two assessment modalities, techniques of video-recording may significantly impact individual items of assessments.
Inadequate statistical power to detect treatment effects in health research is a problem that is compounded when testing for mediation. In general, the preferred strategy for increasing power is to increase the sample size, but there are many situations where additional participants cannot be recruited, necessitating the use of other methods to increase statistical power. Many of these other strategies, commonly applied to analysis of variance and multiple regression models, can be applied to mediation models with similar results. Additional predictors or blocking variables will increase or decrease statistical power, however, depending on whether these variables are related to the mediator, the outcome, or both. The effect of these two methods on the power for tests of mediation is illustrated through the use of simulations. Implications for health researchers using these methods are discussed.
Pharmacists are routinely providing reproductive health counseling in community pharmacies, but studies have revealed significant deficits in their competencies. Therefore, continuing pharmacy education (CPE) could be utilized as a valuable modality to upgrade pharmacists’ capabilities. A randomized controlled trial was designed to compare the efficacy of CPE meetings (lecture based vs. workshop based) on contraception and male sexual dysfunctions. Sixty pharmacists were recruited for each CPE meeting. Small group training using simulated patients was employed in the workshop-based CPE. Study outcomes were declarative/procedural knowledge, attitudes, and satisfaction of the participants. Data were collected pre-CPE, post-CPE, and 2 months afterward and were analyzed using repeated measure analysis of variance and Mann–Whitney U test. Results showed that lecture-based CPE was more successful in improving pharmacists’ knowledge post-CPE (p < .001). In contrast, a significant decrease was observed in the lecture-based group at follow-up (p = .002), whereas the workshop-based group maintained their knowledge over time (p = 1.00). Knowledge scores of both groups were significantly higher at follow-up in comparison with pre-CPE (p < .01). No significant differences were observed regarding satisfaction and attitudes scores between groups. In conclusion, an interactive workshop might not be superior to lecture-based training for improving pharmacists’ knowledge and attitudes in a 1-day CPE meeting.
This study assessed whether postal follow-up to a web-based physician survey improves response rates, response quality, and representativeness. We recruited primary care and gastroenterology chiefs at 125 Veterans Affairs medical facilities to complete a 10-min web-based survey on colorectal cancer screening and diagnostic practices in 2010. We compared response rates, response errors, and representativeness in the primary care and gastroenterology samples before and after adding postal follow-up. Adding postal follow-up increased response rates by 20–25 percentage points; markedly greater increases than predicted from a third e-mail reminder. In the gastroenterology sample, the mean number of response errors made by web responders (0.25) was significantly smaller than the mean number made by postal responders (2.18), and web responders provided significantly longer responses to open-ended questions. There were no significant differences in these outcomes in the primary care sample. Adequate representativeness was achieved before postal follow-up in both samples, as indicated by the lack of significant differences between web responders and the recruitment population on facility characteristics. We conclude adding postal follow-up to this web-based physician leader survey improved response rates but not response quality or representativeness.
In light of increasing interest in evidence-based management, we conducted a scoping review of systematic reviews (SRs) and meta-analyses (MAs) to determine the availability and accessibility of evidence for health care managers; 14 MAs and 61 SRs met the inclusion criteria. Most reviews appeared in medical journals (53%), originated in the United States (29%) or United Kingdom (22%), were hospital-based (55%), and targeted clinical providers (55%). Topics included health services organization (34%), quality/patient safety (17%), information technology (15%), organization/workplace management (13%), and health care workforce (12%). Most reviews addressed clinical topics of relevance to managers; management-related interventions were rare. The management issues were mostly classified as operational (65%). Surprisingly, 96.5% of search results were not on target. A better classification within PubMed is needed to increase the accessibility of meaningful resources and facilitate evidence retrieval. Health care journals should take initiatives encouraging the publication of reviews in relevant management areas.
The purpose of this article is to introduce and describe a statistical model that researchers can use to evaluate underlying mechanisms of behavioral onset and other event occurrence outcomes. Specifically, the article develops a framework for estimating mediation effects with outcomes measured in discrete-time epochs by integrating the statistical mediation model with discrete-time survival analysis. The methodology has the potential to help strengthen health research by targeting prevention and intervention work more effectively as well as by improving our understanding of discretized periods of risk. The model is applied to an existing longitudinal data set to demonstrate its use, and programming code is provided to facilitate its implementation.
Assessments of subjects by multiple raters are completed in several health research and practice arenas. As observed in interrater comparisons of checklists during assessment, raters generally make mistakes in assessment. Despite these errors, the average scores on all the checklists have been historically used to represent the subjects’ true scores. The effect of the assessment rater error on these average scores has been seldom studied. In the present study, the effect of random rater error on subjects’ average scores was investigated mathematically. The analysis based on a mathematical model revealed that average scores were truly representative only in special cases where raters made no assessment errors. In addition, average scores were poor estimates of the true score due to random assessment error. A new, more precise formula to estimate subjects’ true scores was proposed. The method to apply this formula to actual checklist assessment data was demonstrated.
This study investigated construct validity and internal consistency of the Student Practice Evaluation Form–Revised Edition Package (SPEF-R) which evaluates students’ performance on practice education placements. The SPEF-R has 38 items covering eight domains, and each item is rated on a 5-point rating scale. Data from 125 students’ final placement evaluations in their final year study were analyzed using the Rasch measurement model. The SPEF-R exhibited satisfactory rating scale performance and unidimensionality across the eight domains, providing construct validity evidence. Only 2 items misfit Rasch model’s expectations (both related to students’ performance with client groups, which were often rated as not observed). Additionally, the internal consistency of each SPEF-R domain was found to be excellent (Cronbach’s α = .86 to .91) and all individual items had reasonable to excellent item-total correlation coefficients. The study results indicate that the SPEF-R can be used with confidence to evaluate students’ performance during placements, but continued validation and refinement are required.
The Patient Reported Outcomes Measurement Information System (PROMIS) initiative aims to provide reliable and precise item banks measuring patient-reported outcomes in different health domains. The aim of the present work was to provide a German translation of the PROMIS item banks for satisfaction with participation and to psychometrically test these German versions. Cognitive interviews followed a forward–backward translation. Distribution characteristics, unidimensionality, Rasch model fit, reliability, construct validity, and internal responsiveness were tested in 262 patients with chronic low back pain undergoing rehabilitation. Results for the final 13- and 10-item German static scales (Satisfaction with
This study serves as an investigation of the reliability of symptom data as reported by individuals with chronic fatigue syndrome (CFS), across three recall time frames (the past week, the past month, and the past 6 months), and at two assessment points (with 1 week in between each assessment). Multilevel model analyses were used to determine the optimal recall time frame, in terms of test -retest reliability, for each of the Fukuda et al. (1994) case defining symptoms. Results suggested that the optimal time frame for reliably reporting CFS symptoms was six months for sore throat, lymph node pain, muscle pain, post-exertional malaise, headaches, memory/concentration difficulties, and unrefreshing sleep. For joint pain, the optimal time frame was one month. Researchers who are interested in the assessment of CFS symptoms need to take recall time frame into account, especially when the intended goal is to standardize and improve the methods used to reliably and accurately diagnose this complex illness.
This article reflects on the current state of process evaluations of health behavior interventions and argues that evaluation practice in this area could be improved by drawing on the social science literature to a greater degree. While process evaluations of health behavior interventions have increasingly engaged with the social world and sociological aspects of interventions, there has been a lag in applying relevant and potentially useful approaches from the social sciences. This has limited the scope for health behavior process evaluations to address pertinent contextual issues and methodological challenges. Three aspects of process evaluations are discussed: the incorporation of contexts of interventions; engagement with the concept of "process" in process evaluation; and working with theory to understand interventions. Following on from this, the article also comments on the need for new methodologies and on the implications for addressing health inequalities.
This article describes a protocol that was employed to validate the content of the National Board Dental Hygiene Examination (NBDHE) using two separate practice analyses. The protocol consisted of the following phases: (1) the careful definition of the domain of knowledge and skills required for successful entry-level dental hygiene practice, (2) the conduct of a survey to gather information regarding the judgments of practicing dental hygienists on the important knowledge, abilities, and skills required of entry-level dental hygiene practice, (3) the analysis of the survey data, (4) the integration of the survey findings with the examination content using a two-dimensional matrix, (5) development of the updated specifications of the examination based on the practice analysis data, and (6) implementation of the updated specifications for subsequent item and test development. Survey data from 1,284 and 1,388 full-time practicing dental hygienists for the 2002 and 2009 practice analyses, respectively, were analyzed. A two-dimensional matrix was used to map the content of the examination to the practice of dental hygiene based on the practice analysis data. The mapping showed that the distribution of items in the content specifications reflected actual dental hygiene practice. Thus, the adequacy of the content validity of the NBDHE was confirmed.
Interviewers administered the Rosenberg Self-Esteem scale (RSES) to five groups of Black (formal township and informal settlement), White, Indian, and mixed race adult residents of Greater Pretoria. The results demonstrated that the RSES was psychometrically sound for the five groups. The minimal effects of sociodemographic characteristics on global self-esteem showed that the RSES and its two dimensions, self-competence (SC) and self-liking (SL), were suitable in this setting. All five groups scored above the theoretical midpoint of the RSES, indicating that generally positive self-evaluations appear to be universal. The relationships between positively and negatively worded items, SC, and SL attested to the following: internal structure reliability, congruence between positive and negative items, no negative biases in response, and concordance between SC and SL dimensions. The significant differences between informal settlement residents and the other four groups on global self-esteem, positively and negatively worded items, and SC and SL were possibly due to physiological needs taking precedence over higher order needs.
Assessing the value of clinical and translational research funding on accelerating the translation of scientific knowledge is a fundamental issue faced by the National Institutes of Health (NIH) and its Clinical and Translational Awards (CTSAs). To address this issue, the authors propose a model for measuring the return on investment (ROI) of one key CTSA program, the clinical research unit (CRU). By estimating the economic and social inputs and outputs of this program, this model produces multiple levels of ROI: investigator, program, and institutional estimates. A methodology, or evaluation protocol, is proposed to assess the value of this CTSA function, with specific objectives, methods, descriptions of the data to be collected, and how data are to be filtered, analyzed, and evaluated. This article provides an approach CTSAs could use to assess the economic and social returns on NIH and institutional investments in these critical activities.
The success case studies approach examines in depth what works well in a program by describing cases and examining factors leading to successful outcomes. In this article, we describe the use of success case studies as part of an evaluation of the transformation of a health sciences research support infrastructure. Using project-specific descriptions and the researchers’ perceptions of the impact of improved research infrastructure, we added depth of understanding to the quantitative data required by funding agencies. Each case study included an interview with the lead researcher, along with review of documents about the research, the investigator, and their collaborators. Our analyses elucidated themes regarding contributions of the Clinical and Translational Science Awards program of the National Institutes of Health to scientific achievements and career advancement of investigators in one academic institution.
Nonresponse bias in survey research can result in misleading or inaccurate findings and assessment of nonresponse bias is advocated to determine response sample representativeness. Four methods of assessing nonresponse bias (analysis of known characteristics of a population, subsampling of nonresponders, wave analysis, and linear extrapolation) were applied to the results of a postal survey of U.K. hospital organizations. The purpose was to establish whether validated methods for assessing nonresponse bias at the individual level can be successfully applied to an organizational level survey. The aim of the initial survey was to investigate trends in the implementation of radiographer abnormality detection schemes, and a response rate of 63.7% (325/510) was achieved. This study identified conflicting trends in the outcomes of analysis of nonresponse bias between the different methods applied and we were unable to validate the continuum of resistance theory as applied to organizational survey data. Further work is required to ensure established nonresponse bias analysis approaches can be successfully applied to organizational survey data. Until then, it is suggested that a combination of methods should be used to enhance the rigor of survey analysis.
This article shows how to compute statistical power for testing the main effect of treatment in three-arm cluster randomized trials. Using orthogonal coding, we derive the exact test statistic of the treatment effect and its non-central distribution. The non-centrality parameter in the omnibus test is found to be related to the non-centrality parameters in the contrast tests. A study of physician and pharmacist comanagement of patients’ blood pressure is used as an example to show the power computation in a three-arm cluster randomized trial.
Clinical research management (CRM) is a critical resource for the management of clinical trials and it requires proper evaluation. This article advances a model of evaluation that has three local levels, plus one global level, for evaluating the value of CRM. The primary level for evaluation is that of the study or processes level. The managerial or aggregate level concerns management of the portfolio of trials under the control of the CRM office. The third, often overlooked level of evaluation, is the strategic level, whose goal is encapsulated in the phrase, "doing the right trials, while doing trials right." The global ("plus one") evaluation level concerns the need to evaluate the ever-increasing number of multi-institutional and multinational studies. As there are host of evaluation metrics, this article provides representative examples of metrics at each level and provides methods that can aid in the selecting appropriate metrics for an organization.
The present investigation examined associations between intensities of exercise involvement and posttraumatic stress (PTS) symptom cluster severity (reexperiencing, avoidance/numbing, and hyperarousal). The sample was comprised of 108 adults (54.6% women; Mage = 23.9, SD = 10.22, range = 18–62), who endorsed exposure to a Diagnostic and Statistical Manual of Mental Disorders (Fourth edition, Text Revision) posttraumatic stress disorder Criterion A traumatic life event but did not meet criteria for any current Axis I psychopathology. After controlling for gender and lifetime number of trauma exposure types experienced, results indicated that vigorous-intensity exercise, but not light- or moderate-intensity exercise, was significantly inversely associated with hyperarousal symptom cluster severity. This study adds to the scarce, yet growing, body of exercise—PTS literature—by illuminating the inverse associations of vigorous-intensity exercise, specifically, and PTS hyperarousal symptom severity among trauma-exposed individuals.
Hospital readmission is an important indicator of health care quality and currently used in determining hospital reimbursement rates by Centers for Medicare & Medicaid Services. Given the important policy implications, a better understanding of factors that influence readmission rates is needed. Racial disparities in readmission have been extensively studied, but income and marital status (a postdischarge care support indicator) disparities have received limited attention. By employing three Poisson regression models controlling for different confounders on 8,718 patients in a veterans-integrated health care network, this study assessed racial, income, and martial disparities in relation to total number of readmissions. In contrast to other studies, no racial and income disparities were found, but unmarried patients experienced significantly more readmissions: 16%, after controlling for the confounders. These findings render unique insight into health care policies aimed to improve race and income disparities, while challenging policy makers to reduce readmissions for those who lack family support.
The Simple Screening Instrument for Substance Abuse (SSI-SA) is gaining widespread use as a self-report measure of substance abuse; yet, little information exists regarding the instrument’s psychometric properties. This study examined the SSI’s psychometric properties within a population of 6,664 adult Medicaid enrollees in Florida, who responded to a survey conducted as part of a statewide evaluation of Medicaid services. The SSI-SA had excellent internal consistency (.85). Evidence of the SSI’s validity was strong; SSI-SA scores distinguished among individuals with and without substance abuse needs and were significantly correlated with a measure of functioning in daily living. Using the recommended SSI-SA cutoff score of 4 or higher to indicate the presence of a substance abuse problem, the SSI-SA had respectable sensitivity (.82) and specificity (.90).
Health care organizations continue to implement organization-wide educational approaches to enhance patient safety with less attention on evaluating the impact of these approaches. In this context, a study was conducted to measure the impact of an organization-wide patient safety network approach on patient safety event reporting. A time-series analysis with reported rates of adverse events (major and moderate), near misses, sentinel events, and incidents from 2 years prior through 13 months following implementation was conducted. Study findings include a significant increase in reporting of patient safety events (an approximately 50% increase in overall reporting of safety events was observed; p < .001), especially near misses (an approximately 100% increase following implementation; p = .002). Study findings suggest that a multifaceted networked approach does contribute to improving patient safety event reporting.
Previous research has demonstrated the importance of ensuring that programs are implemented as intended by program developers in order to achieve desired program effects. The current study examined implementation fidelity of Pathways to Health (Pathways), a newly developed obesity prevention program for fourth- through sixth-grade children. We explored the associations between self-reported and observed implementation fidelity scores and whether implementation fidelity differed across the first 2 years of program implementation. Additionally, we examined whether implementation fidelity affected program outcomes and whether teacher beliefs were associated with implementation fidelity. The program was better received, and implementation fidelity had more effects on program outcomes in fifth grade than in fourth grade. Findings suggest that implementation in school-based obesity programs may affect junk food intake and intentions to eat healthfully and exercise. School support was associated with implementation fidelity, suggesting that prevention programs may benefit from including a component that boosts school-wide support.
The general purpose of this secondary analysis of a prior systematic review was to determine the degree to which group dynamics–based physical activity interventions align with research processes and outcomes that are more likely to facilitate the translation of research into practice. To accomplish this, a systematic search was conducted within Science Citation Index Expanded, Social Sciences Citation Index, Arts and Humanities Citation Index, and MEDLINE databases to identify articles published prior to 2010 that used at least one group dynamics–based strategy (e.g., group interaction, group goal setting) in physical activity promotion. These 17 intervention studies were identified and coded based on the reach, effectiveness, adoption, implementation, maintenance framework and Pragmatic–Explanatory Continuum Indicator Summary (for trial design characteristics). Reporting was infrequent for external validity factors (i.e., representativeness, adoption, cost, maintenance) but more frequent for internal validity factors (e.g., inclusion criteria). Intervention costs were not reported. Studies were more likely to be pragmatic (i.e., designed to determine the effects of an intervention under the usual conditions in which it will be applied) in areas of participant compliance and practitioner adherence and explanatory (i.e., designed to determine the effects of an intervention under ideal conditions) and in areas of practitioner expertise and flexibility of intervention protocol. While a number of these interventions were tested in more pragmatic settings, external validity factors were still underreported.
There is a need to standardize methods for assessing fidelity and adaptation. Such standardization would allow program implementation to be examined in a manner that will be useful for understanding the moderating role of fidelity in dissemination research. This article describes a method for collecting data about fidelity of implementation for school-based prevention programs, including measures of adherence, quality of delivery, dosage, participant engagement, and adaptation. We report about the reliability of these methods when applied by four observers who coded video recordings of teachers delivering All Stars, a middle school drug prevention program. Interrater agreement for scaled items was assessed for an instrument designed to evaluate program fidelity. Results indicated sound interrater reliability for items assessing adherence, dosage, quality of teaching, teacher understanding of concepts, and program adaptations. The interrater reliability for items assessing potential program effectiveness, classroom management, achievement of activity objectives, and adaptation valences was improved by dichotomizing the response options for these items. The item that assessed student engagement demonstrated only modest interrater reliability and was not improved through dichotomization. Several coder pairs were discordant on items that overall demonstrated good interrater reliability. Proposed modifications to the coding manual and protocol are discussed.
There is compelling empirical evidence in support of the use of grouped self-assessment data to measure program outcomes. However, other credible research has clearly shown that self-assessments are poor predictors of individual achievement such that the validity of self-assessments has been called into question. Based on the reanalysis of two previously published studies and an analysis of two original studies, we show that grouped self-assessments may be good predictors of and hence valid measures of performance at the group level, an outcome commonly used in program evaluation studies. We found statistically significant correlation coefficients (between 0.56 and 0.87), when comparing across performance items using the group means of self-assessments with the group means of individual achievement on criterion tests. We call for further research into the conditions and circumstances in which grouped self-assessments are used, so that they can be employed more effectively and confidently by program evaluators, decision makers, and researchers.
The aim of this study was to translate and validate the Alzheimer’s Disease Knowledge scale (ADKS) in a population of Greek general practitioners (GPs). The international standards for the forward and back translation approach were followed. For the validation step, 112 GPs, treating dementia in their daily practices, were enrolled from Crete. The questionnaire was assessed for the following psychometric properties: intraclass reliability, test–retest reliability, and construct and face validity. Internal consistency of the Greek ADKS was satisfactory (α = .65). A high repeatability of the instrument was found during the retest with 27 GPs (intraclass correlation coefficient = 1.0). Factor analysis showed that all the items from the original instrument can be used in the Greek version. The inter-item correlation revealed a high cross-correlation between the items of the questionnaire (α > .6). The data confirmed the validity of the Greek version of the ADKS for measuring GPs’ knowledge on the diagnosis and management of dementia.
The health status of ambulance personnel (AP) has an important impact on the quality of patient care. The aim of this study is to analyze the self-fperceived health of AP. A cross-sectional survey was conducted among Hungarian AP, in which anonymous data (n = 364 subjects) were evaluated by descriptive statistics and multivariate logistic regression analyses. Those AP who reported engaging in any amount of exercise experienced better self-rated health (odds ratio [OR]: 1.7 confidence interval [CI] 95% [1.2, 2.7]) and self-rated physical fitness (OR: 2.0 CI 95% [1.2, 2.9]), and reported less limitation in daily activities due to health problems (OR: 2.4 CI 95% [1.4, 4.0]). Those AP who reported feeling more overall stress reported 2.1 times (CI 95% [1.3, 3.2]) worse health and 1.9 (CI 95% [1.2, 2.8]) times worse self-rated physical fitness. Possibly, physical fitness protocols should be implemented and required or advised for all AP.
The spread of evidence-based practice throughout the world has resulted in the wide adoption of empirically supported interventions (ESIs) and a growing number of controlled trials of imported and culturally adapted ESIs. This article is informed by outcome research on family-based interventions including programs listed in the American Blueprints Model and Promising Programs. Evidence from these controlled trials is mixed and, because it is comprised of both successful and unsuccessful replications of ESIs, it provides clues for the translation of promising programs in the future. At least four explanations appear plausible for the mixed results in replication trials. One has to do with methodological differences across trials. A second deals with ambiguities in the cultural adaptation process. A third explanation is that ESIs in failed replications have not been adequately implemented. A fourth source of variation derives from unanticipated contextual influences that might affect the effects of ESIs when transported to other cultures and countries. This article describes a model that allows for the differential examination of adaptations of interventions in new cultural contexts.
Medical decisions, including physicians’ prescribing behaviors, are shaped by a complex interplay of clinical and nonclinical factors. We aim to determine how physician, patient, and relationship characteristics influence physicians’ decisions to accommodate brand-name prescription drug requests. We applied multivariate logistic regression to data from the Attitudinal and Behavioral Effects of Direct-to-Consumer Promotion of Prescription Drugs physician survey. We used a national probability sample of 500 primary care and specialty physicians reporting on a clinical encounter that involved a prescription drug request. Independent variables include physician’s assessment of the patient’s understanding of risks and benefits of a requested medication, whether the patient had the condition the drug treats, duration of the clinical relationship, and physician’s age, area of practice, years of experience, and gender. These variables were used to predict whether the physician prescribed the requested drug. Physicians were more willing to accommodate requests when they believed that patients had a clear understanding of the drug’s risks and when patients had the condition the drug treats. Primary care practitioners, compared to specialists, had higher odds of prescribing a requested drug. We conclude that clinical and communicative factors shape physicians’ decisions to prescribe requested brand-name drugs. Findings offer insight into the influence that direct-to-consumer advertising can have in medical encounters, and may guide efforts to enhance physician–patient communication and shared decision making.
`Varenicline use has been shown to produce greater long-term smoking cessation rates than bupropion but has no clear differences compared to the transdermal nicotine patch. We performed this study to compare the effectiveness of varenicline with the nicotine patch at 3 and 6 months of follow-up of patients in an outpatient smoking cessation program provided by a hospital in Southern Taiwan. The sample consisted of 463 patients who attended the smoking cessation program at the outpatient family medicine clinic at Kaohsiung Veterans General Hospital between March 2006 and December 2008. All patients were aged ≥18 years and either smoked ≥10 cigarettes per day or scored ≥4 on the Fagerström Test for Nicotine Dependence. Patients were seen by a physician for up to 8 sessions in 90 days. Medication use was guided by patient preference (208 opted for varenicline and 255 for the nicotine patch). The primary outcomes of the study were self-reported 7-day point prevalence abstinence rates at 3 and 6 months from the first clinic visit. Varenicline users had a significantly higher abstinence rate than those using nicotine patch at 3-month (47.1% vs. 30.6%; odds ratio [OR] = 2.02, 95% confidence interval [CI] = [1.38, 2.96]) and 6-month follow-up (41.3% vs. 30.6%; OR = 1.60, 95% CI [1.09, 2.32]). Both groups had similar incidences of adverse events. Varenicline use in a sample of treatment-seeking-dependent smokers was associated with significantly higher abstinence rates than the nicotine patch.
Little is known about the current training and barriers in resuscitation skills among practicing ward nurses. A convenience sample of 459 ward nurses, recruited from 11 academic teaching hospitals in Korea, were surveyed to assess current training and barriers to optimal resuscitation performance on the wards. The Perceived Barriers scale was developed, refined, and its psychometric properties were assessed. Approximately 36% of nurses had received simulation-based resuscitation skills training. Exploratory factor analysis identified four barriers accounting for 58.4% of the variance: insufficient training (37.7%), lack of competence (9.8%), lack of self-confidence (5.9%), and workload and tension (5.1%). Strategic planning and resuscitation skills training should be incorporated into staff development programs to reduce barriers to optimal resuscitation performance and cope with work demands for ward nurses.
We propose a method for creating groups against which outcomes of local pretest–posttest evaluations of evidence-based programs can be judged. This involves assessing pretest markers for new and previously conducted evaluations to identify groups that have high pretest similarity. A database of 802 prior local evaluations provided six summary measures for analysis. The proximity of all groups using these variables is calculated as standardized proximities having values between 0 and 1. Five methods for creating standardized proximities are demonstrated. The approach allows proximity limits to be adjusted to find sufficient numbers of synthetic comparators. Several index cases are examined to assess the numbers of groups available to serve as comparators. Results show that most local evaluations would have sufficient numbers of comparators available for estimating program effects. This method holds promise as a tool for local evaluations to estimate relative effectiveness.
It is well known that the differences-in-differences (DD) estimator is based on the assumption that in the absence of treatment, the average outcomes for the treated group and the control group will follow a common trend over time. That can be problematic, especially when the selection for the treatment is influenced by the individual’s unobserved behavior correlating with the medical utilization. The aim of this study was to develop an index for controlling a patient’s unobserved heterogeneous response to reform, in order to improve the comparability of treatment assignment. This study showed that a DD estimator of the reform effects can be decomposed into effects induced by moral hazard and by changes in health risk within the same treated/untreated group. This article also presented evidence that the constructed index of the price elasticity of the adjusted clinical group has good statistical properties for identifying the impact of reform.
Growing Up in New Zealand, a longitudinal study following nearly 7,000 children, has faced some unique challenges in identifying, enrolling, and retaining a large and diverse antenatal cohort. Identification of a study region with population demographics that enabled enrollment of an appropriately diverse sample was required as was intensive community and participant engagement in order to promote the study. Complementary methods used included direct engagement with prospective participants and the community and indirect engagement via media. Thus far, retention rates above 95% have been achieved by maintaining a multimethod approach that includes valuing participants and building trusting relationships, strong brand recognition, community engagement, maintenance of participant contact and location records, ensuring high-quality interactions between the participants and the study, pretesting measures and methods prior to the main cohort, and using participant feedback to inform the measures and methods used in future waves of data collection.
This study investigated the perceptions of short-term assignments of medical services among participating health care professionals dispatched from Taiwan to underdeveloped areas. Structured questionnaires were mailed to four groups of professionals (physicians, pharmacists, nurses, and public health personnel) who had participated in any of 88 medical missions dispatched to 24 allied nations. A total of 278 returns were valid for analysis. Among them, 222 respondents reported that they had participated in just one overseas medical mission (79.9%). The majority of physicians, pharmacists, and nurses listed humanitarianism as their foremost incentive for participation. In contrast, public health personnel most frequently reported that they had been assigned to the mission abroad. Pharmacists, nurses, and public health personnel most commonly stated that their top goal was health care; but physicians said that aiding Taiwan’s diplomatic relations was their main motive. While all groups generally recognized language proficiency and cultural awareness as important for conducting successful short-term medical aid missions (STMMs), many members of groups did not rate their own capabilities in those area as sufficient, especially pharmacists (p < .001). Orientation for participants and training for local health workers were seen as relatively insufficient. In conclusion, there are considerable differences in the thoughts about STMMs across four key groups of heath personnel. The findings can help inform efforts to integrate evidence into the deployment of STMMs.
It is now presumed that youth do not move directly from adolescence to adulthood, but rather pass through a transitional period, "emerging adulthood." The Revised Inventory of the Dimensions of Emerging Adulthood (IDEA-R) is a self-report instrument developed to examine the attributes of this period. "At-risk" youth appear to enter emerging adulthood developmental tasks at a slightly earlier age than general population youth. In the present study, a 21-item version of the IDEA was administered to a sample of 1676 "at-risk" continuation (alternative) high school students in Southern California. Principal component factor analysis with orthogonal rotation revealed three factors the authors labeled "Identity Exploration," "Experimentation/Possibilities," and "Independence." Overall, the measure demonstrated high internal consistency. Construct validity analyses indicated that the measure was correlated with demographics, risk behaviors, and psychological measures. The authors conclude that the IDEA-R is a useful instrument for measuring emerging adulthood in at-risk populations.