The research article, among other academic genres, has attracted researchers’ attention over the past decades. Although sections such as introduction, method, and results and discussion have been addressed in such studies, the mixed methods research (MMR) discussion genre remains underexplored. Thus, the present study aimed at exploring the generic organizational patterns in applied linguistics MMR articles. A qualitatively driven mixed methods approach was utilized to explore 38 MMR discussions. As a result, there emerged a five-move model for genre macro-organization and rhetorical move structure of the MMR discussions. The study concludes by presenting implications for writing effective MMR discussions.
The purpose of this article is to illustrate data transformation in a mixed methods research phenomenological study, investigating how athletes use concrete and abstract spontaneous imagery in and around competition. To achieve this, we combined the application of co-occurring codes and numerical transformation in a novel way. A thematic analysis of qualitative interviews with 12 elite athletes identified concrete imagery to focus on strategy generation, error correction, technique, and preparation, and abstract imagery to focus on desirability, symbolic and verbal representations, and regulation of affect, arousal, and mastery. Statistical analysis identified that subjective effectiveness of imagery significantly differed for sport type (reactive/static) and competition times. Researchers wishing to apply statistical analyses to qualitative data are encouraged to employ our methodology.
While health services researchers are using mixed methods research in large-scale studies with "big data" and incorporating data transformation for merging qualitative and quantitative data sets, these developments are not widely known to the broader mixed methods research community. Our purpose in this article is to introduce health services research to the broader mixed methods audience, to examine the potential for novel innovations in mixed methods research procedures, and to illustrate these points through a project on care management that used a convergent mixed methods design. In addition to traditional analytical procedures, we illustrate two qualitative to quantitative data transformation procedures, one using normalization process theory and a second, fuzzy set qualitative comparative analysis.
Despite the utility of applying mixed methods research to understand complex phenomenon, few studies have applied this approach to health policy and in Africa. This article illustrates the application of mixed methods research to inform health policy in Ghana with the intent of complementarity. Through an exploratory sequential mixed methods research design involving 24 focus group interviews and 417 household surveys, we developed criteria for identifying very poor households for health insurance premium exemptions in Ghana. The qualitative procedures identified communities’ concerns regarding being very poor: food insecurity, lack of seeds to sow, compromised access to education, financial insecurity, and status as unemployed widows with children. The survey findings illustrated the distribution and predictors of poverty in the Kassena-Nankana District. Based on these findings, the authors proposed a four-question survey for the Kassena-Nankana District Health Insurance Scheme to administer to determine extreme poverty. Based on these recommendations, the local government has a unique opportunity to increase the very poor’s access to and utilization of health care services.
Although investigators in health sciences are increasingly interested in using mixed methods, greater adoption requires funding opportunities. Funding allocations can signal recognition of a particular methodology and proficiency of investigators in using such methods. We examined trends in prevalence of mixed methods research funded by federal agencies in the United States. We reviewed abstracts to describe the content and terminology related to mixed methods. The number of self-identified mixed methods awards (n = 535; 2009-2014) increased modestly from previous trends (n = 226; 1997-2008). Abstracts were highly variable in their depth of description and use of mixed methods terminology. We propose five elements to address when developing a scientific abstract for a mixed methods study to facilitate clear description of study design and methods.
This article outlines the rationale for adopting a mixed methods approach within randomized controlled trials (RCTs) and explores challenges associated in doing so. Taking the example of the EASE Back feasibility and pilot study (Evaluating Acupuncture and Standard care for pregnant womEn with BACK pain: ISRCTN49955124), we detail why and how we operationalized a concurrent-sequential mixed methods research design. We present key findings from the exploratory research (focus groups and interviews) and explain how these were integrated with descriptive findings (a national survey of physical therapists) in order to inform and refine the design of the explanatory phase (the pilot RCT). We conclude with a discussion of lessons learned and implications for future research design and conduct.
This quantitatively driven sequential mixed methods study articulates the role of theory in mixed methods research and assesses the contribution of a mixed methods design to examining gender differences in men and women’s genetic testing decisions and the psychosocial factors impacting health-seeking behaviors post-testing. A quantitative online survey of BRCA-positive mutation men and women was followed by a qualitative component consisting of semi-structured telephone interviews regarding genetic testing and post-testing decision making. Findings reveal gender differences in testing motivations: women focus on health; men focus on their family’s needs. Gender differences also appeared in psychological states and surgical decisions, revealing women’s more negative psychosocial reactions to a positive BRCA test result and higher rate of selecting preventative surgery.
The Southeastern United States is home to one of the most culturally resistant arenas for undocumented immigrant students to pursue postsecondary education. Using a transformative mixed methods approach, we explore the multidimensional dynamics of contention that are present as campus administrators navigate the process of serving a group of students who are marginalized due to their unresolved immigration status. Our article contributes to the methodological literature by exemplifying how transformative mixed methods are powerful tools for understanding how the oppression of vulnerable populations is institutionalized in organizational settings.
The aim of this methodological description is to illustrate the application of mixed methods research in architectural design using a hybrid model consisting of a taxonomy development model and an embedded quasi-experimental model. The research employed qualitative and quantitative methods to explore the design of an architectural intervention and determine its effectiveness in improving environmental knowledge, attitudes, and behavior among primary school students. It also outlined strategies adopted to overcome the challenges related to the use of a hybrid design. Combining sequential and embedded mixed methods designs is a relatively new approach in architectural research, providing new insights that contribute to future mixed methods exploration in the field of design and planning.
Development of joint displays is a valued approach to merging qualitative and quantitative findings in mixed methods research. This study aimed to illustrate a case series mixed methods display and the utility of using mixed methods for broadening our understanding of domestic violence and abuse. Using a convergent design, 532 gay and bisexual men participated in a Health and Relationship Survey in a U.K. sexual health service and 19 in an interview. Quantitative and qualitative data were analyzed separately and integrated at the level of interpretation and reporting. There were inconsistencies in perceptions and reports of abuse. Men were supportive of selective enquiry for domestic violence and abuse by practitioners (62.6%; 95% confidence interval = 58.1% to 66.7%) while being mindful of contextual factors.
This article explores the application of prospective and retrospective elements of enquiry at different time points in longitudinal mixed methods research. It discusses how the method facilitates shifts in the dominance of quantitative and qualitative approaches and focuses attention on change and on interpretation. The article presents exploratory research designed to untangle different meanings of justice from the perspective of men and women who have been victims of violence and who then became involved in a criminal justice process. Both individual- and group-level analyses are used to show justice as a multidimensional phenomenon that unfolds and opens in context as well as over time. However, how best to report complex findings from longitudinal mixed methods research remains a challenge.
There were two purposes for this mixed methods study: to investigate (a) the realistic meaning of awareness and understanding as the underlying constructs of general knowledge of the learning process and (b) a procedure for data consolidation. The participants were 11th-grade high school and first-year university students. Integrated data collection and data transformation provided for positive but small correlations between awareness and understanding. A comparison of the created combined and integrated new data sets showed that the integrated data set provided for an expected statistically significant outcome, which was in line with the participants’ developmental difference. This study can contribute to the mixed methods research because it proposes a procedure for data consolidation and a new research design.
This article demonstrates how a digital environment offers new opportunities for transforming qualitative data into quantitative data in order to use data mining and information visualization for mixed methods research. The digital approach to mixed methods research is illustrated by a framework which combines qualitative methods of multimodal discourse analysis with quantitative methods of data mining and information visualization in a multilevel, contextual model that will result in an integrated, theoretically well-founded, and empirically evaluated technology for analyzing large data sets of multimodal texts. The framework is applicable to situations in which critical information needs to be extracted from geotagged public data: for example, in crisis informatics, where public reports of extreme events provide valuable data sources for disaster management.
The field of mixed methods research abounds with opportunities for creative development in terms of methodological advances and potential to contribute to important and complex societal problems. Inspired by issues that arose in the Mixed Methods International Research Association task force report on the future of mixed methods, this article contemplates its implications for challenges and opportunities for the mixed methods community for advancement in philosophy and methodology, innovative designs, technological advancements and big data, preparation of mixed methods researchers, and responsiveness to complex societal problems. We invite the mixed methods research community to participate in the conversation about these important, as yet unanswered, questions relevant for the future of mixed methods research and the world.
As an emergent care model combining conventional with complementary therapies, integrative interventions challenge evaluation, necessitating approaches capable of capturing complex, multilevel interactions. This article evaluates the effects of a Tibetan yoga intervention on lymphoma patients’ quality of life and cancer experience. Our methodological aims were to explore differences in therapeutic effect between treatment and control group using qualitative data, and explain equivocal findings between data sets. Use of both data transformation techniques—qualitizing and quantitizing—within an experimental embedded design comparing and integrating data between data sets and treatment groups allowed us to develop this innovative evaluative approach. Findings clarify convergence and divergence between data sets, explore participants’ complex cancer experience, and capture dimensions and intervention effects inaccessible through either method alone.
This article describes a mixed methods study of community-based participatory research (CBPR) partnership practices and the links between these practices and changes in health status and disparities outcomes. Directed by a CBPR conceptual model and grounded in indigenous-transformative theory, our nation-wide, cross-site study showcases the value of a mixed methods approach for better understanding the complexity of CBPR partnerships across diverse community and research contexts. The article then provides examples of how an iterative, integrated approach to our mixed methods analysis yielded enriched understandings of two key constructs of the model: trust and governance. Implications and lessons learned while using mixed methods to study CBPR are provided.
Onwuegbuzie and colleagues proposed the Instrument Development and Construct Validation (IDCV) process as a mixed methods framework for creating and validating measures. Examples applying IDCV are lacking. We provide an illustrative case integrating the Rasch model and cognitive interviews applied to the development of the Transformative Experience Questionnaire (TEQ). TEQ measures student engagement with the content they are learning. IDCV process yielded a high degree of utility for informing the psychometric properties of the TEQ. The quantitative data yielded supportive content, construct, and concurrent-related validity evidence using a sample of middle (n = 115) and high school students (n = 81). The qualitative data revealed item rewording was needed. Moderate to strong test–retest reliability was found for both forms.
Mixed methods research is increasingly common in emergency medical services, but methodological expertise among prehospital researchers has been found lacking. The purpose of this article is to describe unique challenges that the authors encountered in the application of mixed methods to a multisite, international study of prehospital language barriers. Lessons learned include the role of formative research in identifying cultural and organizational norms that affect researcher engagement with emergency medical service agencies, the necessity of developing approaches for member checking and assessing respondent validity, and the importance of promoting mixed methods as a rigorous methodology in international settings.
In several ways, randomized controlled trials represent a high standard of rigor in clinical biomedical research. Randomized controlled trials fail, however, to yield knowledge applicable to specific individuals. This article presents a methodological rationale for a mixed methods approach to n-of-1 clinical studies that attends to the preferences and concerns of individuals while attaining high standards of qualitative and quantitative rigor. An illustrative research design involving a hypothetical music therapy intervention for a quality of life outcome is examined in some detail and a concluding argument is made that the proposed mixed methods approach is especially appropriate for early-stage research interventions intended to generate explanatory hypotheses.
Developing psychometrically sound instruments can be difficult, especially if little is known about the constructs of interest. When constructs of interest are unclear, a mixed methods approach can be useful. Qualitative inquiry can be used to explore a construct’s meaning in a way that informs item writing and allows the strengths of one analysis method to compensate for the weaknesses of the other. Mixing method applications can be complex, however, there are few examples within the literature pertaining to the mix of interviews, Rasch modeling, and classical test theory. This article demonstrates how to mix qualitative inquiry with Rasch modeling (and classical test theory) in order to develop an instrument that measures a complex construct: patient trust.
The purpose of this study was to explore the role of nonspousal family support on mental health among older, church-going African American men. The mixed methods objective was to employ a design that used existing qualitative and quantitative data to explore the interpretive context within which social and cultural experiences occur. Qualitative data (n = 21) were used to build a conceptual model that was tested using quantitative data (n = 401). Confirmatory factor analysis indicated an inverse association between nonspousal family support and distress. The comparative fit index, Tucker–Lewis fit index, and root mean square error of approximation indicated good model fit. This study offers unique methodological approaches to using existing, complementary data sources to understand the health of African American men.
Mixed methods researchers share a commitment to knowing their sampling frames and minimizing discovery failure, especially when using surveys. Notwithstanding advances in sampling strategies, the geographic clustering of perceptions has not been fully considered for its relevance to sampling. This article examines the value of spatial autocorrelation analysis to guide sampling decisions. Spatial autocorrelation refers to the clustering of (dis)similar phenomena and signals the likely existence of perception subgroups. Through a spatial autocorrelation analysis of Dallas, Texas, the authors identify sampling frames for collecting data about perceptions of West Nile Virus eradication measures. They furnish some empirical confirmation of the geographic clustering of perceptions and argue for designs that identify perception clustering, which can affect qualitative sampling as well as advance the integration of quantitative and qualitative research.
We describe the Imitation Game, a new research method that collects both qualitative and quantitative data, and which can be used as a mixed methods procedure in many disciplines. Drawing on two projects, one investigating gender, the other sexuality, we show that the quantitative and qualitative aspects of the Game combine in four different ways, from more quantitative to more qualitative, involving increasing cultural understanding by the researchers. Crucially, deep cultural input is initially supplied by the players of the Game, who act as "proxy researchers," enabling data to be gathered quickly and efficiently. The analysis has its roots in sociology of scientific knowledge and studies of expertise and experience, hence the emphasis on the cultural foundations of methods and the nature of expertise.
Methodologists have urged researchers who use mixed methods to justify their methodological choices and provide greater clarity about the philosophical underpinnings and implications of their approaches. This article outlines the reasoning process undertaken in an endeavor to develop philosophical clarity for an applied, interdisciplinary, mixed methods research project about the communication of scientific evidence in the legal system. I used Greene’s domains of methodology for social inquiry as a framework for addressing reflexive questions about assumptions. Flowing from the domains of values and philosophies, the logic of inquiry was developed before the implications for the integration of findings and reporting of research were outlined. Early engagement in reflexive questioning provided a foundation for methodological refinement throughout the ongoing research journey.
This article advances the conditional incompatibility thesis, which when left unaddressed, poses challenges to the pragmatic maxim as a guiding framework for mixed methods research. The conditional incompatibility thesis stands in opposition to two claims, the first pertaining to the position that incompatibility can be avoided by adherence to a "whatever-works" maxim. Also questioned is the claim that quantitative and qualitative data are inherently incompatible. Arguing that there are conditions under which incompatibility occurs, we illustrate within the context of latent variable modeling how particular techniques, methods, and/or decisions fail to be philosophically neutral. Offered are methods through which researchers can be more mindful of, and thus transparent about, the influence of philosophical perspectives in their work.
By distinguishing between a survey and—a newly introduced term—a "probe," we recast the relationship between qualitative and quantitative approaches to social science. The difference turns on the uniformity of the phenomenon being examined. Uniformity is a fundamental idea underlying all scientific research but is rarely spoken about. The idea explains the different approaches to social science and has implications for monomethod and mixed method research designs. At the broadest level, it clarifies the different logics of quantitative and qualitative research and how they combine. For individual researcher, it provides a new rationale for mixing small and large samples in the same research project and, in the right circumstances, justifies efficient research designs based on small samples.
Health and social service practitioners have not readily incorporated research-based behavioral interventions in HIV prevention practice due to lack of awareness, lack of training, and challenges translating research findings into practice. Practitioners’ involvement in research is associated with their willingness to use research to guide practice. Likewise practitioners’ personal and organizational characteristics have been shown to be associated with use of research findings in practice. Factors associated with practitioners’ use of evidence-based interventions (EBIs) in HIV prevention, however are not well understood. While research involvement has been recommended to help practitioners overcome barriers to the use of EBIs, the types of research involvement that result in practitioners’ actual use of EBIs is not known. This is partly because most studies are either qualitative or quantitative and ignore the voices of practitioners, which can be best unearthed with qualitative research followed by survey research including larger numbers of participants. By using a mixed methods approach, this study fills this gap by showing associations between practitioners’ and organizational characteristics, specific areas of research involvement, and practitioners’ use of EBIs. The use of mixed methods was extremely helpful to integrate in-depth qualitative interview data from practitioners across 10 community-based agencies providing social and health services, with cross-sectional survey data from practitioners across 24 agencies in New York City. We used a sequential approach for data collection and a concurrent approach for data analysis, content analysis of in-depth interviews, and multivariate linear regression analysis. Practitioners who performed research tasks/procedures similar to their professional duties as well as their level of education, knowledge about EBIs, and their agency capacity were all positively associated with HIV evidence–based interventions. Training practitioners in research, engaging practitioners in research tasks that resemble their professional practices, and capacity building within organizations to foster academic-research partnerships could optimize practitioners’ use of EBIs.
This article argues that Bourdieu’s theory of practice offers a unified epistemological foundation for mixed methods research by emphasizing the reflexive and iterative nature of knowing, and the relational aspects of knowledge construction. The increasing presence of spatial data and tools in research fields that focus on sociospatial phenomena suggests that visual representation can facilitate the resituating of objective patterns within a subjective context of geographic and symbolic space. The article presents the foundations of the theory of practice as a unifying framework for mixed methods research that incorporates spatiality. The article then offers an example of empirical research that characterizes changing seed systems in West Africa using the theory of practice to guide mixed methods research and analysis.
The INTERMEDE Project brought together a number of research teams to study the interaction between a patient and their general practitioner, and how this can produce social inequalities in health. The ultimate objective of the project was to formalize a core of common findings by integrating qualitative and quantitative results. The methodology chosen for the integration was inspired by the Delphi participatory method. It involves several rounds of questions and feedback in writing between all members of project teams, in order to compare contradictory opinions and identify key concepts arising from the project. This interdisciplinary research has provided a more nuanced understanding of the mechanisms underlying physician–patient interaction by revealing the convergences of the various disciplinary approaches.
There has been much debate about the role of paradigms in mixed methods research. In the face of past calls for each researcher to operate within a single paradigm, it turns out that some researchers/practitioners find many positive features in more than one paradigm. This "multiparadigmatic perspective" used in mixed methods research needs a systematic framework for the practice of engaging in difference. Also, individuals committed to a single paradigm need a philosophical/theoretical framework for working in multiparadigmatic teams. This article provides such a framework. It is a metaparadigm, and it is labeled dialectical pluralism 2.0 or more simply dialectical pluralism. The word "pluralism" refers to the acceptance and expectancy of difference in virtually every realm of inquiry, including reality, and the age-old word "dialectical" refers to the operative process which is both dialectical and dialogical. Dialectical pluralism provides a way for researchers, practitioners, clients, policy makers, and other stakeholders to work together and produce new workable "wholes" while, concurrently, thriving on differences and intellectual tensions.
This article argues that methodological pluralism (MP) can benefit from a deeper and more systematic integration of reflexive processes. In particular, reflexivity can facilitate meaningful and problem-specific ways of combining methods across different disciplinary fields, types of expertise, and practices. To develop our argument, we distinguish between two dimensions of reflexivity: critical (questioning of values, assumptions, and sociopolitical context underlying research methodology) and transformative (investigating pathways for change by mobilizing social experimentation and learning). We discuss two cases of research on environmental valuation that mobilizes reflexivity. We conclude by emphasizing the specific role of critical and transformative reflexivity in guiding methodological choices.
This article introduces the realist survey, an extension of the realistic interview. In a realistic interview, the researcher adapts his or her own theory in response to comments by interviewees, explaining the extent to which the theory applies or does not apply to themselves. The realist survey expands the reach of the realistic interview by including application to generally accepted theories, and it enhances the generalizability of its outcomes through the use of quantitative survey data. As a result, this extended realist interview/survey is a tool that can be used by researchers in widely varying contexts. A realist survey on the effects of the often replicated technology acceptance model shows that the original model is not the best realist fit.
The purpose of this explanatory mixed methods study was to examine the perceived value of mixed methods research for graduate students. The quantitative phase was an experiment examining the effect of a passage’s methodology on students’ perceived value. Results indicated students scored the mixed methods passage as more valuable than those who scored the quantitative or qualitative passage. The qualitative phase involved focus groups to better understand students’ perceptions of the perceived value of mixed methods. Findings suggested graduate students view mixed methods passages as having rigorous methods, a newer history, and providing a deeper meaning of the phenomenon. This study adds to the literature base by revealing what value graduate students assign to quantitative, qualitative, and mixed methods research.
Permanent supportive housing (PSH) has been recognized by the U.S. federal government as the "clear solution" to chronic homelessness. Whether and how access to PSH affects HIV risk is unclear. This mixed methods pilot study uses a convergent parallel design in order to better understand social relationships and HIV risk during the transition from homelessness to PSH. Findings suggest that (a) Sexual activity and HIV risk behaviors increase with housing, (b) Social network size and composition appear to change as individuals transition into PSH, and (c) There is tension between moving forward and leaving behind the past once a person has housing. This study demonstrates that PSH constitutes a specific risk environment that has not been previously investigated.
Certain methodological techniques blend quantitative and qualitative elements, defying explicit characterization as either. Placing focus on the distinction between methods for data collection and methods for analysis ameliorates this confusion and identifies the occasionally ambiguous quantitative/qualitative designation of highly convergent methods as largely subject to researchers’ goals. We use free listing and prototype analysis, two methods which are variously characterized as either quantitative or qualitative, to investigate the conceptual relationship between love, romantic love, and sex. A methodological framework is presented in which open-ended questionnaires and structured interviews are analyzed comparatively to clarify the relationship between the three concepts. Results suggest that romantic love is conceptualized as a synthesis of prototypical "caring" love and noncrude attributes of sex.
This study used a mixed methods approach to examine the perceptions of fairness and satisfaction young offenders have toward their lawyers in the Hong Kong criminal justice system, drawing on procedural justice theory. A sample of 168 young offenders aged between 14 and 21 years old was surveyed. Regression analyses indicated that participants who had private lawyers were more satisfied compared to those who had duty lawyers. Two elements of procedural justice, participation, and trustworthiness, demonstrated the strongest influence on youths’ satisfaction with their lawyers. Supplementary interviews with 30 young defendants further explained the discrepancy, revealing that the limited amount of time spent by duty lawyers in meetings and their detached focus contributed to clients’ low perceptions of participation and trustworthiness.
Using a mixed methods approach, the researcher gathered a set of narrative responses from focus groups that supported the claim of underreporting campus discrimination on a survey. Multiple studies have shown that underrepresented minority students are likely to bond with same-ethnic peers in a racially tense campus climate. This mixed method is a follow-up investigation into, why, in a post-9/11 environment, Arab and Muslim American community college students demonstrated less variation in the level of perceived discrimination in relationship to the percentage of same-ethnic and/or same-faith campus friendship groups. Post hoc comparisons revealed that Arab and Muslim students exhibited significantly higher mean scores on a perceived discrimination scale than other ethnic groups.
A mixed methods approach was applied to examine the effects of a naturally occurring teamwork intervention supported with training. The first objective was to integrate qualitative process evaluation and quantitative effect evaluation to examine how and why the training influence intervention outcomes. The intervention (N = 328) was supplemented with four training conditions (no training, team member training, team leader training, and a combination of training types). The second objective was to examine whether different training conditions support team member training in isolation, but not in combination, led to positive outcomes. The integrated analysis of qualitative and quantitative data indicated that a number of contextual factors interacted with training experiences and outcomes to influence the success of team intervention.
This article reports the results of a nested analysis conducted to evaluate whether or not electoral donations are considered legal bribes. Introduced by Lieberman, nested analysis brings together the strengths of the regression analysis and the case study research by integrating large-N approaches (LNA) with small-N approaches (SNA). The nested analysis uses a sequential sampling model (QUANTITATIVE -> QUALITATIVE) and a nested sampling design (case selection "on/off the line"). Here, Lieberman’s original model was extended to deal with an apparent paradox that emerged from the analysis. This inquiry included a cross-national examination among 78 countries, denoted as LNA, followed by an intranational analysis conducted in Colombia, where an SNA survey with 302 respondents and an SNA case study were carried out.
The study of political and social violence and conflict has expanded in recent decades, concurrent with a rise in the use of mixed methods research (MMR) throughout the social sciences. This article examines how methods are best integrated in studies of violence and conflict, critically reviewing examples from previous prominent works and suggesting directions for future research. I explore the benefits of MMR for understanding structures, agency, and processes related to violence and conflict, and the opportunity MMR offers to influence a broader academic and policy audience. MMR can improve the accordance of theories and empirical studies with the complexities of social reality and enhance understanding of the causes, consequences, and potential remedies of violence and conflict.
While recent mixed methods publications advocate for researchers’ explicit discussion of their paradigmatic foundations, more guidance is needed regarding how these paradigms can be used. This article comparatively analyzes four major paradigmatic perspectives discussed in mixed methods literature: pragmatism, transformative-emancipation, dialectics, and critical realism. It offers a discussion of each perspective’s implications for mixed methods and how they can be used to influence research based on recent publications. While there are several similarities, such as emphasizing divergent results and allowing for researcher choice in methods, each perspective offers a unique set of advantages and disadvantages. Emphasizing how paradigms can be used then promotes more explicit engagement with them in future research.
In 2009, Mexico decriminalized small-scale drug possession, instituting drug treatment diversion in lieu of incarceration. To assess initial reform impact, our mixed methods study integrated a structured questionnaire with in-depth interviews assessing legal knowledge, police encounters, and risk behaviors among people who inject drugs (PWID) in Tijuana. Between 2010 and 2013, we recruited 737 adults; 32 participated in qualitative interviews. Only 11% reported being aware of the reform; virtually none experienced its operational components. Narratives underscored the law’s irrelevance to PWID; 699 (98%) saw police practice as generally inconsistent with formal law. Instead of treatment diversion, police encounters were associated with risk behaviors, including syringe sharing (odds ratio [OR] = 1.26; 95% confidence interval [CI] = 1.09-1.46) and polydrug use (OR = 2.11; 95% CI = 1.38-3.22). As drug policy reforms gain global momentum, ancillary structural interventions are needed to improve their public health benefit.
The study of social networks has included an array of mixed methods research in recent years. More often than not, however, research design is not discussed in any great depth. This article discusses procedural issues relating to employing both social network analysis and qualitative methods in a whole network study of cross-community youth leaders in a particularly segregated area of Belfast, Northern Ireland. A compound, sequential, three-step research design was used where distinct research phases were employed to account for the realities of researching an ambiguous network. This article will highlight design issues that tend to be overlooked in network research and offer practical responses that can be of benefit to researchers facing similar challenges.
In this article, we describe a research approach to undertaking a small area census to identify informal economy activity, using a mixture of quantitative and qualitative tools. The method focuses on enterprise activity. The approach enables the researcher to record a broader spectrum of informal micro-enterprises through identifying businesses in situ within an area of sufficient scale to broadly reflect area-level market conditions and business dynamics. The approach comprises an enterprise census, a survey of all identified micro-enterprises in key sectors, in-depth interviews, and participatory research techniques. The article reports on the application of this method in eight case sites, located in township settlements within five major cities in South Africa. The research identified 9,400 individual enterprises, entailing 10,220 primary and secondary activities, distributed within a population of 325,000 and comprising 97,000 households. The approach permits significant advances to our understanding of the spatial dynamics of the informal sector. The research data has enabled the researcher to make original contributions to understanding informal enterprise activities in grocery retailing, liquor trade, and traditional medicine sectors.
Qualitative comparative analysis (QCA) was developed over 25 years ago to bridge the qualitative and quantitative research gap. On searching PubMed and the Journal of Mixed Methods Research, this review identified 30 original research studies that used QCA. Perceptions that QCA is complex and provides few relative advantages over other methods may be limiting QCA adoption. Thus, to overcome these perceptions, this article demonstrates how to perform QCA using data from 15 institutions that implemented universal tumor screening programs to identify patients at high risk for hereditary colorectal cancer. In this example, QCA revealed a combination of conditions unique to effective universal tumor screening programs. Results informed additional research and provided a model for improving patient follow-through after a positive screen.
The purpose of this study was to develop codes of ethics for Iranian nurses. Researchers used mixed methods in four stages during 2009-2010 to develop a series of ethical codes for Iranian nurses in Isfahan, Iran. 55 ethical codes in 12 categories were extracted and listed as (1) believe in human dignity, (2) respect for patients’ privacy, (3) empathy for patient, (4) autonomy in decision making, (5) accurate and precise nursing care, (6) being conscience, (7) human relationship, (8) professional commitment, (9) promoting justice, (10) preventing harm and/or injury, (11) honesty and confidentiality, and (12) maintaining and promoting professional values. Study findings serve to the global nursing community on the Islamic view of ethics.
Although the mixing of quantitative and qualitative data is an essential component of mixed methods research, the process of integrating both types of data in meaningful ways can be challenging. The purpose of this article is to describe the use of data labels in mixed methods research as a technique for the integration of qualitative and quantitative data within equivalent-status mixed methods research designs. The music education study described utilized a combination of focus group, survey, and case study data to provide a fuller view of instrumental music teaching within the urban context. Using this study as an exemplar, the author discusses the application of data labels to points of convergence between qualitative and quantitative strands of mixed methods studies in order to achieve integration between both data sets.
The purpose of this mixed methods sequential explanatory study was to explore how specific dimensions of supervisor support (mentoring, coaching, social support, and task support) influence the transfer of learned knowledge and skills to the job. Quantitative data were collected from employees (N = 48) who develop curriculum at an educational organization and analyzed using multiple regression analysis; this was followed by in-depth interviews (n = 10) that focused on participants’ lived experiences of supervisor support, transfer motivation, and training transfer. Findings suggest that mentoring, coaching, social support, and task support directly and/or indirectly facilitate training transfer and provide greater insight into the underlying mechanisms that account for how and why supervisor support influences training transfer.
This community-based concurrent mixed methods study identified specific cases of discrimination and explored HIV stigma. In-depth interviews were conducted with individuals experiencing discrimination, and a quantitative survey was conducted in the immediate neighborhood. Focus group discussions were held with opinion makers. Survey respondents not attending school and believing in mosquito-borne HIV transmission had twice the odds of having fear of contracting HIV through noninvasive contact. In contrast, correct knowledge of mother-to-child HIV transmission was associated with judgmental attitudes, and knowledge of the role of condoms in HIV prevention was associated with shame. Qualitative investigations corroborated these findings, documented the denial of rights of people living with HIV, and revealed that discrimination resulting from fear of contagion directed against women had a moral dimension. Thematic issues and communication strategies were identified and translated into an intervention development.
We present outcomes from a mixed methods research project in agricultural sciences. An atypical methodology for sciences was developed as the only way to avoid embedded assumptions commonly seen in sustainability investigations. Eighty-three upper echelon participants from the wine grape industry participated in 14 focus groups in five countries: Australia, Chile, New Zealand, South Africa, and the United States. Quantitative measures were compared with results from qualitatively coded participant utterances using two content analysis software tools: Leximancer and NVivo. This article presents the strategies and method applied in this investigation to define sustainability. A consensual sustainability definition prior to the establishment of assessment systems is essential. The model developed seems to be viable for similar sustainability investigations of individual organizations.
This article uses a Kuhnian framework to explain the adoption of the transformative paradigm in pragmatically informed mixed methods research. We argue that pragmatism represents a model of "normal science" among many mixed methods researchers and that Kuhn’s concept of the scientific anomaly provides an instructive metaphor for understanding what we interpret to be a failure of pragmatic mixed methods researchers to adequately account for the axiological foundation for their work. The transformative paradigm thus can be read as providing pragmatic researchers with an axiological "fix" that sidesteps the larger question of how to establish a philosophically consistent means of specifying research value. We discuss this argument in light of pragmatist philosophy, the recent history of mixed methods research, and the future development of the field.
Indigenous communities raise concerns that they are overresearched and tired of research always asking the same questions and reproducing the same answers, thus pressuring researchers to open the discourse on mixed methods research so as to enable new debates and approaches to emerge. A postcolonial indigenous paradigm provides a theoretical framework that informed a mixed methods research approach to design and test the efficacy of a school-based risk-reduction intervention for 14- to 17-year-old adolescents in Botswana. Indigenous methods were used to collect cultural knowledge and to build relationships; these approaches allowed for the integration of the largely marginalized knowledge systems with dominant knowledge systems through a decolonization and indigenization research process.
Using both quantitative and qualitative analyses, we examine the role of a college access intervention in the enrollment and persistence outcomes of low-income students in Chile modeled partially after a Texas admissions program. We find that, although students from the program have a mean cumulative GPA significantly lower than that of their peers who entered college through the regular admissions system, most program students nevertheless meet the institution’s academic requirements. Qualitative evidence collected through student interviews presents some of the personal challenges and struggles that these students face when making the transition from high school to university while also highlighting the role of the program in student decisions to persist in college through the end of the first year.
This study focuses on the extent to which mixed methods have been used in studies of rural women’s struggle for land rights in Brazil based on an analysis of a database of theses managed by the Brazilian government. The analysis focused on three aspects of the studies: identification of methodology, representation of the rural women’s voices, and whether the studies highlighted the women’s role as active agents. Mixed methods guided the analysis strategy of the studies themselves. The results indicate that 74% of the studies use qualitative methods, 22% use mixed methods, and only 4% were exclusively quantitative. Consideration of the studies quality reveals that mixed methods potentially provide a better understanding of the complexity in the struggle for land by women.
This research, based on a case study of vulnerable children in Korea, used a mixed methods transformative approach to explore strategies to support and help disadvantaged children. The methodological approach includes three phases: a mixed methods contextual analysis, a qualitative dominant analysis based on Sen’s capability approach and critical communicative methodology to identify relevant value systems that support and impede resilience, and an action research approach to develop and implement the interventions. The adoption of mixed methods extends understandings of how to design, implement, and evaluate programs for vulnerable children.
Over a decade ago, researchers in Roma studies shifted their approach, from an exclusionary stance to a more communicative one. Despite major positive changes since then, researchers still do not adequately reflect the voices of the majority of the world’s Roma. In this article, we draw on a communicative mixed methods case study, conducted within the INCLUD-ED project in one of Spain’s most underprivileged neighborhoods, to show how it was possible to engage in intersubjective dialogue with grassroots Roma throughout the entire research process. Through such engagement, it is possible to develop new knowledge that can transform local conditions, as they shift from diagnosing social exclusion to identifying the approaches that work best to reduce it.
Deaf people, members of a complex global language minority, have been excluded from positions of power in the field of humanitarian aid and development assistance, as well as from playing a significant role in the conduct of research that affects their lives. Deaf people rarely lead or participate as researchers in studies about their own communities or about the assistance they receive. Transformative mixed methods research strategies focus on the development of culturally respectful relationships to enhance collaboration between members of dominant and marginalized communities in order to improve the accuracy and usefulness of the findings. In this article, the application of transformative mixed methods in the deaf community provides examples of how research reveals complex realities and how researchers employ mixed methods strategies to appropriately engage members of marginalized communities in the context of development assistance to support human rights.
We discuss the use of mixed methods research to further understanding of displaced Colombian gay and bisexual men and transwomen, a marginalized population at risk. Within the framework of communicative action, which calls for social change through egalitarian dialog, we describe how our multinational, interdisciplinary research team explored the subjective, objective, and social worlds of participants through life history interviews and surveys. We also describe the unique Colombian context, conscious efforts to establish egalitarian relationships among research team members, and efforts to disseminate and reflect on findings. Through discussion of our research process and results, we aim to demonstrate how mixed methods research can be utilized to facilitate noncoercive discourse and contribute to social change.
In 2007, a 4-year longitudinal study of all people admitted to the two New Zealand spinal units commenced. It aims to (a) explore interrelationship(s) of body, self, and society for people with spinal cord injury (SCI) and (b) investigate how entitlement to rehabilitation and compensation through New Zealand’s Accident Compensation Corporation affects socioeconomic and health outcomes. Working within the emancipatory-transformative research paradigm, an underlying aim was to build research capacity among people with SCI by recruiting them onto the research team (in addition to the principal investigator who has a SCI). The primary focus of this article is not on the findings of the study but on building research capacity in the SCI population of New Zealand, which emerged from our commitment to the study being ethical and emancipatory. This will involve a consideration of what constitutes ethical disability research, the research paradigm chosen, assembling the research team, and the flexibility provided by the use of mixed methods to achieve these ends.
Appreciative inquiry is a strategy which takes a positive approach to organizational development. It aims to identify good practice, design effective development plans, and ensure implementation. This article examines the potentials and limitations of using the appreciative inquiry in a mixed methods research design for developing school effectiveness and care in three secondary schools in Lebanon.
While most research on cultural models isolates qualitative and quantitative methodologies, this study applies mixed techniques to integrate qualitative and quantitative data to analyze cultural dimensionality. In a sequential research design, the data obtained in qualitative interviews, free-listing, and focus groups are validated by using triangulation method and further formalized to create survey items. Correspondence analysis of multi-item scales is then employed to explore the Swedish cultural model of prosocial cooperation and to represent the dynamic relationship among its facets and different categories of informants. Besides its contents-rich structured output, the advantages of this technique include increased reliability of qualitative findings. The study illustrates applicability of mixed methods to research the patterns of stability and change in cultural models.
The purpose of this study was to develop, test, and validate a questionnaire for assessing social workers’ exposure to client violence, which we call the Client Violence Questionnaire (CVQ). Following established procedures for scale development, four distinct stages of research were conducted, combining qualitative and quantitative methods. The contribution of this study is threefold—methodological, conceptual, and practical. The instrument offers practitioners and academic researchers the opportunity to apply the scale both for internal monitoring and knowledge sharing as well as further research. The development process of the CVQ scale demonstrates how the qualitative method can serve as a distinct research stage and at the same time support and enhance the quantitative one, thus contributing to the validity and applicability of the instrument.
The objective of this article is to show how mixed methods can be used to develop a deeper understanding of the construct, absorptive capacity (AC). We used qualitative data from eight case studies to identify which types of AC knowledge firms have. Then we identified variables for measuring AC. We identified two types of AC: "technological" AC and "non-core" AC. Using a Karnaugh Map and a Fuzzy Logic Design, we developed a quantitative model to measure overall AC. By focusing on interpretive integration, our results from using both qualitative and quantitative methods yielded complementary rather than contradictory findings, and the goal of "knowing more" about AC was achieved.
This convergent parallel design mixed methods case study of four community colleges explores the relationship between organizational capacity and implementation of data-driven decision making (DDDM). The article also illustrates purposive sampling using replication logic for cross-case analysis and the strengths and weaknesses of quantitizing qualitative data in a mixed methods design. The findings suggest that community colleges’ organizational capacity for data-driven decision making is a function of human and social capital, but not physical capital. Methodologically, the data analyses suggest that researchers considering quantitizing qualitative data should consider an exploratory sequential design to better understand the phenomenon under study before reducing qualitative data to numbers.
While qualitative comparative analysis (QCA) has often been portrayed as an alternative to mainstream statistical methods, there is a growing debate on how the techniques can be combined. This article shows how, using a sequential design, the integration of a large-N survival analysis with a smaller-N QCA yielded new insights about the reasons for project delay in a multiproject organization. While the regression served to trace significant explanatory variables and their net effects, the QCA nuanced these findings by demonstrating how variables combine across distinct categories of cases. As a result, the differing logics underlying externally and internally generated delays were elucidated. Implications for our understanding of multiproject organizing and for mixed methods research are discussed.
The utilization of mixed methods in prehospital research is relatively new. Its use may enhance research findings, but it is not without its challenges. This study used online databases to systemically search for literature relating to the application of mixed methods in prehospital research, in order to understand the place of mixed methods research in the prehospital setting. The prehospital field appears to be embracing mixed methods as an approach to research due to its ability to address health care questions in complex, diverse environments. However, supplemental literature in this area is limited, with mixed methods expertise being found in other health science areas. Researchers should endeavor to continue to contribute to this area through high-quality, rigorous mixed methods studies.
Despite a heightened emphasis on building evaluation capacity and evaluation quality, there is a lack of tools available to identify high-quality evaluation. In the context of testing the Systems Evaluation Protocol (SEP), quantitative rubrics were designed and tested to assess the quality of evaluation plans and models. Interview data were also collected and analyzed using a priori codes. A mixed methods approach was used to synthesize quantitative and qualitative data and explore trends. Consistencies between data types were found for attitude and capacity, and disconnects were found for knowledge, cyberinfrastructure, time, and quality. This approach to data integration represents a novel way to tap the generative potential of divergence that arises when different methods produce contradictory results.
According to Guba and Lincoln, renunciation of mixed methods is characteristic of a moribund culture with the possible exception of human sciences. The question I raise here is whether a person can be considered a humanist only if he or she mixes methods, and I analyze Leibniz’s search for a universal system of justice to test this hypothesis. Considering the difference between a methodology and a mere approach, I also consider the issue of whether acquiring knowledge of any methodology always entails an approach. I stress the relationship between the concepts of "paradigm" and "culture" as a set of habits linked to mixed methods, and I conclude with an explanation of how mixing methods can be first sequential and, then, concurrent.
Organizational communication processes are complex, but all too often, researchers oversimplify the study of these processes by relying on a single method. Particularly when scholars and practitioners partner together to solve organizational problems, meaningful results require methodological flexibility and diversity. As an exemplar of the fit between mixed methods research and engaged scholarship, the present project used surveys, observations, and interviews to study the process of dissent in two organizations. Results emphasized that each organization’s mission was a key factor in how employees expressed disagreement and how others responded. Emergent research questions and sequential data collection revealed insights that would have been missed in single method studies and provided more complete insights for the nonacademic practitioners involved in the study.
The present study provides an example of the integrated data analysis technique of creating and interpreting merged profiles. By using this approach to merging data sources, we gained unique insight into goal disruption theory (GDT). Qualitative data suggest that military personnel harbor a wide range of desired end-states. Quantitative data support a component of GDT, suggesting that participants who have a strong need for desired end-state displayed greater purposive harm endurance. Interpretation of merged profiles revealed caveats to this relationship, in particular that not all end-states are equally motivating. Results illustrate the benefits of the integrated data analysis technique of creating and interpreting merged profiles. Utilization of the merged profiles illuminated relationships that would not have been exposed otherwise.
We explore development of elementary preservice teachers’ reflective practices as they solved problems encountered while teaching in a reading clinic. Written reflections (N = 175) were collected across 8 weeks from 23 preservice teachers and analyzed to investigate relationships among problem exploration, teaching adaptations, and problem resolution. In this sequential mixed methods design, exploratory qualitative analysis revealed co-occurrence of problem exploration, instructional adaptation, and problem resolution. Confirmatory quantitative analysis found significant relationships: preservice teachers who engaged in more problem exploration or description of instructional adaptations reported more problem resolutions the following week. Results support mixed method, longitudinal analyses to analyze preservice teachers’ written reflections, and use of written reflections with responsive feedback to develop preservice teachers’ agency for problem solving.
Increasing recognition of the role of social conditions in health has led to calls for methods that can be used to change social conditions. Popular education has demonstrated great promise as a methodology that can be used to address the underlying social and structural determinants of health. To date, most studies of popular education have used qualitative methods and case study designs, making them less compelling for decision makers. La Palabra es Salud (The Word Is Health) compared the relative effectiveness of popular and conventional education using a participatory, quasi-experimental, mixed methods design. Use of this model can enhance our understanding of popular education and raise its profile among researchers and practitioners in multiple disciplines, thus potentially extending its benefits.
This article examines the prevalence and characteristics of mixed methods research in the relatively new subdiscipline of sport management. A mixed methods study is undertaken to evaluate the epistemological/philosophical, methodological, and technical levels of mixed methods design in sport management research. The results indicate that mixed methods research is still rarely used, poorly legitimized and often weakly designed in this field. Our conclusions lead to the hypotheses that the more central a research field is, the higher the prevalence of mixed methods, and that mixed methods only slowly trickle down from central to more peripheral subdisciplines. Implications of the research findings for both mixed methods scholars and sport management researchers are discussed, and directions for future research are proposed.
This study investigates the prevalence and causes of inconsistency in translation memories (TM) using a sequential explanatory mixed methods design. The initial quantitative phase introduces a novel method and typology for measuring and categorizing inconsistencies in TM data. The data are the product of professional computer-aided translation of software documentation. In the follow-on qualitative phase, interviewees compare the quantitative results with their professional experience of TMs. Their confirmation of the quantitative results improves the validity of the study. The interview data also increase the utility of the research, suggesting possible causes and solutions for inconsistency. Results are presented interactively, followed by a short discussion of the findings and their consequences.
Increasingly, researchers are recognizing the benefits of expanding research designs that are rooted in one tradition (i.e., monomethod design) into a design that incorporates or interfaces with the other tradition. The flexibility of phenomenologically driven methods provides one such example. Indeed, phenomenological research methods work extremely well as a component of mixed methods research approaches. However, to date, a mixed methods version of phenomenological research has not been formally conceptualized. Thus, the purpose of this article is twofold. First, we provide a philosophical justification for using what we call mixed methods phenomenological research (MMPR). Second, we provide examples of MMPR in practice to underline a number of potential models for MMPR that can practically be used in future research.
This small n observational study used a concurrent mixed methods approach to address a void in the literature with regard to the qualitative meaningfulness of the data yielded by absolute magnitude estimation scaling (MES) used to rate subjective stimuli. We investigated whether respondents’ scales progressed from less to more and provided for meaningful discriminations, similarities and differences in how they defined their scales, and whether their definitions were congruent with how they used their scales. The respondents completed a think-aloud task while using MES and a follow-up interview regarding how they used MES and defined their scales. The respondents’ scales did not yield meaningful data. Implications for survey research and the use of mixed methods in rating scale construction are discussed.
Although research shows college preparatory programs’ effectiveness regarding academic achievement and college awareness, much less is known about whether programs affect college attendance and persistence. GEAR UP (Gaining Early Awareness and Readiness for Undergraduate Programs) and other college preparatory programs claim that this is a primary outcome, yet very few studies on persistence in higher education have been done. Through the use of an explanatory sequential mixed methods design, our study found that significantly more students (including low-socioeconomic background) from a GEAR UP college preparatory program showed greater college attendance and persistence outcomes over those who did not participate and also provides potential explanations why this may be the case. Mixing quantitative and qualitative data allowed for meta-inferences about the isolation of poverty, goal setting, resilience, and program impacts on minorities to be drawn.
Achievement goal theory is one of the most popular theories of achievement motivation. Techniques researchers have used to assess goals include standardized questionnaires and interviews. One curious finding is that participants whose self-report questionnaire responses strongly indicate they operate with a performance goal do not make performance goal responses in subsequent interviews. In this article, we consider the nature of this divergence using a mixed methods approach and discuss how a third technique, the Implicit Association Test, might help address divergent goal responses. More broadly, we suggest that implicit measures may offer an additional and/or alternative technique for assessing the prevalence of psychological constructs thought to be underpinned by processes involving social cognition.
Background: A systematic literature review was conducted on mixed methods area. Objectives: The overall aim was to explore how qualitative information from interviews has been analyzed using quantitative methods. Methods: A contemporary review was undertaken and based on a predefined protocol. The references were identified using inclusion and exclusion criteria and specific key terms in 11 search databases. Results: Evidence was synthesized from 14 references that included the methods used for quantifying qualitative information, analyzing it statistically and the rationale behind this. Gaps in the existing literature and recommendations for future research were identified. Conclusions: This review highlights the need for a new mixed method based on advanced statistical modeling method that will explore complex relationships arising from qualitative information.
Elementary school children between 9 and 12 years of age were interviewed on what they believed to be the causes of learning difficulties and were invited to take part in the analysis of the data. We achieved this with Trochim’s concept mapping approach that combines qualitative and quantitative data analyses. Study results indicated that children were more knowledgeable than expected. Although each participant gave relatively few ideas, they collectively generated a list of 42 unique statements, which they categorized into a meaningful structured conceptualization. Results showed that children were competent and reliable participants in the concept mapping process. Educational, research, and methodological implications are discussed.
Fields from political science to critical education policy studies have long explored power relations in policy processes, showing who influences policy agendas, policy creation, and policy implementation. Yet showing particular actors’ influence on specific points in a policy text remains a methodological challenge. This article presents a five-stage, embedded mixed methods design for establishing influence on educational policy moving from a policy text outward. I use an example analysis of Australia’s policy making on boys’ education—the report Boys: Getting it Right (2002)—to show how data transformation measures, both quantitizing and qualitizing, within a larger qualitative study helped identify influence. This mixed design, I argue, can be useful in other research contexts, with variations for data availability and researcher resources.
Partnerships between HIV researchers and service providers are essential for reducing the gap between research and practice. Community-Based Participatory Research principles guided this cross-sectional study, combining 40 in-depth interviews with surveys of 141 providers in 24 social service agencies in New York City. We generated the Provider-Researcher Partnership Model to account for provider- and agency-level factors’ influence on intentions to form partnerships with researchers. Providers preferred "balanced partnerships" in which researchers and providers allocated research tasks and procedures to reflect diverse knowledge/skill sets. An organizational culture that values research can help enhance providers’ intentions to partner. Providers’ intentions and priorities found in this study may encourage researchers to engage in and policy makers to fund collaborative research.
In spite of recent methodological developments related to quality assurance in mixed methods research, practical examples of how to implement quality criteria in designing and conducting sequential QUAN -> QUAL mixed methods studies to ensure the process is systematic and rigorous remain scarce. This article discusses a three-step procedure for securing the quality of the meta-inferences generated from sequential employment of quantitative and qualitative methods and offers several validation strategies specific to a sequential QUAN -> QUAL mixed methods design: applying a systematic process for selecting participants for qualitative follow-up, elaborating on unexpected quantitative results, and observing interaction between qualitative and quantitative study strands. The discussed procedures are illustrated using a mixed methods study of graduate student engagement in learning applied research methods online.
Differential item functioning (DIF) can undermine the validity of cross-lingual comparisons. While a lot of efficient statistics for detecting DIF are available, few general findings have been found to explain DIF results. The objective of the article was to study DIF sources by using a mixed method design. The design involves a quantitative phase in which DIF was analyzed followed by a qualitative phase conducting cognitive interviews. To illustrate the proposal, polytomous DIF was analyzed in the scales from the PISA (Programme for International Student Assessment) Student Questionnaire (Organisation for Economic Co-operation and Development). Evidence obtained allowed DIF to be connected with differences in the interpretation patterns of participants from the different linguistic groups. Finally, benefits of mixed methods design for analyzing equivalence in cross-lingual assessments are discussed.
Although combining methods is nothing new, more contributions about why and how to mix methods for validation purposes are needed. This article presents a case of validating the inferences drawn from the Participatory Evaluation Measurement Instrument, an instrument that purports to measure stakeholder participation in evaluation. Although the process was intended to be almost exclusively quantitative, one of its components unexpectedly turned into a mixed methods study. This, in turn, spurred on a cycle of instrument revision and further quantitative validation. Whereas the validation evidence is modest and tentative, it suggests that the revised version of the Participatory Evaluation Measurement Instrument offers a better fit with the respondents’ opinions regarding the participation level of selected evaluation cases. The article concludes with a brief discussion on the added value of mixed methods for validation purposes.
In mixed methods research (MMR), integrating the quantitative and the qualitative components of a study is assumed to result in additional knowledge (or "yield"). This narrative review examines the extent to which MMR is used in the field of childhood trauma and provides directions for improving mixed methods studies in this field. A systematic literature search resulted in 13 studies that were achieving four different research objectives: (a) measures and meaning, (b) intervention evaluation, (c) theory building, and (d) measurement instrument development and validation. Although some studies produced yield by integrating the components, there is room for improvement and better use of MMR’s potential. We conclude by presenting reommendations for improving the application and dissemination of MMR in childhood trauma.
In this study, we used a mixed methods approach to explore the determinants of relationship patterns and risky sex among homeless men living in downtown Los Angeles. This involved analysis of qualitative interviews focused on gender ideology and sexual events (n = 30) as well as structured interviews (n = 305) focused on homeless men’s sexual partners, sexual behaviors, and social networks. We found that men valued committed relationships but were frustrated by their lack of access to female partners (accentuated by the stigma attached to homeless males) and also by structural and logistical barriers to relationships. Some men reported monogamous relationships; such relationships were less likely with homeless or substance-using partners and more likely with partners central to men’s social networks.
In several subdomains of the social, behavioral, health, and human sciences, research questions are increasingly answered through mixed methods studies, combining qualitative and quantitative evidence and research elements. Accordingly, the importance of including those primary mixed methods research articles in systematic reviews grows. It is generally known that the critical appraisal of articles is an essential step in the development of a methodologically sound review. This article provides an overview of the available critical appraisal frameworks developed to evaluate primary mixed methods research articles. In addition, we critically compare and evaluate these frameworks and the quality criteria they include.