publications

Educational technology is ubiquitous in schools, but there is insufficient information about the effectiveness of interventions that use educational technology to support literacy in elementary school. Given the importance of literacy development in these grades, we conducted a meta-analysis on the effects of educational technology interventions on reading comprehension and writing proficiency as well as the constrained and unconstrained skills that underlie these two literacy constructs. We also conducted analyses to determine whether effects of educational technology interventions differ according to characteristics of the educational interventions or participants in the studies in our corpus. Findings indicate positive effects of educational technology interventions on constrained skills, unconstrained skills, reading comprehension, and writing proficiency though effect sizes differed dramatically across outcomes. Findings also suggest that effects differ according to particular characteristics, providing direction for future research and development of educational technology interventions to support literacy in elementary school.

Network psychometrics leverages pairwise Markov random fields to depict conditional dependencies among a set of psychological variables as undirected edge-weighted graphs. Researchers often intend to compare such psychometric networks across subpopulations, and recent methodological advances provide invariance tests of differences in subpopulation networks. What remains missing, though, is an analogue to an effect size measure that quantifies differences in psychometric networks. We address this gap by complementing recent advances for investigating whether psychometric networks differ with an intuitive similarity measure quantifying the extent to which networks differ. To this end, we build on graph-theoretic approaches and propose a similarity measure based on the Frobenius norm of differences in psychometric networks’ weighted adjacency matrices. To assess this measure’s utility for quantifying differences between psychometric networks, we study how it captures differences in subpopulation network models implied by both latent variable models and Gaussian graphical models. We show that a wide array of network differences translates intuitively into the proposed measure, while the same does not hold true for customary correlation-based comparisons. In a simulation study on finite-sample behavior, we show that the proposed measure yields trustworthy results when population networks differ and sample sizes are sufficiently large, but fails to identify exact similarity when population networks are the same. From these results, we derive a strong recommendation to only use the measure as a complement to a significant test for network similarity. We illustrate potential insights from quantifying psychometric network similarities through cross-country comparisons of human values networks.

By analyzing 25,671 journals largely absent from common journal counts, as well as Web of Science and Scopus, this study demonstrates that scholarly communication is more of a global endeavor than is commonly credited. These journals, employing the open-source publishing platform Open Journal Systems (OJS), have published 5.8 million items; they are in 136 countries, with 79.9% in the Global South and 84.2% following the OA diamond model (charging neither reader nor author). A substantial proportion of journals operate in more than one language (48.3%), with research published in a total of 60 languages (led by English, Indonesian, Spanish, and Portuguese). The journals are distributed across the social sciences (45.9%), STEM (40.3%), and the humanities (13.8%). For all their geographic, linguistic, and disciplinary diversity, 1.2% are indexed in the Web of Science and 5.7% in Scopus. On the other hand, 1.0% are found in Cabell’s Predatory Reports, while 1.4% show up in Beall’s (2021) questionable list. This paper seeks to both contribute and historically situate the expanded scale and diversity of scholarly publishing in the hope that this recognition may assist humankind in taking full advantage of what is increasingly a global research enterprise.

Whether faculty research affects college student learning has long been the subject of debate. Previous studies use subjective measures of student learning; focus on correlation rather than causation; and typically focus on one college, thus lacking generalizability. Using unique, large-scale survey and assessment data that we collected from nationally representative samples of STEM undergraduates in China, India, and Russia, as well as a causal identification strategy that accounts for differential sorting of students to faculty, we present generalizable estimates of the effect of faculty research on objective, standardized measures of student learning. Results show that faculty research has a negative effect on student learning, suggesting direct trade-offs between the university’s dual mission of producing research and learning.

Universities contribute to economic growth and national competitiveness by equipping students with higher-order thinking and academic skills. Despite large investments in university science, technology, engineering and mathematics (STEM) education, little is known about how the skills of STEM undergraduates compare across countries and by institutional selectivity. Here, we provide direct evidence on these issues by collecting and analysing longitudinal data on tens of thousands of computer science and electrical engineering students in China, India, Russia and the United States. We find stark differences in skill levels and gains among countries and by institutional selectivity. Compared with the United States, students in China, India and Russia do not gain critical thinking skills over four years. Furthermore, while students in India and Russia gain academic skills during the first two years, students in China do not. These gaps in skill levels and gains provide insights into the global competitiveness of STEM university students across nations and institutional types.

This AERA Open special topic concerns the large emerging research area of education data science (EDS). In a narrow sense, EDS applies statistics and computational techniques to educational phenomena and questions. In a broader sense, it is an umbrella for a fleet of new computational techniques being used to identify new forms of data, measures, descriptives, predictions, and experiments in education. Not only are old research questions being analyzed in new ways but also new questions are emerging based on novel data and discoveries from EDS techniques. This overview defines the emerging field of education data science and discusses 12 articles that illustrate an AERA-angle on EDS. Our overview relates a variety of promises EDS poses for the field of education as well as the areas where EDS scholars could successfully focus going forward.

The debate over the science of reading has focused primarily on decoding (i.e., connecting letters and sounds to read words) and whether to use phonics to teach it. However, research on reading has included much more than decoding. Language comprehension, which allows readers to derive meaning from text, is an equally critical component of reading. Research has suggested that explicit instruction on the components of language comprehension—vocabulary and semantics, morphology, and syntax—can support language and reading comprehension. To inform the field on the science of reading as it pertains to language comprehension, in this meta-analysis of recent language comprehension interventions (n = 43) in U.S. elementary schools, the authors examined whether effects vary depending on participant and intervention characteristics. Findings suggest positive effects on custom measures of vocabulary, listening comprehension, and reading comprehension but not on standardized measures of these outcomes. Results also indicate positive effects for English learners and promise for multicomponent interventions and those that include technology. Much more research is needed on how best to support language comprehension for underserved populations (e.g., students from low-income backgrounds) and how interventions can be optimized to support generalizable language and literacy outcomes. Implications for policy and practice are discussed.

The rapid proliferation of information and communication technologies in economic, political, and social life has led to an increasing demand for computing professionals worldwide. It has also seen a corresponding expansion in undergraduate computer science (CS) programs. However, despite rapid increases in the quantity of CS graduates, little is known about their quality. In particular, little is known about the major-specific competencies, knowledge, and skills of CS graduates from different countries, types of programs, and backgrounds. Such evidence can ultimately inform employers seeking to hire qualified computing professionals within a globally competitive labor market, as well as policymakers and administrators seeking to improve the quality and diversity of CS programs in an international context.We assess and compare computer science skills among final-year computer science undergraduates (seniors) in four major economic and political powers that produce approximately half of the science, technology, engineering, and mathematics graduates in the world. We find that seniors in the United States substantially outperform seniors in China, India, and Russia by 0.76–0.88 SDs and score comparably with seniors in elite institutions in these countries. Seniors in elite institutions in the United States further outperform seniors in elite institutions in China, India, and Russia by ~0.85 SDs. The skills advantage of the United States is not because it has a large proportion of high-scoring international students. Finally, males score consistently but only moderately higher (0.16–0.41 SDs) than females within all four countries.

This book chapter presents a three-phase analysis of 521 journals that use the open source publishing platform Open Journal Systems (OJS) while appearing on Beall’s list of predatory publishers and journals and/or in Cabells Predatory Reports, both of which purport to identify journals that charge authors article processing fees (APC) to publish in the pretense of a peer-reviewed journal. In 2020, 25,671 journals were actively using OJS, with 81.3 percent in the Global South. As members of the Public Knowledge Project (PKP), which develops this freely available publishing platform, the authors feel a responsibility to explore what platform developers can do to address both the real problem of duplicitous journals and the over-ascription of the “predatory” label to publishers and journals. This chapter represents an assessment and intervention drawing on data collected from OJS installations. The first phase involved working with 14 of the 50 publishers (28.0 percent) and two of the 51 standalone journals (3.9 percent) on Beall’s list that use OJS to assess the extent of the evidence involved in assigning the label “predatory.” The second phase, devoted to assessing the number of journals using OJS labelled as “predatory” revealed that 521 (2.0 percent) of the journals using OJS are on Beall’s and/or Cabells’ lists. The two phases point to journals’ lack of editorial transparency that obscures which journals are guilty and which are misjudged. This leads to a third phase involving strategies for verifying and communicating journal adherence to scholarly standards by involving trade organizations, such ORCiD and Crossref. The goal is to provide a publicly accessible industry standard for more reliably assessing journal quality.

This dataset contains a summary of information about known public installations of Open Journal Systems, Open Monograph Press, and Open Preprint Systems.

By analyzing 25,671 journals largely absent from journal counts and indexes, this study demonstrates that scholarly communication is more of a global endeavor than is commonly credited. These journals, employing the open source publishing platform Open Journal Systems (OJS), have published 5.8 million items and represent 136 countries, with 79.9 percent publishing in the Global South and 84.2 percent following the OA diamond model (charging neither reader nor author). More than half (54.6 percent) of the journals operate in more than one language, while publishing research in 60 languages (led by English, Indonesian, Spanish, and Portuguese). The journals are distributed across the social sciences (45.9 percent), STEM (40.3 percent), and the humanities (13.8 percent). For all their geographic, linguistic, and disciplinary diversity, the Web of Science indexes 1.2 percent of the journals and Scopus 5.7 percent. On the other hand, Cabells Predatory Reports includes 1.0 percent of the journals, while Beall lists 1.4 percent of them as predatory. A recognition of the expanded scope and scale of scholarly publishing will help ensure that humankind takes full advantage of what is increasingly a global research enterprise.

We present a method to conduct automated surveys over WhatsApp, a globally popular messaging service. WhatsApp surveys offer potential advantages since they incur relatively low costs to respondents and researchers, are easy to use for respondents who are already familiar with WhatsApp chat features, and facilitate continued engagement with mobile populations as users can retain their WhatsApp number even if they change SIM cards and phone numbers. Yet, there is limited knowledge on how well WhatsApp surveys perform empirically. We test the WhatsApp method using two original panel surveys of refugees in Colombia and the U.S. and find satisfactory response rates and retention over a nine-month follow-up period in these mobile populations. We also conduct a mode experiment in Colombia comparing WhatsApp to short message service (SMS) and interactive voice response (IVR) surveys. We find that WhatsApp had a 12 and 27 percentage points higher response rate than IVR and SMS, respectively, resulting from higher initial engagement and lower break-off. We conclude by discussing advantages and limitations of the WhatsApp method and offer documentation and a public code repository to support researchers and practitioners in applying the method in other contexts.

This chapter presents a three-phase analysis of 521 journals that use the open source publishing platform Open Journal Systems (OJS) while appearing on Beall’s list of predatory publishers and journals and/or in Cabells Predatory Reports, both of which purport to identify journals that charge authors article processing fees (APC) to publish in the pretense of a peer-reviewed journal. In 2020, 25,671 journals were actively using OJS, with 81.3 percent in the Global South. As members of the Public Knowledge Project (PKP), which develops this freely available publishing platform, the authors feel a responsibility to explore what platform developers can do to address both the real problem of duplicitous journals and the over-ascription of the “predatory” label to publishers and journals. This chapter represents an assessment and intervention drawing on data collected from OJS installations. The first phase involved working with 14 of the 50 publishers (28.0 percent) and two of the 51 standalone journals (3.9 percent) on Beall’s list that use OJS to assess the extent of the evidence involved in assigning the label “predatory.” The second phase, devoted to assessing the number of journals using OJS labelled as “predatory” revealed that 521 (2.0 percent) of the journals using OJS are on Beall’s and/or Cabells’ lists. The two phases point to journals’ lack of editorial transparency that obscures which journals are guilty and which are misjudged. This leads to a third phase involving strategies for verifying and communicating journal adherence to scholarly standards by involving trade organizations, such ORCiD and Crossref. The goal is to provide a publicly accessible industry standard for more reliably assessing journal quality.

How exposure to diversity affects social segregation has long been the subject of debate. Previous studies focus on correlation rather than causation, use subjective measures of segregation, and typically lack a focus on institution norms enabling diverse interactions that in turn affect segregation. Using the context of the world’s largest affirmative action program and large-scale social network data collected longitudinally from a nationally representative sample of 20,239 engineering undergraduates in India, we employ a randomized experimental design to understand the effects of classmate diversity and faculty diversity on student segregation. We report three key findings – i) socially advantaged students are considerably more segregated than socially disadvantaged students, and these differences sustain throughout college, ii) exposing students to a more diverse composition of classmates reduces segregation for both socially advantaged and disadvantaged students, and iii) exposure to diverse faculty, on the other hand, does not affect segregation for either student group.