Study Objective: To characterize typical menstrual cycle characteristics in adolescents and determine how these differ with age at menarche or years since menarche (gynecologic age). Methods: We surveyed 13 to 18-year-old U.S. users of the Clue app (N = 6,486) and linked their responses to app-recorded cycle data (N=38,916 cycles). We analyzed cycle characteristics including cycle length, cycle variability, period length, experience of heavy flow, and dysmenorrhea in relation to gynecologic age and menarcheal age using mixed effects models. Results: With increasing gynecologic age, we observed dose-dependent associations of lower odds of cycle irregularity (defined as cycles that were highly variable, short, or long) and higher odds of reporting >= 1 day of heavy flow. Individuals <1 year post-menarche had lower odds of heavy flow (Odds Ratio (OR) = 0.3; 95% confidence interval (CI): 0.1, 0.6), and increased odds of having a highly variable cycle (OR = 2.6; 95% CI: 1.3, 5.2) or short cycles (OR = 5.0; 95% CI: 2.3, 11.0) compared to those who were 6+ years post-menarche. We also found associations with early and late age at menarche. Compared to menarcheal age of 14+ years, menarcheal age <= 10 years was associated with shorter cycle length (beta = -1.63 days; 95% CI: -2.51, -0.75), increased odds of dysmenorrhea (OR = 3.2; 95% CI: 2.3, 4.6), and decreased odds of high cycle variability (OR = 0.8; 95% CI: 0.6, 1.0). Conclusion: Cycle characteristics in adolescence are associated with menarcheal age and gynecologic age. Notably, highly variable cycles are common, especially among those with younger gynecologic age or older menarcheal age.

Machine learning (ML) models can fail in unexpected ways in the real world, but not all model failures are equal. With finite time and resources, ML practitioners are forced to prioritize their model debugging and improvement efforts. Through interviews with 13 ML practitioners at Apple, we found that practitioners construct small targeted test sets to estimate an error's nature, scope, and impact on users. We built on this insight in a case study with machine translation models, and developed Angler, an interactive visual analytics tool to help practitioners prioritize model improvements. In a user study with 7 machine translation experts, we used Angler to understand prioritization practices when the input space is infinite, and obtaining reliable signals of model quality is expensive. Our study revealed that participants could form more interesting and user-focused hypotheses for prioritization by analyzing quantitative summary statistics and qualitatively assessing data by reading sentences.

Emerging methods for participatory algorithm design have proposed collecting and aggregating individual stakeholders’ preferences to create algorithmic systems that account for those stakeholders’ values. Drawing on two years of research across two public school districts in the United States, we study how families and school districts use students’ preferences for schools to meet their goals in the context of algorithmic student assignment systems. We find that the design of the preference language, i.e. the structure in which participants must express their needs and goals to the decision-maker, shapes the opportunities for meaningful participation. We define three properties of preference languages – expressiveness, cost, and collectivism – and discuss how these factors shape who is able to participate, and the extent to which they are able to effectively communicate their needs to the decision-maker. Reflecting on these findings, we offer implications and paths forward for researchers and practitioners who are considering applying a preference-based model for participation in algorithmic decision making.

Machine translation models are embedded in larger user-facing systems. Although model evaluation has matured, evaluation at the systems level is still lacking. We review literature from both the translation studies and HCI com-munities about who uses machine translation and for what purposes. We emphasize an important difference in evaluating machine translation models versus the physical and cultural systems in which they are embedded. We then propose opportunities for improved measurement of user-facing translation systems. We pay particular attention to the need for design and evaluation to aid engendering trust and en-hancing user agency in future machine translation systems.

Many people use health tracking apps to keep track of their menstrual cycles, often in the hopes of better understanding their own health, and being able to identify when something might be wrong. However, it can be very difficult to interpret this data alone. Meanwhile, it is becoming increasingly common for researchers to use data from these apps to learn more about menstrual health. In this work we ask, how could more participatory approaches to conducting menstrual health research benefit both participants and researchers? We identify key challenges and risks of this kind of engagement, and propose four design guidelines for human-in-the-loop data analysis tools that engage participants with large-scale, quantitative menstrual health research: surface and elicit feedback on the data cleaning and analysis procedure; convey information relative to other users and clinical guidance; structure engagement to ensure valid analyses; and support social engagement and learning. For each of these, we highlight key open research questions relevant to the HILDA and visualization research communities. We plan to for evaluate and iterate on these guidelines through design workshops with users, researchers, and healthcare providers.

Language barriers between patients and clinicians contribute to disparities in quality of care. Machine Translation (MT) tools are widely used in healthcare settings, but even small mistranslations can have life-threatening consequences. We study how MT is currently used in medical settings through a qualitative interview study with 20 clinicians–physicians, surgeons, nurses, and midwives. We find that clinicians face challenges stemming from lack of time and resources, cultural barriers, and medical literacy rates, as well as accountability in cases of miscommunication. Clinicians have devised strategies to aid communication in the face of language barriers including back translation, non-verbal communication, and testing patient understanding. We propose design implications for machine translation systems including combining neural MT with pre-translated medical phrases, integrating translation support with multimodal communication, and providing interactive support for testing mutual understanding.

Machine translation (MT) is now widely and freely available, and has the potential to greatly improve cross-lingual communication. In order to use MT reliably and safely, end users must be able to assess the quality of system outputs and determine how much they can rely on them to guide their decisions and actions. However, it can be difficult for users to detect and recover from mistranslations due to limited language skills. In this work we collected 19 MT-mediated role-play conversations in housing and employment scenarios, and conducted in-depth interviews to understand how users identify and recover from translation errors. Participants communicated using four language pairs: English, and one of Spanish, Farsi, Igbo, or Tagalog. We conducted qualitative analysis to understand user challenges in light of limited system transparency, strategies for recovery, and the kinds of translation errors that proved more or less difficult for users to overcome. We found that users broadly lacked relevant and helpful information to guide their assessments of translation quality. Instances where a user erroneously thought they had understood a translation correctly were rare but held the potential for serious consequences in the real world. Finally, inaccurate and disfluent translations had social consequences for participants, because it was difficult to discern when a disfluent message was reflective of the other person's intentions, or an artifact of imperfect MT. We draw on theories of grounding and repair in communication to contextualize these findings, and propose design implications for explainable AI (XAI) researchers, MT researchers, as well as collaboration among them to support transparency and explainability in MT. These directions include handling typos and non-standard grammar common in interpersonal communication, making MT in interfaces more visible to help users evaluate errors, supporting collaborative repair of conversation breakdowns, and communicating model strengths and weaknesses to users.

Public school districts across the United States have implemented school choice systems that have the potential to improve underserved students' access to educational opportunities. However, research has shown that learning about and applying for schools can be extremely time-consuming and expensive, making it difficult for these systems to create more equitable access to resources in practice. A common factor surfaced in prior work is unequal access to information about the schools and enrollment process. In response, governments and non-profits have invested in providing more information about schools to parents, for instance, through detailed online dashboards. However, we know little about what information is actually useful for historically marginalized and underserved families. We conducted interviews with 10 low-income families and families of color to learn about the challenges they faced navigating an online school choice and enrollment system. We complement this data with four interviews with people who have supported families through the enrollment process in a wide range of roles, from school principal to non-profit staff ("parent advocates"). Our findings highlight the value of personalized support and trusting relationships to delivering relevant and helpful information. We contrast this against online information resources and dashboards, which tend to be impersonal, target a broad audience, and make strong assumptions about what parents should look for in a school without sensitivity to families' varying circumstances. We advocate for an assets-based design approach to information support in public school enrollment, which would ask how we can support the local, one-on-one support that community members already provide.

Machine Translation (MT) has the potential to help people overcome language barriers and is widely used in high-stakes scenarios, such as in hospitals. However, in order to use MT reliably and safely, users need to understand when to trust MT outputs and how to assess the quality of often imperfect translation results. In this paper, we discuss research directions to support users to calibrate trust in MT systems. We share findings from an empirical study in which we conducted semi-structured interviews with 20 clinicians to understand how they communicate with patients across language barriers, and if and how they use MT systems. Based on our findings, we advocate for empirical research on how MT systems are used in practice as an important first step to addressing the challenges in building appropriate trust between users and MT tools.

As people all over the world adopt machine translation (MT) to communicate across languages, there is increased need for affordances that aid users in understanding when to rely on automated translations. Identifying the information and interactions that will most help users meet their translation needs is an open area of research at the intersection of Human-Computer Interaction (HCI) and Natural Language Processing (NLP). This paper advances work in this area by drawing on a survey of users’ strategies in assessing translations. We identify three directions for the design of translation systems that support more reliable and effective use of machine translation: helping users craft good inputs, helping users understand translations, and expanding interactivity and adaptivity. We describe how these can be introduced in current MT systems and highlight open questions for HCI and NLP research.

Across the United States, a growing number of school districts are turning to matching algorithms to assign students to public schools. The designers of these algorithms aimed to promote values such as transparency, equity, and community in the process. However, school districts have encountered practical challenges in their deployment. In fact, San Francisco Unified School District voted to stop using and completely redesign their student assignment algorithm because it was frustrating for families and it was not promoting educational equity in practice. We analyze this system using a Value Sensitive Design approach and find that one reason values are not met in practice is that the system relies on modeling assumptions about families’ priorities, constraints, and goals that clash with the real world. These assumptions overlook the complex barriers to ideal participation that many families face, particularly because of socioeconomic inequalities. We argue that direct, ongoing engagement with stakeholders is central to aligning algorithmic values with real world conditions. In doing so we must broaden how we evaluate algorithms while recognizing the limitations of purely algorithmic solutions in addressing complex socio-political problems.

Emerging methods for participatory algorithm design have proposed collecting and aggregating individual stakeholder preferences to create algorithmic systems that account for those stakeholders' values. Using algorithmic student assignment as a case study, we argue that optimizing for individual preference satisfaction in the distribution of limited resources may actually inhibit progress towards social and distributive justice. Individual preferences can be a useful signal but should be expanded to support more expressive and inclusive forms of democratic participation.

Systems neuroscience studies involving in-vivo models often require realtime data processing. In these studies, many events must be monitored and processed quickly, including behavior of the subject (e.g., movement of a limb) or features of neural data (e.g., a neuron transmitting an action potential). Unfortunately, most realtime platforms are proprietary, require specific architectures, or are limited to low-level programming languages. Here we present a hardware-independent, open-source realtime computation platform that supports high-level programming. The resulting platform, LiCoRICE, can process on order 10e10 bits/sec of network data at 1 ms ticks with 18.2 µs jitter. It connects to various inputs and outputs (e.g., DIO, Ethernet, database logging, and analog line in/out) and minimizes reliance on custom device drivers by leveraging peripheral support via the Linux kernel. Its modular architecture supports model-based design for rapid prototyping with C and Python/Cython and can perform numerical operations via BLAS/LAPACK-optimized NumPy that is statically compiled via Numba’s pycc. LiCoRICE is not only suitable for systems neuroscience research, but also for applications requiring closed-loop realtime data processing from robotics and control systems to interactive applications and quantitative financial trading.