HCI research has explored AI as a design material, suggesting that designers can envision AI's design opportunities to improve UX. Recent research claimed that enterprise applications offer an opportunity for AI innovation at the user experience level. We conducted design workshops to explore the practices of experienced designers who work on cross-functional AI teams in the enterprise. We discussed how designers successfully work with and struggle with AI. Our findings revealed that designers can innovate at the system and service levels. We also discovered that making a case for an AI feature's return on investment is a barrier for designers when they propose AI concepts and ideas. Our discussions produced novel insights on designers' role on AI teams, and the boundary objects they used for collaborating with data scientists. We discuss the implications of these findings as opportunities for future research aiming to empower designers in working with data and AI.

column Share on UX designers pushing AI in the enterprise: a case for adaptive UIs Authors: John Zimmerman Carnegie Mellon University Carnegie Mellon UniversityView Profile , Changhoon Oh Carnegie Mellon University Carnegie Mellon UniversityView Profile , Nur Yildirim Carnegie Mellon University Carnegie Mellon UniversityView Profile , Alex Kass Accenture AccentureView Profile , Teresa Tung Accenture AccentureView Profile , Jodi Forlizzi Carnegie Mellon University Carnegie Mellon UniversityView Profile Authors Info & Claims InteractionsVolume 28Issue 1January - February 2021 pp 72–77https://doi.org/10.1145/3436954Online:23 December 2020Publication History 2citation3,308DownloadsMetricsTotal Citations2Total Downloads3,308Last 12 Months768Last 6 weeks67 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Hiring is one of the important challenges in the context of online labor marketplace. Unlike traditional hiring, where workers are hired either as a full time employee or as a contractor, hiring from online marketplaces are done for individual jobs of short duration. As these marketplaces are open for anyone, hiring becomes challenging due to the large number of freelancers applying for a posted job. Quite often, clients use ratings of the freelancers while hiring. However, we have observed that ratings are skewed towards higher values and do not provide valuable insights about freelancers' abilities to do a quality work. Therefore, we propose a multidimensional assessment framework which evaluates freelancers on several dimensions. The proposed framework, not only uses the current information about the freelancer, but also utilizes the past jobs he has performed. The framework is evaluated on the data collected from a popular online marketplace. Our analysis, performed on 7254 jobs and 96,271 applicants, shows that the assessment made by the proposed framework outperforms the baseline algorithm.

Diversity and inclusion are becoming major focus areas for most of the organizations these days. It has shown to bring several positive impacts to organizations such as highly engaged and motivated employees, improved team dynamics, sustainable team structure, and better work outcome. Pay and career opportunity are some of the indicators to assess diversity and inclusion practice. In this paper, we study gender pay gap in freelancing marketplace. Freelancing marketplaces are open to everyone and gender neutral; hence, it is expected that gender pay gap should not be much over these platforms as freelancers can quote or negotiate the price for the jobs as per their wish. However, our study, performed on 37,599 freelancers, reveals a gap in pay between male and female freelancers. Moreover, the study shows that female freelancers undervalue themselves compared to male freelancers having similar profile. Our study suggests that there is a need to address this large scale pay gap issue by guidance and counseling of female freelancers.

The emergence of online labor markets has concentrated a lot of attention on the prospect of using crowdsourcing for software development, with a potential to reduce costs, improve time-to-market, and access high-quality skills on demand. However, crowdsourcing of software development is still not widely adopted. A key barrier to adoption is a lack of confidence that a task will be completed on time with the required quality standards. While good managers can develop good, intuitive estimates of task completion when assigning work to their team members, they might lack similar intuition for individuals drawn from an online crowd. The phrase, "Post and Hope" is thus sometimes used when talking about the crowdsourcing of software-development tasks. The objective of this paper is to show the value of replacing the traditional, intuitive assessment of a team's capability with a quantitative assessment of the crowd, derived through analysis of historical performance on similar tasks. This analysis will serve to transform "Post and Hope" to "Post and Expect." We demonstrate this by analyzing data about tasks performed on two popular crowdsourcing platforms: Topcoder and Upwork. Analysis of historical data from these platforms indicates that the platforms indeed demonstrate some level of predictability in task completion. We have identified certain factors that consistently contribute to task completion on both the platforms. Our findings suggest that a data-driven decision processes can play an important role in successful adoption of crowdsourcing practice for software development.

Crowdsourcing is an emerging area which leverages collective intelligence of the crowd. Although crowdsourcing provides several benefits, it also brings uncertainty in any project execution. The uncertainty may be because of the time taken in on-boarding workers and lack of confidence in workers. The On-boarding time specifically becomes important when tasks are of short duration as it is not worth spending too much of time in on-boarding a worker for short task. In this paper, we empirically analyze 59,597 tasks data from Upwork, an online marketplace, to understand major factors that impact On-boarding time. We identified that certain factors, such as Feedback, Hiring rate, Total hours spent, Length of requirement etc., affect the On-boarding time. We applied two predictive models to predict the On-boarding time. Our study provides insights for researchers, organizations, etc. who are looking to accomplish their tasks through crowdsourcing and helps them to better understand factors which influence the On-boarding time.

In this paper we study the trustworthiness of the crowd for crowdsourced software development. Through the study of literature from various domains, we present the risks that impact the trustworthiness in an enterprise context. We survey known techniques to mitigate these risks. We also analyze key metrics from multiple years of empirical data of actual crowdsourced software development tasks from two leading vendors. We present the metrics around untrustworthy behavior and the performance of certain mitigation techniques. Our study and results can serve as guidelines for crowdsourced enterprise software development.

We present and evaluate a software development methodology that addresses key challenges for the application of Crowd sourcing to an enterprise application development. Our methodology presents a mechanism to systematically break the overall business application into small tasks such that the tasks can be completed independently and in parallel by the crowd. Our methodology supports automated testing and automatic integration. We evaluate our methodology by developing a web application through Crowd sourcing. The methodology was tested through two Crowd sourcing models: one through contests and the other through hiring freelancers. We present various metrics of the Crowd sourcing experiment and compare against the estimate for the traditional software development methodology.

We discuss our experiences with deploying a tool called the Requirements Analysis Tool (RAT), which automatically reviews requirements documents for clarity and content based issues using a variety of syntactic and semantic techniques. The tool has been deployed at over 500 large software projects. We provide an overview of our syntactic approach, which is based on enforcing restrictions on both sentence structure and vocabulary in a way that is carefully chosen to align with best practices. We discuss how RAT analyzes natural language text to find defects such as terminological inconsistencies and missing contextual information. Structured content from requirements is then represented as a semantic graph and RAT performs semantic analysis to help users perform interaction analysis. We present a number of case studies based on real world deployments of RAT which demonstrate number of improvements in the projects' requirements ranging from clearer sentence structure to more complete requirements.

This book is the third volume in a series that provides a hands-on perspective on the evolving theories associated with Roger Schank and his students. The primary focus of this volume is on constructing explanations. All of the chapters relate to the problem of building computer programs that can develop hypotheses about what might have caused an observed event. Because most researchers in natural language processing don't really want to work on inference, memory, and learning issues, most of their sample text fragments are chosen carefully to de-emphasize the need for non text-related reasoning. The ability to come up with hypotheses about what is really going on in a story is a hallmark of human intelligence. The biggest difference between truly intelligent readers and less intelligent ones is the extent to which the reader can go beyond merely understanding the explicit statements being communicated. Achieving a creative level of understanding means developing hypotheses about questions for which there may be no conclusively correct answer at all. The focus of the lab, during the period documented in this book, was to work on getting a computer program to do that. The volume adopts a case-based approach to the construction of explanations which suggests that the main steps in the process of explaining a given anomaly are as follows: * Retrieve an explanation that might be relevant to the anomaly. * Evaluate whether the retrieved explanation makes sense when applied to the current anomaly. * Adapt the explanation to produce a new variant that fits better if the retrieved explanation doesn't fit the anomaly perfectly.

This position paper provides an informal framework for thinking about Cross-Enterprise Collaboration (CEC), which is an increasingly crucial factor in driving business results. We argue that sustaining effective CEC generally requires careful consideration of technology to support both person-to-person and system-to-system interaction. We outline the main ingredients of CEC, identify common CEC patterns, and discuss some key technologies that enable collaboration across enterprise boundaries.