While the Internet of Things (IoT) demands robust protocols, verifying network behavior at scale presents a significant challenge. Physical hardware deployments are cost-prohibitive, time-consuming, slightly-adaptable and difficult to manage. To this purpose, several simulation environments have been developed, which take into consideration different aspect of the network i.e. physical layer, communication protocol, environmental conditions, etc. In this work, we are interested in investigate the behaviour of network based on OpenThread. Even if the existing official simulation environments OpenThread Network Simulator (OTNS) allows to simulate network based on the last release of the protocol, it lacks of native mechanisms for granular statistical data collection-specifically concerning per-packet energy consumption and automated experimentation. To bridge these operational gaps, this work introduces a software toolkit designed to automate experimental execution and quantitative metric extraction. The technical implementation utilizes containerization to ensure cross-platform reproducibility and environment stability, a custom network management abstraction to achieve easy control over network traffic generation, and the consolidation of fragmented resources into a unified technical reference to lower the barrier to entry for researchers. The toolkit's efficacy was validated by comparing two divergent topologies, demonstrating the ability to measure network reliability, temporal dynamics, and correlation between link layer RSSI and network layer stability.

Wireless sensor networks (WSNs) offer an efficient solution for monitoring agricultural conditions and have the potential to support future long-term space missions and extraterrestrial colonization. In this work, we present the design of a monitoring platform that integrates OpenThread, a generalpurpose Internet of Things (IoT) protocol, with low-cost physical and chemical sensors. Additionally, a customized meteorological shield is developed to minimize environmental interference with sensor measurements. The proposed platform is designed to monitor key environmental parameters, including temperature, relative humidity, and common volatile compounds, in real-world conditions. A particular focus is given to optimizing the shield design to ensure accurate sensor readings while providing adequate protection.

In microtask crowdsourcing, Human Intelligence Tasks (HITs) are commonly allocated on a first-come, first-served basis: they are published on the platform and the fastest workers select the most attractive ones first. This step has not received much attention from the scientific community yet, though it can become particularly taxing for workers when they compete to secure the most sought-after tasks. There are many strategies to ensure one's access to tasks and their effects on the labour process as a whole are not well understood. For instance, platforms with a sizeable task reservation queue allow workers to gain preferential access to a large number of tasks, which in turn may cause a shortage of work for the rest of the crowd. For the requesters, this means lower rates of completion and a lack of worker diversity. We explore workers' strategies for accessing and reserving tasks using monitoring techniques from both client and server sides. We investigate how these strategies affect task execution, in terms of availability, completion time, and answer quality, by deploying 1000 image annotation HITs in Amazon Mechanical Turk including objective and subjective tasks. We observe that workers who do not use automated catching techniques tend to have higher annotation quality, are more focused, spend more effort on text editing, and provide a higher diversity of output than workers using such tools. This study also reveals the tragedy of the commons effect among platform members due to the use of catching techniques: workers using automated catching techniques reserve and complete a substantially higher portion of the available tasks, but the over-reservation of HITs restricts all workers of reservation opportunities, and compromise their own future labour capacity as well. We observe a high inefficiency in job completions, as the majority of the times a task is being reserved by a worker, it will not get actually performed and will need to be republished for further allocation. Finally, we propose solutions to mitigate the negative effects of these phenomena on the labour process.

Electronic Patient-Reported Outcome Measures (ePROMs) are widely used in telemonitoring for efficient patient status assessment, without the need of clinician intervention. However, the quality of collected data is often compromised by issues such as patient comprehension, response fatigue, and varying levels of digital proficiency. We’re looking at ways to overcome these challenges by using crowdsourcing techniques to evaluate the quality and reliability of patient responses. Our idea is to enhance PROMs by adding elements inspired by crowdsourcing, such as asking repeated or counterfactual questions, gauging self-reported confidence, and checking internal response consistency. This is all about improving how we understand and assess patient feedback without changing the original questionnaire’s structure. To show how this can work in practice, we’ve designed a proposal using the SNOT-22 questionnaire, which is used in otolaryngology to manage chronic upper airway diseases. Our plan involves adding extra questions and using metadata analysis to indirectly but effectively enhance response quality.

Microtask crowdsourcing platforms enable rapid, large-scale completion of simple tasks by a globally distributed workforce. This study investigates the factors influencing knowledge-sharing behaviours among crowdworkers, integrating the Unified Theory of Acceptance and Use of Technology (UTAUT) with Social Exchange Theory (SET) to provide a comprehensive understanding of these dynamics. Using Structural Equation Modelling (SEM) to analyse survey data from 413 crowdworkers, the study identifies key drivers such as Performance Expectancy (PE), Effort Expectancy (EE), and Rewards, which significantly impact both Knowledge-sharing Intention (KSI) and Behaviour (KSB). Our findings highlight the importance of user-friendly and accessible digital tools in promoting active knowledge-sharing within online communities. Effort Expectancy directly influences Knowledge-sharing Behaviour, highlighting the importance of usability in sustaining platform adoption. This research confirms the robustness of the UTAUT model and extends it with social exchange elements to offer new insights into human aspects of information systems.

Wireless sensor networks can be a low-cost and efficient solution for monitoring environmental parameters in agriculture. In this work, we analyze the potentialities of using a network based on a general-purpose Internet of Things protocol such as OpenThread and on low-cost general-purpose physical and chemical sensors. This article aims to test and verify the platform functionalities when monitoring environmental parameters such as temperature, relative humidity, and visible and infrared irradiance in a real environment. We designed wireless sensor nodes and a data collection and visualization dashboard. We tested the sensors' response under controlled settings, conducted connectivity and network topology tests, validated the system functionality via in-field measurements, and discussed the main issues and potential capabilities of these sensors and network architecture.

This study explores the correlation between residents’ subjective assessments of urban neighbourhoods, obtained through virtual walkthroughs, and objective measures of deprivation. Our study was set within a specific city in the United Kingdom, with neighbourhoods selected based on Indices of Multiple Deprivation (IMD). We invited residents in the UK through Prolific, a crowdsourcing platform. Employing complete case analysis, TF-IDF keyword extraction, the Kruskal–Wallis test, and Spearman’s rank-order correlation, our study examines the alignment between subjective assessments and existing deprivation measures (IMD). The results reveal a nuanced relationship, suggesting potential subjective biases influencing residents’ perceptions. Despite these complexities, the study highlights the value of virtual walkthroughs in offering a holistic overview of neighbourhoods. While acknowledging the limitations posed by subjective biases, we argue that virtual walkthroughs provide insights into residents’ experiences that potentially complement traditional objective measures of deprivation. By capturing the intricacies of residents’ perceptions, virtual walkthroughs contribute to a more comprehensive understanding of neighbourhood deprivation. This research informs future endeavours to integrate subjective assessments with objective measures for robust neighbourhood evaluations.

PurposeUnderlying much recent development in data science and artificial intelligence (AI) is a dependence on the labour of precarious crowdworkers via platforms such as Amazon Mechanical Turk. These platforms have been widely critiqued for their exploitative labour relations, and over recent years, there have been various efforts by academic researchers to develop interventions aimed at improving labour conditions. The aim of this paper is to explore US-based crowdworkers' views on two proposed interventions: a browser plugin that detects automated quality control "Gold Question" (GQ) checks and a proposal for a crowdworker co-operative. Design/methodology/approachThe authors interviewed 20 US-based crowdworkers and undertook a thematic analysis of collected data. FindingsThe findings indicate that US-based crowdworkers tend to have negative and mixed feelings about the GQ detector, but were more enthusiastic about the crowdworker co-operative. Originality/valueDrawing on theories of precarious labour, this study suggests an explanation for the findings based on US-based workers' objective and subjective experiences of precarity. The authors argue that for US-based crowdworkers "constructive" interventions such as a crowdworker co-operative have more potential to improve labour conditions.

Crowdworkers on platforms like Amazon Mechanical Turk face growing competition as a result of the global excess supply of digital labour. As a result, many crowdworkers turn to automated scripts, which help them locate better tasks faster and to boost their earnings. However, to date, it is not clear whether and to what extent the use of such scripts influence the opportunities for those crowdworkers who do not use them. This an important aspect that warrants further exploration because it can have negative implications for the health of crowdwork platforms. In this study, we use Discrete Event Simulation to identify and quantify the unintended consequences of the excessive use of automated scripts. Our findings show that, while the use of scripts allows some crowdworkers to identify and accept far more tasks than others, in the long run, this behaviour results in their competence persistence and reputational persistence and progressively to detrimental impacts for those workers who do not use scripts, and who may ultimately be forced to exit the platform. As a result, automated scripts have negative consequences, whereby their excessive use leads to a tragedy of the commons for all platform stakeholders, including the crowdworkers, the job requesters and the platform itself.

Micro-task crowdsourcing marketplaces like Figure Eight (F8) connect a large pool of workers to employers through a single online platform, by aggregating multiple crowdsourcing platforms (channels) under a unique system. This paper investigates the F8 channels’ demographic distribution and reward schemes by analysing more than 53k crowdsourcing tasks over four years, collecting survey data and scraping marketplace metadata. We reveal an heterogeneous per-channel demographic distribution, and an opaque channel commission scheme, that varies over time and is not communicated to the employer when launching a task: workers often will receive a smaller payment than expected by the employer. In addition, the impact of channel commission schemes on the relationship between requesters and crowdworkers is explored. These observations uncover important issues on ethics, reliability and transparency of crowdsourced experiment when using this kind of marketplaces, especially for academic research.

Wireless sensor networks can be an low-cost and efficient solution for monitoring environmental parameters in agriculture. In this work, we analyze the potentialities of using a network based on a general purpose internet of things protocol as OpenThread. The aim of this paper is to test and verify the platform functionalities during the monitoring of environmental parameters such as: temperature, relative humidity, visible and infrared irradiance in a real environment.

AI and robots have the potential to transform Higher Education (HE) but pose many ethical and implementation challenges. To ensure the widest debate about our choices for the future of HE with these technologies, engaging ways to present the issues are needed and this article is part of an exploration of the potential of fictional narratives to do so. Specifically, the purpose of this article is to enrich understanding of quality in such fiction-based research, through analysing responses to a collection of fictions from a group of expert readers. A starting point was synthesising previous attempts to articulate notions of quality. The discussions with the readers suggest that the key qualities were substantive contribution, credibility, resonance, ambiguity and aesthetics; rich rigour and sincerity need also to be considered. Fiction has a place in educational research because it enables one to imagine vividly different possibilities, presents issues in an open-ended way, and is engaging.

AbstractCrowdwork platforms such as Amazon Mechanical Turk (AMT) are a crucial infrastructural component of our global data assemblage. Through these platforms, low-paid crowdworkers perform the vital labour of manually labelling large-scale and complex datasets, labels that are needed to train machine learning and AI models (Tubaro et al., Big Data & Society, 7(1), 2020) and which enable the functioning of much digital technology, from niche applications to global platforms such as Google, Amazon and Facebook.In this chapter, we reflect on how a ‘design justice’ approach might be valuable to build on insights gained from a series of exploratory discussions we have engaged in with US-based crowdworkers about how a crowdworker co-operative might work in practice, and begin to sketch out a potential software architecture that could form the basis of future participative approaches to the design and development of a crowdworker co-operative.We begin by describing and reflecting on our own evolving methodology and how it fits with the ‘design justice’ lens we propose for future work. Following this, we present findings from our discussions with crowdworkers about how a crowdwork co-operative might work in practice, including what values workers would like to see embedded in the design. We then finish with the outline of a prototype software architecture for a crowdworker co-operative that could be used as a starting point in future design work in collaboration with crowdworkers.

Due to the increasing amount of information shared online every day, the need for sound and reliable ways of distinguishing between trustworthy and non-trustworthy information is as present as ever. One technique for performing fact-checking at scale is to employ human intelligence in the form of crowd workers. Although earlier work has suggested that crowd workers can reliably identify misinformation, cognitive biases of crowd workers may reduce the quality of truthfulness judgments in this context. We performed a systematic exploratory analysis of publicly available crowdsourced data to identify a set of potential systematic biases that may occur when crowd workers perform fact-checking tasks. Following this exploratory study, we collected a novel data set of crowdsourced truthfulness judgments to validate our hypotheses. Our findings suggest that workers generally overestimate the truthfulness of statements and that different individual characteristics (i.e., their belief in science) and cognitive biases (i.e., the affect heuristic and overconfidence) can affect their annotations. Interestingly, we find that, depending on the general judgment tendencies of workers, their biases may sometimes lead to more accurate judgments.

A user accessing an online recommender system typically has two choices: either agree to be uniquely identified and in return receive a personalized and rich experience, or try to use the service anonymously but receive a degraded non‐personalized service. In this paper, we offer a third option to this “all or nothing” paradigm, namely use a web service with a public group identity, that we refer to as an OpenNym identity, which provides users with a degree of anonymity while still allowing useful personalization of the web service. Our approach can be implemented as a browser shim that is backward compatible with existing services and as an example, we demonstrate operation with the Movielens online service. We exploit the fact that users can often be clustered into groups having similar preferences and in this way, increased privacy need not come at the cost of degraded service. Indeed use of the OpenNym approach with Movielens improves personalization performance.

In Information Retrieval (IR) evaluation, preference judgments are collected by presenting to the assessors a pair of documents and asking them to select which of the two, if any, is the most relevant. This is an alternative to the classic relevance judgment approach, in which human assessors judge the relevance of a single document on a scale; such an alternative allows to make relative rather than absolute judgments of relevance. While preference judgments are easier for human assessors to perform, the number of possible document pairs to be judged is usually so high that it makes it unfeasible to judge them all. Thus, following a similar idea to pooling strategies for single document relevance judgments where the goal is to sample the most useful documents to be judged, in this work we focus on analyzing alternative ways to sample document pairs to judge, in order to maximize the value of a fixed number of preference judgments that can feasibly be collected. Such value is defined as how well we can evaluate IR systems given a budget, that is, a fixed number of human preference judgments that may be collected. By relying on several datasets featuring relevance judgments gathered by means of experts and crowdsourcing, we experimentally compare alternative strategies to select document pairs and show how different strategies lead to different IR evaluation result quality levels. Our results show that, by using the appropriate procedure, it is possible to achieve good IR evaluation results with a limited number of preference judgments, thus confirming the feasibility of using preference judgments to create IR evaluation collections.

Smallholder farmers provide the majority of food production in sub-Saharan Africa. They will be severely impacted by climate change, especially because they are dependent on rain-fed irrigation. We provide a summary of challenges and opportunities in designing smart farming infrastructure in this context. We observe that innovation in technology and knowledge production is necessary to increase the efficacy of water usage and land management. Such solutions must take into account the technological constraints and their regional variability to be able to provide sustainable and scalable solutions. Such solutions also need to embrace the notion of openness, encouraging collaborative endeavour and avoiding proprietary

The scientific literature peer review workflow is under strain because of the constant growth of submission volume. One response to this is to make initial screening of submissions less time intensive. Reducing screening and review time would save millions of working hours and potentially boost academic productivity. Many platforms have already started to use automated screening tools, to prevent plagiarism and failure to respect format requirements. Some tools even attempt to flag the quality of a study or summarise its content, to reduce reviewers’ load. The recent advances in artificial intelligence (AI) create the potential for (semi) automated peer review systems, where potentially low-quality or controversial studies could be flagged, and reviewer-document matching could be performed in an automated manner. However, there are ethical concerns, which arise from such approaches, particularly associated with bias and the extent to which AI systems may replicate bias. Our main goal in this study is to discuss the potential, pitfalls, and uncertainties of the use of AI to approximate or assist human decisions in the quality assurance and peer-review process associated with research outputs. We design an AI tool and train it with 3300 papers from three conferences, together with their reviews evaluations. We then test the ability of the AI in predicting the review score of a new, unobserved manuscript, only using its textual content. We show that such techniques can reveal correlations between the decision process and other quality proxy measures, uncovering potential biases of the review process. Finally, we discuss the opportunities, but also the potential unintended consequences of these techniques in terms of algorithmic bias and ethical concerns.

Crowdsourcing has been leveraged in various tasks and applications, primarily to gather information from human annotators in exchange for a monetary reward. The main challenge associated with crowdsourcing is the low quality of the results, which can stem from multiple reasons, including bias, error, and adversarial behavior. Researchers and practitioners can apply quality control methods to prevent and detect low-quality responses. For example, worker selection methods utilize qualifications and attention check questions before assigning a task. Similarly, task routing identifies the workers who can provide a more accurate response to a given task type using recommender system techniques. In practice, posterior quality control methods are the most common approach to deal with noisy labels once they are obtained. Such methods require task repetition, i.e., assigning the task to multiple crowd-workers, followed by an aggregation mechanism (aka truth inference) to select the most likely answer or request an additional label. A large number of techniques have been proposed for crowdsourcing aggregation covering several types of task types. This tutorial aims to present common and recent label aggregation techniques for multiple-choice questions, multi-class labels, ratings, pairwise comparison, and image/text annotation. We believe that the audience will benefit from the focus on this specific research area to learn about the best techniques to apply in their crowdsourcing projects.