Security and performance are critical nonfunctional requirements for software systems. Thus, it is crucial to include verification activities during software development to identify defects related to such requirements, avoiding their occurrence after release. Software verification, including testing and reviews, encompasses a set of activities that have a purpose of analyzing the software searching for defects. Security and performance verification are activities that look at defects related to these specific quality attributes. Few empirical studies have been focused on how is the state of the practice in security and performance verification. This paper presents the results of a case study performed in the context of Brazilian organizations aiming to characterize security and performance verification practices. Additionally, it provides a set of conjectures indicating recommendations to improve security and performance verification activities.
Internet of Things (IoT) systems continuously collect a large amount of data from heterogeneous "smart objects" through standardised service interfaces. A key challenge is how to use these data and relevant event logs to construct continuously adapted usage profiles and apply them to enhance testing methods, i.e., prioritization of tests for the testing of continuous integration of an IoT system. In addition, these usage profiles provide relevance weightings to analyse architecture and behaviour of the system. Based on the analysis, testing methods can predict specific system locations that are susceptible to error, and therefore suggest where expanded runtime monitoring is necessary. Furthermore, IoT aims to connect billions of "smart devices" over the network. Testing even a small IoT system connecting a few dozens of smart devices would require a network of test Virtual Machines (VMs) possibly spreading across the Fog and the Cloud. In this paper we propose a framework for testing of each IoT layer in a separate VM environment, and discuss potential difficulties with optimal VM allocation.
Resilience is an important property of cyber defence systems operating in complex, adversarial environments. In previous work, we have developed the AWaRE approach for enabling cyber resilience via distributed, agent-oriented, constraint-based self-management. A key requirement and feature of AWaRE is the programmatic definition of constraint problems and their mapping to suitable coordination architectures in a way that supports run-time self-representation and self-organization. This can be satisfied via a domain-specific language (DSL); however, existing DSLs, e.g. for automated deployment and self-adaptivity, are not suitable for this purpose. In this paper, we present an expressive, Java-embedded DSL named ADSL that enables the design-time specification of system components, inter-component relationships and constraints, constraint problem decompositions, managing agent architectures, and sub-problem to agent mappings – and the run-time use of these specifications to achieve self-management in accordance with the models@run.time paradigm. Our initial validation of ADSL in the context of an enterprise scenario demonstrates the suitability of the language.
Automated machine translation software and services have become widely available and increasingly popular. Due to the complexity and flexibility of natural languages, automated testing and quality assessment of this type of software is extremely challenging, especially in the absence of a human oracle or a reference translation. Furthermore, even if a reference translation is available, some major evaluation metrics, such as BLEU, are not reliable on short sentences, the type of sentence now prevailing on the Internet. To alleviate these problems, we have been using a metamorphic testing technique to test machine translation services in a fully automatic way without the involvement of any human assessor or reference translation. This article reports on our progress, and presents some interesting preliminary experimental results that reveal quality issues of English-to-Chinese translations in two mainstream machine translation services: Google Translate and Microsoft Translator. These preliminary results demonstrate the usefulness and potential of metamorphic testing for applications in the natural language processing domain.
Program analyses typically need to identify a single entry method that triggers program executions. However, many modern applications do not have such a single entry point. Instead, they have to be deployed in a container that interacts with them through an application programming interface (API). We present a tool that generates a driver that supplies such an entry method for Java EE web applications. The generated driver simulates an arbitrary client interacting with the web application through the container. We describe the modular design of the driver generator, and existing support for features like Java server pages, servlets, filters and listeners. We report on our experience of using the driver generator for large real-world web applications.
Learning management systems (LMS) are widely used in many educational environments. Although usability is widely acknowledged as a desired quality attribute, it has not been deeply investigated in the context of LMS systems where it has to align with pedagogy. In this research we evaluate extensions to an LMS designed to support the implementation of constructive alignment for technical units using a Task Oriented Portfolio approach. The LMS application was extended with a range of open learner model-based visualisations to help staff and students monitor progress towards achieving unit intended learning outcomes throughout unit delivery. This paper reports on a case study of a formative usability testing of the prototype tool by using User eXperience (UX) metrics. The results unveil the potential usability issues and ways to improve the design of the prototype tool to better cater the actual target user needs.
Dynamic symbolic execution (DSE) is a non-standard execution mechanism which, loosely, executes a program symbolically and, simultaneously, on concrete input. DSE is attractive because of several uses in software engineering, including the generation of test data suites with large coverage relative to test suite size. However, DSE struggles in the face of execution path explosion, and is often unable to cover certain kinds of difficult-to-reach program points. Invariant inlay is a technique that aims to improve a DSE tool by interspersing code with invariants, generated automatically using off-the-shelf tools for static program analysis using abstract interpretation. To capitalise fully on a static analyzer, invariant inlay applies certain instrumentations and testability transformations to the program source. In this paper we outline the invariant inlay approach, and how we have evaluated the idea, in order to determine its usefulness for programs with complex control flow.
Continuous integration (CI) is a key practice where software developers integrate frequently via a shared repository to enable automated build, test, and release of software features. At the same time, digital economies are moving towards a service-oriented model with which software projects have become complex service-based systems orchestrated through service composition. While enabling CI in a centralised software development environment has been a common practice, little work has been done to optimally support CI in cross-organisational service-based software systems whose constituents are usually owned and managed by different organisations to only expose their interfaces. This paper presents a new decentralised interface-oriented CI model that is particularly optimised for supporting CI of cross-organisational service-based software systems. To demonstrate the viability and the effectiveness of the proposed approach, the paper further presents a proof-of-concept prototype that provides tool support, followed by an experimental evaluation that compares the prototype against an established technology stack for implementing CI using the service-oriented approach.
Developing software that is accepted by its intended users can be a challenge. It is not only important to facilitate peoples tasks or achieve their functional goals, but also such technology has to meet their values, satisfy their expectations, and encourage them to accept and trust the software. In this paper, we propose a novel psychologically-driven requirements engineering approach that captures and models stakeholders personal values, motivations and emotions. Using a grounded theory approach, we analyze information about stakeholders and the domain, model the results using psychologically-driven goal models, which facilitate the communication with stakeholders, and then we use this model as a basis for engineering concrete requirements. In conjunction with a team of experts on depression care, we evaluated our approach on an application for early diagnosis of depression. Results from our evaluation showed that the consideration of emotions, values, and motivations lead to improved outcomes, and were strongly endorsed by various stakeholders, including users, general practitioners, and psychologists.
App reviews are important as they contain valuable information for improving the quality of such software systems. To this end, most users providing app reviews expect a response; to the extent that prior research has shown that when developers respond to app reviews these responses improve app ratings and users' satisfaction. However, unfortunately, user reviews largely go unanswered for most apps due to the high prevalence of reviews. This challenge may be addressed by creating a system that automatically generates responses having learned from the responses already posted by developers for a given app. These generated responses may then be modified by developers if required. This work presents a system that recommends socially-acceptable responses based on principles adopted from three domains: information retrieval, social norms and userinterface design. We then evaluate the newly developed system against Google Play's de facto review response system, which requires developers to write responses manually. The evaluation of the two systems involved measuring participants' feedback on three aspects – usability, cognitive load and performance. The goal of the evaluation was to investigate whether users prefer the new system over the existing system. Our outcomes show that there were statistically significant differences between the two systems on all three aspects evaluated, with users preferring the newly proposed system over Google Play's response system. In particular, the proposed system has the potential to reduce the overall workload of developers considerably.
Open source software development projects typically use generic defect reporting forms, such as BugZilla and Jira, even for specialized kinds of defects such as usability issues. These issues typically tend to be reported as unstructured text, mix multiple defect attributes, lack precise cause, and overlook emotional impact on users. The poor quality of these defect reports impacts on the ability of developers to address them efficiently and effectively. In this paper, we describe a novel guided defect reporting method for capturing usability defects. We evaluate our approach using an expert judgment approach where experienced developers and researchers evaluate the clarity of usability defects reported via conventional defect reporting tools and our method. Our results show this is a promising approach at capturing more clear usability defect descriptions.
Pseudo-code written in natural language and mathematical expressions is a useful description of source code. Pseudocode aids programmers in understanding the code written in a programming language they are not familiar with. However, writing pseudo-code for each code statement is labour intensive. In this paper, we propose a novel approach to automatically generate pseudo-code from source code using Neural Machine Translation. Our model is built upon the deep learning encoder-decoder using the attention-based Long Short-Term Memory architecture to capture the long-term dependencies in both source code and pseudo-code. An empirical evaluation on a real Python dataset demonstrates the applicability of our approach in practice.
In order to improve and increase their utility, software products must evolve continually and incrementally to meet the new requirements of current and future users. Online reviews from users of the software provide a rich and readily available resource for discovering candidate new features for future software releases. However, it is challenging to manually analyze a large volume of potentially unstructured and noisy data to extract useful information to support software release planning decisions. This paper investigates machine learning techniques to automatically identify text that represents users' ideas for new features from their online reviews. A binary classification approach to categorize extracted text as either a feature or non-feature was evaluated experimentally. Three machine learning algorithms were evaluated in the experiments: Naïve Bayes (with multinomial and Bernoulli variants), Support Vector Machines (with linear and multinomial variants) and Logistic Regression. Variations on the configurations of k-fold cross validation, the use of n-grams and review sentiment were also experimentally evaluated. Based on binary classification of over a thousand separate reviews of two products, Trello and Jira, linear Support Vector Machines with review sentiment as an input, using n-gram (1,4) together with k-fold 10 cross validation gave the best performance. The results have confirmed the feasibility and accuracy of semi-automated extraction of candidate requirements from a large volume of unstructured and noisy online user reviews. The next steps planned are to experiment with machine supported grouping, prioritizing and visualizing the extracted features to best support release planners' work, as well as extending the sources of candidate requirements.
For software architects involved in maintaining existing software systems, understanding the architecture is a crucial task. They have to comprehend the evolution of a system and the rationale behind design decisions to avoid architecture erosion. This is challenging due to frequently outdated documentation and incomplete knowledge about the system, which especially applies to long-living systems. Therefore, it is necessary to retrieve the missing information straight from the existing system. Most existing approaches are limited to reconstruct the structure of a software system by preferably analyzing static dependencies between components, without considering their behavior. However, the usage of technologies determines the behavior of a software system. In this paper, we propose an approach to identify technology features in existing systems. Used technology features are fundamental facts which support architects to understand the software architecture and to find alternative technology solutions. We provide tool support and present first results of the identification of technology features in open source systems.
Context: Social media has dramatically changed the landscape of software engineering, and there is increasing use of blogs in software development. Objective: To examine opportunities for using blogs as a source of evidence in software engineering research. Method: We informally review practitioners use of blogs, and review the research literature. Results: We identify benefits and challenges to the use of blogs, consider quality criteria, and describe an automated methodology, based on the case survey, to gather and analyse blog-based evidence. Conclusion: Further work is required on automating the case survey methodology to ensure that large volumes of higher-quality blog content can be used effectively in research.
Software solutions for big data analysis not only provide many opportunities for industry but might help to increase social and economics sustainability. In this paper, we discuss the lessons learned from a project on the analysis of the possible impact of weather conditions on the medicine consumption. We elaborated a framework and successfully applied it on a number of case studies.
A question familiar to any developer facing legacy code for the first time is, where to start? Understanding the entire code base of any large project is impractical, so techniques for identifying the most relevant code are needed. We discuss one potential technique, which we call "object counts:" a report of the number of objects created during the execution of an object-oriented program. We show how object counts can help when comparing small designs, and comprehending a larger design-specifically JUnit. Our results indicate that the small amount of data provided by object counts can provide significant insight into a system's design.
The goal of secure software engineering is to create software that keeps performing as intended even when exposed to attacks. Threat modeling is considered to be a key activity, but can be challenging to perform for developers, and even more so in agile software development. Hence, threat modeling has not seen widespread use in agile software projects. The goal of this paper is to investigate the challenges facing adoption of threat modeling using the Microsoft approach with STRIDE. We performed a case study in a company comprising five agile development projects. We identified 21 challenges to threat modeling that emerged from our observations. We then mapped these challenges to challenges found in the literature. Some challenges overlap the findings from the literature; the extra challenges we have found in our exploratory study came mostly from the activities of asset identification and also from our observations on what happened after the threat modeling meetings. This study shows that we still have to address many challenges in order to get a proper adoption of threat modeling in agile development projects.
