A key goal of performance testing is the detection of performance degradations (i.e., regressions) compared to previous releases. Prior research has proposed the automation of such analysis through the mining of historical performance data (e.g., CPU and memory usage) from prior test runs. Nevertheless, such research has had limited adoption in practice. Working with a large industrial performance testing lab, we noted that a major hurdle in the adoption of prior work (including our own work) is the incorrect assumption that prior tests are always executed in the same environment (i.e., labs). All too often, tests are performed in heterogenous environments with each test being run in a possibly different lab with different hardware and software configurations. To make automated performance regression analysis techniques work in industry, we propose to model the global expected behaviour of a system as an ensemble (combination) of individual models, one for each successful previous test run (and hence configuration). The ensemble of models of prior test runs are used to flag performance deviations (e.g., CPU counters showing higher usage) in new tests. The deviations are then aggregated using simple voting or more advanced weighting to determine whether the counters really deviate from the expected behaviour or whether it was simply due to an environment-specific variation. Case studies on two open-source systems and a very large scale industrial application show that our weighting approach outperforms a state-of-the-art environment-agnostic approach. Feedback from practitioners who used our approach over a 4 year period (across several major versions) has been very positive.

Thesis (Master, Electrical & Computer Engineering) -- Queen's University, 2011-01-31 15:53:02.732

The backbone of cloud computing platforms like Amazon S3 and Salesforce is formed by Ultra-Large-Scale (ULS) systems, i.e., complex, globally distributed infrastructure consisting of heterogeneous sets of software and hardware nodes. To ensure that a ULS system can scale to handle increasing service demand, it is important to understand the system's performance behaviour, for example to pro-actively plan for hardware upgrades. A good performance model should address concerns from all stakeholders at the level appropriate to their knowledge, interest, and experience. However, this is not straightforward, since stakeholders of ULS systems have a wide range of backgrounds and concerns: software developers are more interested in the performance of individual software components in the system, whereas managers are concerned about the performance of the entire system in different configurations. In this paper, we adapt the “4+1 View” model for software architecture to performance analysis models by building simulation models with multiple layers of abstraction. As a proof-of-concept, we conducted case studies on an open source RSS (Really Simple Syndication) Cloud system that actively delivers notifications of newly published content to subscribers, and on a hypothetical, industry-inspired performance monitor for ULS systems. We show that our layered simulation models are effective in identifying performance bottlenecks and optimal system configurations, balancing across performance objectives.

Detecting software clones in large scale projects helps improve the maintainability of large code bases. The source code representation (e.g., Java or C files) of a software system has traditionally been used for clone detection. In this paper, we propose a technique that transforms the source code to an intermediate representation, and then reuses established source-based clone detection techniques to detect clones in the intermediate representation. The clones are mapped back to the source code and are used to augment the results reported by source-based clone detection. We demonstrate the performance of our new technique using systems from the Bellon clone evaluation benchmark. The result shows that our technique can detect Type 3 clones. Our technique has higher recall with minimal drop in precision using Bellon corpus. By examining the complete clone groups, our technique has higher precision than the standalone string based and token based clone detectors.

A business process, such as the process followed when ordering a book, describes the order of executing tasks (e.g., check inventory, verify credit card, and ship book). Business applications implement the business processes for the daily operations of an organization. Organizations must continuously modify their business applications to accommodate changes to business processes. However, business applications are often designed and developed without referring to the documented definitions of business processes. Modifying business applications is a time-consuming and error-prone task. To correctly perform this task, developers require an in-depth understanding of multi-tiered applications and the definitions of the business processes that they implement. In this paper, we present an approach that automatically recovers business process definitions from multi-tiered business applications. Given the starting UI screen of a particular business process, the approach recovers the process definition by tracing the flow of control throughout the different tiers of the business application. We demonstrate the effectiveness of our approach through a case study using 15 business applications from three large open-source projects. Our case study demonstrates that our approach can recover business process definitions from the implementation with high precision and recall. Copyright © 2009 John Wiley & Sons, Ltd. We present an approach which automatically recovers business process definitions from multi-tiered business applications. Given the starting UI screen of a particular business process, the approach recovers the process definition by tracing the flow of control throughout the different tiers of the business application. We demonstrate the effectiveness of our approach through a case study using 15 business applications from three large open-source projects. Copyright © 2009 John Wiley & Sons, Ltd.

Business applications automate various business processes within an organization. Business analysts frequently modify business processes to maintain a competitive edge. Estimating the time needed to modify a business process is not a trivial task, since changes to the business process result in changes to the source code of which the business analyst has limited knowledge. We propose a change impact metric which estimates the code to be modified as a result of a business process change. We demonstrate the effectiveness of our proposed change impact metric through a case study, using seven large business applications from the OFBiz open source project.

A business process is composed of a set of interrelated tasks which are joined together by control flow elements. E-commerce systems implement business processes to automate the daily operations of an organization. Organizations must continuously modify their e-commerce systems to accommodate changes to business processes. However, modifying e-commerce systems is a time consuming and error prone task. To correctly perform this task, developers require an in-depth understanding of multi-tiered e-commerce systems and the business processes that they implement. In this paper, we present a business process explorer tool which automatically recovers business processes from three tier e-commerce systems. Developers can explore the recovered business processes and browse the corresponding source code. We integrate our tool with IBM WebSphere Business Modeler (WBM), a leading commercial tool for business process management and modeling. Business analysts could then visualize and analyze the recovered processes using WBM. The business process explorer eases the co-evolution of business processes and their e-commerce system implementation.