2 機械学習システムにおける技術的負債に関する既存研究 2.1 システム設計レベルの技術的負債の分類 Sculleyらは,機械学習システムのシステム設計レベルで生じるリスク要因を 5つ 挙げている [5] [6]. • Complex Models Erode Boundaries • Data Dependencies Cost More than Code Dependencies • Feedback Loops • Conguration Debt • Dealing with Changes in the External World また,機械学習システムのシステムレベルのアンチパターンとして,以下を 7つ挙 げている. • Glue Code • Pipeline Jungles • Dead Experimental Codepaths • Abstraction Debt • Plain-Old-Data Type Smell • Multiple-Language Smell • Prototype Smell

Machine learning offers a fantastically powerful toolkit for building useful complex prediction systems quickly. This paper argues it is dangerous to think of these quick wins as coming for free. Using the software engineering framework of technical debt, we find it is common to incur massive ongoing maintenance costs in real-world ML systems. We explore several ML-specific risk factors to account for in system design. These include boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, configuration issues, changes in the external world, and a variety of system-level anti-patterns.

Machine learning offers a fantastically powerful toolkit for building complex systems quickly. This paper argues that it is dangerous to think of these quick wins as coming for free. Using the framework of technical debt, we note that it is remarkably easy to incur massive ongoing maintenance costs at the system level when applying machine learning. The goal of this paper is highlight several machine learning specific risk factors and design patterns to be avoided or refactored where possible. These include boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, changes in the external world, and a variety of system-level anti-patterns. 1 Machine Learning and Complex Systems Real world software engineers are often faced with the challenge of moving quickly to ship new products or services, which can lead to a dilemma between speed of execution and quality of engineering. The concept of technical debt was first introduced by Ward Cunningham in 1992 as a way to help quantify the cost of such decisions. Like incurring fiscal debt, there are often sound strategic reasons to take on technical debt. Not all debt is necessarily bad, but technical debt does tend to compound. Deferring the work to pay it off results in increasing costs, system brittleness, and reduced rates of innovation. Traditional methods of paying off technical debt include refactoring, increasing coverage of unit tests, deleting dead code, reducing dependencies, tightening APIs, and improving documentation [4]. The goal of these activities is not to add new functionality, but to make it easier to add future improvements, be cheaper to maintain, and reduce the likelihood of bugs. One of the basic arguments in this paper is that machine learning packages have all the basic code complexity issues as normal code, but also have a larger system-level complexity that can create hidden debt. Thus, refactoring these libraries, adding better unit tests, and associated activity is time well spent but does not necessarily address debt at a systems level. In this paper, we focus on the system-level interaction between machine learning code and larger systems as an area where hidden technical debt may rapidly accumulate. At a system-level, a machine learning model may subtly erode abstraction boundaries. It may be tempting to re-use input signals in ways that create unintended tight coupling of otherwise disjoint systems. Machine learning packages may often be treated as black boxes, resulting in large masses of “glue code” or calibration layers that can lock in assumptions. Changes in the external world may make models or input signals change behavior in unintended ways, ratcheting up maintenance cost and the burden of any debt. Even monitoring that the system as a whole is operating as intended may be difficult without careful design.

Babel is an open-source language interoperability framework tailored to the needs of high-performance scientific computing. As an integral element of the Common Component Architecture, it is employed in a wide range of scientific applications where it is used to connect components written in different programming languages. In this paper we describe how we extended Babel to support interoperable tuple data types (structs). Structs are a common idiom in (mono-lingual) scientific application programming interfaces (APIs); they are an efficient way to pass tuples of nonuniform data between functions, and are supported natively by most programming languages. Using our extended version of Babel, developers of scientific codes can now pass structs as arguments between functions implemented in any of the supported languages. In C, C++, Fortran 2003/2008 and Chapel, structs can be passed without the overhead of data marshaling or copying, providing language interoperability at minimal cost. Other supported languages are Fortran 77, Fortran 90/95, Java and Python. We will show how we designed a struct implementation that is interoperable with all of the supported languages and present benchmark data to compare the performance of all language bindings, highlighting the differences between languages that offer native struct support and an object-oriented interface with getter/setter methods. A case study shows how structs can help simplify the interfaces of scientific codes significantly.

Predicting ad click-through rates (CTR) is a massive-scale learning problem that is central to the multi-billion dollar online advertising industry. We present a selection of case studies and topics drawn from recent experiments in the setting of a deployed CTR prediction system. These include improvements in the context of traditional supervised learning based on an FTRL-Proximal online learning algorithm (which has excellent sparsity and convergence properties) and the use of per-coordinate learning rates. We also explore some of the challenges that arise in a real-world system that may appear at first to be outside the domain of traditional machine learning research. These include useful tricks for memory savings, methods for assessing and visualizing performance, practical methods for providing confidence estimates for predicted probabilities, calibration methods, and methods for automated management of features. Finally, we also detail several directions that did not turn out to be beneficial for us, despite promising results elsewhere in the literature. The goal of this paper is to highlight the close relationship between theoretical advances and practical engineering in this industrial setting, and to show the depth of challenges that appear when applying traditional machine learning methods in a complex dynamic system.

High-performance scientific applications are usually built from software modules written in multiple programming languages. This raises the issue of language interoperability which involves making calls between languages, converting basic types, and bridging disparate programming models. Babel provides a feature-rich, extensible, high-performance solution to the language interoperability problem currently supporting C, C++, FORTRAN 77, Fortran 90/95, Fortran 2003/2008, Python, and Java. Babel supports object-oriented programming features and interface semantics with runtime enforcement. In addition to in-process language interoperability, Babel includes remote method invocation to support hybrid parallel and distributed computing paradigms.

Babel is an open-source language interoperability framework tailored to the needs of high-performance scientific computing. As an integral element of the Common Component Architecture (CCA) it is used in a wide range of research projects. In this paper we describe how we extended Babel to support interoperable tuple data types (structs). Structs are a common idiom in scientific APIs; they are an efficient way to pass tuples of nonuniform data between functions, and are supported natively by most programming languages. Using our extended version of Babel, developers of scientific code can now pass structs as arguments between functions implemented in any of the supported languages. In C, C++ and Fortran 2003, structs can be passed without the overhead of data marshaling or copying, providing language interoperability at minimal cost. Other supported languages are Fortran 77, Fortran 90, Java and Python. We will show how we designed a struct implementation that is interoperable with all of the supported languages and present benchmark data compare the performance of all language bindings, highlighting the differences between languages that offer native struct support and an object-oriented interface with getter/setter methods.

Babel is an open-source language interoperability framework tailored to the needs of high-performance scientific computing. Its primary focus is on fast in-process communication across various languages. In doing so, some additional call overhead is often inevitable. For several pairs of languages, however, shortcuts exist that allow for more efficient function calls. As Babel is a dynamic framework, the particular set of languages involved is often only known at runtime. In this work, we present a simple yet very effective optimization that can be used to reduce the call overhead between various pairs of languages. In particular, our optimization is applicable if caller and callee are implemented in the same language. We implement and evaluate these techniques for C++ and Python. When applicable, our optimization virtually eliminates the overhead for a small memory cost. Compared to previous versions of Babel, this means a speedup ranging from about 5x for simple numerical argument types up to roughly 125x for strings.

Register allocation has gained renewed attention in the recent past. Several authors propose a separation of the problem into decoupled sub-tasks including spilling, allocation, assignment, and coalescing. This approach is largely motivated by recent advances in SSA-based register allocation that suggest that a decomposition does not significantly degrade the overall allocation quality. The algorithmic challenges of intra-procedural spilling have been neglected so far and very crude heuristics were employed. In this work, (1) we introduce the constrained min-cut (CMC) problem for solving the spilling problem, (2) we provide an integer linear program formulation for computing an optimal solution of CMC, and (3) we devise a progressive Lagrangian solver that is viable for production compilers. Our experiments with Spec2k and MiBench show that optimal solutions are feasible, even for very large programs, and that heuristics leave significant potential behind for small register files.

Instruction selection is a well-studied compiler phase that translates the compiler's intermediate representation of programs to a sequence of target-dependent machine instructions optimizing for various compiler objectives ( e.g. speed and space). Most existing instruction selection techniques are limited to the scope of a single statement or a basic block and cannot cope with irregular instruction sets that are frequently found in embedded systems. We consider an optimal technique for instruction selection that uses Static Single Assignment ( SSA ) graphs as an intermediate representation of programs and employs the Partitioned Boolean Quadratic Problem ( PBQP ) for finding an optimal instruction selection. While existing approaches are limited to instruction patterns that can be expressed in a simple tree structure, we consider complex patterns producing multiple results at the same time including pre/post increment addressing modes, div-mod instructions, and SIMD extensions frequently found in embedded systems. Although both instruction selection on SSA-graphs and PBQP are known to be NP-complete, the problem can be solved efficiently - even for very large instances. Our approach has been implemented in LLVM for an embedded ARMv5 architecture. Extensive experiments show speedups of up to 57% on typical DSP kernels and up to 10% on SPECINT 2000 and MiBench benchmarks. All of the test programs could be compiled within less than half a minute using a heuristic PBQP solver that solves 99.83% of all instances optimally.

The crossing number of a graph is the minimum number of edge crossings in any drawing of the graph in the plane. Extensive research has produced bounds on the crossing number and exact formulae for special graph classes, yet the crossing numbers of graphs such as K11 or K9,11 are still unknown. Finding the crossing number is NP-hard for general graphs and no practical algorithm for its computation has been published so far. We present an integer linear programming formulation that is based on a reduction of the general problem to a restricted version of the crossing number problem in which each edge may be crossed at most once. We also present cutting plane generation heuristics and a column generation scheme. As we demonstrate in a computational study, a branch-and-cut algorithm based on these techniques as well as recently published preprocessing algorithms can be used to successfully compute the crossing number for small- to medium-sized general graphs for the first time.

Modern compression standards such as H.264, DivX, or VC-1 provide astonishing quality at the costs of steadily increasing processing requirements. Therefore, efficient solutions for mobile multimedia devices have to effectively leverage instruction level parallelism (LLP), which is often achieved by the deployment of EPIC (explicitly parallel instruction computing) architectures. A characteristical architectural feature to increase the available ILP in the presence of control flow is predicated execution. Compilers targeting those hardware platforms are responsible to carefully convert control flow into conditional/predicated instructions - a process called if-conversion. We describe an effective if-conversion algorithm for the CHILI - a novel hardware architecture specifically designed for digital video processing and mobile multimedia consumer electronic. Several architectural characteristics such as the lack of branch prediction units, large delay slots, and the provided predication model are significantly different from previous work, typically aiming mainstream architectures such as Intel Itanium. The algorithm has been implemented for an optimizing compiler based on LLVM. Experimental results using a cycle accurate simulator for the well known benchmark suite MiBench and several multimedia codecs show a speed improvement of about 18% on average. On the same programs, our compiler achieves a speedup of 21% in comparison to the existing code generator based on gcc.

With increasing complexity of modern embedded systems, the availability of highly optimizing compilers becomes more and more important. At the same time, application specific instruction-set processors (ASIPs) are used to fine-tune hardware platforms to the intended application, demanding the availability of retargetable components throughout thewhole tool chain. A very promising approach is to model the target architecture using a dedicated description language that is rich enough to generate hardware components and the required tool chain, e.g., assembler, linker, simulator, and compiler. In this work we present a new structural architecture description language (ADL) that is used to derive the architecture dependent components of a compiler backend - most notably an instruction selector based on tree pattern matching. We combine our backend with gcc, thereby opening up the way for a large number of readily available high level optimizations. Experimental results show that the automatically derived code generator is competitive in comparison to a handcrafted compiler backend.

Explicit memory management of Java objects is a frequent source of programming errors in Java native libraries. In this article we present a static verification tool for the automatic detection of frequent kinds of errors such as missing registration of global references. The tool implements a control-flow aware interprocedural escape analysis in order to determine which references have to be explicitly registered for the garbage collector followed by an analysis that makes sure that those references are created on all possible control flow paths. The tool has been applied to some large native libraries and we were able to detect a fairly large number of programming errors.

The crossing number of a graph is the minimum number of edge crossings in any drawing of the graph into the plane. This very basic property has been studied extensively in the literature from a theoretic point of view and many bounds exist for a variety of graph classes. In this paper, we present the first algorithm able to compute the crossing number of general sparse graphs of moderate size and present computational results on a popular benchmark set of graphs. The approach uses a new integer linear programming formulation of the problem combined with strong heuristics and problem reduction techniques, This enables us to compute the crossing number for 91 percent of all graphs on up to 40 nodes in the benchmark set within a time limit of five minutes per graph.

Acknowledgments First of all I want to thank my advisor Petra Mutzel and her group at the department of Algorithms and Data Structures at the Technical University of Vienna for the possibility to write my thesis about this interesting topic and for the outstanding support during the last years. Her great experience and expert knowledge in the field of Graph Drawing was a great help and her enthusiasm for this topic was always a great motivation. This work originated from ideas developed by her group and a number of contributing people and the results could never have been achieved without her support. My special thanks also belong to Gunnar Klau and René Weisskircher. In addition to this thesis they provided me with a number of interesting topics for practical classes and their courses have always been interesting and instructive. Especially I want to thank Gunnar, who took upon him to proofread this thesis and some other papers, and for the large number of fruitful discussions. The major part of the practical work of this thesis was done during a six month stay at the University of La Laguna in Tenerife. I want to thank Prof. Dr. María Belén Melián Batista and her group at the institute for Estadística, Investigación Operativa y Computación for providing me with the necessary facilities, their support and for the possibility to spend a great time on this wonderful island. Furthermore I want to thank Christoph Buchheim, who provided me with a proof-of-concept implementation of our algorithm and for his large number of useful suggestions and ideas. Thanks also to Prof. Barth for representing Prof. Mutzel and to Prof. Raidl and Prof. Krall for holding the final exam. Last but not least I want to thank all my friends for the wonderful time we spent together and my parents for their support during all those years. i Short Abstract In this thesis we study the Crossing Minimization Problem. We are looking for a drawing for a given graph in the plane, such that a minimum number of edges cross. Although this problem was shown to be NP hard, algorithms are needed in practice. Application areas occur, e.g., in the area of Graph Drawing and VLSI design. While the problem mostly is attacked using heuristic approaches in practice, we present an algorithm that is able to solve medium sized instances to provable optimality in reasonable computation time. …

We consider the NP-hard label number maximization problem lnm: Given a set of rectangular labels, each of which belongs to a point feature in the plane, the task is to find a labeling for a largest subset of the labels. A labeling is a placement such that none of the labels overlap and each is placed so that its boundary touches the corresponding point feature. The purpose of this paper is twofold: We present a new force-based simulated annealing algorithm to heuristically solve the problem and we provide the results of a very thorough experimental comparison of the best known labeling methods on widely used benchmark sets. The design of our new method has been guided by the goal to produce labelings that are similar to the results of an experienced human performing the same task. So we are not only looking for a labeling where the number of labels placed is high but also where the distribution of the placed labels is good. Our experimental results show that the new algorithm outperforms the other methods in terms of quality while still being reasonably fast and confirm that the simulated annealing method is well-suited for map labeling problems.

We present a force-based simulated annealing algorithm to heuristically solve the NP-hard label number maximization problem lnm: Given a set of rectangular labels, each of which belongs to a point-feature in the plane, the task is to find a labeling for a largest subset of the labels. A labeling is a placement such that none of the labels overlap and each is placed at its point-feature. The abstraction of the problem to the virtual force system allows us to implement additional aesthetic criteria and to compute placements with good label distribution in a short amount of time. Furthermore, our algorithm can be applied as a postprocessing step to improve existing labelings. We find that the results often look similar to those of a human cartographer. Extensive computational experiments on widely used benchmark data demonstrate that our new algorithm produces solutions where the number of placed labels is close to optimality and where the distribution of the labels is better than in labelings computed by algorithms that only maximize the number of placed labels. The experiments also show that the algorithm is able to solve large problems fast. This demonstrates its applicability in dynamic real-time environments, e.g., in navigation systems.