Though the 2010s saw many research publications about languages such as JavaScript and Python, there currently appears to be a general loss of interest in dynamic languages, with popular new languages such as Rust and Zig being statically typed, and AOT compilation often being viewed as a preferable option to JIT compilation. There is a legitimate question as to whether we are headed towards, or already in, a dynamic language "winter", with reduced interest from industry and reduced access to funding for dynamic language research. However, despite this, many of the most popular languages (Python, JS, Julia, etc.) are still dynamically typed. In this talk, we discuss questions such as potential causes for a dynamic language winter, what makes statically typed languages so attractive at this time, the major strengths of dynamic languages that could help turn the tide, and what may come after.
Programming languages offer a number of abstractions such as dynamic typing, sandboxing, and automatic garbage collection which, however, come at a performance cost. Looking back, the most influential programming languages were proposed at a time when Moore’s Law was still in place. Nowadays, post-Moore’s law, scalability, and elasticity become crucial requirements, leading to an increasing tension between programming language design and implementation, and performance. It is now time to discuss the impact of programming languages and language runtimes in the context of scalable and elastic cloud computing platforms with the goal of forecasting their role in the new cloud era.
Since the first bug was discovered in the Mark Harvard II electromechanical computer it was clear that finding bugs and debugging of computer systems would be an extremely challenging task. Today, various reports indicated that programmers spend approximately 50% of their time on debugging related tasks resulting in an annual cost of $312 billion. Given these astronomical amounts of resources being put into debugging, any technique that improves debugging efficiency is tremendously valuable. In the last decades various new debugging techniques have been put forward to ease debugging and finding the root cause of a failures. Techniques like record-replay, deltadebugging, model checking, tracing, visualisation, fuzzing, automated debugging, and many more help programmers to be more effective while debugging. Recently, we have seen that some of techniques are slowly finding their way into mainstream debugging practices. In this talk we first give an overview of recent exiting debugging techniques, show their advantages and limitations to then reflect on the challenges and opportunities for further research.
Dynamic languages have evolved quite a bit over the past few decades. While there’s always room for improvement, the current generation of languages have rich semantics and expressive syntax, making for a pleasant developer experience. Developers can clearly represent ideas, decreasing the maintenance burden while supporting rapid development. Dynamic languages such as Python, Ruby, JavaScript, PHP, and Lua power a substantial portion of web applications and services today. However, diminishing returns in terms of single-core performance and memory bandwidth improvements combined with the limited computational resources available in budget-minded cloud computing have highlighted the inefficiencies of language interpreters. To remain relevant in the decades to come, dynamic language VMs must make a concerted effort to reduce overhead and make effective use of performance features made available by the underlying platform. Dynamic optimization through JIT compilation has proven to be an effective mechanism for improving dynamic language performance, but building and maintaining a JIT compiler is an expensive undertaking. Meta-compilation promises to reduce those costs, but incurs other costs that hamper adoption in industry. Through the lens of a company deploying thousands of Ruby projects into production, we assess the limitations of current VMs, highlight the most impactful advancements, and consider what’s most important for the coming decades.
Over the past decade software development has shifted from a process centered around writing code to a process that increasingly involves composition of external packages and managing the integration of code from other team members. The next decade-plus will be defined by the shift from a process where humans are the central developers of code into one where AI agents, likely Large Language Model (LLM) based, will be the major creators of code and humans will shift to a supervisory role as curators, integrating rich framework-functionality and code developed by AI programming agents. In this new world we must ask ourselves – are programming languages as they exist today fit for purpose and how do they evolve to meet the needs of this future programming model. This talk represents an opinionated take on the question and attempts to outline specific areas of investigation that need to be addressed by the PL community as part of this journey including: What programming language features help/hinder AI agents when understanding and generating code? What programming language features help/hinder human agents when working with an AI Copilot? What programming language tools are needed to empower AI agents in creating grounded and reliable outputs? How can intents be expressed as part of the program representation – examples, constraints, natural language, external documents? How do we empower end-users as part of this transformation? What programming language features are needed to support new AI driven workflows – live coding, interactive requirement gathering, AI TDD? Effectively answering these questions plays a key role in determining if AI driven programming represents a revolution in how software is developed or is limited to being a programming productivity aid for existing development workflows. As such our community should play a central role in understanding this space and leading in the development of this technological transformation!
WebAssembly (Wasm) is a virtual machine whose defining characteristic is that it is low-level: Wasm is designed to abstract the hardware below, not language concepts above. This is a prerequisite for providing predictable performance and for avoiding language bias without feature creep. At the same time, it is a hard requirement that Wasm is safe and portable, which sometimes necessitates raising its abstraction level above the raw metal. Yet ultimately, the intention is that language runtimes are largely implemented _on top_ of Wasm, in Wasm itself. Dynamic languages pose a challenge for this model, because achieving acceptable performance for them often requires every dirty trick from the books. Not all of these techniques are easily ported to Wasm with some of its abstractions, or they incur higher cost because a Wasm engine cannot know or trust invariants in the higher-level runtime and may need to perform redundant checks to maintain its own safety. In particular, Wasm will need to supply additional mechanisms to efficiently support techniques like jit compilation or inline caches.
Polyglot programming is the practice of writing an application with multiple languages to capture additional functionality and efficiency not available to a single language. This happens more often than people think. Some reasons are: to support different platforms (e.g., Android, iOS), to be more efficient on some parts, to take advantage of features unique to a different ecosystem (e.g., dedicated APIs). But are we ready for polyglot programming? This talk will try to explore the open issues from the point of view of both the multiple programming language integration and from the software engineering development for polyglot programming.
Virtual Machines, Dynamic Compllers, and Implementation Frameworks make dynamic langauges easier and more efficient to optimize. Meanwhile, IDEs, provers, dependent types, type inferencers, and (so-called) "generative AI" mean programmers can express - statically - more information about the dynamic behaviour of their programs. Component libraries in these languages will come with assertions and proofs of their behaviour, and their advocates fantasise about transforming programming into the composition of dependently-typed higher-order yoneda morphisms, ensuring programs are correct-by-construction (where that construction is carried out by yet more generative AI). In this talk, I'll speculate about what the resulting world will be like for programmers. Rather than a static world of platonic mathematical abstractions, I argue that the opposite will be true: that all languages will be dynamic.
Applications written in dynamic languages are becoming larger and larger and companies increasingly use multi-million line codebases in production. At the same time, dynamic languages rely heavily on dynamic optimizations, particularly those that reduce the overhead of method calls. In this work, we study the call-site behavior of Ruby benchmarks that are being used to guide the development of upcoming Ruby implementations such as TruffleRuby and YJIT. We study the interaction of call-site lookup caches, method splitting, and elimination of duplicate call-targets. We find that these optimizations are indeed highly effective on both smaller and large benchmarks, methods and closures alike, and help to open up opportunities for further optimizations such as inlining. However, we show that TruffleRuby’s splitting may be applied too aggressively on already-monomorphic call-sites, coming at a run-time cost. We also find three distinct patterns in the evolution of call-site behavior over time, which may help to guide novel optimizations. We believe that our results may support language implementers in optimizing runtime systems for large code-bases built in dynamic languages.
With the wide adoption of the language server protocol, the desire to have IDE-style tooling even for niche and research languages has exploded. The Truffle language framework facilitates this desire by offering an almost zero-effort approach to language implementers to providing IDE features. However, this existing approach needs to execute the code being worked on to capture much of the information needed for an IDE, ideally with full unit-test coverage. To capture information more reliably and avoid the need to execute the code being worked on, we propose a new parse-based design for language servers. Our solution provides a language-agnostic interface for structural information, with which we can support most common IDE features for dynamic languages. Comparing the two approaches, we find that our new parse-based approach requires only a modest development effort for each language and has only minor tradeoffs for precision, for instance for code completion, compared to Truffle's execution-based approach. Further, we show that less than 1,000 lines of code capture enough details to provide much of the typical IDE functionality, with an order of magnitude less code than ad hoc language servers. We tested our approach for the custom parsers of Newspeak and SOM, as well as SimpleLanguage's ANTLR grammar without any changes to it. Combining both parse and execution-based approaches has the potential to provide good and precise IDE tooling for a wide range of languages with only small development effort. By itself, our approach would be a good addition to the many libraries implementing the language server protocol to enable low-effort implementations of IDE features.
Communicating Sequential Process (CSP) is nowadays a popular concurrency model in which threads/processes communicate by exchanging data through channels. Channels help in orchestrating concurrent processes but do not solve per-se data races. To prevent data races in the channel model, many programming languages rely on type systems to express ownership and behavioural restrictions such as immutability. However, dynamically-typed languages require run-time mechanisms because of the lack of type information at compile-time. In this paper, we propose to augment channels with four different permission transfer semantics. We explore two mechanisms to implement such permission transfers at run time: write barriers and partial-read barriers. To validate our approach we implemented a channel framework in Pharo, and we extended it with different permission transfer semantics. We report on performance measurements of both (a) the transfer overhead on a single object and on a graph of objects, and (b) the per-object access overhead incurred by ownership checks. This work stands as a cornerstone of future work on adaptive optimizations for permission transfer channels.
The R programming language is widely used for statistical computing. To enable interactive data exploration and rapid prototyping, R encourages a dynamic programming style. This programming style is supported by features such as first-class environments. Amongst widely used languages, R has the richest interface for programmatically manipulating environments. With the flexibility afforded by reflective operations on first-class environments, come significant challenges for reasoning and optimizing user-defined code. This paper documents the reflective interface used to operate over first-class environment. We explain the rationale behind its design and conduct a large-scale study of how the interface is used in popular libraries.
Many functional languages-including Racket, Clojure, and Scala-provide a persistent-map datatype with an implementation based on Hash Array Mapped Tries (HAMTs). HAMTs enable efficient functional lookup, insertion, and deletion operations with a small memory footprint, especially when taking advantage of implementation techniques that have been developed since the original HAMT implementation. Racket's latest HAMT implementation is based on an intermediate data structure, a stencil vector, that supports an especially compact representation of HAMTs with help from the compiler and memory manager. That is, stencil vectors provide an abstraction to improve HAMT performance without burdening the compiler with all of the complexity and design choices of a HAMT implementation. Benchmark measurements show that HAMTs in Racket have performance comparable to other state-of-the-art implementations, while stencil-vector HAMTs are more compact and run as fast as alternative representations in Racket. Although we only report on Racket, our experience suggests that a stencil-vector datatype in other dynamic-language implementations might improve HAMT performance in those implementations.
The first Futamura projection enables compilation and high performance code generation of user programs by partial evaluation of language interpreters. Previous work has shown that online partial evaluation can yield the same peak performance as a specialized JIT compiler. However, this comes with the downside of additional compile time: Online partial evaluation of language interpreters has to specialize interpreter code on the fly to the dynamic types used at run time to create efficient target code. As a result, the time spent on partial evaluation itself is a significant contributor to the overall compile time of a method. The second Futamura projection solves this problem by self-applying partial evaluation on the partial evaluation algorithm, effectively generating language-specific compilers from interpreters. This typically reduces compilation time compared to the first projection. Previous work employed the second projection to some extent, however we are not aware of any usage of the generic second Futamura projection in a state-of-the-art language runtime. To solve the problems of self-application and code-size explosion, this paper proposes CompGen, an approach based on code generation of subsets of language interpreters. It is loosely based upon the idea of the second Futamura projection. Our implementation of CompGen for GraalVM shows that our usage of a novel code-generation algorithm allows us to generate efficient compilers that emit fast target programs which easily outperform the first Futamura projection in compilation time. We evaluated our approach with the high-performance JavaScript implementation of GraalVM and standard JavaScript benchmarks, showing that our approach achieves >2X speedups of partial evaluation.
Union and intersection types are a staple of gradually typed languages such as TypeScript. While it's long been recognized that union and intersection types are difficult to verify statically, it may appear at first that the dynamic part of gradual typing is actually pretty simple. It turns out however, that in presence of higher-order contracts union and intersection are deceptively difficult. The literature on higher-order contracts with union and intersection, while keenly aware of the fact, doesn't really explain why. We point and illustrate the problems and tradeoffs inherent to union and intersection contracts, via example and a survey of the literature.
We all have questions. About today's temperature, scores of our favorite baseball team, the Universe, and about vaccine for COVID-19. Life, physical, and natural scientists have been trying to find answers to various topics using scientific methods and experiments, while computer scientists have built language models as a tiny step towards automatically answering all of these questions across domains given a little bit of context. In this paper, we propose an architecture using state-of-the-art Natural Language Processing language models namely Topic Models and Bidirectional Encoder Representations from Transformers (BERT) that can transparently and automatically retrieve articles of relevance to questions across domains, and fetch answers to topical questions related to COVID-19 current and historical medical research literature. We demonstrate the benefits of using domain-specific supercomputers like Tensor Processing Units (TPUs), residing on cloud-based infrastructure, using which we could achieve significant gains in training and inference times, also with very minimal cost.
Solutions to the Schrodinger equation can be used to predict the electronic structure of molecules and materials and therefore infer their complex physical and chemical properties. Variational Quantum Monte Carlo (VMC) is a technique that can be used to solve the weak form of the Schrodinger equation. Applying VMC to systems with N electrons involves evaluating the determinant of an N by N matrix. The evaluation of this determinant scales as O(N-3) and is the main computational cost in the VMC process. In this work, we investigate an alternative VMC technique based on the Vandermonde determinant. The Vandermonde determinant is a product of pairwise differences and so evaluating it scales as O(N-2). Therefore, this approach reduces the computational cost by a factor of N. The Vandermonde determinant was implemented in PyTorch and the performance was assessed in approximating the ground state energy of various quantum systems against existing techniques. The performance is evaluated in a variety of systems, starting with the one-dimensional particle in a box, and then considering more complicated atomic systems with multiple particles. The Vandermonde determinant was also implemented in PauliNet, a deep-learning architecture for VMC. The new method is shown to be computationally efficient, and results in a speed-up as large as 5X. In these cases, the new ansatz obtains a reasonable approximation for wavefunctions of atomic systems, but does not reach the accuracy of the Hartree-Fock method that relies on the Slater determinant. It is observed that while the use of neural networks in VMC can result in highly accurate solutions, further work is necessary to determine an appropriate balance between computational time and accuracy.
Galaxy mergers, the dynamical process during which two galaxies collide, are among the most spectacular phenomena in the Universe. During this process, the two colliding galaxies are tidally disrupted, producing significant visual features that evolve as a function of time. These visual features contain valuable clues for deducing the physical properties of the galaxy mergers. In this work, we propose DeepGalaxy, a visual analysis framework trained to predict the physical properties of galaxy mergers based on their morphology. Based on an encoder-decoder architecture, DeepGalaxy encodes the input images to a compressed latent space z, and determines the similarity of images according to the latent-space distance. DeepGalaxy consists of a fully convolutional autoencoder (FCAE) which generates activation maps at its 3D latent-space, and a variational autoencoder (VAE) which compresses the activation maps into a 1D vector, and a classifier that generates labels from the activation maps. The backbone of the FCAE can be fully customized according to the complexity of the images. DeepGalaxy demonstrates excellent scaling performance on parallel machines. On the Endeavour supercomputer, the scaling efficiency exceeds 0.93 when trained on 128 workers, and it maintains above 0.73 when trained with 512 workers. Without having to carry out expensive numerical simulations, DeepGalaxy makes inferences of the physical properties of galaxy mergers directly from images, and thereby achieves a speedup factor of ~10 5 .
Pattern matching allows programs both to extract specific information from complex data types, as well as to branch on the structure of data and thus apply specialized actions to different forms of data. Originally designed for strongly typed functional languages with algebraic data types, pattern matching has since been adapted for object-oriented and even dynamic languages. This paper discusses how pattern matching can be included in the dynamically typed language Python in line with existing features that support extracting values from sequential data structures.
Python 3 is a highly dynamic language, but it has introduced a syntax for expressing types with PEP484. This paper explores how developers use these type annotations, the type system semantics provided by type checking and inference tools, and the performance of these tools. We evaluate the types and tools on a corpus of public GitHub repositories. We review MyPy and PyType, two canonical static type checking and inference tools, and their distinct approaches to type analysis. We then address three research questions: (i) How often and in what ways do developers use Python 3 types? (ii) Which type errors do developers make? (iii) How do type errors from different tools compare? Surprisingly, when developers use static types, the code rarely type-checks with either of the tools. MyPy and PyType exhibit false positives, due to their static nature, but also flag many useful errors in our corpus. Lastly, MyPy and PyType embody two distinct type systems, flagging different errors in many cases. Understanding the usage of Python types can help guide tool-builders and researchers. Understanding the performance of popular tools can help increase the adoption of static types and tools by practitioners, ultimately leading to more correct and more robust Python code.
