We provide a brief primer for the idea behind formalising hierarchical causality in the context of complex systems. Here actors are not simply agents. Actors instantiate causation classes. Agents implement local dynamics in given levels or organisation in a given system. Hierarchical causality then describes how actor-level roles constrain, select, and organise agent-level behaviour across levels. The system then necessarily requires three additional structures. First, causation classes to abstract a given form of causal influence that an actor instantiates. Second, aggregation operators to move across the levels. Third, discrete event-time maps are required because the system comprises events, and the relation between local event counts and any global clock must be specified. Our formulation here is purposefully simple and discrete.

We consider reflexivity in hierarchical causal systems in which higher-level states constrain the lower-level dynamics that remain admissible [Wilcox and Gebbie (2014),Gebbie (2026)]. We ask how such state-dependent top-down constraints are realised when local activity is event driven while causal claims are made in calendar time. If an actor-conditioned constraint governs both the admissible event transitions and their timing, the event process is carried into calendar time by a state-dependent time change. If either the admissible constraint or the compatible timing law is not unique, the induced calendar-time causal law need not be unique even when the local event dynamics admit a Markov representation. Hierarchical causality keeps explicit the cross-level constraint and event coordination from which such a calendar-time law must be constructed. Within this hierarchical representation, reflexivity is the endogenous closure of the causal loop through state-dependent top-down constraint and an event-to-calendar time map.

The Epps effect is key phenomenology relating to high frequency correlation dynamics in financial markets. We argue that it can be used to provide insight into whether tick data is best represented as samples from Brownian diffusions, or as samples from truly discrete events represented as connected point processes. We derive the Epps effect arising from asynchrony and provide a refined method to correct for the effect. We then propose three experiments which show how to discriminate between possible underlying representations. These in turn demonstrate how a simple Hawkes representation recovers phenomenology reported in the literature that cannot be recovered using a Brownian representation without additional ad hoc model complexity. However, complex ad hoc noise models built on Brownian motions cannot in general be discriminated relative to a Hawkes representation. Nevertheless, we argue that high frequency correlation dynamics are most faithfully recovered when tick data is represented as a web of interconnected discrete events rather than being samples from continuous Brownian diffusions even when combined with noise.

We extend a Discrete Time Random Walk (DTRW) numerical scheme to simulate the anomalous diffusion of financial market orders in a simulated order book. Here using random walks with Sibuya waiting times to include a time-dependent stochastic forcing function with non-uniformly sampled times between order book events in the setting of fractional diffusion. This models the fluid limit of an order book by modelling the continuous arrival, cancellation and diffusion of orders in the presence of information shocks. We study the impulse response and stylised facts of orders undergoing anomalous diffusion for different forcing functions and model parameters. Concretely, we demonstrate the price impact for flash limit-orders and market orders and show how the numerical method generates kinks in the price impact. We use cubic spline interpolation to generate smoothed price impact curves. The work promotes the use of non-uniform sampling in the presence of diffusive dynamics as the preferred simulation method.

Backtests on historical data are the basis for practical evaluations of portfolio selection rules, but their reliability is often limited by reliance on a single sample path. This can lead to high estimation variance. Resampling techniques offer a potential solution by increasing the effective sample size, but can disrupt the temporal ordering inherent in financial data and introduce significant bias. This paper investigates the critical questions: First, How large is this bias for Sharpe Ratio estimates?, and then, second: What are its primary drivers?. We focus on the canonical rolling-window mean-variance portfolio rule. Our contributions are identifying the bias mechanism, and providing a practical heuristic for gauging bias severity. We show that the bias arises from the disruption of train-test dependence linked to the return auto-covariance structure and derive bounds for the bias which show a strong dependence on the observable first-lag autocorrelation. Using simulations to confirm these findings, it is revealed that the resulting Sharpe Ratio bias is often a fraction of a typical backtest's estimation noise, benefiting from partial offsetting of component biases. Empirical analysis further illustrates that differences between IID-resampled and standard backtests align qualitatively with these drivers. Surprisingly, our results suggest that while IID resampling can disrupt temporal dependence, its resulting bias can often be tolerable. However, we highlight the need for structure-preserving resampling methods.

We investigate and extend the results of Golts and Jones (2009) that an $\alpha$-weight angle resulting from unconstrained quadratic portfolio optimisations has an upper bound dependent on the condition number of the covariance matrix. This implies that better conditioned covariance matrices produce weights from unconstrained mean-variance optimisations that are better aligned with each assets expected return. We provide further clarity on the mathematical insights that relate the inequality between the $\alpha$-weight angle and the condition number and extend the result to include portfolio optimisations with gearing constraints. We provide an extended family of robust optimisations that include the gearing constraints, and discuss their interpretation.

We consider the learning dynamics of a single reinforcement learning optimal execution trading agent when it interacts with an event driven agent-based financial market model. Trading takes place asynchronously through a matching engine in event time. The optimal execution agent is considered at different levels of initial order-sizes and differently sized state spaces. The resulting impact on the agent-based model and market are considered using a calibration approach that explores changes in the empirical stylised facts and price impact curves. Convergence, volume trajectory and action trace plots are used to visualise the learning dynamics. Here the smaller state space agents had the number of states they visited converge much faster than the larger state space agents, and they were able to start learning to trade intuitively using the spread and volume states. We find that the moments of the model are robust to the impact of the learning agents except for the Hurst exponent, which was lowered by the introduction of strategic order-splitting. The introduction of the learning agent preserves the shape of the price impact curves but can reduce the trade-sign auto-correlations when their trading volumes increase.

We use random walks to simulate the fluid limit of two coupled diffusive limit order books to model correlation emergence. The model implements the arrival, cancellation and diffusion of orders coupled by a pairs trader profiting from the mean-reversion between the two order books in the fluid limit for a Lit order book with vanishing boundary conditions and order volume conservation. We are able to demonstrate the recovery of an Epps effect from this. We discuss how various stylised facts depend on the model parameters and the numerical scheme and discuss the various strengths and weaknesses of the approach. We demonstrate how the Epps effect depends on different choices of time and price discretisation. This shows how an Epps effect can emerge without recourse to market microstructure noise relative to a latent model but can rather be viewed as an emergent property arising from trader interactions in a world of asynchronous events.

We extend a Discrete Time Random Walk (DTRW) numerical scheme to simulate the anomalous diffusion of financial market orders in a simulated order book. Here using random walks with Sibuya waiting times to include a time-dependent stochastic forcing function with non-uniformly sampled times between order book events in the setting of fractional diffusion. This models the fluid limit of an order book by modelling the continuous arrival, cancellation and diffusion of orders in the presence of information shocks. We study the impulse response and stylised facts of orders undergoing anomalous diffusion for different forcing functions and model parameters. Concretely, we demonstrate the price impact for flash limit-orders and market orders and show how the numerical method generate kinks in the price impact. We use cubic spline interpolation to generate smoothed price impact curves. The work promotes the use of non-uniform sampling in the presence of diffusive dynamics as the preferred simulation method.

The clarion call for causal reduction in the study of capital markets is intensifying. However, in self-referencing and open systems such as capital markets, the idea of unidirectional causation (if applicable) may be limiting at best, and unstable or fallacious at worst. In this work, we critically assess the use of scientific deduction and causal inference within the study of empirical finance and financial econometrics. We then demonstrate the idea of competing causal chains using a toy model adapted from ecological predator/prey relationships. From this, we develop the alternative view that the study of empirical finance, and the risks contained therein, may be better appreciated once we admit that our current arsenal of quantitative finance tools may be limited to ex post causal inference under popular assumptions. Where these assumptions are challenged, for example in a recognizable reflexive context, the prescription of unidirectional causation proves deeply problematic.

Blackjack or "21" is a popular card-based game of chance and skill. The objective of the game is to win by obtaining a hand total higher than the dealer's without exceeding 21. The ideal blackjack strategy will maximize financial return in the long run while avoiding gambler's ruin. The stochastic environment and inherent reward structure of blackjack presents an appealing problem to better understand reinforcement learning agents in the presence of environment variations. Here we consider a q-learning solution for optimal play and investigate the rate of learning convergence of the algorithm as a function of deck size. A blackjack simulator allowing for universal blackjack rules is also implemented to demonstrate the extent to which a card counter perfectly using the basic strategy and hi-lo system can bring the house to bankruptcy and how environment variations impact this outcome. The novelty of our work is to place this conceptual understanding of the impact of deck size in the context of learning agent convergence.

We consider the dynamics and the interactions of multiple reinforcement learning optimal execution trading agents interacting with a reactive Agent-Based Model (ABM) of a financial market in event time. The model represents a market ecology with 3-trophic levels represented by: optimal execution learning agents, minimally intelligent liquidity takers, and fast electronic liquidity providers. The optimal execution agent classes include buying and selling agents that can either use a combination of limit orders and market orders, or only trade using market orders. The reward function explicitly balances trade execution slippage against the penalty of not executing the order timeously. This work demonstrates how multiple competing learning agents impact a minimally intelligent market simulation as functions of the number of agents, the size of agents' initial orders, and the state spaces used for learning. We use phase space plots to examine the dynamics of the ABM, when various specifications of learning agents are included. Further, we examine whether the inclusion of optimal execution agents that can learn is able to produce dynamics with the same complexity as empirical data. We find that the inclusion of optimal execution agents changes the stylised facts produced by ABM to conform more with empirical data, and are a necessary inclusion for ABMs investigating market micro-structure. However, including execution agents to chartist-fundamentalist-noise ABMs is insufficient to recover the complexity observed in empirical data.

We deploy and demonstrate the CoinTossX low-latency, high-throughput, open-source matching engine with orders sent using the Julia and Python languages. We show how this can be deployed for small-scale local desk-top testing and discuss a larger scale, but local hosting, with multiple traded instruments managed concurrently and managed by multiple clients. We then demonstrate a cloud based deployment using Microsoft Azure, with large-scale industrial and simulation research use cases in mind. The system is exposed and interacted with via sockets using UDP SBE message protocols and can be monitored using a simple web browser interface using HTTP. We give examples showing how orders can be be sent to the system and market data feeds monitored using the Julia and Python languages. The system is developed in Java with orders submitted as binary encodings (SBE) via UDP protocols using the Aeron Media Driver as the low-latency, high throughput message transport. The system separates the order-generation and simulation environments e.g. agent-based model simulation, from the matching of orders, data-feeds and various modularised components of the order-book system. This ensures a more natural and realistic asynchronicity between events generating orders, and the events associated with order-book dynamics and market data-feeds. We promote the use of Julia as the preferred order submission and simulation environment.

We present the constrained portfolio selection problem as a learning problem requiring hyper-parameter specification. In practice, hyper-parameters are typically selected using a validation procedure, of which there are several widely-used alternatives. However, the performance of different validation procedures is problem dependent and has not been investigated for the portfolio selection problem. This study examines the behaviour of common validation procedures, including holdout, k-fold cross-validation, Monte Carlo cross-validation, and repeated k-fold cross-validation for estimating performance and selecting hyper-parameters for constrained portfolio selection. The results demonstrate that repeated k-fold cross-validation is the best performing procedure and recommend using 5 repetitions with 3 ≤ k ≤ 10 in practice.

We consider the problem of fast time-series data clustering. Building on previous work modeling the correlation-based Hamiltonian of spin variables we present an updated fast non-expensive Agglomerative Likelihood Clustering algorithm (ALC). The method replaces the optimized genetic algorithm based approach (f-SPC) with an agglomerative recursive merging framework inspired by previous work in Econophysics and Community Detection. The method is tested on noisy synthetic correlated time-series data-sets with built-in cluster structure to demonstrate that the algorithm produces meaningful non-trivial results. We apply it to time-series data-sets as large as 20,000 assets and we argue that ALC can reduce compute time costs and resource usage cost for large scale clustering for time-series applications while being serialized, and hence has no obvious parallelization requirement. The algorithm can be an effective choice for state-detection for online learning in a fast non-linear data environment because the algorithm requires no prior information about the number of clusters.

Time and the choice of measurement time scales is fundamental to how we choose to represent information and data in finance. This choice implies both the units and the aggregation scales for the resulting statistical measurables used to describe a financial system. It also defines how we measure the relationship between different traded instruments. As we move from high-frequency time scales, when individual trade and quote events occur, to the mesoscales when correlations emerge in ways that can conform to various latent models; it remains unclear what choice of time and sampling rates are appropriate to faithfully capture system dynamics and asset correlations for decision making. The Epps effect is the key phenomenology that couples the emergence of correlations to the choice of sampling time scales. Here we consider and compare the Epps effect under different sampling schemes in order to contrast three choices of time: calendar time, volume time and trade time. Using a toy model based on a Hawkes process, we are able to achieve simulation results that conform well with empirical dynamics. Concretely, we find that the Epps effect is present under all three definitions of time and that correlations emerge faster under trade time compared to calendar time, whereas correlations emerge linearly under volume time.

We use an adversarial expert based online learning algorithm to learn the optimal parameters required to maximise wealth trading zero-cost portfolio strategies. The learning algorithm is used to determine the dynamics of a large population of technical trading strategies that can survive historical back-testing as well as form an overall aggregated portfolio trading strategy from the set of underlying trading strategies implemented on daily and intraday Johannesburg Stock Exchange data. The resulting population time-series are investigated using unsupervised learning for dimensionality reduction and visualisation. A key contribution is that the overall aggregated trading strategies are tested for statistical arbitrage using a novel hypothesis test, on both daily sampled and intraday time-scales. The (low frequency) daily sampled strategies fail the arbitrage tests after costs, while the (high frequency) intraday sampled strategies are not falsified as statistical arbitrages after costs. The estimates of trading strategy success, cost of trading and slippage are considered along with an offline benchmark portfolio algorithm for performance comparison. In addition, the algorithms generalisation error is analysed by recovering a probability of back-test overfitting estimate using a nonparametric procedure. The work aims to explore and better understand the interplay between different technical trading strategies from a data-informed perspective.

The extent to which a matching engine can cloud the modelling of underlying order submission and management processes in a financial market remains an unanswered concern with regards to market models. Here we consider a 10-variate Hawkes process with simple rules to simulate common order types which are submitted to a matching engine. Hawkes processes can be used to model the time and order of events, and how these events relate to each other. However, they provide a freedom with regards to implementation mechanics relating to the prices and volumes of injected orders. This allows us to consider a reference Hawkes model and two additional models which have rules that change the behaviour of limit orders. The resulting trade and quote data from the simulations are then calibrated and compared with the original order generating process to determine the extent with which implementation rules can distort model parameters. Evidence from validation and hypothesis tests suggest that the true model specification can be significantly distorted by market mechanics, and that practical considerations not directly due to model specification can be important with regards to model identification within an inherently asynchronous trading environment.

Mean-variance portfolio decisions that combine prediction and optimisation have been shown to have poor empirical performance. Here, we consider the performance of various shrinkage predictors by their efficient frontiers under different distributional assumptions to study the impact of reasonable departures from Gaussianity. Namely, we investigate the impact of first-order autocorrelation, second-order autocorrelation, skewness, and excess kurtosis. We show that the shrinkage predictors tend to rescale the sample efficient frontier, which can change based on the nature of local perturbations from Gaussianity. This rescaling implies that the standard approach of evaluating decision rules for a fixed level of risk aversion can give a misleading impression of comparative performance, and more so in a dynamic market setting. Our results suggest that comparing efficient frontiers has serious implications which oppose the prevailing thinking in the literature. Namely, that sample estimators out-perform Stein type estimators of the mean, and that improving the prediction of the covariance has greater importance than improving that of the means.

An agent-based model with interacting low frequency liquidity takers inter-mediated by high-frequency liquidity providers acting collectively as market makers can be used to provide realistic simulated price impact curves. This is possible when agent-based model interactions occur asynchronously via order matching using a matching engine in event time to replace sequential calendar time market clearing. Here the matching engine infrastructure has been modified to provide a continuous feed of order confirmations and updates as message streams in order to conform more closely to live trading environments. The resulting trade and quote message data from the simulations are then aggregated, calibrated and visualised. Various stylised facts are presented along with event visualisations and price impact curves. We argue that additional realism in modelling can be achieved with a small set of agent parameters and simple interaction rules once interactions are reactive, asynchronous and in event time. We argue that the reactive nature of market agents may be a fundamental property of financial markets and when accounted for can allow for parsimonious modelling without recourse to additional sources of noise.