Historical research relies on archival sources, which are increasingly kept in digital formats. This chapter discusses the digital sources available to management researchers and where to find them. A key difference when working with digital sources is whether they were originally created digitally. Digitized sources are generally images of physical documents with basic text recognition. Born-digital sources have always existed digitally and often need to be searched computationally. We highlight the need for critical source analysis in the digital era, meaning that researchers should question why some sources are available digitally. Second, the nature of digital sources means that they can be analysed differently and that the degree of technology involved in interpreting them may vary significantly. Ultimately, while new collections and tools make digital collections more accessible, historical research approaches can build on a strong methodological tradition of identifying bias and silences through critical source analysis.

Historians of business and management increasingly conduct research in digital archives. This article reviews some of the challenges and opportunities associated with the use of born-digital archives. As an example, we focus on scholarly use of large-scale organizational e-mail collections. In addition to allowing researchers to answer traditional questions about innovation, strategy and organizational development, e-mail also permits more granular investigation of new questions, such as those relating to the timing and flow of information inside organizational networks. Knowledge discovery in e-mail requires new search tools. We describe EMCODIST, a prototype tool that we have developed to support search and discovery in e-mail. Scholars interested in learning more are directed to a grant-funded website where versions of the EMCODIST tool support different types of searches.

Digital transformation in government has brought an increase in the scale, variety, and complexity of records and greater levels of disorganised data. Current practices for selecting records for transfer to The National Archives (TNA) were developed to deal with paper records and are struggling to deal with this shift. This article examines the background to the problem and outlines a project that TNA undertook to research the feasibility of using commercially available artificial intelligence tools to aid selection. The project AI for Selection evaluated a range of commercial solutions varying from off-the-shelf products to cloud-hosted machine learning platforms, as well as a benchmarking tool developed in-house. Suitability of tools depended on several factors, including requirements and skills of transferring bodies as well as the tools’ usability and configurability. This article also explores questions around trust and explainability of decisions made when using AI for sensitive tasks such as selection.

Email archives are important historical resources, but access to such data poses a unique archival challenge and many born-digital collections remain dark, while questions of how they should be effectively made available remain. This paper contributes to the growing interest in preserving access to email by addressing the needs of users, in readiness for when such collections become more widely available. We argue that for the content of email to be meaningfully accessed, the context of email must form part of this access. In exploring this idea, we focus on discovery within large, multi-custodian archives of organisational email, where emails’ network features are particularly apparent. We introduce our prototype search tool, which uses AI-based methods to support user-driven exploration of email. Specifically, we integrate two distinct AI models that generate systematically different types of results, one based upon simple, phrase-matching and the other upon more complex, BERT embeddings. Together, these provide a new pathway to contextual discovery that accounts for the diversity of future archival users, their interests and level of experience.

Preservation of emails poses particular challenges to future discovery as alternative historical sources. Emails represent communications between individuals and contain a wealth of information when viewed as an organisation-wide collection. Existing search tools can extract named entities and keyword searches but are less effective when it comes to extracting patterns and contextual information across multiple custodians. To address this, we present EMCODIST, a discovery tool for searching the contextual information across emails using attention-based models of Natural Language Processing (NLP). The EMCODIST aims to steer end-users to personalise their searches towards a concept. In this paper, we explain the definition of the ‘context’ for emails which is also suitable for object-oriented computational modelling. The tool is evaluated based on the relevancy of the emails extracted.

One of the challenges faced in digital preservation is to identify the file types when the files can be opened with simple text editors and their extensions are unknown. The problem gets complicated when the file passes through the test of human readability, but would not make sense how to put to use! The Text File Format Identification (TFFI) project was initiated at The National Archives to identify file types from plain text file contents with the help of computing intelligence models. A methodology that takes help of AI and machine learning to automate the process was successfully tested and implemented on the test data. The prototype developed as a proof of concept has achieved up to 98.58% of accuracy in detecting five file formats.

When data stores and users are distributed geographically, it is essential to organize distributed data cache points at ideal locations to minimize data transfers. To answer this, we are developing an adaptive distributed data caching framework that can identify suitable data chunks to cache and move across a network of community cache locations.

The Network of Cancer Genes (NCG) is a manually curated repository of 2372 genes whose somatic modifications have known or predicted cancer driver roles. These genes were collected from 275 publications, including two sources of known cancer genes and 273 cancer sequencing screens of more than 100 cancer types from 34,905 cancer donors and multiple primary sites. This represents a more than 1.5-fold content increase compared to the previous version. NCG also annotates properties of cancer genes, such as duplicability, evolutionary origin, RNA and protein expression, miRNA and protein interactions, and protein function and essentiality. NCG is accessible at http://ncg.kcl.ac.uk/ .

One of the challenges in dealing with distributed large data is to transfer massive amounts of data from multiple data server(s) to users. Unless data transfers are planned, organized and regulated carefully, they can become a potential bottleneck and may necessitate changes in queries and database design which involves costly maintenance work. This is a pronounced problem in the case of virtual observatories where data is to be brought from multiple astronomical databases from all around the world. In this paper, we present adaptive middle ware caching using sub-query fragmentation. When groups of users working on related projects query multiple databases, often their queries are overlapped only partially. We develop a cooperative cache framework with dynamic maintenance algorithms to capture user query patterns in the workload that adapts itself to provide as much data available from cache units as possible. Initial results in the simulated environment with known query inputs show significant reduction in the data to be transferred in comparison with full query caching.

Innovations in science and technology is increasing the demand on huge data transfers and hence number of data caches. In this paper, we consider the community caching solution, CommCache, where many groups of users are working together on related projects distributed all over the world. We demonstrate the use of proactive caches for data placement problem with the help of multi-agent coordination.

Caching frequently used data is a common practice to improve query performance in database systems. But traditional algorithms used for cache management prove to be insufficient in distributed environment where groups of users require similar or related data from multiple databases. Repeated data transfers can become a bottleneck leading to long query response time and high resource utilization. Our work focuses on adaptive algorithms to decide on optimal grain of data to be cached and cache refreshment techniques to reduce data transfers. In this paper, we present agent based simulation to investigate and in consequence improve cache management in the distributed database environment. Dynamic grain size and decisions on cache refreshment are made as a result of coordination and interaction between agents. Initial results show better response time and higher data availability compared to traditional caching techniques.