Combining Semantic Web and Machine Learning for Auditable Legal Key Element Extraction

Based on a real world use case, we developed and evaluated a hybrid AI system that aims to extract key elements from legal permits by combining methods from the Semantic Web and Machine Learning. Specifically, we modelled the available background knowledge in a custom Knowledge Graph, which we exploited together with the usage of different language- and text-embedding-models in order to extract different information from official Austrian permits, including the Issuing Authority, the Operator of the facility in question, the Reference Number, and the Issuing Date. Additionally, we implemented mechanisms to capture automatically auditable traces of the system to ensure the transparency of the processes. Our quantitative evaluation showed overall promising results, while the in-depth qualitative analysis revealed concrete error types, providing guidance on how to improve the current prototype.