Domain models are essential artifacts for representing relevant abstractions of a given application domain in Model-driven Engineering (MDE). Recent advances in large language models (LLMs) have enabled the automatic generation of such models from textual domain descriptions; however, the resulting models often fail to capture the human intention and resolve ambiguities present in natural language. This paper presents ToT-Q, a framework that extends our previous work to integrate the generative capabilities of LLMs, the guidance of a rule-based agent, and human expertise. In essence, ToT-Q employs Tree-of-Thought (ToT) prompting with an LLM to generate an initial candidate model. It then identifies uncertainty within the model and uses a rule-based agent to engage the user in a Q A refinement dialogue. Through targeted questions, the agent clarifies user intent and iteratively improves the model. We conduct an empirical study with 67 university participants who refine five domain models comparing ToT-Q against an LLM-only baseline. Model quality is evaluated against reference solutions. Participants with low-to-medium modeling expertise use our tool to assess how it supports users with different modeling expertise. Our results demonstrate that ToT-Q improves the model, particularly for elements where LLMs struggle. Domain knowledge is more critical than modeling expertise for successful refinement, and participants find the refinement process clear, useful, and manageable.

Large Language Models (LLMs) have the potential to support the transformation of natural language legal text into a regulatory model, a task conventionally known to be time consuming and error prone when done manually.In this paper, we introduce CLERK: a Companion LLM Expert for modeling Regulatory Knowledge existing in natural language legal texts. CLERK captures regulatory knowledge in the format of Legal Goal Requirements Language (GRL) models.CLERK offers three key contributions, utilizing established prompting techniques: (1) Adopting the Tree-of-Thought (ToT) prompting framework, CLERK streamlines the regulatory modeling process by breaking down complex steps into manageable tasks and focusing on those essential for constructing a Legal GRL model only. (2) The ToT framework enables self-evaluation of intermediate outputs. (3) CLERK enhances consistency and clarity, by leveraging additional in-context learning prompting techniques, such as few-shot prompting and output formatting with an explicit syntax definition.Experiments with eight regulatory articles from two domains (healthcare and energy communities) display a notable improvement brought about by CLERK compared to previous approaches. This improvement pertains to identifying relevant actors, goals and their deontic modalities, as well as the relationships among goals.

The increasing advent of approaches that adopt Large Language Models for text-to-model generation is accompanied by a variety of experimental designs for their evaluation. Inspired by “The Prompt Report”, which provides a systematic account of prompting techniques for prompt engineering, we present a structured synthesis of experimental designs used to evaluate LLM-based text-to-model approaches. Through a systematic literature review, we compile a compendium of experimental designs, organized into six tree-structured categories. This compendium integrates both experimental design principles from traditional empirical research and challenges unique to LLM usage. For example, specific to LLMs, we identify studies that conduct an ablation study to isolate the effect of individual prompting techniques, as well as those that address cascading errors introduced by intermediate LLM outputs. The compendium is intended to inform the design of future evaluations in this emerging field, offering researchers structured support and guidance.

The use of Large Language Models (LLMs), combined with advanced prompting strategies, automates the creation of domain models from textual domain descriptions. However, the output is often influenced by mistakes and limitations that arise from the inherent characteristics of LLMs, including hallucinations and inconsistencies. Additionally, ambiguities and incompleteness in the input text further affect the quality of the results. We propose a new LLM-based modeling method with human in the loop that aims to combine the strengths of automatic model creation with human supervision and interaction to refine and validate the model. In our approach, the LLM generates an initial draft model from textual descriptions. This draft is then subjected to a feedback loop moderated by a rule-based agent, which engages the user through a Q&A dialogue. The rule-based agent selects the questions based on their potential to clarify the most uncertain aspects of the model up to that point.

The electricity sector is increasingly characterized by the use of Information Technology (IT) for the electricity grid, leading to its transformation into a smart grid. Various pilot smart grid initiatives exist, which have already showcased their technical feasibility. Nevertheless, for the needs of informed decision-making, a subsequent valuation support of those initiatives for all involved stakeholders is also essential. Considering the importance and complexity of such a valuation, the authors propose a MOdel-based, multi-perspectiVe method for the valuation of INitiatives in the smart Grid (MOVING). MOVING complements well-established smart grid valuation methods with conceptual modeling, to cater for a systematic analysis of actor goals, value exchanges, and IT. They illustrate the MOVING method with a well-documented smart grid initiative called NRGcoin, which is a blockchain-based incentive mechanism to encourage local production and consumption of renewable electricity.

Domain modeling is typically an iterative process where modeling experts interact with domain experts to complete and refine the model. Recently, we have seen several attempts to assist, or even replace, the modeler with a Large Language Model (LLM). Several LLM prompting strategies have been attempted, but with limited success. In this paper, we advocate for the adoption of a Tree-of-Thoughts (ToT) strategy to overcome the limitations of current approaches based on simpler prompting strategies. With a ToT strategy, we can decompose the modeling process into several sub-steps using for each step a specialized set of generators and evaluators prompts to optimize the quality of the LLM output. As part of our adaptation, we provide a Domain-Specific Language (DSL) to facilitate the formalization of the ToT process for domain modeling. Our approach is implemented as part of an open source tool available on GitHub.

Verification in the realm of enterprise modeling (EM) ensures both the consistency of EM language specifications (i.e., meta models and additional well-formedness constraints), as well as of enterprise models. The consistency of enterprise models, which integrate different perspectives on an enterprise, ensures that they contain the necessary, in line with domain-specific rules, information for carrying out a variety of model-driven enterprise analyses. Meta modeling platforms are instrumental in carrying out such verification, especially when multiple languages are applied in tandem, as is inherent to enterprise modeling. This paper reports on our practical experiences of using formal methods for verification in the context of EM. Motivated by the required verification capabilities, we show for one example platform, ADOxx, how it can be chained together with Alloy, an example of lightweight formal method, to capitalize on complementary platform strengths. Namely, ADOxx for language specification and use, and Alloy for verification capabilities. We show the verification, both, on the meta model level, in terms of checking the consistency of language specifications, and on the model level, in terms of checking models against well-formedness constraints. We illustrate the chaining of ADOxx and Alloy on the basis of consistency checks of two languages applied in tandem, namely the value modeling language e3value and the IT infrastructure modeling language, ITML. We also carry out experiments with three further languages to reflect upon the performance of Alloy, and its capability to uncover inconsistencies.

This paper offers an assessment of the extent to which conceptual modeling can be used for a conjoint assessment of regulatory and economic viability of new projects in the electricity sector, with a particular focus on developing energy communities. To this end, we establish a set of challenges resulting out of a confrontation of, on the one hand, the observed relevance of conjointly assessing the regulatory and economic viability for electricity sector projects, and on the other hand, the fact that no dedicated efforts exist which explicitly target such a conjoint assessment. Then, using a realistic scenario we show how two selected conceptual modeling languages can be used for a conjoint assessment: Legal GRL, for regulatory viability, and e 3 value , for economic viability. Finally, we discuss lessons learned from our experience, among others, a need for a taxonomy for energy sector specific regulation and the use of value network patterns.

The energy sector is characterized by considerable regulatory variation. Thus, for developing a new energy sector project, typical early phase business development activities need to be supplemented with a careful regulatory assessment. In this paper, we investigate to what extent contextual requirements engineering can be useful to support regulatory analyses in the energy sector. To this end, we first derive a set of requirements, and then show both the potentials and shortcomings of contextual requirements engineering for regulatory analysis. A demand side flexibility scenario, focusing on assessing the rights and obligations imposed by the European Clean Energy Package, is used for motivation and illustration purposes. Finally, we provide a discussion on working towards a modeling method, which intertwines business development and regulatory assessment.

Enterprise models have the potential to constitute a valuable asset for organizations, e.g., in terms of enabling a variety of analyses. A prerequisite for realizing this potential is that an enterprise model is syntactically, semantically and pragmatically valid. To ensure these three types of validity, verification and validation (V&V) mechanisms are required to be in place while designing the enterprise modeling method, e.g., to validate identified requirements, to check created enterprise models against syntactic rules, or to ensure intra- and inter-model consistency. Therefore, the objective of this paper is to systematically embed verification and validation (V&V) techniques into the design of (enterprise) domain-specific modeling methods (DSMMs). To this end, we integrate steps and considerations of well-established DSMM engineering processes, and enrich them with V&V techniques based upon our earlier experiences and a literature analysis.

To support the value assessment of technically feasible smart grid initiatives there exist several valuation methods. To determine whether those methods address all concerns relevant for smart grid valuation, we carry out a literature analysis aiming at (1) identifying existing valuation methods and the steps they propose, (2) identifying important valuation considerations, and (3) confronting these considerations with artifacts proposed by the existing valuation methods to identify open issues, requirements, and remaining challenges. Based on the conducted analysis we identify, among others, the following main deficiencies: (1) only a limited scope of concerns relevant to valuation is covered, particularly a systematic consideration of stakeholders goals, value exchange scenarios, and the IT infrastructure is lacking; and (2) a lack of instruments dedicated to fostering accessibility of valuation, in terms of establishing a shared understanding, communicating results, or actively involving different stakeholders in the process. Based on the findings, we suggest the application of conceptual modeling as an instrument to address the identified deficiencies. Therefore, we reflect on the role that current modeling approaches can play in smart grid valuation. This paper is a part of a larger project whose ultimate goal is to develop a model-based method for multi-perspective valuation of smart grid initiatives. The purpose of this paper is to establish a foundation for the realization of the envisioned method. The design of the model-based valuation method itself, its application and evaluation, are subjects of future work.

A domain model provides an explicit knowledge representation of (selected aspects of) some domain of interest. The transition to the digital age results in an increased need for domain models that are machine understandable. We posit that, at the same time, there is an increasing need for non-experts (in modeling) to be able to create such models, or at least be able to understand the created models, and take ownership of their meaning and implications. This situation causes a ‘modeling bottleneck’ in that it is not reasonable to expect all non-experts to become modeling experts. This is where we turn to AI as an enabling technology to support non-experts in domain modeling related tasks; i.e. AI Assisted Domain Modeling. We foresee a symbiotic collaboration between human intelligence, symbolic AI and subsymbolic AI; essentially resulting in a triple-helix of human, symbolic, and subsymbolic intelligence. The aim of this workshop paper is to structurally explore the potential role of (symbolic and subsymbolic) AI to support domain conceptualization. To do so, we will combine three perspectives on domain modeling: (1) a framework relating the different conceptions (harbored in the mind of a modeler) regarding the domain to be modeled, and the model itself, (2) the role of normative frames towards modeling activities, and (3) modeling as a structured dialogue between an (automated) system analyst and a domain expert.

Organizations increasingly have to cope with the digital transformation, which is ubiquitous in today's society. Strategic analysis is an important first step towards the success of digital transformation initiatives, whereby all the elements (e.g., business processes and IT infrastructure) that are required to achieve the transformation can be aligned to the strategic goals and decisions. In this paper, we work towards a modeling method to perform model-based strategic analysis. We explicitly account for information technology (IT) infrastructure because of its key role for digital transformation. Specifically, (1) based on a conducted study on business scholar literature and existing work in conceptual modeling, a set of requirements is first identified; (2) then, we propose a modeling method that integrates, among others, goal modeling, strategic modeling, and IT infrastructure modeling. The method exploits, among others, three previously designed domain specific modeling languages in the Multi-Perspective Enterprise Modeling (MEMO) family: GoalML, SAML and ITML; (3) we illustrate the use of the modeling method in terms of a digital transformation initiative in the electricity sector; and finally, (4) we evaluate the proposed modeling method by comparing it with the conventional SWOT analysis and reflecting upon the fulfillment of the identified requirements.

Information Technology (IT) is increasingly used in the electricity grid to cope with its multiple challenges, leading to its transformation into a so-called “smart grid”. While there exist various technically feasible pilot smart grid initiatives, a subsequent assessment of their “value” is a non-trivial task given the notion of value in smart grid projects. The notion of value usually encompasses, among others, readily quantifiable benefits as well as qualitative ones, of different types (economic, social, environmental), which must be assessed for single actors as well as for a network of actors. To support this assessment, several smart grid valuation methods have been proposed, and subsequently adopted in practice. Although those methods are actively used, a question appears to what extent they address all important factors relevant for smart grid valuation. To answer this question, in this technical report, we carry out a literature analysis aiming at (1) identifying existing valuation methods and the steps they propose, (2) identifying important valuation considerations, and (3) confronting these considerations with artifacts proposed by the existing valuation methods to identify open issues that should be tackled. Based on the conducted analysis, we identify, among others, the following main deficiencies: (1) only a limited scope of concerns relevant to valuation is covered, particularly a systematic consideration of stakeholders goals, value exchange scenarios, and IT infrastructure is lacking; and (2) a lack of instruments dedicated to fostering accessibility of valuation, in terms of establishing a shared understanding, communicating results, or actively involving different stakeholders in the process. The findings reported here correspond to the first stage of a larger project aiming at the development of a modeling method for the multi-perspective valuation of smart grid initiatives.

Organizations increasingly have to cope with the digital transformation, which is ubiquitous in today’s society. Strategic analysis is an important first step towards the success of digital transformation initiatives, whereby all the elements (e.g., business processes and IT infrastructure) that are required to achieve the transformation can be aligned to the strategic goals and decisions. In this paper, we work towards a modeling method to perform model-based strategic analysis. We explicitly account for information technology (IT) infrastructure because of its key role for digital transformation. Specifically, (1) based on a conducted study on business scholar literature and existing work in conceptual modeling, a set of requirements is first identified; (2) then, we propose a modeling method that integrates, among others, goal modeling, strategic modeling, and IT infrastructure modeling. The method exploits, among others, three previously designed domain specific modeling languages in the Multi-Perspective Enterprise Modeling (MEMO) family: GoalML, SAML and ITML; (3) we illustrate the use of the modeling method in terms of a digital transformation initiative in the electricity sector; and finally, (4) we evaluate the proposed modeling method by comparing it with the conventional SWOT analysis and reflecting upon the fulfillment of the identified requirements.

Creating models and transforming them using current MDE techniques is not easy: it generally requires mastering several non-trivial languages such as a metamodeling languages and a model transformation language. We propose a two-pronged approach for tackling language complexity for the case of model-to-text transformations. We first allow the user to define the metamodel in an example-driven fashion in which (s)he incrementally builds a set of examples and automatically infers the metamodel from them. The example-driven approach is based on a new object-modelling notation named OYAML that is both human- and machine-readable. Second we break down the complexity of writing the transformation itself by separately defining the functional decomposition of the transformation function using a new modelling language named FUDOMO. This will then allow the user to describe the precise behaviour in a general purpose programming language that (s)he is familiar with. Because they do not need to be very expressive, OYAML and FUDOMO are small languages when compared to commonly used metamodeling and model-to-text transformation languages. We provide a web-based tool, also named FUDOMO, that assists the user in this example-driven approach to model-to-text transformations and currently supports the use of Javascript and Python for defining the precise behaviour of model transformations.

While ADOxx is a popular platform for the creation and use of enterprise modeling languages, it provides only limited support for a well-formedness check of created enterprise models. In this paper, we propose to complement the meta modeling platform ADOxx with Alloy, which natively provides extensive model checking capabilities, so as to enable a well-formedness check of enterprise models created in ADOxx. Using the $$e^{3}{value}$$ modeling language as a point of departure, we particularly provide (a) a partial ADOxx implementation of $$e^{3}{value}$$ , (b) a proof-of-concept XML2Alloy parser, which allows for converting $$e^{3}{value}$$ models created in ADOxx into Alloy format, so that (c) $$e^{3}{value}$$ well-formedness constraints stated in Alloy can be used to check the validity of an $$e^{3}{value}$$ model with the Alloy Evaluator. Beyond the specific proof-of-concept, we also discuss further possibilities of using ADOxx in conjunction with Alloy, particularly in checking the soundness of meta models underlying an enterprise modeling language.

Digitalization of the energy grid, leading to a so-called smart grid, offers notable benefits to the electricity system itself, and creates economic, social, and environmental value for different parties. Nevertheless, there is still a lack of understanding the value proposition underlying a technically feasible smart grid initiative. In this paper we identify requirements towards an approach for understanding the value proposition of a smart grid initiative, and propose a first step towards such an approach in the form of a landscape of modeling languages complemented by a process model for valuation. We evaluate the proposed approach, among others, by applying it to a blockchain-based initiative in the energy sector.