On Coordinating LLMs and Platform Knowledge for Software Modernization and New Developments

Emerging generative and fine-tuning LLMs services have been widely benchmarked and used for various software development tasks. These LLMs services are powerful but have different output qualities for software development tasks and may not be able to deal with complex development tasks in edge-cloud software modernization and new developments due to their generative capabilities and lack of up-to-date (domain) knowledge. Many queries and solutions related to target platforms, deployment configurations, policies, data regulation, observability, to name just a few, are not well integrated with these LLMs, but are accessed by the developer through other sources. In this work, we discuss situations where the gaps between the needs and the offerings from LLMs can be compensated by Platform Knowledge, which captures knowledge about, e.g., software, service and infrastructure catalogs, architectural decision records and code patterns. We propose COLLMS - a framework for coordinating LLMs services and Platform Knowledge. At the starting point of the framework, we will discuss challenges for achieving the coordination centered around Platform Knowledge, LLMs management and integration, quality-aware coordination of LLMs, and observability and knowledge updating.