AgentMILO: a knowledge-based framework for complex MILP modelling conversations with LLMs

This paper presents AgentMILO, an LLM-based conversational agent designed to assist non-expert users in modelling complex Mixed-Integer Linear Programming (MILP) problems, with a focus on production planning and single-traveller routing problem types. By incorporating expert designed knowledge graphs, AgentMILO improves the problem modelling process by guiding users through asking relevant questions to elicit problem specifications from users and formulation of precise MILP models. Through experiments with ten distinct auto-answering agents acting as users, we compared AgentMILO against a general LLM agent without knowledge graphs. The results show that AgentMILO consistently outperforms the general LLM model in clarity in guiding users, question quality, and ease of interaction, while also delivering more precise MILP formulations. In contrast, the general model struggled with inconsistent performance and frequent failures in guiding users effectively. AgentMILO's architecture allows for integration into various domains, with the potential to utilise expert knowledge to design new knowledge graphs tailored to specific scenarios. The complete development framework for AgentMILO, including the prompts, knowledge graphs, tested problem instances, generated conversations, and full experimental setup, is available at https://github.com/arc2022-deakin/AgentMILO.