Abstract
Modern organizations depend on analytics pipelines for decision-making, yet integrating advanced ML/AI techniques often increases complexity and hinders optimal performance. This paper introduces PipeMind, a multi-agent framework that leverages large language models (LLMs) and specialized agents to provide real-time feedback and continuous optimization. Focusing on execution time, resource utilization, and throughput, PipeMind autonomously monitors pipeline performance, identifies bottlenecks, and recommends improvements. Its web-based infrastructure, built on RESTful APIs, seamlessly integrates with external data sources and cloud services. We demonstrate a working prototype with quantitative evaluations and a high-frequency trading case study, underscoring PipeMind's potential to enable fully autonomous analytics pipeline management.
| Original language | English |
|---|---|
| Title of host publication | WWW Companion '25 |
| Subtitle of host publication | Companion proceedings of the ACM Web Conference 2025 |
| Place of Publication | New York |
| Publisher | Association for Computing Machinery |
| Pages | 2891-2894 |
| Number of pages | 4 |
| ISBN (Electronic) | 9798400713316 |
| DOIs | |
| Publication status | Published - 2025 |
| Event | 34th ACM Web Conference, WWW Companion 2025 - Sydney, Australia Duration: 28 Apr 2025 → 2 May 2025 |
Conference
| Conference | 34th ACM Web Conference, WWW Companion 2025 |
|---|---|
| Country/Territory | Australia |
| City | Sydney |
| Period | 28/04/25 → 2/05/25 |
Keywords
- Analytics Pipelines
- Multi-Agent Systems
- Large Language Models
- Performance Optimization
- Real-Time Feedback
- Scalability
Fingerprint
Dive into the research topics of 'PipeMind: toward a multi-agent framework for real-time feedback and continuous optimization in analytics pipelines'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver