Amazon Web Services (AWS) has wrapped up its Generative AI Accelerator Program, which helps identify technological trends while supporting startups. Over ten weeks, 80 startups received access to cloud credits, expertise from companies like Nvidia, Mistral, and Meta, and a network of experts. Two key trends identified this year are the shift from traditional fine-tuning to foundation models and the growing use of Multi-Agent Systems (MAS).
Diverse Applications Despite Reduction in Fine-Tuning
In developing artificial intelligence, a common approach now focuses on training foundation models for longer periods. These models, based on larger datasets, develop a deeper understanding compared to traditional models. Previously, AI was often refined through repeated fine-tuning, where pre-trained models were adapted to specific tasks or applications. Foundation models, however, are versatile and do not require repeated specific adjustments. AWS has also expanded its offerings to include such models.
The use of foundation models was often not feasible due to lack of scalability and computing power. While their initial training costs are higher, the long-term expenses for repeated fine-tuning are eliminated. Critics of fine-tuning, like Andrej Karpathy, co-founder of OpenAI, argue that it often merely imitates human evaluators’ labels without enabling a deep understanding, a quality that should be the potential of AI.
Multi-Agent Systems: Flexibility Through Specialization
Alongside the development of foundation models, the use of Multi-Agent Systems (MAS) has emerged as another trend. These systems consist of autonomous units, known as agents, specialized in specific tasks and able to solve complex problems together. Unlike monolithic models, MAS are modular. Individual agents can be flexibly added, replaced, or adjusted depending on the application. Each agent is tailored to its specific task, making the system efficient without needing a comprehensive, expensive model.
Swami Sivasubramanian, Vice President for Data and Machine Learning Services at AWS, illustrates the benefit of MAS with a humorous example: a system that finds free events with food. One agent analyzes the user’s food preferences, another evaluates event dates and locations, while another prioritizes restaurants based on food quality. Other agents plan the route and register the user. If needed, an additional agent could check if the user’s friends are attending the event.
Startups as Guides for AI Trends
“Generative AI is here to stay,” says Jon Jones, AWS Vice President and Global Head of Startups, in an interview. He adds, “It will be one of the biggest trends in technology history.” Jones emphasizes that both AWS and other companies benefit from collaborating with startups to recognize future technologies early. This allows them to shape these technologies in line with their business goals.