Red Hat has announced new features in OpenShift AI version 2.15, including a Model Registry for managing predictive and generative AI models, metadata, and model artifacts. OpenShift AI is a platform for developing and operating AI applications in a hybrid cloud environment, focusing on data collection, model training, fine-tuning, deployment, monitoring, and hardware acceleration.
One of the new features in OpenShift AI 2.15 is Data Drift Detection, which identifies changes in input data distribution for existing ML models. This feature alerts data scientists when live data significantly deviates from training data, allowing model reliability checks and adjustments to real data. Additionally, Bias Detection helps create fair and unbiased AI by monitoring model fairness in practice.
Fine-Tuning with Low Rank Adaption (LoRA) allows quicker fine-tuning of large language models (LLMs) like Llama 3. NVIDIA NIM support accelerates distributed AI application deployment using microservices and is part of the NVIDIA AI Enterprise Software Platform. OpenShift also supports AMD GPUs, providing access to an AMD ROCm Workbench Image for model development and additional options for GPU use to enhance performance in compute-intensive tasks.
Red Hat aims to enable platform-independent AI models, recognizing that large language models are often hardware-dependent. To address this, Red Hat acquired Neural Magic, a MIT spin-off, and plans to introduce “Open Hybrid AI.” This involves abstracting LLMs to run on any platform, creating “virtual LLMs” (vLLMs). This concept, developed at the University of California, Berkeley, supports various hardware platforms, including processors from AMD, Intel, and Nvidia, as well as custom chips from Amazon Web Services and Google.
While large LLMs require substantial computational power, Red Hat focuses on “Small Language Models” (SML) for simpler AI applications. These models are trained specifically for certain tasks, requiring less computational power and can run on servers or new AI PCs. Red Hat collaborates with Intel in this endeavor.
IBM’s Granite-7B language model is integrated into InstructLab, allowing customization for specific business needs without losing prior training. IBM reports that Granite model fine-tuning for COBOL to Java translation initially required 14 rounds over nine months. With InstructLab, this was reduced to one week for improved performance.
The use of open source for AI is viewed critically, partly due to dependence on major companies like Microsoft, Facebook, Google, and Tesla. Developing and maintaining LLMs requires immense resources beyond community capabilities. Open source is not yet a full alternative to proprietary products. The LF AI & Data Foundation found open-source models generally perform worse than closed-source models. Security is another concern, with over 30% of open-source models showing vulnerabilities, compared to fewer risks in closed models.
Open-source AI faces challenges like “openwashing,” where products claim to be open source but don’t adhere to open source principles. According to the LF AI & Data Foundation, an open-source AI system must be usable for any purpose without permission, have accessible functionality, be modifiable, and be shareable with or without changes. Models meeting these criteria include Pythia (Eleuther AI), OLMo (AI2), Amber, CrystalCoder (LLM360), and T5 (Google). Others, like BLOOM (BigScience), Starcoder2 (BigCode), and Falcon (TII), may qualify if they change licenses or legal terms. Models like Llama2 (Meta), Grok (X/Twitter), Phi-2 (Microsoft), and Mixtral (Mistral) do not meet open-source principles.
The Foundation highlights a major issue with closed-source models: the lack of transparency in algorithms and data, potentially leading to legal issues. Open-source providers argue they have an advantage, as their software is free from third-party claims, including AI solutions.