In recent years, many AI companies like OpenAI, Google, and Meta have continued to scale up their models. Running large language models with billions of parameters and making them available to all users at the same time requires enormous computing power. But what if we didn’t have this power and were still using old technology?
This question was explored by the programmers at EXO Labs. They acquired an old Windows 98 computer to test this idea. According to a blog post, the hardware is over 25 years old. They purchased it for about 119 British pounds on eBay. Inside the Windows 98 machine is an Intel Pentium II with 128 megabytes of RAM. Compared to today’s home computers, this is a very minimal performance. If we compare this with AI data centers, whose power consumption is constantly rising, the project seems almost impossible. However, a small version of Llama 2 with a total of 260,000 parameters can actually be run on the computer with a few tweaks.
The tweaks include connecting old PS/2 hardware because the existing USB ports did not work. Llama 2 was then transferred to the PC via FTP. This was only a workaround because discs were not recognized by the PC, and the existing hard drive with four terabytes of storage was too large for the FAT32 file system under Windows 98.
The result: With the old Windows 98, around 40 tokens per second can be generated using the 260K model. The programmers also tested the old hardware with a language model with 15 million parameters. Here, the hardware struggled and generated only one token per second. Based on a benchmark, the programmers also calculated how long the Windows 98 PC would last with a Llama 3.2 model and thus one billion parameters. The result is 0.0093 tokens per second, which is unusable for serious AI use.
This experiment shows that even with outdated technology, it is possible to run simple AI models locally. However, the performance is very limited compared to modern standards. For those interested in experimenting with AI on old hardware, it is a fun challenge but not practical for real-world applications.
Running AI models locally can be an interesting endeavor for enthusiasts who want to see how far they can push old technology. It demonstrates the potential of AI even on limited hardware, although practical applications would require more powerful systems.
For those looking to run AI models locally on more modern hardware, there are several tools available that make this task more feasible and efficient. These tools are designed to work with current technology and provide better performance and usability for AI applications.
Overall, while this experiment with Windows 98 is more of a novelty, it highlights how far technology has come and the possibilities that exist for AI development. It is a testament to the adaptability and innovation within the tech community, always finding new ways to explore and expand the capabilities of AI.