Fortune 500 companies now use multiple LLMs, with a growing preference for open-source models. AI budgets have tripled, focusing on practical implementation over innovation. Key challenges include hallucinations and data security. Allganize offers tailored AI solutions to optimize performance and reduce costs.
How many LLM models do you use? Unlike last year, when Fortune 500 companies only used OpenAI, this year they are using more than three LLM models. OpenAI's Chat GPT is still used a lot in production, but for companies still in the testing phase, Google's Gemini model is used more than OpenAI. It's also worth noting that companies are increasingly favoring open-source models. Fine-tuning and RAGs are becoming more and more important.
Andreessen Horowitz's survey of Fortune 500 and other enterprise leaders explores how companies are adapting their use of generative AI, their purchasing, and their budgeting. While some still have doubts about generative AI, companies have nearly tripled their budget in six months while also shifting their focus in favor of open source models.
According to Andreessen Horowitz's survey report, "16 Changes to the Way Companies Build and Buy Generative AI," enterprises spent an average of $7 million on LLMs, APIs, hosting, and fine-tuning models in FY23. In '24, this increased 2.5 times to $18 million.
How was this increased funding spent? 2023 was mostly about "innovation" and one-off spending on trials. In 2024, we are seeing a reduction in innovation budgets by about a quarter and a shit towards buying generative AI software.
While the innovation budget was a kind of PoC (Proof of Concept) to find out how generative AI could be introduced into our organization and to conduct various experiments, this year it will actually be used for software and IT infrastructure to roll out AI to practitioners. In addition, customer service is increasingly spending on generative AI to reduce labor costs. One company increased its generative AI spend by 8x, saving up to $6 per call with LLM-based customer service, a 90% cost savings.
As the budget has increased, measuring ROI is also a concern for companies. While the companies surveyed are using Net Promoter Score (NPS) and customer satisfaction by default, they are also looking for ways to reduce costs, generate revenue, and improve accuracy. Fifty-six percent of customers say they haven't yet found an accurate way to measure it, but they see it as a positive result, such as increased productivity and efficiency. To date, only one-third measure ROI through cost savings.
How many LLMs does your company use? Just six months ago, most companies were using one model (usually OpenAI) or two, but in this survey, they are using three (36%), four (28%) and five or more (29%).
Open-source performance is not yet as high as OpenAI's GPT-4, so OpenAI is used in live enterprise products and solutions. As you can see in the graph below, we often see Google's Gemini model in the testing phase. You can see that the open-source model Llama is also being used a lot in the testing stage.
Why has the number of LLMs in use by companies increased? This is partly due to the proliferation of Big Tech models, but also because companies have increased their preference for open-source models (46%). Companies are looking for ways to optimize based on performance, scale, and cost, and they also want to quickly take advantage of the rapidly changing technology trends of LLMs. They want to use a combination of current state-of-the-art and open-source models to get the best results.
Six months ago, 80-90% of the LLM market share was closed models, with OpenAI accounting for the majority. This year, 60% of enterprise leaders are interested in using open source and fine tuning based on open source models. Some companies have gone from using a closed vs. open source model to 80/20 and setting a goal of going 50/50 in 2024.
Companies are adopting an open source model because of control and customization rather than cost. This is because you can control the security of your own data and effectively fine-tune it to suit your needs. Businesses are still reluctant to share proprietary data with closed model providers due to regulatory or data security concerns. Companies where IP is at the core of their business model tend to be particularly conservative.
In 2023, custom models such as BloombergGPT appeared, and companies were discussing building their own LLMs. This year, with the number of high-quality, open-source models increasing, people are choosing to fine-tune (72%) or use RAGs (Retrieval Augmented Generation). It looks like the demand for fine-tuning will continue to grow.
If you look at the generative AI use cases in the enterprise in the graph below, it is used more than 60% of the time for text summarization and knowledge management within the company. The use of chatbots and recommendation algorithms as external services is less preferred at less than 40%.
That's because two concerns still remain when generative AI is used by businesses:
1) Hallucinations and security
2) Sensitive customers information(medical and financial)
Businesses are looking for ways to reduce hallucinations with RAGs and build fine-tuned models on-premise for security issues, but do not often introduce generative AI into the services that their customers use directly. There are many cases where generative AI is used to improve the work productivity of internal employees - by far the most popular use cases are assistants for customer support, marketing copywriting, co-piloting, and more.
If 2023 was the year of testing generative AI, this year is the stage where generative AI becomes a reality.
Allganize is a B2B AI company that creates industry-specific models with open-source model-based fine tuning, and delivers the market-leading technology to help companies successfully deploy Gen AI without hallucinations and errors and costly delays and fine tuning.
B2B AI solutions are evolving to bring LLMs in the workplace to solve problems and produce sophisticated and accurate results. The art of finding answers in complex tables of corporate documents is an area where Allganize is leading the market and has set itself apart from OpenAI.
AllAge's Ali LLM app market, which allows you to select and use various LLMs to suit your company's work, and allows you to use more than 100 work automation tools at once, is also evolving towards full-stack AI tools.
If you're curious about AI-native workflow tools, contact Allganize!
Learn more about LLM apps for businesses you can start using today