Claude 3 vs GPT-4 vs Gemini Blitzkrieg, from Coding Skills to Price!

In this article, we’ll compare Antropic's Claude 3, Google's Gemini Ultra, and OpenAI's GPT-4, all of which were released recently. Is OpenAI and ChatGPT losing their advantage?

1. Antropic claims that "Claude 3 has surpassed GPT-4"!

Anthropic’s Claude 3 series of AI models is designed to meet the diverse needs of enterprise customers through a balance of intelligence, speed, and cost-effectiveness. The lineup includes the high-end Opus, the mid-range Sonnet, and the upcoming economical model Haiku.

Antropic CEO Amodei explained that Opus outperforms top AI models such as GPT-4, GPT-3.5, and Gemini Ultra in various benchmarks. It is also said to rank first in academic benchmarks such as GSM-8k for mathematical reasoning and MMLU for expert-level knowledge.

If you look at the chart above, you can see that the Claude 3 Haiku model is better than GPT-4V or Gemini.
Claude 3 has multimodal input, so you can understand text, images, PDFs, and more. It can process more data than GPT-4 (about 150,000 words at a time (200k context window)), and it also comes with improved memory memory with over 99% accuracy.

Let's summarize the features of Claude 3:

• Enables enhanced analytics, forecasting, content creation, and multilingual communication.
• Handles a variety of visual formats, including new vision features, charts and diagrams.
• Leverages sub-agents to perform complex, multimodal analysis.
• Less likely to reject prompts, improved accuracy and recall.

2. Claude 3 vs GPT-4 vs Gemini, let's compare!

1) Comparison of the strengths and limitations of the three models

In response to GPT-4, which has shown the best performance, Claude 3 touts its strengths in OCR and visual data interpretation. Gemini had the upper hand before Claude 3 and now has to compete with the Antrhopic LLM.

Visual image performance is becoming increasingly important because when enterprises deploy AI, they expect it to interpret complex tables, charts, diagrams, and more.

• Claude 3  some text
  • Strengths: Optical character recognition (OCR), nuanced understanding of complex queries, improved benchmark performance, accurate visual recognition such as license plate numbers in images, and analysis of up to 20 images at a time.
  • Limitations: Lack of analysis of low-resolution images, lack of detection of subtle details such as weather conditions in images, learning from data before August 2023, inability to search the latest web.
  • Antropic's claims: Outperform ChatGPT and Gemini in coding and OCR.
     
• GPT-4some text
  • Strengths: Extensive knowledge base, powerful conversational features, excellent performance in a wide range of text-based applications, including writing, summarizing, and question answering, and user-friendly.
  • Limitations: Lagging behind certain technical benchmarks, smaller context window than Claude.
     
• Gemini 1.0 Ultra some text
  • Strengths: Strong performance in vision tasks and general AI functions.
  • Limitations: Lower competitive advantage in OCR area, competing with Claude 3.

2) Comparison of Coding Performance of the Three Models

It's becoming more and more commonplace to use LLMs, including “co-pilots”, in coding. When it comes to coding, it's important to get accurate results, but it's also important to have a good coding style. Understanding the details of programming tasks and being able to execute them in context is a great help for developers.

• Claude 3
  - Significant advances in handling specialized tasks such as complex queries, OCR, and image inference 

- Haiku, Sonnet, and Opus tiering allows users to choose the model that best suits their specific needs, from simple queries to complex analytics.
 

• GPT-4
  - Excellent for creating conversational AI that can engage in detailed discussions, answer a wide range of questions, and generate human-like text.
   
• Gemini
  - Competitive advantage in mixed processing of text and visual information.
  - Compete with Claude 3 in visual information processing, with deeper contextual understanding and improved accuracy.

3) Cost and accessibility

• Claude 3 (price per million input tokens, per million output tokens each)
  - Opus: $15 / $75
  - Sonnet: $3/ $15
  - Haiku: $0.25/ $1.25
   
• GPT-4
  - GPT-4 Turbo: $10 / $30
  - GPT-4: $30 / $60

3. Which LLM should I use now?

Claude 3 can have multiple inputs (images, etc.), but it does not have multiple outputs. This means that it doesn't generate images instead of text. This may be a reflection of the fact that there were errors and hallucinations in the creation of images in Gemini not long ago and the company wanting to play it safe.

However, as mentioned earlier, it is increasingly important to import and interpret increasingly complex tables, charts, and diagrams from PDFs in the use of AI in enterprises, and to produce advanced results. It is particularly interesting to observe the competition between Gemini and Claude 3, as both models excel in OCR Capabilities. 

B2B AI solutions are evolving to combine LLMs in the workplace to solve problems and produce sophisticated and accurate results.
The art of finding answers in complex tables of corporate documents is an area where Allganize is doing better than OpenAI's retriever.

Allganize’s Ali LLM app market, which allows you to decide which LLMs to use to suit your company's work and which includes more than 100 work automation apps, is also rapidly evolving towards becoming a full-stack AI option for enterprises.

If you're curious about AI-native workflow tools, contact Allganize!

Learn more about LLM apps for businesses you can start using today

‍