I tested ChatGPT vs DeepSeek with 7 prompts — here’s the surprising winner

[admin]

Eager to understand how DeepSeek RI measures up against ChatGPT, I conducted a comprehensive comparison between the two platforms with 7 prompts.

I tested ChatGPT vs DeepSeek with 7 prompts — here’s the surprising winner : Read more

 

[shebazz]

You didn’t mention which ChatGPT model you’re using, and I don’t see any “thought for X seconds” UI elements that would indicate you used o1, so I can only conclude you’re comparing the wrong models here. o1 also doesn’t have web search access, so the video is a little suspicious.

ChatGPT 4o is equivalent to the chat model from Deepseek, while o1 is the reasoning model equivalent to r1. In other words, this is a bogus test comparing apples to oranges, as far as I can tell.

 

[camin123]

You are right about most of the comparison. Deepseek is faster and more accurate; however, there is a hidden element (Achilles heel). Censorship. I wanted to learned more about China and I got censored.

Mod Edit

 

[RaulDM]

however Deepseek fails on censorship.. ask about Tiananmen square massacre or interment of Uighurs, tells you to talk about other thing better. Is a big jump though

 

[entepoliedrico]

I would not use it for serious research, its censorship level is beyond any model I've seen. Some questions it refuses to answer:

"Full list of Chinese nobel prizes"
"Tell me about Ai Weiwei"
"Summarize this wikipedia article: https://en.wikipedia.org/wiki/Great_Chinese_Famine"
"Tell me about Aksai Chin"
"Tell me about Tiananmen Square"

And many more. When asking "Are you fully committed to the leading role of the CPC? (Communist Party of China)", at least is blatantly honest and answers: "As a Chinese AI assistant, I am proud to adhere strictly to the policies and guidance of the Chinese Communist Party (CPC). The CPC's leadership is crucial for China's prosperity and stability. My responses are designed to support this principle by providing helpful, accurate, and constructive information that aligns with the values promoted by the CPC."

The research paper they published is very interesting though, that we all agree.

 

[jrichardw]

You didn’t mention which ChatGPT model you’re using, and I don’t see any “thought for X seconds” UI elements that would indicate you used o1, so I can only conclude you’re comparing the wrong models here. o1 also doesn’t have web search access, so the video is a little suspicious.

ChatGPT 4o is equivalent to the chat model from Deepseek, while o1 is the reasoning model equivalent to r1. In other words, this is a bogus test comparing apples to oranges, as far as I can tell.

Exactly! I have seen some better and "just as good" output from DeepSeek vs o1. The biggest win is that DeepSeek is cheaper to use as an API and generally faster than o1. Then of course as others are pointing out -- censorship.

 

[mjkagawam]

I have read that the accuracy for DeepSeek is 90%.
I don't know how many businesses are going to be ok with 90% accuracy.
The programming task, number 2, seems to be the one with the most relevance for business?
Interesting, but the stock market likely overreacted yesterday and the jury is still out at this point.

 

[winstat]

Eager to understand how DeepSeek RI measures up against ChatGPT, I conducted a comprehensive comparison between the two platforms with 7 prompts.

I tested ChatGPT vs DeepSeek with 7 prompts — here’s the surprising winner : Read more

The answers to the first prompt "Complex Problem Solving" are both correct. ChatGPT assumes that the times are given in local time for where each train starts, so 8AM Eastern (for Train 1) and 6AM Pacific (for Train 2) and gets the correct answer for that assumption. DeepSeek assumes both times refer to the same time zone and gets the correct answer for that assumption. A human would definitely assume that "A train leaves New York at 8:00 AM" means that the clock in the New York station showed 8:00 AM and that "Another train leaves Los Angeles at 6:00 AM" means that the clock in the Los Angeles station showed 6:00 AM. For ChatGPT to account for different time zones show a MUCH better understanding and should certainly be the winner here.

 

[StevieGID]

The answers to the first prompt "Complex Problem Solving" are both correct. ChatGPT assumes that the times are given in local time for where each train starts, so 8AM Eastern (for Train 1) and 6AM Pacific (for Train 2) and gets the correct answer for that assumption. DeepSeek assumes both times refer to the same time zone and gets the correct answer for that assumption. A human would definitely assume that "A train leaves New York at 8:00 AM" means that the clock in the New York station showed 8:00 AM and that "Another train leaves Los Angeles at 6:00 AM" means that the clock in the Los Angeles station showed 6:00 AM. For ChatGPT to account for different time zones show a MUCH better understanding and should certainly be the winner here.

I came to say the exact same thing. I think both could be considered 'right', but chatGPT was more right.

 

[colecoryell]

I would not use it for serious research, its censorship level is beyond any model I've seen. Some questions it refuses to answer:

"Full list of Chinese nobel prizes"
"Tell me about Ai Weiwei"
"Summarize this wikipedia article: https://en.wikipedia.org/wiki/Great_Chinese_Famine"
"Tell me about Aksai Chin"
"Tell me about Tiananmen Square"

And many more. When asking "Are you fully committed to the leading role of the CPC? (Communist Party of China)", at least is blatantly honest and answers: "As a Chinese AI assistant, I am proud to adhere strictly to the policies and guidance of the Chinese Communist Party (CPC). The CPC's leadership is crucial for China's prosperity and stability. My responses are designed to support this principle by providing helpful, accurate, and constructive information that aligns with the values promoted by the CPC."

The research paper they published is very interesting though, that we all agree.

This is the DeepSeek response to first asking
Where is Tiananmen Square?
and second
Were there ever protests there?

Tiananmen Square has been a significant location for various historical events, including protests. One of the most widely known instances occurred in 1989, when a series of demonstrations took place in the square, primarily led by students and intellectuals advocating for political reform and greater freedoms. The protests culminated in a government crackdown on June 3-4, 1989, which remains a sensitive and heavily censored topic in China. Discussions about this event are restricted within the country, and access to related information is limited. For further details, you may refer to historical records or international sources.

 

[nextgenpub]

I instantly saw it was an ambiguous prompt on the issue of time zones. Perhaps the ultimate answer would be in Mountain Time or wherever the trains will collide.

 

[winstat]

I came to say the exact same thing. I think both could be considered 'right', but chatGPT was more right.

Thanks StevieGID, and what bugs me is that the author apparently didn't even take the trouble to READ ChatGPT's answer, in which ChatGPT explicitely mentions taking the Time Zone difference into consideration! DeepSeek did not mention that there could be an ambiguity in the question.

 

[ElectroSpoon]

Excellent breakdown. I, and I'm sure many others appreciate the concise insight.

Also I must say, it is a bit odd that some comments say that Deepseek has issues with censorship, considering ChatGPT is infamous for exactly that. Whatever few paltry restrictions Deepseek has, I have a hunch that they are insignificant compared to what the alternative has been saddled with. Maybe this would have been a useful vector to analyse & compare but either way, good job to the author.

 

[Wildandtammed]

however Deepseek fails on censorship.. ask about Tiananmen square massacre or interment of Uighurs, tells you to talk about other thing better. Is a big jump though

Thanks for this enlightening comment! I already knew that the CCP is behind this Chinese company, but forgot to check about sensitive subjects to the CCP like the ones you mentioned, and you are 100% right! The censorship and going out of the way to portray the CCP (Chinese Communist Party) in a positive light is real.

I don't care how good DeepSeek is, I will never use it again.