p.1 #1 · Using Artificial Intelligence like ChatGPT to inform your purchase decsions - warning
We live in a world increasingly dependent upon artificial intelligence (esp Generative AI) to help us make decisions.
I decided to test ChatGPT (via paid Plus plan, using ChatGPT 5.2) as a decision support engine, and I was shocked by the result and advise it gave - a number of recommedations and assertions were factually wrong.
As an exercise I opted for to ask what are the advantages of buying a minty used a7RV over a brand new a7V in Febuary 2026, strengths and weaknesses of each camera, etc.
While it got a lot of data correct and presented it in a good and structured format, but some of the results presented back were just factually wrong?! It stated things like that the RV had better ergonomics (they are the same), that the a7V has "older sensor read out tech" (not true), that the a7RV is more "future prof" (how?), etc.
It failed to bring up things like rolling shutter at all, when asked about it, it stated that the a7V has a non-stacked sensor (most likely mistaking the a7V for the a7IV?!) which is not true either. It kept going on how the a7RV has a better sensor read out tech and perfoms much better on e-shutter than the a7V.
When called out on the errors, multiple times, ChatGPT answered:
What went wrong on my side
I relied on pattern-matching instead of verification.
The a7V is a late-2025 body with a new partially-stacked sensor. I defaulted to older A7-series mental models instead of treating it as a new architecture.
I prioritized “structured comparison” over the decisive factor.
Rolling shutter is the practical differentiator between a7V and a7R V for silent shutter. That should have been paragraph #1, not an afterthought.
I didn’t challenge my own assumptions.
When you said “the a7R V behaves like the a7IV and is terrible in e-shutter,” that should have triggered a reset on my reasoning, not a defense.
You were doing the role I should have done: reality-checking the model.
It completely missed talking about pre-capture, the new processor, faster fps, etc, etc.
As a comparision, I ran the same exact questions throught Google Gemini PRO (also paid verion) and it got the answers and assessment correct, this time at least.
At the end, we all know about AI hallucinations and other issues but unless you actually know the topics or facts, it is very hard to know when GenAI services get it wrong.
My point is simply that one should/must be very careful when relying on any AI service to make infomred purchasing decions, especially if you don't know the topic that well. In this case, ChatGPT got it materially wrong time after time. Imagine asking it for help buying a new or used car, house hold appliances, researching areas to buy a house - whatever you can thing of - and it just guesses and make sh-t up!?
p.1 #4 · Using Artificial Intelligence like ChatGPT to inform your purchase decsions - warning
patotts wrote:
I decided to test ChatGPT (via paid Plus plan, using ChatGPT 5.2) as a decision support engine, and I was shocked by the result and advise it gave - a number of recommedations and assertions were factually wrong.
You really were shocked? I'd be shocked if the software weighted any decisions other than whoever is paying for it to make them money and whatever fools are blabbering the most on the internet. It's still subject to garbage in-garbage out, same as it ever was.
p.1 #5 · Using Artificial Intelligence like ChatGPT to inform your purchase decsions - warning
The problems are diverse.
There are very many layers to a decision of the sort you asked AI to help you with, and how they are ranked and valued is so dependent on the individual buyer’s expectations and needs that asking an AI what product to get is almost useless.
I suspect that it might do better with more general questions along the lines of, to make up an example, the pluses and minuses of using a ILC versusor a smart phone…
There’s another problem, too. There’s no consensus about what camera or brand or model is “best.” Different photographers will swear that Brand A is better than Brand B or that Brand B is better than Brand A. The same holds true for types of cameras and much more. Since AIs build their data bases from material that they can find in the online world, the underlying data for their output is, itself, not definitive about things like this.
p.1 #6 · Using Artificial Intelligence like ChatGPT to inform your purchase decsions - warning
Can you share the prompts you used for each platform?
Thanks
patotts wrote:
We live in a world increasingly dependent upon artificial intelligence (esp Generative AI) to help us make decisions.
I decided to test ChatGPT (via paid Plus plan, using ChatGPT 5.2) as a decision support engine, and I was shocked by the result and advise it gave - a number of recommedations and assertions were factually wrong.
As an exercise I opted for to ask what are the advantages of buying a minty used a7RV over a brand new a7V in Febuary 2026, strengths and weaknesses of each camera, etc.
While it got a lot of data correct and presented it in a good and structured format, but some of the results presented back were just factually wrong?! It stated things like that the RV had better ergonomics (they are the same), that the a7V has "older sensor read out tech" (not true), that the a7RV is more "future prof" (how?), etc.
It failed to bring up things like rolling shutter at all, when asked about it, it stated that the a7V has a non-stacked sensor (most likely mistaking the a7V for the a7IV?!) which is not true either. It kept going on how the a7RV has a better sensor read out tech and perfoms much better on e-shutter than the a7V.
When called out on the errors, multiple times, ChatGPT answered:
What went wrong on my side
I relied on pattern-matching instead of verification.
The a7V is a late-2025 body with a new partially-stacked sensor. I defaulted to older A7-series mental models instead of treating it as a new architecture.
I prioritized “structured comparison” over the decisive factor.
Rolling shutter is the practical differentiator between a7V and a7R V for silent shutter. That should have been paragraph #1, not an afterthought.
I didn’t challenge my own assumptions.
When you said “the a7R V behaves like the a7IV and is terrible in e-shutter,” that should have triggered a reset on my reasoning, not a defense.
You were doing the role I should have done: reality-checking the model.
It completely missed talking about pre-capture, the new processor, faster fps, etc, etc.
As a comparision, I ran the same exact questions throught Google Gemini PRO (also paid verion) and it got the answers and assessment correct, this time at least.
At the end, we all know about AI hallucinations and other issues but unless you actually know the topics or facts, it is very hard to know when GenAI services get it wrong.
My point is simply that one should/must be very careful when relying on any AI service to make infomred purchasing decions, especially if you don't know the topic that well. In this case, ChatGPT got it materially wrong time after time. Imagine asking it for help buying a new or used car, house hold appliances, researching areas to buy a house - whatever you can thing of - and it just guesses and make sh-t up!?
p.1 #8 · Using Artificial Intelligence like ChatGPT to inform your purchase decsions - warning
I surely don't trust these platform blindly, but this wasn't a case of asking it to interpret color science between brands, or anything having to do with tastes, but rather the strength and weaknesses of comparing two Sony cameras.
This is like being a tourist in e.g. Tokyo and someone giving wrong directions when asked instead of admitting that they don't know the way or place you are looking for.
I've been in and aorund machine learning, data analytics, AI etc for 10+ years. I am indeed shocked with a service invents answers, or ChatGPT actually said itself that it "guessed" instead of asking for more questions, clarifications, etc to avoid misinformation or hallucinations.
p.1 #9 · Using Artificial Intelligence like ChatGPT to inform your purchase decsions - warning
AI sucks! I've never gotten a product recommendation from it that I thought was good. I'm really into science and math and the results I get in areas I'm expert in are just factually incorrect. I'd rate it about the same level has an average anonymous Reddit user. It's like the worst aspect of my bank's phone tree combined with Reddit Sh3t-posting. Unfortunately AI is in the wild now and scrambling social media, search engines, scientific papers, our interactions with companies and even human resources!!! I think we are going to be in serous trouble soon! We'll look back in fondness for the days we could chat on FM or Twitters just like we look back in fondness as at the pre-COVID days when a everything didn't cost and arm-and-a-leg and governments in Western Countries didn't suppress free speech. It's going to be like the movie "Brazil!"
p.1 #10 · Using Artificial Intelligence like ChatGPT to inform your purchase decsions - warning
The difficult thing about AI is that sometimes it is very useful, often it is right, and it can provide information and context that you might not otherwise have… but it can also be quite wrong or misleading, and if you don’t already have some degree f expertise in the area about which you query it, you may not understand the context for the information it supplies or catch on when it “hallucinates.” (That’s the technical term of art for when AI systems make up complete bullsh*t, when is a known problem.)
p.1 #11 · Using Artificial Intelligence like ChatGPT to inform your purchase decsions - warning
netexpress wrote:
AI sucks! I've never gotten a product recommendation from it that I thought was good. I'm really into science and math and the results I get in areas I'm expert in are just factually incorrect. I'd rate it about the same level has an average anonymous Reddit user. It's like the worst aspect of my bank's phone tree combined with Reddit Sh3t-posting. Unfortunately AI is in the wild now and scrambling social media, search engines, scientific papers, our interactions with companies and even human resources!!! I think we are going to be in serous trouble soon! We'll look back in fondness for the days we could chat on FM or Twitters just like we look back in fondness as at the pre-COVID days when a everything didn't cost and arm-and-a-leg and governments in Western Countries didn't suppress free speech. It's going to be like the movie "Brazil!" ...Show more →
I say we (common folk) will start paying tributes to the elite and fight for bread scrums, just like in the movie "The Hunger Games".
p.1 #12 · Using Artificial Intelligence like ChatGPT to inform your purchase decsions - warning
guywithgas wrote:
I say we (common folk) will start paying tributes to the elite and fight for bread scrums, just like in the movie "The Hunger Games".
What happens to regular people if the worst fantasies of the AI tech bros are achieved is a serious question.
I’m mainly thinking of employment. If “they” can replace human workers — taxi drivers, computer programmers, help line agents, grocery store checkers, and the list goes on — who will be left with a sufficient income to buy the products and services that the AI people will sell or product?
If the logical endpoint of all of this is to replace as many jobs as possible, humanity is in for a very tough ride.
p.1 #13 · Using Artificial Intelligence like ChatGPT to inform your purchase decsions - warning
I am shocked that you are shocked
SkyNet is pretty decent at tasks that have an objective goodness function. I am surprised how good they are at fairly in depth engineering discussions. Or excellent at coding fairly complicated snippets. Or summarizing text.
But they suck when they have to make decisions in gray areas. Which is what product recommendations are. What is shocking to me is that plenty of smart people think we will soon trust them to make hotel and restaurant reservations and buy plane tickets unsupervised. Supposedly having full access to our information will make them better.
If you believe that I have a bridge for sale. Repeat a simple experiment I did. My liked songs list is about 2000 songs long. I exported it to a text file and uploaded to all of the Terminators. Asked them to recommend songs they have high confidence I will like. To eliminate easy choices and stress the decision process I restricted the choices to songs released within the past 5 years. I could dismiss well over 50% of the recommendations within 2s of hearing them.
p.1 #14 · Using Artificial Intelligence like ChatGPT to inform your purchase decsions - warning
I sitll chose to use the word "shocked" :-)
The reality is that AI, at this point, is more about prediction than intelligence. And that the prediciton alogrithm is skewed for the middle of the bell curve, i.e. trying to produce an answer where there is a high probabiliity that the service produce an answer that you are satisfied with. It is trying not to suck, nor will it ever (at this stage) present any breakthrough thinking.
I guess I am waiting/hoping is a humble AI service. One that admits to not knowing the answer. One that puts a probability rating next to the answer. One that asks for more prompts, inputs or context.
p.1 #15 · Using Artificial Intelligence like ChatGPT to inform your purchase decsions - warning
Given the response from Chat-GPT, one can easily become extremely worried about it's use in many fields where an incorrect response could lead to disastrous results. The jeanie is out of the box and AI will be misused. Some by ill intent people. In other cases, blind acceptance will yield unwanted results. On a positive note AI has shown to be a helpful tool in pouring through mounds of data to help ferret out potential medicine opportunities and the recent review of all Hubble Telescope photos to point out anomalies that have been missed. Unfortunately we are left with determining the good from the bad.
p.1 #16 · Using Artificial Intelligence like ChatGPT to inform your purchase decsions - warning
gdanmitchell wrote:
What happens to regular people if the worst fantasies of the AI tech bros are achieved is a serious question.
I’m mainly thinking of employment. If “they” can replace human workers — taxi drivers, computer programmers, help line agents, grocery store checkers, and the list goes on — who will be left with a sufficient income to buy the products and services that the AI people will sell or product?
If the logical endpoint of all of this is to replace as many jobs as possible, humanity is in for a very tough ride.
p.1 #17 · Using Artificial Intelligence like ChatGPT to inform your purchase decsions - warning
Any information you get from AI needs to be verified before making any important decision based on it. The amount of errors and pure fiction found in most AI responses is terrifying.
p.1 #18 · Using Artificial Intelligence like ChatGPT to inform your purchase decsions - warning
schlotz wrote:
Given the response from Chat-GPT, one can easily become extremely worried about it's use in many fields where an incorrect response could lead to disastrous results. The jeanie is out of the box and AI will be misused. Some by ill intent people. In other cases, blind acceptance will yield unwanted results. On a positive note AI has shown to be a helpful tool in pouring through mounds of data to help ferret out potential medicine opportunities and the recent review of all Hubble Telescope photos to point out anomalies that have been missed. Unfortunately we are left with determining the good from the bad. ...Show more →
I worry a lot about the use of AI tools for things like screening job applicants, etc.
- - -
guywithgas wrote:
I'm afraid, it is "when" not "if"!
p.1 #19 · Using Artificial Intelligence like ChatGPT to inform your purchase decsions - warning
AI is trained on all the bad reviews, videos, social media clips, speculation, rumors, and sprinkle in some actual expert reviews and spec data from the manufacturer. It doesn't know who is right, but it does know how often something is mentioned. If a thousand people without actual hands-on experience on reddit say the ergonomics are bad, AI will assume the one expert that has actually used the camera is wrong.
For a lot of things, it has learned from bad information. Then it'll hallucinate and try to pander an answer you'll be agreeable to.
!!!
Unfortunately AI is in the wild now and scrambling social media, search engines, scientific papers, our interactions with companies and even human resources!!! I think we are going to be in serous trouble soon! We'll look back in fondness for the days we could chat on FM or Twitters just like we look back in fondness as at the pre-COVID days when a everything didn't cost and arm-and-a-leg and governments in Western Countries didn't suppress free speech. It's going to be like the movie "Brazil!"