Retool, a development platform for business software, recently published the results of its State of AI survey. Over 1,500 people took part, all from the tech industry:...
Over half of all tech industry workers view AI as overrated::undefined
I’ve found that relying on it is a mistake anyhow, the amount of incorrect information I’ve seen from chatgpt has been crazy. It’s not a bad thing to get started with it but it’s like reading a grade school kids homework, you need to proofread the heck out of it.
That 80% also doesn’t take that much effort. Automation can still be helpful depending on how much effort it is to repeatedly do it, but that 20% is really where we need to see progress for a massive innovation to happen.
I actually disagree. Ai is great at doing the parts that are easy to do mentally but still take time to do. This “fancy autocomplete” is where it shines and can accelerate the work of a professional by an order of magnitude
I have found that it’s like having a junior programmer assistant. It’s great for “write me python code for opening an in file from a command line argument, reading the contents into a key/value dict array, then closing the file.” It’s terrible for “write me a python code for pulling data into a redis database.”
I find it’s wrong 50% of the time for certain command line switches, Linux file structure, and aws cli.
I find it’s terrible for advanced stuff like, “using aws cli and jq, take all volumes in a vpc, and display the volume id, volume size in gb, instance id it’s attached to, private IP address of the instance, whether is a gp3 or gp2, and the vpc id in a comma separated format, sorted by volume size.”
Even worse at, “take all my gp2 volumes and make them gp3.”
I recently used it to update my resume with great success. But I also didn’t just blindly trust it.
Gave it my resume and then asked it to edit my resume to more closely align with a guide I found on Harvards website. Gave it the guide as well and it spit out a version of mine that much more closely resembled the provided guide.
Spent roughly 5 minutes editing the new version to correct for any problems it had and boom. Half an hour of worked parsed down to sub 10
I then had it use my new resume (I gave it a copy of the edited version) and asked it to write me a cover letter for a job (I provided the job description)
Boom. Cover letter. I spent about 10 minutes editing that piece. And then that new resume and cover letter lead to an interview and subsequent job offer.
No. It’s not and hasn’t been for at least a year. Maybe the ai your dealing with is, but it’s shown understanding of concepts in ways that make no sense for how it was created.
Gotta go.
It does a shockingly good analogue of “understanding” at the very least. Have you tried asking chatgpt to solve analogies? Those show up in all kinds of intelligence tests.
We don’t have agi, definitely, but this stuff has come a very long way and it’s quite close to being genuinely useful.
Even if we completely reject the “it’s ai,” we more or less have a natural language interface for computers that isn’t a shallow trick and that’s awesome.
Well here’s the question. Is it solving them, or just regurgitating the answer? If it solves them it should be able to accurately solve completely novel analogies.
yes, it’s has. the most famous example is the stacking of the laptop and the markers. you may not have access but it’s about to eclipse us imho. I’m no technological fanboy either. 20 years ago I argued that I wouldn’t be possible to understand human speech. now that is a everyday occurrence.
As you can see, the past year has shed a lot of light on the topic.
One of my favorite facts is that it takes on average 17 years before discoveries in research find their way to the average practitioner in the medical field. While tech as a discipline may be more quick to update itself, it’s still not sub-12 months, and as a result a lot of people are continuing to confidently parrot things that have recently been shown in research circles to be BS.
Best assessment I’ve heard: Current AI is an aggressive autocomplete.
I’ve found that relying on it is a mistake anyhow, the amount of incorrect information I’ve seen from chatgpt has been crazy. It’s not a bad thing to get started with it but it’s like reading a grade school kids homework, you need to proofread the heck out of it.
I feel like the AI in self-driving cars is the same way. They’re like driving with a 15 year old that just got their learners permit.
Turns out that getting a computer to do 80% of a good job isn’t so great. It’s that extra 20% that makes all the difference.
That 80% also doesn’t take that much effort. Automation can still be helpful depending on how much effort it is to repeatedly do it, but that 20% is really where we need to see progress for a massive innovation to happen.
I actually disagree. Ai is great at doing the parts that are easy to do mentally but still take time to do. This “fancy autocomplete” is where it shines and can accelerate the work of a professional by an order of magnitude
I have found that it’s like having a junior programmer assistant. It’s great for “write me python code for opening an in file from a command line argument, reading the contents into a key/value dict array, then closing the file.” It’s terrible for “write me a python code for pulling data into a redis database.”
I find it’s wrong 50% of the time for certain command line switches, Linux file structure, and aws cli.
I find it’s terrible for advanced stuff like, “using aws cli and jq, take all volumes in a vpc, and display the volume id, volume size in gb, instance id it’s attached to, private IP address of the instance, whether is a gp3 or gp2, and the vpc id in a comma separated format, sorted by volume size.”
Even worse at, “take all my gp2 volumes and make them gp3.”
I recently used it to update my resume with great success. But I also didn’t just blindly trust it.
Gave it my resume and then asked it to edit my resume to more closely align with a guide I found on Harvards website. Gave it the guide as well and it spit out a version of mine that much more closely resembled the provided guide.
Spent roughly 5 minutes editing the new version to correct for any problems it had and boom. Half an hour of worked parsed down to sub 10
I then had it use my new resume (I gave it a copy of the edited version) and asked it to write me a cover letter for a job (I provided the job description)
Boom. Cover letter. I spent about 10 minutes editing that piece. And then that new resume and cover letter lead to an interview and subsequent job offer.
AI is a tool not an all in one solution.
https://www.cs.cornell.edu/info/people/fcc/humor/history.html
I just reviewed a PR today and the code was… bad, like unusually bad for mycoworkers and left some comments.
Then my coworker said he used chatgpt without really thinking on what he was copypasting.
Nice one! I have heard it called a fuzzy JPG of the internet.
And that’s entirely correct
No. It’s not and hasn’t been for at least a year. Maybe the ai your dealing with is, but it’s shown understanding of concepts in ways that make no sense for how it was created. Gotta go.
No it hasn’t.
It does a shockingly good analogue of “understanding” at the very least. Have you tried asking chatgpt to solve analogies? Those show up in all kinds of intelligence tests.
We don’t have agi, definitely, but this stuff has come a very long way and it’s quite close to being genuinely useful.
Even if we completely reject the “it’s ai,” we more or less have a natural language interface for computers that isn’t a shallow trick and that’s awesome.
Well here’s the question. Is it solving them, or just regurgitating the answer? If it solves them it should be able to accurately solve completely novel analogies.
Novel analogies. Very easy to prove this independently for yourself.
yes, it’s has. the most famous example is the stacking of the laptop and the markers. you may not have access but it’s about to eclipse us imho. I’m no technological fanboy either. 20 years ago I argued that I wouldn’t be possible to understand human speech. now that is a everyday occurrence.
Maybe if you Interpret it’s output as such.
Too bad it’s bullshit.
If you are actually interested in the topic, here’s a few good reads:
Do Large Language Models learn world models or just surface statistics? (Jan 2023)
Actually, Othello-GPT Has A Linear Emergent World Representation (Mar 2023)
Eight Things to Know about Large Language Models (April 2023)
Playing chess with large language models (Aug 2023)
Language Models Represent Space and Time (Oct 2023)
The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets (Oct 2023)
As you can see, the past year has shed a lot of light on the topic.
One of my favorite facts is that it takes on average 17 years before discoveries in research find their way to the average practitioner in the medical field. While tech as a discipline may be more quick to update itself, it’s still not sub-12 months, and as a result a lot of people are continuing to confidently parrot things that have recently been shown in research circles to be BS.