Please remove it if unallowed
I see alot of people in here who get mad at AI generated code and I am wondering why. I wrote a couple of bash scripts with the help of chatGPT and if anything, I think its great.
Now, I obviously didnt tell it to write the entire code by itself. That would be a horrible idea, instead, I would ask it questions along the way and test its output before putting it in my scripts.
I am fairly competent in writing programs. I know how and when to use arrays, loops, functions, conditionals, etc. I just dont know anything about bash’s syntax. Now, I could have used any other languages I knew but chose bash because it made the most sense, that bash is shipped with most linux distros out of the box and one does not have to install another interpreter/compiler for another language. I dont like Bash because of its, dare I say weird syntax but it made the most sense for my purpose so I chose it. Also I have not written anything of this complexity before in Bash, just a bunch of commands in multiple seperate lines so that I dont have to type those one after another. But this one required many rather advanced features. I was not motivated to learn Bash, I just wanted to put my idea into action.
I did start with internet search. But guides I found were lacking. I could not find how to pass values into the function and return from a function easily, or removing trailing slash from directory path or how to loop over array or how to catch errors that occured in previous command or how to seperate letter and number from a string, etc.
That is where chatGPT helped greatly. I would ask chatGPT to write these pieces of code whenever I encountered them, then test its code with various input to see if it works as expected. If not, I would ask it again with what case failed and it would revise the code before I put it in my scripts.
Thanks to chatGPT, someone who has 0 knowledge about bash can write bash easily and quickly that is fairly advanced. I dont think it would take this quick to write what I wrote if I had to do it the old fashioned way, I would eventually write it but it would take far too long. Thanks to chatGPT I can just write all this quickly and forget about it. If I want to learn Bash and am motivated, I would certainly take time to learn it in a nice way.
What do you think? What negative experience do you have with AI chatbots that made you hate them?
That is the general reason, i use llms to help myself with everything including coding too, even though i know why it’s badIf you’re not an experienced developer, it could be used as a crutch rather than actually learning how to write the code.
The real reason? People are just fed up with AI in general (that has no real-world use to most people) being crammed down their throats and having their personal code (and other data) being used to train models for megacorps.
There are probably legitimate uses out there for gen AI, but all the money people have such a hard-on for the unethical uses that now it’s impossible for me to hear about AI without an automatic “ugggghhhhh” reaction.
A lot of the criticism comes with AI results being wrong a lot of the time, while sounding convincingly correct. In software, things that appear to be correct but are subtly wrong leads to errors that can be difficult to decipher.
Imagine that your AI was trained on StackOverflow results. It learns from the questions as well as the answers, but the questions will often include snippets of code that just don’t work.
The workflow of using AI resembles something like the relationship between a junior and senior developer. The junior/AI generates code from a spec/prompt, and then the senior/prompter inspects the code for errors. If we remove the junior from the equation to replace with AI, then entry level developer jobs are slashed, and at the same time people aren’t getting the experience required to get to the senior level.
Generally speaking, programmers like to program (many do it just for fun), and many dislike review. AI removes the programming from the equation in favour of review.
Another argument would be that if I generate code that I have to take time to review and figure out what might be wrong with it, it might just be quicker and easier to write it correctly the first time
Business often doesn’t understand these subtleties. There’s a ton of money being shovelled into AI right now. Not only for developing new models, but for marketing AI as a solution to business problems. A greedy executive that’s only looking at the bottom line and doesn’t understand the solution might be eager to implement AI in order to cut jobs. Everyone suffers when jobs are eliminated this way, and the product rarely improves.
Generally speaking, programmers like to program (many do it just for fun), and many dislike review. AI removes the programming from the equation in favour of review.
This really resonated with me and is an excellent point. I’m going to have to remember that one.
Two reasons:
- my company doesn’t allow it - my boss is worried about our IP getting leaked
- I find them more work than they’re worth - I’m a senior dev, and it would take longer for me to write the prompt than just write the code
I just dont know anything about bash’s syntax
That probably won’t be the last time you write Bash, so do you really want to go through AI every time you need to write a Bash script? Bash syntax is pretty simple, especially if you understand the basic concept that everything is a command (i.e. syntax is
<command> [arguments...]; likeif <condition>where<condition>can be[ <special syntax> ]or[[ <test syntax> ]]), which explains some of the weird corners of the syntax.AI sucks for anything that needs to be maintained. If it’s a one-off, sure, use AI. But if you’re writing a script others on your team will use, it’s worth taking the time to actually understand what it’s doing (instead of just briefly reading through the output). You never know if it’ll fail on another machine if it has a different set of dependencies or something.
What negative experience do you have with AI chatbots that made you hate them?
I just find dealing with them to take more time than just doing the work myself. I’ve done a lot of Bash in my career (>10 years), so I can generally get 90% of the way there by just brain-dumping what I want to do and maybe looking up 1-2 commands. As such, I think it’s worth it for any dev to take the time to learn their tools properly so the next time will be that much faster. If you rely on AI too much, it’ll become a crutch and you’ll be functionally useless w/o it.
I did an interview with a candidate who asked if they could use AI, and we allowed it. They ended up making (and missing) the same mistake twice in the same interview because they didn’t seem to actually understand what the AI output. I’ve messed around with code chatbots, and my experience is that I generally have to spend quite a bit of time to get what I want, and then I still need to modify and debug it. Why would I do that when I can spend the same amount of time and just write the code myself? I’d understand the code better if I did it myself, which would make debugging way easier.
Anyway, I just don’t find it actually helpful. It can feel helpful because it gets you from 0 to a bunch of code really quickly, but that code will probably need quite a bit of modification anyway. I’d rather just DIY and not faff about with AI.
If you’re a seasoned developer who’s using it to boilerplate / template something and you’re confident you can go in after it and fix anything wrong with it, it’s fine.
The problem is it’s used often by beginners or people who aren’t experienced in whatever language they’re writing, to the point that they won’t even understand what’s wrong with it.
If you’re trying to learn to code or code in a new language, would you try to learn from somebody who has only half a clue what he’s doing and will confidently tell you things that are objectively wrong? Thats much worse than just learning to do it properly yourself.
We built a Durable task workflow engine to manage infrastructure and we asked a new hire to add a small feature to it.
I checked on them later and they expressed they were stuck on an aspect of the change.
I could tell the code was ChatGPT. I asked “you wrote this with ChatGPT didn’t you?” And they asked how I could tell.
I explained that ChatGPT doesn’t have the full context and will send you on tangents like it has here.
I gave them the docs to the engine and to the integration point and said "try using only these and ask me questions if you’re stuck for more than 40min.
They went on to become a very strong contributor and no longer uses ChatGPT or copilot.
I’ve tried it myself and it gives me the wrong answers 90% of the time. It could be useful though. If they changed ChatGPT to find and link you docs it finds relevant I would love it but it never does even when asked.
Phind is better about linking sources. I’ve found that generated code sometimes points me in the right direction, but other times it leads me down a rabbit hole of obsolete syntax or other problems.
Ironically, if you already are familiar with the code then you can easily tell where the LLM went wrong and adapt their generated code.
But I don’t use it much because its almost more trouble than its worth.
Lots of good comments here. I think there’s many reasons, but AI in general is being quite hated on. It’s sad to me - pre-GPT I literally researched how AI can be used to help people be more creative and support human workflows, but our pipelines around the AI are lacking right now. As for the hate, here’s a few perspectives:
- Training data is questionable/debatable ethics,
- Amateur programmers don’t build up the same “code muscle memory”,
- It’s being treated as a sole author (generate all of this code for me) instead of like a ping-pong pair programmer,
- The time saved writing code isn’t being used to review and test the code more carefully than it was before,
- The AI is being used for problem solving, where it’s not ideal, as opposed to code-from-spec where it’s much better,
- Non-Local AI is scraping your (often confidential) data,
- Environmental impact of the use of massive remote LLMs,
- Can be used (according to execs, anyways) to replace entry level developers,
- Devs can have too much faith in the output because they have weak code review skills compared to their code writing skills,
- New programmers can bypass their learning and get an unrealistic perspective of their understanding; this one is most egregious to me as a CS professor, where students and new programmers often think the final answer is what’s important and don’t see the skills they strengthen along the way to the answer.
I like coding with local LLMs and asking occasional questions to larger ones, but the code on larger code bases (with these small, local models) is often pretty non-sensical, but improves with the right approach. Provide it documented functions, examples of a strong and consistent code style, write your test cases in advance so you can verify the outputs, use it as an extension of IDE capabilities (like generating repetitive lines) rather than replacing your problem solving.
I think there is a lot of reasons to hate on it, but I think it’s because the reasons to use it effectively are still being figured out.
Some of my academic colleagues still hate IDEs because tab completion, fast compilers, in-line documentation, and automated code linting (to them) means you don’t really need to know anything or follow any good practices, your editor will do it all for you, so you should just use vim or notepad. It’ll take time to adopt and adapt.
Now, I obviously didnt tell it to write the entire code by itself. […]
I am fairly competent in writing programs.
Go ahead using it. You are safe.
My objections:
- It doesn’t adequately indicate “confidence”. It could return “foo” or “!foo” just as easily, and if that’s one term in a nested structure, you could spend hours chasing it.
- So many hallucinations-- inventing methods and fields from nowhere, even in an IDE where they’re tagged and searchable.
Instead of writing the code now, you end up having to review and debug it, which is more work IMO.
I have a coworker who is essentially building a custom program in Sheets using AppScript, and has been using CGPT/Gemini the whole way.
While this person has a basic grasp of the fundamentals, there’s a lot of missing information that gets filled in by the bots. Ultimately after enough fiddling, it will spit out usable code that works how it’s supposed to, but honestly it ends up taking significantly longer to guide the bot into making just the right solution for a given problem. Not to mention the code is just a mess - even though it works there’s no real consistency since it’s built across prompts.
I’m confident that in this case and likely in plenty of other cases like it, the amount of time it takes to learn how to ask the bot the right questions in totality would be better spent just reading the documentation for whatever language is being used. At that point it might be worth it to spit out simple code that can be easily debugged.
Ultimately, it just feels like you’re offloading complexity from one layer to the next, and in so doing quickly acquiring tech debt.
Exactly my experience as well. Using AI will take about the same amount of time as just doing it myself, but at least I’ll understand the code at the end if I do it myself. Even if AI was a little faster to get working code, writing it yourself will pay off in debugging later.
And honestly, I enjoy writing code more than chatting with a bot. So if the time spent is going to be similar, I’m going to lean toward DIY every time.
- AI Code suggestions will guide you to making less secure code, not to mention often being lower quality in other ways.
- AI code is designed to look like it fits, not be correct. Sometimes it is correct. Sometimes it’s close but has small errors. Sometimes it looks right but is significantly wrong. Personally I’ve never gotten ChatGPT to write code without significant errors for more than trivially small test cases.
- You aren’t learning as much when you have ChatGPT do it for you, and what you do learn is “this is what chat gpt did and it worked last time” and not “this is what the problem is and last time this is the solution I came up with and this is why that worked”. In the second case you are far better equipped to tackle future problems, which won’t be exactly the same.
All that being said, I do think there is a place for chat GPT in simple queries like asking about syntax for a language you don’t know. But take every answer it gives you with a grain of salt. And if you can find documentation I’d trust that a lot more.
Also if you are interested, here are those scripts I wrote with chatGPT
I’ve found it to be extremely helpful in coding. Instead of trying to read huge documentation pages, I can just have a chatbot read it and tell me the answer. My coworker has been wanting to learn Powershell. Using a chatbot, his understanding of the language has greatly improved. A chatbot can not only give you the answer, but it can break down how it reached that conclusion. It can be a very useful learning tool.
I agree AI is a godsend for non coders and amateur programmers who need a quick and dirty script. As a professional, the quality of code is oftentimes 💩 and I can write it myself in less time than it takes to describe it to an AI.
Keep in mind that at the core of an LLM is it being a probability autocompletion mechanism using the vast training data is was fed. A fine tuned coding LLM would have data more in line to suit an output of coding solutions. So when you ask for generation of code for very specific purposes, it’s much more likely to find a mesh of matches that will work well most of the time. Be more generic in your request, and you could get all sorts of things, some that even look good at first glance but have flaws that will break them. The LLM doesn’t understand the code it gives you, nor can it reason if it will function.
Think of an analogy where you Googled a coding question and took the first twenty hits, and merged all the results together to give an answer. An LLM does a better job that this, but the idea is similar. If the data it was trained on was flawed from the beginning, such as what some of the hits you might find on Reddit or Stack Overflow, how can it possibly give you perfect results every time? The analogy is also why a much narrow query for coding may work more often - if you Google a niche question you will find more accurate, or at least more relevant results than if you just try a general search and past together anything that looks close.
Basically, if you can help the LLM hone in its probabilities on the better data from the start, you’re more likely to get what may be good code.
