Programming

From my experience it's great at doing things that have been done 1000x before (which makes sense given the training data), but when it comes to building something novel it really struggles, especially if there's 3rd party libraries involved that aren't commonly used. So you end up spending a lot of time and money hand holding it through things that likely would have been quicker to do yourself.

I'm pretty sure every time you use AI for programming your brain atrophies a little, even if you're just looking something up. There's value in the struggle.

So they can definitely speed you up, but be careful how you use it. There's no value in a programmer who can only blindly recite LLM output.

There's a balance to be struck in there somewhere, and I'm still figuring it out.

You can either spend your time generating prompts, tweaking them until you get what you want and then using more prompts to refining the code until you end up with something that does what you want...

or you can just fucking write it yourself. And there's the bonus of understanding how it works.

AI is probably fine for generating boiler plate code or repetitive simple stuff, but personally I wouldn't trust it any further than that.

In the grand scheme of things, I think AI code generators make people less efficient. Some studies have come out that indicate this. I've tried to use various AI tools, as I do like fields of AI/ML in general, but they would end up hampering my work in various ways.

I don't see how it could be more effecient to have AI generate something that you then have to review and make sure actually works over just writing the code yourself, unless you don't know enough to code it yourself and just accept the AI generated code as-is without further review.

I'm enjoying it, mostly. It's definitely great at some tasks and terrible at orhers. You get a feel for what those are after a while:

1. Throwaway projects - proof of concepts, one-off static websites, that kind of thing: absolutely ideal. Weeks of dev becomes hours, and you barely need to bother reviewing it if it works.

2. Research (find a tool for doing XYZ) where you barely know the right search terms: ideal. The research mode on claude.ai is especially amazing at this.

3. Anything where the language is unfamiliar. AI bootstraps past most of the learning curve. Doesn't help you learn much, but sometimes you don't care about learning the codebase layout and you just need to fix something.

4. Any medium sized project with a detailed up front description.

What it's not good for:

1. Debugging in a complex system
2. Tiny projects (one line change), faster to do it yourself
3. Large projects (500+ line change) - the diff becomes unreviewable fairly quickly and can't be trusted (much worse than the same problem with a human where you can at least trust the intent)

I'm not against AI use in software development... But you need to understand what the tools you use actually do.

An LLM is not a dev. It doesn't have the capability to think on a problem and come up with a solution. If you use an LLM as a dev, you are an idiot pressing buttons on a black box you understand nothing about.

An LLM is a predictive tool. So use it as a predictive tool.

• Boilerplate code? It can do that, yeah. I don't like to use it that way, but it can do that.
• Implementing a new feature? Maybe, if you're lucky, it has been trained on enough data that it can put something together. But you need to consider its output completely untrustworthy, and therefore it will require so much reviewing that it's just better to write it yourself in the first place.
• Implementing something that solves a problem not solved before? Just don't. Use your own brain, for fuck's sake. That's what you have been trained on.

The one use of AI, at the moment, that I actually like and actually improves my workflow is JetBrains' full line completion AI. It very often accurately predicts what I want to write when it's boilerplate-ish, and shuts up when I write something original.

From my experience it’s really great at bootstrapping new projects for you. It’s good at getting you sample files and if you’re using cursor just building out a sample project.

It’s decent at being an alternative to google/SO for syntax or previously encountered errors. There’s a few things it hallucinates but generally it can save time as long as you don’t trust it blindly.

It struggles when you give it complex tasks or not-straightforward items. Or things that require a lot of domain knowledge. I once wanted to see what css classes were still in use across a handful of react components and it just shat the bed.

The people who champion AI as a human replacement will build a quick proof of concept with it and proclaim “oh shit this is awesome!” And not realize that that’s the easy part of software engineering.

The truth, as always, is in the middle. Plenty of programmers hate it, and plenty of programmers like it. It just so happens that these kinds of spaces attracts more or the hate crowd, and paid shill articles attract more or the like crowd ¯⁠\⁠_⁠(⁠ツ⁠)⁠_⁠/⁠¯

The hate is ridiculous as is the hype.

It's a new tool that is often useful when used correctly. Don't use it to write entire applications - that's a recipe for disaster.

But if you're learning a new language it's amazing. You have an infinitely patient and immediately available tutor that can teach you a language's syntax, best practices, etc. I don't know why that would make you "ill" besides all the shame "real developers" seem to want to lump on anybody who uses AI. If you're not concerned about passing some "I don't use an IDE" nerd's purity test you'll be fine.

I use it mainly to tweak things I can't be bothered to dig into, like Jekyll or Wordpress templates. A few times I let it run and do a major refactor of some async back-end code. It botched the whole thing. Fortunately, easy to rewind everything from remote git repo.

Last week I started a brand new project, thought I'd have it write the boilerplate starter code. Described in detail what I was looking for. It sat there for ten minutes saying 'Thinking' and nothing happened. Killed it and created it myself. This was with Cursor using Claude. I've noticed it's gotten worse lately, maybe because of the increased costs.

Its an absolute gamechanger, IMO - the research phase of any task is reduced to effectively nothing, and I get massive amounts of work done when I walk away from my desk, because I plan for and keep lists of longer tasks to accomplish during those times.

You need to review every line of code it writes, but that's no different than it ever was when working with junior devs 🤷‍♂️ but now I get the code in minutes instead of weeks and the agents actually react to my comments.

We're using this with a massive monorepo containing hundreds of thousands of lines of code, and in tiny tool repos that serve exactly one purpose. If our code quality checks and standards werent as strict as they have been for the past decade, I think it wouldn't work well with the monorepo.

The important part is that my company is paying for it - I have no clue what these tools cost. I am definitely more productive, there is absolutely no debate there IMO. Is the extra productivity worth the extra cost? I have literally no idea.

Not total bullshit, but it's not great for all use cases:

For coding tasks the output looks good on the surface but often I end up changing stuff, meaning it would have been faster up do myself.

For coding I know little about (currently writing some GitHub actions), it's great at explaining alternatives, pros and cons, to give me a rudimentary understanding of stuff

I've also used it to transcribe tutorial screencasts, and then afterwards having a secondary LLM use the transcription to generate documentation (include in prompt: "when relevant, generate examples, use markdown tables, generate plantuml etc)

Yes. But I'm not paying for premium like some of my cowokres. I use it to avoid the grunt work, and to avoid things I know I'd have to google.

I used some coworkers account for a while and auto complete is amazing. I it guesses wrong you just keep tipping as usual. If its right, hit tab and saves you like 20 seconds.

On the other hand I have cokowkers that do not check the chatgpt output and the PRs make no sense. Example: instead of making a variable type any (which is forbidden in our codebase) they did

Let a : date|number|string|object|(...) = fetchData()

Definitely depends on the sub-sector of the industry you're in. There's no shortage of stories of people who swear by it, or who are having it forced on them by management.

Me personally' I've never wanted or been pressured to use it, and I work for a company with "AI" in the damn name. To be fair, though, this company was around doing general machine-learning stuff before the current LLM craze exploded. Also, I work with a small team that was only bought by this company a few years ago, and thus far has been allowed to remain practically independent. Also also, the business domain my team works in is finance and accounting, where there's bot much practical application for ML, and where you REALLY can't afford to fuck around and find out, with business data.

My favorite use is actually just to help me name stuff. Give it a short description of what the thing does and get a list of decent names. Refine if they're all missing something.

Also useful for finding things quickly in generated documentation, by attaching the documentation as context. And I use it when trying to remember some of the more obscure syntax stuff.

As for coding assistants, they can help quickly fill in boilerplate or maybe autocomplete a line or two. I don't use it for generating whole functions or anything larger.

So I get some nice marginal benefits out of it. I definitely like it. It's got a ways to go before it replaces the programming part of my job, though.

Not a programmer, but I used it at my last job to get over humps where I was stuck on PowerShell scripts. AI can show you a path you didn't know or hadn't thought about. The developers seemed to be using it the same way. Great tool if you don't completely lean on it and you know enough to judge the output.

I've had good luck having it write simple scripts that I could easily handle myself. For example, I needed a script to chop a directory full of log files up into archives, with some constraints. That sort of thing.
I haven't tried it on anything more substantial.
This was using Copilot because I haven't found a good coding model that will run locally on 16GB VRAM.

I’m somewhat new to the field ~1.5 years, so my opinion doesn’t hold too much weight.

But in the embedded field I’ve found AI to not be as helpful as it seems to be for many others. The one BIG thing is has helped me with is I can give it a data sheet and it’ll spit out all the register fields that I need, or help me quickly find information that I’m too lazy to Ctrl-f, which saves a couple minutes.

It has not proven it’s worth when it comes to the firmware itself. I’ve tried to get it to instantiate some peripheral instances and they never ended up working, no matter how I phrased the prompt or what context i’ve given it.

I use it now and again but not integrated into an ide and not to write large bits of code.

My uses are like so

Rewrite this rant to shut the PO/PM up. Explain why this is a waste of time

Convert this excel row into a custom model.

Given these tables give me the sql to do xyz.

Sometimes for troubleshooting an environment issue

Do I need it , no. But if it saves me some time on bullshit tasks then thats more time for me

I used supermaven (copilot competitor) for awhile and it was sorta ok sometimes, but I turned it off when I realized I’d forgotten how to write a switch case. Autocomplete doesn’t know your intent, so it introduces a lot of noise that I prefer to do without.

I’ve been trying out Claude code for a couple months and I think I like it ok for some tasks. If you use it to do your typing rather than your thinking, then it’s pretty decent. Give it small tasks with detailed instructions and you generally get good results. The problem is that it’s most tempting to use when you don’t have the problem figured out and you’re hoping it will, but thats when it gives you overconvoluted garbage. About half the time this garbage is more useful than starting from scratch.

It’s good at sorting out boilerplate and following explicit patterns that you’ve already created. It’s not good at inventing and implementing those patterns in the first place.

I use it to vet ideas, concepts, approaches, and paradigms. It's amazing for rubber ducking. I don't use it for wholesale code gen though.

And as a documentation companion it's pretty rad. Not always right but generally gets things in the correct direction.

Some definitely are. But I think a lot aren't. Hell, a lot of programmers still don't even use an IDE.

I don't know why it would make you ill.

I use llms from both ends. It helps me plan an think through complex code architecture and helps me do the little stuff i do too infrequent to remember. Putting it all together is usually all me.

I use it sparingly and only to automate things I know how to do very well, so reviewing its work become easier.

I've been using phind as a technical-focused AI search engine, which is a great addition to my toolset.

I'm mindful of using it vs searching [ref docs etc], not only in the kind of search and answer I'm looking for but also energy consumption impact, but it's definitely very useful. I'm a senior dev though, and know what to expect and I am able to assess plausibility, and phind provides sources I can inspect too.

As for code assistance, I find it plausible that it can be useful, even if from my personal experience I'm skeptical.

I watched an Microsoft talk from two devs, which was technically sound and plausible in that it was not just marketing but they talked about their experience, including limits of AI, and where they had to and to what degree they had to deal with hallucinations and cleanup. They talked about where they see usefulness in AI. They were both senior, and able to assess plausibility, and do corrections where necessary. What I remember; they used it to bounce ideas back and forth, to do an implementation draft they then go over and complete, etc.

Microsoft can do the investment of AI setup, code sharing to model, AI instructions/meta-description setup investment, etc.

My personal experience was in using copilot for Rust code, for Nushell plugins. I'm not very familiar with Rust, and it was very confusing, and with a lot of hallucinations.

The PR descriptions CodeRabbit did were verbose and not useful for smaller PRs I made. That has been a while ago.

At work we have a voluntary work group exploring AI. The whole generate your whole app kind of thing seems plausible for UI prototypes. But nothing more. And for that it's probably expensive.

I'm not sure how much the whole thing does or can do for efficiency. Seems situational - in terms of environment, setup, capabilities, and kind of work and approach.