this post was submitted on 09 Mar 2024
540 points (95.9% liked)
memes
10449 readers
2574 users here now
Community rules
1. Be civil
No trolling, bigotry or other insulting / annoying behaviour
2. No politics
This is non-politics community. For political memes please go to !politicalmemes@lemmy.world
3. No recent reposts
Check for reposts when posting a meme, you can only repost after 1 month
4. No bots
No bots without the express approval of the mods or the admins
5. No Spam/Ads
No advertisements or spam. This is an instance rule and the only way to live.
Sister communities
- !tenforward@lemmy.world : Star Trek memes, chat and shitposts
- !lemmyshitpost@lemmy.world : Lemmy Shitposts, anything and everything goes.
- !linuxmemes@lemmy.world : Linux themed memes
- !comicstrips@lemmy.world : for those who love comic stories.
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Do those really require ML? For an e-commerce with millions of entries maybe, but for a CLI I don't see it.
"need"? Of course not. Though I do see it being capable of much more sophisticated autocomplete. Like a tab-complete that is aware of what you've already typed in the command and gives you only compatible remaining flags, or could tab-complete information available in the environment, like recognize it's running in Kubernetes and let you tab through running hosts or commands that'd make sense from 'here', etc, etc.
Sure, it's all things a very nice and complicated algorithm could do, but ... that's all "AI" thus far. There have been zero actual artificial intelligences created.
So I didn't do it in the CLI direct, but I had a whole lot of files in a collection that obviously had duplicates.
So I first used fdupes, which got a lot of them. But there were a lot of duplicates still. I noted a bunch were identifiable by having identical file size, but just some different metadata, so I made a quick work of presenting only files with identical stuff and went about reviewing and deleting.
Then I still see a lot of duplicates, because the metadata might be slightly different. Sizes were close, but non dupes also were close. I might have proceeded to write a little something to strip it the metadata to normalize, but decided to feed it to an LLM and ask to identify likely duplicates. It failed to find them all, and erroneously declared duplicates, but it did make the work go faster. Of course in this scenario a missed duplicate isn't a huge deal, so I had to double check their results and might have missed some things, but good enough for the effort.
Sometimes my recall isn't quite good enough for ctrl-r, but maybe an LLM could do better. Of course a better "search engine" also could do well. Also a mind numbingly obvious snippet could be generated without the tedium. Again, having to be careful to reviee because the LLMs are useful, but unreliable.