Why spend 30 seconds manually editing some text when you can spend 30 minutes clobbering together a pipeline involving awk, sed and jq
this post was submitted on 05 Sep 2025
460 points (98.5% liked)
linuxmemes
27191 readers
2960 users here now
Hint: :q!
Sister communities:
Community rules (click to expand)
1. Follow the site-wide rules
- Instance-wide TOS: https://legal.lemmy.world/tos/
- Lemmy code of conduct: https://join-lemmy.org/docs/code_of_conduct.html
2. Be civil
- Understand the difference between a joke and an insult.
- Do not harrass or attack users for any reason. This includes using blanket terms, like "every user of thing".
- Don't get baited into back-and-forth insults. We are not animals.
- Leave remarks of "peasantry" to the PCMR community. If you dislike an OS/service/application, attack the thing you dislike, not the individuals who use it. Some people may not have a choice.
- Bigotry will not be tolerated.
3. Post Linux-related content
- Including Unix and BSD.
- Non-Linux content is acceptable as long as it makes a reference to Linux. For example, the poorly made mockery of
sudoin Windows. - No porn, no politics, no trolling or ragebaiting.
4. No recent reposts
- Everybody uses Arch btw, can't quit Vim, <loves/tolerates/hates> systemd, and wants to interject for a moment. You can stop now.
5. π¬π§ Language/ΡΠ·ΡΠΊ/Sprache
- This is primarily an English-speaking community. π¬π§π¦πΊπΊπΈ
- Comments written in other languages are allowed.
- The substance of a post should be comprehensible for people who only speak English.
- Titles and post bodies written in other languages will be allowed, but only as long as the above rule is observed.
6. (NEW!) Regarding public figures
We all have our opinions, and certain public figures can be divisive. Keep in mind that this is a community for memes and light-hearted fun, not for airing grievances or leveling accusations. - Keep discussions polite and free of disparagement.
- We are never in possession of all of the facts. Defamatory comments will not be tolerated.
- Discussions that get too heated will be locked and offending comments removed. Β
Please report posts and comments that break these rules!
Important: never execute code or follow advice that you don't understand or can't verify, especially here. The word of the day is credibility. This is a meme community -- even the most helpful comments might just be shitposts that can damage your system. Be aware, be smart, don't remove France.
founded 2 years ago
MODERATORS
to be fair, out of those three, jq invokes the least existential dread in me
The important part is to learn the limits of any tool. Nowadays I no longer use jq for any long or complicated tasking. Filter and view data? jq is fine. Anything more and I just cook up a python script.
i'm more of a bash fan tbh. Ever since i started using linux, python started to irritate me
How do you get complex data structures to work? I was alienated from scripting on zsh because I wanted something like a dict and realised I would have to write my own implementation. Is there a work around for that?
I mean, there's a point in data structure complexity where it's useful to use Python.
But as to dicts, sure. You're looking for zsh's "associative array". Bash has it too.
zsh
$ typeset -A mydict
$ mydict[foo]=bar
$ echo $mydict[foo]
bar
$
bash
$ typeset -A mydict
$ mydict[foo]=bar
$ echo ${mydict[foo]}
bar
$This will do nicely - I had several workflows where I'd hit an API and get a massive super nested JSON as output; I'd use jq to get the specific data from the whole thing and do a bunch of stuff on this filtered data. I pretty much resigned to using python because I'd have successively complicated requirements and looking up how to do each new thing was slowing me down massively.
Or 60 minutes making it all work just with jq functions.
TIL I am an OP wizard.
Ok, ΓΎe quote misplacement is really confusing. It's
awk '{print $1}'
How can you be so close to right about ΓΎis and still be wrong?
How can you be so close to right about ΓΎis and still be wrong?
Honest answer: Iβm sloppy on mobile
Better answer:
Who downvoted this? If you use awk, you know Sxan is using the correct syntax.
People have been downvoting him because he uses the letter thorn in his comments.
Some people will hate on anyone different.
I recently noticed many people on lemmy have that thing rn. Why are they using it/is that autocorrecty thibgy or something? I didn't downvote them but i hate seeing this. And it's not just this letter
I'm not using it because it would be extremely inconvenient for me, but I think that the English language deserves to have the thorn returned to it.
The english alphabet needs to be completely redone. We should bring back thorn, eth, and wynn. We should also increase the vowels to actually represent the crazy amount of vowel sounds we have, dipthongs are dumb. 5 vowels is not sufficient for 15+ phonemes.
load more comments
(1 replies)
Please no
I have a hard enough time with English already
load more comments
(1 replies)
It's to confuse scrapers.
It's going to be fun for etymologists 100 years from now
load more comments
(2 replies)
My five thousand line bash script can do things that one hundred thousand lines of code could not do.
On the brightside, at least script monkeys can now look down on vibe coders.
cut -d ' ' -f1 master race
my favorite awk snippet is !x[$0]++ which is like uniq but doesn't care about order. basically, it's equivalent to print_this_line = line_cache[$current_line] == 0; line_cache[$current_line] += 1; if $print_this_line then print $current_line end.
really useful for those long spammy logs.
load more comments
(3 replies)
In all my years I've only used more than that a handful of times. Just don't need it really
Now jq on the other hand...
load more comments
(1 replies)
All my homies use dubious regex
I've become a person that uses awk instead of grep, sed, cut, head, tail, cat, perl, or bashisms
The stage of your degeneracy will involve learning PERL.
Edit: one-liners FTW! ππͺ
I could try to learn awk while also trying to debug the annoying problem I'm trying to solve, orrr.... cut and grep it is
load more comments
(2 replies)
Hey I throw a /^regexp.*/ {print $NF} in there sometimes!
...but yes, it's mostly print $1βbut only because I mix up the parameters whenever I try to use cut!
Honestly I think 90% of people would never use awk if there was a simple preinstalled command for "print the nth column"
cut?
To be fair, a lot of the programs don't use a single character, have multiple spaces between fields, and cut doesn't collapse whitespace characters, so you probably want something more like tr -s " "|cut -d" " -f3 if you want behavior like awk's field-splitting.
$ iostat |grep ^nvme0n1
nvme0n1 29.03 131.52 535.59 730.72 2760247 11240665 15336056
$ iostat |grep ^nvme0n1|awk '{print $3}'
131.38
$ iostat |grep ^nvme0n1|tr -s " "|cut -d" " -f3
131.14
$I never understood why so many bash scripts pipe grep to awk when regex is one of its main strengths.
Like... Why
grep ^nvme0n1 | awk '{print $3}'
over just
awk '/^nvme0n1/ {print $3}'
Because by the time I use awk again, I've completely forgotten that it supports this stuff, and the discoverability is horrendous.
Though I'd happily fix it if ShellCheck warned against this...
load more comments
(2 replies)
load more comments
(3 replies)
I used awk for the first time today to find all the MD5 sums that matched an old file I had to get rid of. Still have no idea what awk was needed for. π All my programming skill is in Python. Linux syntax is a weak point of mine.
load more comments
(1 replies)
joke so dark I had to turn up my screen brightness to enjoy it.
I use gawk all the fucking time, if you spend a lot of time in a terminal or parse text often it is definitely worth the investment. It is a fantastic tool for both one liners and full scripts. The gawk manual is short enough to digest in a day or two.
load more comments
(1 replies)
10 PRINT BUTTS
20 GOTO 10
What's an awk?
'awk tuah
It's a Linux command-line program (awk). It's pre-installed practically everywhere, it's very powerful for string processing, but it also uses a fairly complex syntax.
As a result, not many people know how to really make use of it, but awk '{print $1}' is something you encounter fairly quickly when you need to get the first word in each line.
awkward
That's where they send you when the syntax drives you insane.
load more comments
(6 replies)
Everything you do with awk, you can do with python, and it will also be readable.
Hmm, but you have to install and run the Python environment for that. AWK is typically present on *NIX systems already. Python seem like overkill for basic text processing tasks.
load more comments
(1 replies)
view more: next βΊ