this post was submitted on 07 Dec 2023
539 points (87.8% liked)

Asklemmy

43945 readers
676 users here now

A loosely moderated place to ask open-ended questions

Search asklemmy ๐Ÿ”

If your post meets the following criteria, it's welcome here!

  1. Open-ended question
  2. Not offensive: at this point, we do not have the bandwidth to moderate overtly political discussions. Assume best intent and be excellent to each other.
  3. Not regarding using or support for Lemmy: context, see the list of support communities and tools for finding communities below
  4. Not ad nauseam inducing: please make sure it is a question that would be new to most members
  5. An actual topic of discussion

Looking for support?

Looking for a community?

~Icon~ ~by~ ~@Double_A@discuss.tchncs.de~

founded 5 years ago
MODERATORS
Cloak@lemmy.ml
mekhos@lemmy.ml
tmpod@lemmy.pt
14specks@lemmy.ml
539
Whats your such opinion (discuss.tchncs.de)
submitted 11 months ago by cryptix@discuss.tchncs.de to c/asklemmy@lemmy.ml
1344 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[โ€“] chiliedogg@lemmy.world -3 points 11 months ago* (last edited 11 months ago) (1 children)

It's not that precision can't be arbitrarily recorded higher in fraction, it's that precision can't be recorded precisely. Decimal is essentially fractional that's written differently and ignoring every fraction that isn't a power of 10.

How can a measurement 3/4 that's precise to 1/4 unit be recorded in decimal using significant figures? The most-correct answer would be 1. "0.8" or "0.75" suggest a precision of 1/10th and 1/100th, respectively, and sig figs are all about eliminating spurious precision.

If you have 2 measurement devices, and one is 5 times more precise than the other, decimal doesn't show it because it can only increase precision by powers of 10.

In the case of 1/64th above, if you just divide it out it shows a false precision of 1/100,000.

[โ€“] trolololol@lemmy.world 6 points 11 months ago (1 children)

0.75 +- .25 is that what you mean? If so here you go, that's how any statician would do.

[โ€“] chiliedogg@lemmy.world 0 points 11 months ago (1 children)

That's not a number - that's a sentence that takes up 3 times as many characters as 3/8.

3/8 is more efficient.

[โ€“] trolololol@lemmy.world 1 points 11 months ago* (last edited 11 months ago) (1 children)

Sure dude

Now do 0.75 +- 0.05 with a fraction

[โ€“] chiliedogg@lemmy.world 1 points 11 months ago* (last edited 11 months ago) (1 children)

15/20