egg|nomz|egg changed the topic of #kspacademia to: https://gist.github.com/pdn4kd/164b9b85435d87afbec0c3a7e69d3e6d | Dogs are cats. Spiders are cat interferometers. | Космизм сегодня! | Document well, for tomorrow you may get mauled by a ネコバス. | <UmbralRaptor> egg|nomz|egg: generally if your eyes are dewing over, that's not the weather. | <ferram4> I shall beat my problems to death with an engineer.
<egg|phone|egg>
Argh back up to 38,2 from 37,5 a couple of hours back
<kmath>
<mrocklin> If you really want to talk about tensors as abstract multi-linear operators then get ready to dust off your differe… https://t.co/vO8SzG6wn3
<rqou>
what is everybody else doing with them?
<rqou>
just a giant bag of numbers?
<rqou>
egg|phone|egg: i thought to speak cantonese you just need to shout angrily around a dinner table :P :P
<bofh>
rqou: I mean, same, multilinear map, here's the universal property, prove some things about Z-modules
<bofh>
rqou: first-year linalg was a wild ride
<rqou>
lol i never did any of those things
<rqou>
i guess my linalg class wasn't as good
<rqou>
we didn't cover tensors at all other than "so we learned about dual spaces, right? if you take the product of vector spaces and dual spaces, you get tensors"
<rqou>
"out of scope for this course"
<UmbralRaptor>
Tensors are some bit of magic you invoke with the Levi-cevitta symbol to make matrix operations commutative.
<UmbralRaptor>
>_>
<rqou>
i don't actually know what that means
<rqou>
my math skills are probably the worst among people here
<UmbralRaptor>
Uh
<SilverFox>
uhhhhhhhhhhhhhhhhhhhhhhhhhh
<rqou>
what>
<rqou>
?
<UmbralRaptor>
rqou: I'm generally unable to follow math conversations between you, bofh, and/or egg.
* UmbralRaptor
is unsure where SilverFox's math skills are at.
<rqou>
huh, then i guess bofh and egg together somehow make everyone here seem like an expert :P
<SilverFox>
try and test them
<UmbralRaptor>
SilverFox: Two sports teams play a best of 7 tournament. One wins in 4. Can you say that the winning team is better? (p≤0.05)?
<UmbralRaptor>
(Assume a binomial distribution)
<SilverFox>
k so
<SilverFox>
what sport?
<UmbralRaptor>
Alternatively, ask egg for some numeric madness?
<UmbralRaptor>
For consistency with 4 of 7, probably baseball or basketball, but that should be unrelated to the question.
<SilverFox>
there isn't enough data to suggest that the team is better than the other, they could just have a good run, the sample size is small for this statistic
<rqou>
ugh, i should really know how to solve this, but probability isn't my strong point
<SilverFox>
ima need like, many tournaments here to make any reasonable conclusion
<SilverFox>
otherwise you might as well just flip a coin and say the only side of a coin is the one it lands on
<UmbralRaptor>
It's probability. "Insufficient data, so ¯\_(ツ)_/¯" is a completely legitimate answer.
<SilverFox>
sick
<SilverFox>
did I pass?
<UmbralRaptor>
Sorta.
<SilverFox>
>70%?
<UmbralRaptor>
I was hoping for you to notice that the 50-50 case isn't disproven in a Statistically Significant™ way because it can have one team blow through with 4 wins 6.25% (=1/2^4) of the time.
<UmbralRaptor>
Anyway, entirely too much of QM (and angles in classical mechanics) still feels like magic. =\
<kmath>
<astrotweeps> Because Astronomy can also be a high-risk activity: a huge spider in the @AAOastro , courtesy of the SAMI team. ⏎ No… https://t.co/jQ0ru4kFIe
<bofh>
oh joy, I decided to check out tomorrow's forecast (note: it's already pouring, set to turn into wet snow ~2am): "Rain, snow and freezing rain, possibly mixed with sleet before 3pm, then freezing rain between 3pm and 4pm, then rain, possibly mixed with snow and freezing rain after 4pm. Chance of precipitation is 100%".
<bofh>
TIL it is possible for the weather to flip you the finger.
<egg|phone|egg>
UmbralRaptor: liver?
<egg|phone|egg>
Rqou: probably works for Viet too
<egg|phone|egg>
Rqou: I wonder whether there's a half decent mapping between the tones of Cantonese and those of south viet (since I know the latter but not the former)
<egg|phone|egg>
Bofh: classical mechanics is nice, what with the cute pettable Hamiltonians
<egg|zzz|egg>
bofh: also I wonder when the initial guess for rootn might be better done with FP arithmetic than with exponent fiddling
<egg|zzz|egg>
bofh: it seems obvious that the fourth root would get an *extremely good* guess from two sqrt
<egg|zzz|egg>
is there something smart to do when n is not a power of two
ferram4 has joined #kspacademia
ferram4_ has quit [Read error: -0x1: UNKNOWN ERROR CODE (0001)]
<egg|zzz|egg>
!wpn whitequark
* Qboid
gives whitequark an optional lake which vaguely resembles a dip pen
<kmath>
<GHeathWhyte> Can't quite decide whether writing out my set texts in cuneiform with a chopstick counts as revision or procrastina… https://t.co/BI84qPW9kY
<egg|zzz|egg>
Unicode transcription left as an exercise to the reader
<egg|zzz|egg>
bofh: hm so seems for the square root you need ~ a couple extra bits, and then for the nth root you need another bit every time n passes another power of two (so e.g. with extended precision you have enough for roots up to the 256th or so)
<egg|zzz|egg>
(or I may have screwed up my analysis completely)
tawny has quit [Ping timeout: 186 seconds]
<egg|zzz|egg>
bofh: um, instead of just dividing the exponent and then tweaking the mantissa, wouldn't an affine map of the entire representation of the floating-point number be a fairly decent guess? Or does the mantissa tweaking yield something better than that
egg|phone|egg has quit [Ping timeout: 190 seconds]
egg|phone|egg has joined #kspacademia
<egg|zzz|egg>
bofh: yeah at least compared to just ceiling(division) on the eggsponent (which doesn't do fancy things to the mantissa, but was consistent with the number of iterates you needed,) it's a whole lot better
<egg|zzz|egg>
bofh: for the cube root, need 4 Newton iterates instead of 6, 3 Halley iterates instead of 4, 2 3rd order Householder iterates instead of 3
<egg|zzz|egg>
bofh: no guarantees on what this affine transformation thing is going to do to subnormal value though
<egg|zzz|egg>
s/ue/ues/
<Qboid>
egg|zzz|egg meant to say: bofh: no guarantees on what this affine transformation thing is going to do to subnormal values though
<kmath>
<c4757p> imagine if microwave ovens weren't already a thing, and today someone proposed a several hundred watt RF emitter in… https://t.co/JonqgbUefy
<UmbralRaptor>
egg|zzz|egg: uh, for the liver thing I may have confused ibuprofen with acetomeniphen.
* egg|zzz|egg
always confused by that name of paracetamol
<bofh>
yeah, it's ultimately better since the way I do it the starting guess is based purely on the exponent, which loses bits.
<egg|zzz|egg>
I mean in integer arithmetic, i.e., if / is flooring integer division and representation is unchecked floating point to integer conversion, with from_representation being its inverse, from_representation((representation(z)-representation(1))/3+representation(1))
<egg|zzz|egg>
for normalized numbers, the error will be below 500 teraULPs for all values of 3 (greater than or equal to 3)
<egg|zzz|egg>
bofh: yes teraULP is a silly unit :-p
<egg|zzz|egg>
(using binary64 here of course)
<UmbralRaptor>
There are many sillier units.
<egg|zzz|egg>
bofh: hm, I'm not sure I understand how Kahan's mapping works (but it seems to be morally similar?)
<egg|zzz|egg>
he has lower relative errors though :-p
<whitequark>
UmbralRaptor: imagining?
<whitequark>
there's a microwave with a tegra inside
<whitequark>
for running opencv
<bofh>
egg|zzz|egg: like it sounds more or less exactly like what you suggested earlier, unless I misunderstood your idea :P
<egg|zzz|egg>
bofh: yeah, except I don't get 3,2%
<egg|zzz|egg>
!wpn bofh
* Qboid
gives bofh a matrix
Snoozee is now known as Majiir
<UmbralRaptor>
whitequark: and no one has gotten strange and exciting burns from a company not bothering with safety considerations?
<Majiir>
!wpn egg|phone|egg
* Qboid
gives egg|phone|egg a visual prompt-critical ram
<bofh>
!wpn egg|zzz|egg, UmbralRaptor, whitequark et al
* Qboid
gives egg|zzz|egg, UmbralRaptor, whitequark et al a degenerate crystal
<egg|zzz|egg>
!wpn Majiir
* Qboid
gives Majiir a Lambert van
<UmbralRaptor>
The ramming process crushes the fissile material above the critical mass, which then destroys the door.
<bofh>
egg|zzz|egg: and yeah, I have a fourth root function that uses rsqrtps twice as its initial guess, then does 3 Newton iterates. saturates single precision nicely.
<egg|zzz|egg>
UmbralRaptor: yeah I was wondering how this worked
<Majiir>
I didn't know Lambertini made vans
<egg|zzz|egg>
UmbralRaptor: gun-type, but with ramming instead of the explosives?
<bofh>
(to be exact, it's an SSE function that computes x^{3/4} over an array)
<UmbralRaptor>
egg|zzz|egg: yes.
<egg|zzz|egg>
bofh: huh, I'd have hoped a single Newton iterate would be saturating after two square roots
<egg|zzz|egg>
oh rsqrtps, that's eggstremely approximate right
<bofh>
oh no wait once again I'm misremembering, I do two iterates
<bofh>
just pulled up the code lol
<Majiir>
You know, after lurking in this channel for a few years, I'm pretty sure this whole channel is running Newton's method to write software using Newton's method, and the top-level one is running way too many iterations.
<egg|zzz|egg>
bofh: I don't get how Kahan obtains a 3,2 % relative error tbh
<egg|zzz|egg>
Majiir: some of the levels probably use Halley's method
<Majiir>
egg++
* egg|zzz|egg
is postincremented
<bofh>
egg|zzz|egg: hey, sometimes I use quasi-Newton methods instead! (mostly L-BFGS) :P
* UmbralRaptor
has vague memories of something with an exponential function in the linear regime.
<egg|zzz|egg>
bofh: but anyway, an affine map on the unchecked conversion seems faster than splitting and operating on the exponent alone, as well as more accurate?
<egg|zzz|egg>
bofh: aside from the fact that I have no idea what those magic Kahan constants are
<bofh>
egg|zzz|egg: I agree
<egg|zzz|egg>
bofh: except don't know whence the constant and whence the 3,2 :-/
<egg|zzz|egg>
hm what if I just use the constant, do I actually get 3,2
<bofh>
egg|zzz|egg: worth empirically testing
<bofh>
egg|zzz|egg: sec, let me read the paper and figure out that constant now.
<egg|zzz|egg>
okay I'm not understanding the "with its binary point in the same place as Y's"
<egg|zzz|egg>
(source: my attempt at implementing my interpretation thereof has a relative error of ~1 all the time :D
<egg|zzz|egg>
)
<egg|zzz|egg>
bofh: okay I think I see what C is supposed to be (2/3 representation(1) * magic), rather than 1/3 as I thought from what's written, that's better on average but not better uniformly than my thing in binary64, certainly don't get 3,2 there...
<egg|zzz|egg>
bofh: and same in binary32; it actually seems worse in the uniform norm than the naive thing?
<egg|zzz|egg>
this is weird
<egg|zzz|egg>
(of course the error is bonkers for subnormal values but that's another discussion)
<bofh>
that *is* weird, again, sec, poking at it myself
<egg|zzz|egg>
bofh: okay I have replicated something that looks like 3,2 after randomly perturbating my interpretation of Kahan's nebulous explanations
<egg|zzz|egg>
bofh: the constant remains magic
TonyC2 has joined #kspacademia
TonyC1 has quit [Ping timeout: 190 seconds]
<egg|zzz|egg>
bofh: the magic constant is 3071303373017859465 as an integer fwiw (or 6 as the last sig dec depending on how you round the division by 3)
TonyC has quit [Ping timeout: 186 seconds]
TonyC1 has joined #kspacademia
<egg|zzz|egg>
bofh: seems like that's like the σ in fast inverse sqrt, optimize for your use case...
TonyC2 has quit [Ping timeout: 186 seconds]
TonyC1 has quit [Ping timeout: 186 seconds]
TonyC has joined #kspacademia
TonyC has quit [Ping timeout: 186 seconds]
TonyC has joined #kspacademia
TonyC1 has joined #kspacademia
TonyC has quit [Ping timeout: 190 seconds]
<bofh>
egg|zzz|egg: finally took a look at the paper (ye gods my ADHD is awful today), by "binary point in the same place as" he means, for instance, for single precision you'd compute lrintf(2**23 * ((127 - 0.1009678)/3))
<bofh>
(or in my case, lrintf(2**23 * ((508 - 0.1984887)/3)), since the Newton iterates for 1/cbrt(x) are immensely faster than those for cbrt(x) b/c there's no division involved)
<bofh>
that being said they are pretty much lifted out of thin air; part of me honestly would suspect Kahan just iterated over all single-precision floats 0 < x < 1 to find the one that minimized relative error (since that code nowadays would run in a few mins tops), but the paper is from 1991 so...
<egg|zzz|egg>
bofh: I thought as you did, but somehow this only works if I double the bias?!
<egg|zzz|egg>
i.e. the magical constant that works is kahanC = Floor[(2 IEEE754FloatingPoint`Private`bias - 0.1009678``10)/
<egg|zzz|egg>
3*Representation[leastNormal]]
<egg|zzz|egg>
gah that newline
<egg|zzz|egg>
(yes I should export the bias from my package, I know)
<egg|zzz|egg>
bofh: could the retypesetter Mr. Bindel have forgotten a factor of 2?
<egg|zzz|egg>
(going to assume that it could not possibly be the wolf himself)
<bofh>
Huh. What the. Interesting. Wait, how does lrintf(2**24 * ((127 - 0.1009678)/3)) compare to doubling the bias (i.e. scale the whole expression by two)?
<bofh>
Yeah, it might be the case indeed.
<bofh>
sec, let me test.
<egg|zzz|egg>
bofh: any other factoring out of the 2 yields garbage (as in either utter garbage, or as in no constant would be better)
<egg|zzz|egg>
bofh: for fast inverse sqrt you don't want the least relative error, you want the least after a few iterates of Newton, so it's even more magic
<egg|zzz|egg>
bofh: but you can certainly assume properties of continuity for a while to restrict your search space before you start to have to deal with that crap
<bofh>
Okay that definetly seems as though a factor of 2 in the bias was lost.
<bofh>
oh, of course there's a missing factor of two
<bofh>
your float is Y = 2^(e+127) * 1.whatever, if we divide the exponent part by 3 we get 2^((e+127)/3) * 1.other_whatever -> 2^(e/3 + 127/3) * 1.other_whatever, so in order to fix the exponent bias, we need to add 2^(2*127/3) to it so we get 2^(e/3 + 127) * 1.other_whatever.
<bofh>
Yeah that's a typo.
<egg|zzz|egg>
bofh: okay so at least the *correct* constant is nowhere to be found in either decimal of hex for either binary32 or binary64 (not going to search for the extended constant or the VAX FDGH ones thankyouverymuch)
<egg|zzz|egg>
bofh: can we find the constant that would result from the typo?
<egg|zzz|egg>
(I mean the binary32 one can be found coincidentally in tables of experimental data from JAXA but that doesn't count)
tawny has joined #kspacademia
<bofh>
rofl
<bofh>
so I wonder if the constant as stated is actually correct, since if the typesetter just dropped a factor of 2 before the B, it would be.
<egg|zzz|egg>
can't find it for binary32 with the typoed form
<bofh>
Huh.
<egg|zzz|egg>
nor for binary64; nobody actually uses this algorithm?
<egg|zzz|egg>
I mean, a typo in a constant that falls out of the sky makes the thing a dead end unless you're really bored tbh
<egg|zzz|egg>
bofh: also, using this algorithm generically for rootn poses two problems: 1. the magical number that turns 5% into 3% (meh, can live without), 2. calculate (n-1)representation(1)/n without falling into into integer overflows at compile time :-p
<bofh>
leading to a difference of only 21 in the resulting fixed-point constant, I can't imagine that makes much of a difference.
<bofh>
Is 2. actually a problem, realistically speaking?
<egg|zzz|egg>
for rootn, yes? representation(1) is a big number
<egg|zzz|egg>
it starts being an issue with the 7th root I think
<bofh>
Wait, how so?
<egg|zzz|egg>
n * representation(1)
<egg|zzz|egg>
if you can't do that you need to do compile-time rational arithmetic/multiprecision somehow
<egg|zzz|egg>
(all hail Ada, which has that :-p)
<bofh>
And this can't be computed at compile-time with just simple extended precision (i.e. uint128_t)?
<egg|zzz|egg>
(and VHDL, which also has that because it just yoinked half the ARM)
<bofh>
because I don't think the constant will be meaningfully too large in most cases.
<egg|zzz|egg>
bofh: that's all very nice if you somehow have 128-bit integers
<bofh>
you have 64-bit ints, but ugh multiprec division is a pain I just remembered
<egg|zzz|egg>
bofh: yes, I have 64-bit ints, but if you want to be portable those are as wide as things get
<egg|zzz|egg>
uintmax_t is unsigned long long is uint64_t on windows (and long is cursed, use long long)
<egg|zzz|egg>
bofh: see this is why you want your language to have multiprecision rationals at compile time :D
<bofh>
I mean if you have a fetish for compile-time numerics, then yes, that makes sense. :P
<egg|zzz|egg>
bofh: nah, the context here is clearly runtime numerics
<egg|zzz|egg>
bofh: it's genericity though
<egg|zzz|egg>
if you want genericity in numerics, then you may need to compute magic constants at compile time rather than on paper, and for that you need actual arithmetic at compile time
<bofh>
True. I mean you can do the lazy thing and compute them at runtime in a constructor, but that causes *some* perf loss and now you get into the nightmare that is runtime library init routines.
<egg|zzz|egg>
bofh: yeah it's rather a lot nicer if functions are functions (and if you have static turds then you have the guards around their initialization for threading etc.)
<egg|zzz|egg>
bofh: also if you're going to write runtime multiprecision, might as well have it constexpr instead
<egg|zzz|egg>
bofh: but that's the amusing thing in Ada, you have it at compile time, not as runtime libs
<egg|zzz|egg>
(I mean, you can write runtime libs if you want, but Universal_Integer and Universal_Real are very weird animals
<bofh>
...I feel like I should reread your Ada numerics ramble at some point :P
<egg|zzz|egg>
bofh: which ramble eggsactly
<egg|zzz|egg>
bofh: tbh Ada is not really great at floating-point arithmetic specifically, because standardized in 1983 and designed earlier (the standardization process involved translating things in french :D)
<egg|zzz|egg>
bofh: it's a language that's far less UB-happy, so division by 0 *is* defined; but it raises Constraint_Error (Numeric_Error in 83)
<egg|zzz|egg>
bofh: the standard does require something about FP arithmetic though (iirc that it must be consistent with interval arithmetic with the given number of digits or something) but that's it
<bofh>
egg|zzz|egg: no, its integer(/rational(?)) arithmetic. IEEE754 float is very sensible, so I don't really need to go looking for better specs. :P
<egg|zzz|egg>
bofh: yeah, it's interesting to see what languages that lived before that tried to specify though
<egg|zzz|egg>
bofh: ok but now how does one actually go about finding the magical value in Kahan's C constant (and how does it generalize to other values of 3)
<bofh>
egg|zzz|egg: give me a moment to think about that, just spent the past hour trying to walk in the current weather to my office, giving up after falling thru freshly frozen ice from freezing rain covering some 15cm of wet snow multiple times, then taking the bus. :P
<bofh>
(seriously the weather here is drunk)
<egg|zzz|egg>
what is the ethanol concentration of the rain
<bofh>
Insufficiently high to cause further snowmelt :P
<EntropicRaptor>
…even the rain has been corntaminated?
<bofh>
I wonder if that constant is just the value that minimizes relative error as a starting point for a random mantissa value, or something along those lines.
<bofh>
i.e. given a random 23-bit mantissa, what is the constant that gives the best *average* relative error for it.
<bofh>
I mean sure "gradient descent on that expression in the L^{\infty} norm" works, but I want an analytical expression.
<egg|zzz|egg>
bofh: I mean basically on ~anything that's linear in log-log, you can apply this strategy
<bofh>
yeah, but how do you analytically calculate the value that minimizes the uniform norm? or *can* you even?
<egg|zzz|egg>
and in all those cases you get an additional affine tweak :-p
<bofh>
yeah but how do you analytically compute the expression for the optimum affine tweak? :P
<egg|zzz|egg>
bofh: so one thing is that there seems to be a periodicity (this is probably obvious, I'm probably not awake enough to see it)
<bofh>
egg|zzz|egg: multiply 84+2/3-0.033whatever by 3 and take out the /3, you get (254-0.099whatever)/3 -> (2*127-0.099whatever)/3
<egg|zzz|egg>
ah
<egg|zzz|egg>
ah right, they divide first
<bofh>
I have no idea why they wrote it as that, it pointlessly obscures the derivation.
<egg|zzz|egg>
they did it independently?
<bofh>
My question is why their constant slightly differs from Kahan's.
<bofh>
And for that matter, if this actually matters (using Kahan's makes B1 smaller by 21) at all for any input.
<bofh>
/¯\_(ツ)_/¯
<egg|zzz|egg>
bofh: I mean, the "natural" thing is from_representation((representation(z)-representation(1))/3+representation(1)), and then it's a coin toss whether you move the addition inside or outside the division
<bofh>
Yeah, I know. Making the addition be inside is nicer IMO tho.
<egg|zzz|egg>
as for coming up with a different constant, that's interesting, it kind of hints at hazy criteria and exhausting exhaustive search?
<egg|zzz|egg>
bofh: iirc there are some blurbs out there on how to find σ for fast inverse sqrt, let me dig that up
<kmath>
<volatile_void> @ollj This is kind of the definitive blog post on fastinversesquareroot, I think. Ignore the other ones.… https://t.co/1XNzUxg8NG
* EntropicRaptor
🔪 take home exams.
<bofh>
okay, I'll read them all. I think I have the lomont.org paper here somewhere already, lol.
<egg|zzz|egg>
bofh: the other one references it so it's probably more recent
<egg|zzz|egg>
you should probably put it in your ~ so it's easier to find, too
<egg|zzz|egg>
(all hail the ~s of academics)
<bofh>
rofl, I should totally just dump all my papers there (b/c it's at my old uni, not the one I'm currently at, I can't really get in trouble or yelled at over it, so might as well just paperdump).
EntropicRaptor has quit [Quit: Bye]
UmbralRaptop has joined #kspacademia
<egg|zzz|egg>
bofh: or wait until you get tenure and do it at your current institution,
<bofh>
egg|zzz|egg: well ofc I'm also doing that. :P
<UmbralRaptop>
(Assuming that tenure still eggsists in a decade or 2)