UmbralRaptor changed the topic of #kspacademia to: https://gist.github.com/pdn4kd/164b9b85435d87afbec0c3a7e69d3e6d | Dogs are cats. Spiders are cat interferometers. | Космизм сегодня! | Document well, for tomorrow you may get mauled by a ネコバス. | <UmbralRaptor> egg|nomz|egg: generally if your eyes are dewing over, that's not the weather. | <ferram4> I shall beat my problems to death with an engineer. | We can haz pdf
e_14159_ has joined #kspacademia
e_14159 has quit [Ping timeout: 378 seconds]
<WeylandsWings>
why are you not eating?
<raptop>
I did ultimately eat something
awang has quit [Ping timeout: 189 seconds]
<egg|cell|egg>
Meow
<raptop>
nyaaaAAAAAAAAA
egg|laptop|egg has joined #kspacademia
egg|laptop|egg has quit [Remote host closed the connection]
egg|laptop|egg has joined #kspacademia
awang has joined #kspacademia
egg|laptop|egg has quit [Remote host closed the connection]
egg|laptop|egg has joined #kspacademia
egg|laptop|egg has quit [Ping timeout: 190 seconds]
<SnoopJeDi>
raptop, yea this nestyboi looks like it could be a numpy idiom of some sort. Can you describe what it's doing? It looks like it's counting, but I can't quite suss out much beyond that
egg|laptop|egg has quit [Remote host closed the connection]
<raptop>
counting up observations across several observatories, and trying to coordinate things
<raptop>
each night: was $star observed? And was $star observable? (the summary files have this, but in a slightly odd format, so it needs to check observability in terms of the star being up, and the weather being good)
<SnoopJeDi>
where "each night" is the for loop over d2?
<SnoopJeDi>
and the first axis of obs[] and observable[] correspond to...different scopes?
<SnoopJeDi>
I think you might be able to collapse the whole thing using some numpy idioms, namely np.any()
<SnoopJeDi>
one sec, it's probably simplest if I just make a mock example to show off what I'm thinking
<raptop>
checking (this isn't my code)
egg|cell|egg has quit [Ping timeout: 189 seconds]
egg|cell|egg has joined #kspacademia
egg|cell|egg has quit [Ping timeout: 190 seconds]
<SnoopJeDi>
raptop, have a look at this and see if it makes sense: http://ix.io/2hmX/py
<SnoopJeDi>
N.B. that it's taking advantage of how sum() behaves for boolean arrays, but you could just as easily explicitly change the dtype to int, or use np.where() if you wanted to have value X above threshold, value Y below, etc.
<SnoopJeDi>
upshot: it's going to be *way* faster this way because ufuncs go brrrr
<SnoopJeDi>
so if I'm not mistaken, a similar expression can replace the entire bottom half of that. Since you do this twice for two different datasets, it might make sense to roll it up into something with the signature `def count_valid_data(arr):` and then do `tot_obs = count_valid_data(obs); tot_observable = count_valid_data(observable)` ?
<SnoopJeDi>
and then I'd probably take everything under `for f in range(np.shape(filename)[0]):` and make it a `def load_and_clean_data(filename):` or something, just to hide those details away from the data-y part of the code
<SnoopJeDi>
the loop itself there is kinda icky, at the very least np.shape()[0] could just be replaced with len(), but that's still not great...I can't work out too well what it's doing though, and it looks like it'll at least fail in understandable ways for the most part.
<SnoopJeDi>
</codereview>
<raptop>
A star can be observed more than once per night, and what counts as a "night" is kinda fuzzy, given that these observatories are at wildly varying longitudes (and latitudes)
<raptop>
hrm
<SnoopJeDi>
does that sample make sense at least?
<SnoopJeDi>
I may have bungled the actual semantics of what's going on, but all that append() county business looks like it's doing the work any() is made for
<SnoopJeDi>
I threw in a threshold just because, heh.