<HebaruSan[m]>
b.py is missing `from .a import this_is_global`
<HebaruSan[m]>
And it doesn't need the `global` statement
<HebaruSan[m]>
That's how all modules work
<HebaruSan[m]>
`touch __init__.py`
<HebaruSan[m]>
Oh, it'll give you a circular import error when you get that set up
<HebaruSan[m]>
So don't do it that way 😉
<HebaruSan[m]>
That's the point of having a `cache.py` module, it doesn't need to import any of the other stuff
<HebaruSan[m]>
It's a leaf in the dependency tree
<HebaruSan[m]>
I'll take a stab at setting up flask_caching as I understand it if I can find the time
<HebaruSan[m]>
Hence <span class="d-mention d-user">DasSkelett</span> trying repeatedly to get your guys' attention with a suggestion to fix that 🙂
<HebaruSan[m]>
`recent` looks like it might be even slower, it's not just hitting every mod, it's hitting every mod VERSION
<HebaruSan[m]>
Guys don't worry about `current_app`, there's another way of doing it
<HebaruSan[m]>
But to get that count it tests `ModVersion.mod_id == Mod.id` for every ModVersion
<HebaruSan[m]>
Maybe there's some automatic indexing
<HebaruSan[m]>
`/browse/top` uses different logic than the 'top' section of `/<gameshort>`
<HebaruSan[m]>
But it doesn't worry about scores, it just has the db sort by descending download count
<HebaruSan[m]>
Whereas `/browse/top` calls `search_mods` which gets the list from the db and then applies its own custom sort
<HebaruSan[m]>
I'm not blaming the db, I'm blaming the custom sort
<HebaruSan[m]>
Hypothetically if we sped up search_mods, would anything be left that's dramatically slow?
<HebaruSan[m]>
Because I was just fleshing out a flask_caching prototype and realized that the rest of it should be pretty quick
<HebaruSan[m]>
<span class="d-mention d-user">Darklight</span> can you please try switching to this branch and let me know if it crashes? I don't recall the commands to set up a dummy SpaceDock, and you have a good testbed with lots of dummy data.
<HebaruSan[m]>
Oh, I probably need to make it install flask_caching, one moment...
<HebaruSan[m]>
Pushed a new commit to update requirements-backend.txt
<HebaruSan[m]>
You know, if we just did what <span class="d-mention d-user">DasSkelett</span> suggested and stored the score in the db, ALL of search_mods' logic could be moved db-side
<HebaruSan[m]>
And probably become lightning fast
<HebaruSan[m]>
But thanks, that means that it does do the caching but the db objects are thunks
<HebaruSan[m]>
OK, PR closed. And I pushed a change to my branch that MIGHT help with the default_version thing, when you have a chance.
RockyTV[m] has joined #spacedock
<RockyTV[m]>
you can import the current app with `from flask import current_app`
<HebaruSan[m]>
Huh, I get this. I wonder what part of my browser history it's using to decide that.
<HebaruSan[m]>
Meanwhile I think I have to subscribe to that Youtube channel, he talks about all kinds of sci fi spacechips 🤤
<HebaruSan[m]>
*edit:* ~~Meanwhile I think I have to subscribe to that Youtube channel, he talks about all kinds of sci fi spacechips 🤤~~ -> Meanwhile I think I have to subscribe to that Youtube channel, he talks about all kinds of sci fi spaceships 🤤
<HebaruSan[m]>
<span class="d-mention d-user">RockyTV</span> There's a "restore branch" button on GitHub that I think any team member could use if that eventuality came about
<HebaruSan[m]>
In the meantime I don't want to overrepresent how prodution-ready that change is
<HebaruSan[m]>
Actually maybe the "restore branch" button only appears for me since I pushed it to my remote. Do you see it?
<HebaruSan[m]>
If so, yes, but they're syntactically awkward
<HebaruSan[m]>
`cache` is the first argument
<HebaruSan[m]>
C# may be the best language, forgive my bias
<HebaruSan[m]>
I would like to be loyal to C++, but there is just too much craziness there
<HebaruSan[m]>
And also the need to consider whether you're dealing with the stack or the heap all the g-d time
<HebaruSan[m]>
C# loses something with that limitation but the benefits are positively gigantic
<HebaruSan[m]>
😦 my main comment on that will pretty much be, let's use the module that's made for this
<Darklight[m]>
Awake, going to write something to automate the tests first so I can get nice clean numbers and we can compare lots of different versions
<Darklight[m]>
The only standout is api/user/godarklight, but this is not a real issue at *people don't create 10000 mods*
<Darklight[m]>
These are milliseconds
<Darklight[m]>
Give me a bit for alpha to finish benching
<Darklight[m]>
man I'm only doing 5 benchs a test because I know my endpoints take 20 seconds >.<
<Darklight[m]>
This is taking a while bahahaha
<Darklight[m]>
Point proven anyway, skellets approach is way better than the nonsense I did yesterday
<Darklight[m]>
Cachcing is a spook 😛
<Darklight[m]>
It was at least slightly worth barking up that tree though, I had completely forgot how spacedock worked and this kinda refreshed me in some areas
<Darklight[m]>
It maaaay still be worth caching the main ksp page, 100ms response means it chokes at 10 requests a second, but that is where vitas threw heaps of instances at it to get it to work, I suspect a single gunicorn worker will handle this just fine given that it is literally 200 times faster
<Darklight[m]>
Oh wait up
<Darklight[m]>
That's only in some areas, the main page still takes the same time to load because it never went down the get_mods path
<Darklight[m]>
I wonder if it is worth chucking the cache only on the main page, I think we need vitas to log into piwik and tell us how badly the main page is hit
<Darklight[m]>
When you type spacedock into google what page you get
<Darklight[m]>
Probably both / and /gameshort
<Darklight[m]>
/ is 20ms which I think is acceptable
<Darklight[m]>
but /gameshort is 100
<Darklight[m]>
Yeah
<Darklight[m]>
And the gameshort version actually appears just above in the google search in my case
<Darklight[m]>
Yeah google does that 😛
<Darklight[m]>
But in either case they are right next to each other and people may see the KSP -spacedock an click that instead
<Darklight[m]>
Given that I had a fight with flask-cache I'll temporarily mock it up the same way again but for that single endpoint, then if vitas comes back with numbers that suggest we should cache it then it'll be worth it
<Darklight[m]>
I'll do it a bit neater than what I did previously too
<Darklight[m]>
Aka it's going in its own file
<Darklight[m]>
Oooohkay python is kinda weird
<Darklight[m]>
Does python do anonymous functions in a sane way?
<Darklight[m]>
File "/var/www/virtual/spacedock.info/htdocs/SpaceDock/KerbalStuff/blueprints/anonymous.py", line 28, in game
<Darklight[m]>
ga = cache.get_key(gameshort + "_ga", ga_func)
<Darklight[m]>
TypeError: get_key() takes 2 positional arguments but 3 were given
<Darklight[m]>
Yeah, in what universe is this 3 arguments
<Darklight[m]>
OH
<Darklight[m]>
I forgot this
<Darklight[m]>
I am a donkey
<Darklight[m]>
I am so used to c# where we specify static or instance >><
<Darklight[m]>
I am so used to c# where we specify static or instance >.<
<Darklight[m]>
I'm with you there 😛
<Darklight[m]>
Undefined Behaviour
<Darklight[m]>
Making my own spacedock fork so I don't pollute the main repo with trash... I got *a* solution but not sure about it, it takes the load time from 100ms to 1ms though
<Darklight[m]>
```Percentage of the requests served within a certain time (ms)
<Darklight[m]>
50% 2
<Darklight[m]>
66% 2
<Darklight[m]>
75% 2
<Darklight[m]>
80% 2
<Darklight[m]>
90% 3
<Darklight[m]>
95% 4
<Darklight[m]>
98% 7
<Darklight[m]>
99% 11
<Darklight[m]>
100% 1231 (longest request)```
<Darklight[m]>
Not sure what that little hickup was, could have been some vm weirdness, also this is 10000 requests with 10 concurrent requests going at all time
<Darklight[m]>
Not sure what that little hickup was, could have been some vm weirdness, also this is 10000 requests with 10 concurrent requests going at all times
<Darklight[m]>
On a single gunicorn instance
<Darklight[m]>
If I bench it like I benched the others, this is 100 requests 1 request at a time ```Percentage of the requests served within a certain time (ms)
<Darklight[m]>
50% 1
<Darklight[m]>
66% 2
<Darklight[m]>
75% 2
<Darklight[m]>
Odd... prod spacedock actually has a longer response time than my vm, 300ms (3req/s)
<Darklight[m]>
So that's a potential bigger increase
<Darklight[m]>
I am pretty sure alpha and prod have the same behaviour on that route atm. I think I'll leave it be for now, but it's there if you want it
<Darklight[m]>
Or in general
<Darklight[m]>
I'm just curious if it's even worth caching /kerbal-space-program, it takes 300ms and caching it inside of gunicorn can get it down to like 1
<Darklight[m]>
Any really heavily hit pages would be worth doing
<Darklight[m]>
If I still have the password, let me check 😛
<Darklight[m]>
Looks like I got rid of the details a long time ago >.<
<Darklight[m]>
At least now I am using a password manager so can keep them, you can dm me on matrix if you have spare time, but really I'm only curious in where the bulk of the hits go to.
<Darklight[m]>
Eg if everyone directly goes to the mod page then who cares about the making the main one faster type of thing
<Darklight[m]>
It took me a while to work it out, but yeah I think it's worth adding /game-short, / isn't justified as it's already pretty fast anyway
<Darklight[m]>
It took me a while to work it out, but yeah I think it's worth adding /game-short, / isn't justified as it's already pretty fast anyway
<Darklight[m]>
I feel like this should have been done back in the SirCmpwn days rather than bitching to majiir about not having powerful enough hardware 😛
<Darklight[m]>
In that case spacedock might not have existed 😄