VITAS[m] has quit [Quit: Idle timeout reached: 10800s]
godarklight[m] has joined #spacedock
<godarklight[m]>
Woah sorry about that >.<
<godarklight[m]>
Forgot to test the order by change
HebaruSan[m] has joined #spacedock
<HebaruSan[m]>
No worries, it happens
Darklight[m] has joined #spacedock
<Darklight[m]>
Kk, so I guess I generate my speed test thing with the current master compared to which branch again? I hate github and would prefer just checkout out directly 😛
<Darklight[m]>
Are there any other commits we are interested in?
HebaruSan[m] has quit [Quit: Idle timeout reached: 10800s]
VITAS[m] has joined #spacedock
<VITAS[m]>
Darklight: ive given you rights on dmp in matrix. you could beautify it by adding an image and so on :)
godarklight[m] has quit [Quit: Idle timeout reached: 10800s]
Darklight[m] has quit [Quit: Idle timeout reached: 10800s]
VITAS[m] has quit [Quit: Idle timeout reached: 10800s]
DasSkelett[m]1 has joined #spacedock
<DasSkelett[m]1>
Comparing the values of the benchmarks, there actually do seem to be slight improvements with indices, in the range of 2-20ms.
HebaruSan[m] has joined #spacedock
<HebaruSan[m]>
<span class="d-mention d-user">Darklight</span> any numbers regarding disk space? I think that's what initially led us to the idea of testing this.
Darklight[m] has joined #spacedock
<Darklight[m]>
DasSkellet: 2ms over 18 seconds isn't something I would say is that noticable 😛
<Darklight[m]>
HebaruSan err... I am not completely sure how to check that.... does postgres do database vacuuming?
<Darklight[m]>
Maybe it has a nice size commnad
<Darklight[m]>
Maybe it has a nice size command
<DasSkelett[m]1>
But 152ms instead of 237ms is, I'd say.
<DasSkelett[m]1>
And also memory <-
<DasSkelett[m]1>
And also memory <span class="d-mention d-user">Darklight</span>
<Darklight[m]>
Oh that was flask-caching
<Darklight[m]>
Err, one sec, I need to check it against alpha
<DasSkelett[m]1>
The first summary you posted _is_ current alpha, the second one is from HebaruSan's branch.
godarklight[m] has joined #spacedock
<godarklight[m]>
``sqlalchemy.exc.CompileError: Can't emit DROP CONSTRAINT for constraint ForeignKeyConstraint(<sqlalchemy.sql.base.ColumnCollection object at 0x7feeccc78610>, None, table=Table('mod', MetaData(bind=None), schema=None)); it has no name`` thankyou alemibc, very cool
<godarklight[m]>
I can't downgrade to master which is what indicies is based on
<godarklight[m]>
But I can drop the db and refill it with the script, it doesn't take that long...
<HebaruSan[m]>
The indices branch is just based on an older alpha, I can rebase it to current if that would help
<Darklight[m]>
It depends on what you want to merge it with
<godarklight[m]>
I wish this worked like it did in mysql
<godarklight[m]>
Spacedock is not setup to build tables if the database is created for it
<godarklight[m]>
The easy thing to do is grant createdb access
<godarklight[m]>
Which is what I will do because I am too lazy to python in and fix it >.<
HebaruSan[m] has quit [Quit: Client limit exceeded: 6]
<DasSkelett[m]1>
Are you using `./spacedock database initialize`? In this case try `--force`, it creates the tables even if the db already existed beforehand.
<Darklight[m]>
I was doing from KerbalStuff.database import * create_database() create_tables()
<Darklight[m]>
I didn't realise we had other commands
<Darklight[m]>
Filling the db with 10k mods again just so it is nice and slow
<Darklight[m]>
I'm up to 4k
<Darklight[m]>
6k....
<DasSkelett[m]1>
`spacedock.initialize_database` basically also runs `create_database()` and `create_tables()`.
<DasSkelett[m]1>
So if you just call `create_tables()` when you already created the database manually, it should work, no?
<Darklight[m]>
Yeah it does, I thought it called it itself though if the db was missing
<Darklight[m]>
Also we defintiely have a bug, there is no way to make publishers and games active = True in the admin panel, but that's such a little issue and more for anyone else using spacedock code
<DasSkelett[m]1>
Wouldn't call that a bug 😄 it just isn't implemented yet.
<Darklight[m]>
That... is a bit more of an increase
<Darklight[m]>
Dropping this data though as I don't want it on my machine
<HebaruSan[m]>
So ~28%
<HebaruSan[m]>
That's pretty surprising
godarklight[m] has quit [Quit: Client limit exceeded: 6]
DasSkelett[m] has joined #spacedock
<DasSkelett[m]>
Might be because <span class="d-mention d-user">Darklight</span> only tested with a big number of mods, but not with a big number of users (which are wayyyy more than mods on production, 40k vs 2k), a big number of download events, mod updates, follow events, referral events...
<DasSkelett[m]>
Might be because <span class="d-mention d-user">Darklight</span>'s synthetic database only has a big number of mods, but not a big number of users (which are wayyyy more than mods on production, 40k vs 2k), a big number of download events, mod updates, follow events, referral events...
<Darklight[m]>
Also each of my "mods" only had 1 version
<DasSkelett[m]>
Yeah. Anyways, in this case I'd leave the decision whether that's acceptable for the amount of performance increase we get (which is quite a bit as it looks like) to someone else, but from the code-standpoint it looks good.
<Darklight[m]>
Hrmmm... actually looked into git signing because I saw the verified thing pop up
<Darklight[m]>
In the commit log
<DasSkelett[m]>
I feel like 39k of the 40k user accounts on SpaceDock are bots.
<HebaruSan[m]>
Malicious bots?
<DasSkelett[m]>
No, not really. Didn't find any bad link in a user profile, or malware in a "mod" zip yet. But if you look at the user list on the admin page, 90% are usernames and emails like "asUIGnvasdkU", and I 100% bet that there are no legit users behind them.
<HebaruSan[m]>
I look forward to being able to use the admin page 😉
<DasSkelett[m]>
There are even some ad spam accounts, linking to their company's Twitter account and having a ad text in the user description.
<DasSkelett[m]>
Gonna send you a screenshot via PM.
HebaruSan[m] has quit [Quit: Client limit exceeded: 6]
DasSkelett[m] has quit [Quit: Client limit exceeded: 6]