VITAS changed the topic of #spacedock to: Problems?: https://github.com/KSP-SpaceDock/SpaceDock/issues | Matrix/Riot Chat: https://im.52k.de +spacedock:52k.de Feel free to ask for help, we only bite a little bit! | If you want to help, please check https://github.com/KSP-SpaceDock/SpaceDock-Backend/issues/5 :) | <VITAS> inet users have the attentionspan of a squirrel....oh a nut!
<RockyTV> ckan won't cease to exist
<RockyTV> see skyrim and fallout4, they have the creation club, where bethesda rips off mods created by the community and sells them, and you still have nexus mod manager and similar software
<RockyTV> that means we have a deadline: 2020
<RockyTV> spacedock for ksp2 :P
<DasSkelett[m]> Until then we should get multi-game support fixed!
<DasSkelett[m]> Both, CKAN and SpaceDock πŸ˜„
<HebaruSan[m]> I bet/hope they incorporate equivalents of ModuleManager and Kopernicus
<RockyTV> this new api got me excited
<RockyTV> I'm pretty sure I understand it as "modders are going to have access to features that were impossible to have on the original game"
<RockyTV> it'll probably be a q4 game
<RockyTV> I was thinking, since sd is hosted on germany, that means we have to comply with the GDPR
<RockyTV> which means we should implement a function for the user to request its data. would it return the zipfile for mods as well or just the info about mods published, ratings/reviews posted and modpacks created?
<DasSkelett[m]> Regarding the modfiles, no idea if that's needed by the GDPR or not.
<DasSkelett[m]> I would add basically the row of the user db (minus password) too.
<RockyTV> yeah
<RockyTV> user info + mods info + modpacks info
<RockyTV> another thing: should we implement some sort of malicious file scanning for mods?
<RockyTV> HebaruSan[m], I was thinking about using an external site for detecting open source mod licenses
<RockyTV> wait actually it's not a problem, is it?
<RockyTV> I think we should just delete user profiles and the user should define whether he wants his mods to be deleted or not
HebaruSan[m] has quit [Quit: Idle timeout reached: 10800s]
DasSkelett[m] has quit [Quit: Idle timeout reached: 10800s]
AllisTauri[m] has joined #spacedock
<AllisTauri[m]> What is wrong with multi game support exactly?
Bart[m] has joined #spacedock
<Bart[m]> They'll probably use the steam workshop instead of CKAN.
<Bart[m]> It'd be far easier for them that way, none of the hassle, all of the functionality.
VITAS[m] has joined #spacedock
<VITAS[m]> not the point
<Bart[m]> And as far as GDPR goes, how many times have people requested that data VITAS?
<Bart[m]> Not sure if it's really a concern.
<VITAS[m]> i want multigaqme support to exist. but with a gamewide flag to disable the option for ckan
<AllisTauri[m]> Any more tribe besides ckan flag?
<RockyTV> Bart[m], they won't rely on workshop.
<RockyTV> it'll work like Skyrim. they will support several game stores so sticking to workshop for mods is a no go
<RockyTV> they will support it but it won't be a necessity
<RockyTV> AllisTauri[m], last time I tried to create a NMS mod it broke my profile
<AllisTauri[m]> Ok, so I'll dig into this matter
<RockyTV> we should enable ckan only for ksp mods, like VITAS[m] said
<AllisTauri[m]> We should switch from `ckan` column in the mod table top a json metadata one that could hold any additional information about the mod.
<AllisTauri[m]> And then for a game we could, in the same way, define a set of flags/fields that will be available through interface when a mod for that game j is created.
<AllisTauri[m]> ...(truncated)
Bart[m] has quit [Quit: Idle timeout reached: 10800s]
VITAS[m] has quit [Quit: Idle timeout reached: 10800s]
Bart[m] has joined #spacedock
<Bart[m]> That seems rather inefficient in terms of database design.
<Bart[m]> You can't search for specific json fields efficiently.
<Bart[m]> It'd probably be more efficient database wise to create a separate table with properties and link them by ID.
<Bart[m]> That way you can still search through everything quite quickly with a well-constructed query without having to do full text search
<AllisTauri[m]> No, you can)))
<AllisTauri[m]> Modern database design uses json extensively.
<AllisTauri[m]> In postgres you can efficiently select from within json objects.
<AllisTauri[m]> Highly normalized relational database are are more or less obsolete for many applications
<AllisTauri[m]> We use json to store data and metadata in production all the time
<AllisTauri[m]> Tables are for relations and constraints, not to store everything
<Bart[m]> Yes, you can, but the performance considerations are often left aside when you fill in this "can".
politas has joined #spacedock
<AllisTauri[m]> It's hard to compare performance of semi-relational design with json/xml/blob data storage with that of highly normalized DB outside of particular case. In my experience in many modern applications DB normalization decreases both performance and flexibility.
<AllisTauri[m]> Still, do show me performance tests for JSON/B data types in PG-11-12, I'd be obliged
<AllisTauri[m]> On second thought, that's actually irrelevant: the only meaningful performance measurement is the performance of the application as a whole. All that "requests per second" and other metrical nonsense just clouds rational judgement.
<Bart[m]> I just know spacedock is already slow as fuck on many days πŸ˜›
<Bart[m]> I thought the documentation for the json datatype in postgre stated specifically that it wasn't meant for fields that you were specifically going to search on.
<AllisTauri[m]> PostgreSQL: Documentation: 11: 8.14.Β JSON Types
<AllisTauri[m]> Actually there's nothing like that in current docs
<AllisTauri[m]> > I just know spacedock is already slow as fuck on many days πŸ˜›
<AllisTauri[m]> That's what I'm talking about: the app is slow, but what exactly makes it slow is a thing to be determined with profiling tools and code analysis. Nowadays it's almost never the DB, but the way it's used by the code. The things I see in SD are typical: orm is misused, most of the logic lies in slow python code in single-threaded flask app instead of being properly delegated to the DB.
<AllisTauri[m]> Database design in this case is of minor interest, really, since SD will never have enough data, both in terms of amounts and complexity, to make PG sweat even a little.
<Bart[m]> Mhhh, then it must have been changed, because I vividly recall reading a paragraph on it not being intended for common queries. But to be honest, I still think it's bad practice as it obscures what's actually in a table.
<Bart[m]> And SD is over-complicated for what it is.
<Bart[m]> 5 years ago most folks would have written this with a few hundred lines of PHP and virtually no dependencies.
<AllisTauri[m]> Oh boy... These "oldtimers" :)
<AllisTauri[m]> Flask is 9 years old, sqlalchemy is 13.
<AllisTauri[m]> Five years ago it would be written just like most things are written now. Badly, with poor design and, pardon me, more often than not with sheer incompetence. Stack doesn't matter here.
Bart[m] has quit [Quit: Idle timeout reached: 10800s]
AllisTauri[m] has quit [Quit: Idle timeout reached: 10800s]