VITAS changed the topic of #spacedock to: Problems?: https://github.com/KSP-SpaceDock/SpaceDock/issues | Matrix/Riot Chat: https://im.52k.de +spacedock:52k.de Feel free to ask for help, we only bite a little bit! | If you want to help, please check https://github.com/KSP-SpaceDock/SpaceDock-Backend/issues/5 :) | <VITAS> inet users have the attentionspan of a squirrel...
DasSkelett[m] has quit [Quit: Idle timeout reached: 10800s]
HebaruSan[m] has quit [Quit: Idle timeout reached: 10800s]
KineticSloth8StoneBlue[m has quit [Quit: Idle timeout reached: 10800s]
Darklight[m] has joined #spacedock
<Darklight[m]> You don't need AVC version files if you just want to be on ckan, all you have to do in that case is tick the ckan box on spacedock
<Darklight[m]> AVC just brings up the "new version available" thing, which I don't bother about for DMP
HebaruSan[m] has joined #spacedock
<HebaruSan[m]> Shhh, don't discourage him, version files are great for CKAN
<Darklight[m]> Don't you normally just pull from spacedock? 😛
HebaruSan[m] has quit [Quit: Idle timeout reached: 10800s]
VITAS[m] has joined #spacedock
<VITAS[m]> morning
DasSkelett[m] has joined #spacedock
<DasSkelett[m]> Morning
<Darklight[m]> o/
<DasSkelett[m]> <span class="d-mention d-user">Darklight</span> version files give authors the option to specify entire version ranges for compatibility, instead of a single one like SpaceDock.
<DasSkelett[m]> And in 99% of cases the mod _is_ compatible with multiple KSP versions, be it only the whole minor range (1.10.*), but often also multiple minor ranges (currently you often see 1.8-1.10)
<Darklight[m]> Ah gotcha
VITAS[m] has quit [Quit: Idle timeout reached: 10800s]
VITAS[m] has joined #spacedock
<VITAS[m]> wildcards?
<VITAS[m]> cool
<DasSkelett[m]> And another powerful feature is the option to specify a remote version file, hosted e.g. on GitHub. So for example when KSP 1.11 comes out, and the mod authors determine that their mods still work fine and don't need an update, the can update that remote version file to set the max KSP version to 1.11, and CKAN (and AVC) will note that and update the metadata accordingly.
<DasSkelett[m]> You can (and should) also update the compatibility on SpaceDock, but you'll loose the old KSP version there.
<VITAS[m]> there was a docu ive read a long time ago about ckan structures
<Darklight[m]> I am lazy and only care about the latest version though, I never bothered
<Darklight[m]> I only lock releases to the second number these days, I used to lock to the third when ksp released parts in a revision update...
<Darklight[m]> But these days I am like "fuck it, 1.8 1.9 1.10 you can all play together"
<Darklight[m]> You just need the right version to do so :P
<VITAS[m]> so in case of dmp i could use a 1.8 client and join a 1.10 server?
<Darklight[m]> Yeah, but you need the backport-1.8 version I made
<Darklight[m]> I made it because it lacks the freeze bug
<VITAS[m]> so not 1.8 but 1.8*smallprint version
<VITAS[m]> :D
DasSkelett[m] has quit [Quit: Client limit exceeded: 6]
VITAS[m] has quit [Quit: Client limit exceeded: 6]
KineticSloth8StoneBlue[m has joined #spacedock
<KineticSloth8StoneBlue[m> yeah... as a mod user and ~~hoarder~~ Archivist, I *LOVE* version files... wish it was required that EVERY mod have them...
<KineticSloth8StoneBlue[m> Since I have copies of 95% of every version made, for 95% of every KSP mod ever made, version files are critical for me to accurately identify both the version of each package, as well as KSP version compatability for each package 😉
Darklight[m] has quit [Quit: Client limit exceeded: 6]
Flart[m] has joined #spacedock
<Flart[m]> could you make it 1.10.0 ?
<Flart[m]> it always was having patch within
<Flart[m]> it was having patch within in the last few version
<Flart[m]> it was having patch within in the last few versions
<Flart[m]> I updated my script, but still
KineticSloth8StoneBlue[m has quit [Quit: Client limit exceeded: 6]
VITAS[m] has joined #spacedock
<VITAS[m]> demo 0.25?
<VITAS[m]> theres balsa in there
<VITAS[m]> and maybe even factorio
HebaruSan[m] has joined #spacedock
<HebaruSan[m]> Stardew Valley somehow
<VITAS[m]> yes
<VITAS[m]> simply all versions of all games in the db
<HebaruSan[m]> And Stellaris!
Flart[m] has quit [Quit: Client limit exceeded: 6]
DasSkelett[m] has joined #spacedock
<DasSkelett[m]> But <span class="d-mention d-user">Flart</span> has a point, you previously included the patch version `.0` in all releases, but you didn't for 1.10. Would be good to keep it consistent.
<HebaruSan[m]> Agreed
Flart[m] has joined #spacedock
<Flart[m]> <span class="d-mention d-user">VITAS</span> https://spacedock.info/api/kspversions
<VITAS[m]> there are multiple problems
<VITAS[m]> 1.) can you change the game versions to something uniform via admin panel DasSkelett ?
<DasSkelett[m]> No, admin panel doesn't let you edit anything, only add.
<VITAS[m]> 2.) api path shouldnt be called kspversions but have somethign that filters game and then generic game version name
<VITAS[m]> ok ill add it then
<HebaruSan[m]> <span class="d-mention d-user">Flart</span> try https://spacedock.info/api/3102/versions
<VITAS[m]> so it exists
<VITAS[m]> all we have to do is to get rid of kspversions
<VITAS[m]> ?
<VITAS[m]> and ive to rename 1.10 to 1.10.0?
<DasSkelett[m]> Yes I noted that before that /api/kspversions returns all game versions, not only KSP versions. I think I have a fix for that already locally.
<VITAS[m]> is there a need for something that returns all versions?
<VITAS[m]> if not remove it
<VITAS[m]> if yes rename it to something generic
<VITAS[m]> 1.10 is now 1.10.0
<HebaruSan[m]> I feel like a better route would be `/api/game-versions/<game_short>`
<HebaruSan[m]> I feel like a better route would be `/api/game_versions/<game_short>`
<VITAS[m]> or /api/game/<gameshort>/versions
<DasSkelett[m]> Yes definitely <span class="d-mention d-user">HebaruSan</span>. The concept to put the variables _before_ the action is bad API design, it makes adding additional routes afterwards very complicated (as we saw with the latest-modversion-download).
<VITAS[m]> the path without versions could supply general info about a game
<HebaruSan[m]> `/api/games` exists but returns info for ALL games, dubiously save to execute in prod with 20000+ games
<VITAS[m]> im for a tight tree structure
<VITAS[m]> aka combine as many functions under a parent path
<VITAS[m]> makes it easier to remember the syntax
<HebaruSan[m]> *edit:* ~~`/api/games` exists but returns info for ALL games, dubiously save to execute in prod with 20000+ games~~ -> `/api/games` exists but returns info for ALL games, dubiously safe to execute in prod with 20000+ games
<HebaruSan[m]> Easy to remember is not an important consideration for an API in comparison to anything else
<VITAS[m]> we might want to implement a req limit per day and user
<VITAS[m]> and have user auth via token
<HebaruSan[m]> Nah, just make the routes safer
<VITAS[m]> all games could have "pages"
<VITAS[m]> i know auth is a lot to implement but it would allow to limit scrapers
<HebaruSan[m]> `/api/games` should probablyb e filtered by active=True
<HebaruSan[m]> `/api/games` should probably be filtered by active=True
<HebaruSan[m]> which in turn should be indexed
<DasSkelett[m]> Before we start considering some rate limiting for the API, we should consider whether it is actually needed.
<DasSkelett[m]> <span class="d-mention d-user">HebaruSan</span> I think I also have a fix for active=true stashed somewhere.
<HebaruSan[m]> Are scrapers a problem?
<VITAS[m]> yes
<VITAS[m]> 4-5 times a year
<VITAS[m]> i blocked some
<DasSkelett[m]> Additionally to indexing Game.active, we should also think about removing all the unneeded games. IMHO that's not anything that should be in a production database.
<DasSkelett[m]> If your <span class="d-mention d-user">VITAS</span> wants to keep the list for personal archiving, you could copy it to a separate database or export it somehow.
<VITAS[m]> e.g. the russians are one party
<DasSkelett[m]> *edit:* ~~Additionally to indexing Game.active, we should also think about removing all the unneeded games. IMHO that's not anything that should be in a production database.
<DasSkelett[m]> If your <span class="d-mention d-user">VITAS</span> wants to keep the list for personal archiving, you could copy it to a separate database or export it somehow.~~ -> Additionally to indexing Game.active, we should also think about removing all the unneeded games. IMHO that's not anything that should be in a production database.
<DasSkelett[m]> If you <span class="d-mention d-user">VITAS</span> wants to keep the list for personal archiving, you could copy it to a separate database or export it somehow.
<VITAS[m]> and then theres the viatnamese and the chinese
<VITAS[m]> so i know at least 3 copy cat sites
<VITAS[m]> we could remove them but if the api doesnt care about the active flag theres no point in having it
<VITAS[m]> and i like to be able to disable but not remove a game
<HebaruSan[m]> ?
<VITAS[m]> !
<DasSkelett[m]> This editor.js is an external dependency? I'm trying to figure out why it displays this control bar when creating blogs but not when editing them.
<HebaruSan[m]> I think that might be "CodeMirror". I experimented with installing it via `npm` and eventually reverted it
<HebaruSan[m]> Also could not figure out how to get the toolbar to show up
<DasSkelett[m]> Oh well, if you couldn't get it to work either, I'm not gonna waste much time with it.
<HebaruSan[m]> FYI, just doing that gave the text box a gray background, hence the stylesheet change in the same commit
<HebaruSan[m]> If you let me know which PR you want to look at next (if any), I can rebase just that one after merging the blog PR to avoid having to update five alembic scripts with a new down id
<DasSkelett[m]> Sure. Let's do the modpack fixes #296, they're waiting the longest
Flart[m] has quit [Quit: Client limit exceeded: 6]
Github[m] has joined #spacedock
<Github[m]> https://github.com/KSP-SpaceDock/SpaceDock/pull/296 : A test commit for a test PR
<DasSkelett[m]> It's #293 ...
<DasSkelett[m]> Ah damn it
<VITAS[m]> nearly usefull
<HebaruSan[m]> It did save me from clicking on 296
<DasSkelett[m]> The embed could be smaller without a dozen of empty lines and an oversized logo
<VITAS[m]> im using the default bot thats run by matrix
<VITAS[m]> i could set up my own...
<HebaruSan[m]> <span class="d-mention d-user">DasSkelett</span> OK, 293 is now rebased on top of current alpha, with its alembic script sequenced after 303's
<DasSkelett[m]> Haha, just found a hilarious bug while testing the PR (don't think it has anything to do with it though).
<DasSkelett[m]> When clicking "Follow" on a mod for a game with a non-semantic version it redirects you to the registration page.
<HebaruSan[m]> I just saw that too!
<HebaruSan[m]> DIdn't happen at all during dev
<HebaruSan[m]> You figured out the cause already?
<HebaruSan[m]> Ahhh
<DasSkelett[m]> I have a game in the database with the version `seeding-1.0 `
<DasSkelett[m]> However I have yet to find out why it takes you to the registration page...
<DasSkelett[m]> Also prevents you from creating mods for Balsa on production: `500 Internal Server Error: Invalid version: 'demo-0.25'`
<HebaruSan[m]> Maybe it's all these duplicates thanks to the CKAN import script?
<HebaruSan[m]> So the follow/unfollow buttons just assume that any and every exception from the server is caused by not being logged in
<DasSkelett[m]> Ouch
<DasSkelett[m]> We don't json-ify exceptions for /mod, so we have to go with a status code check there I think.
Github[m] has quit [Quit: Client limit exceeded: 6]
VITAS[m] has quit [Quit: Client limit exceeded: 6]
<DasSkelett[m]> And regarding the `versions_behind()` logic: should we ignore that score penalty for games with versions that the library can't parse?
<DasSkelett[m]> And rename `demo-0.25` on production to `0.25`? Or `0.25-preview`, which would be allowed if I read the regex correctly.
HebaruSan[m] has quit [Quit: Client limit exceeded: 6]
VITAS[m] has joined #spacedock
<VITAS[m]> -preview or -demo is ok
<VITAS[m]> just renaming it 0.25 will colide with a possible 0.25 alpha,beta or stable release
<DasSkelett[m]> `-demo` wouldn't be allowed
HebaruSan[m] has joined #spacedock
<HebaruSan[m]> Let's just ignore those versions
<HebaruSan[m]> There are few of them, and they're weird
<DasSkelett[m]> Haha, I've written a similar code. But I'd restrict the except to `except version.InvalidVersion`, we might want to now of other errors (and it avoids that linting warning)
<VITAS[m]> so how do you want to tag demos and betas etc.?
<HebaruSan[m]> Idk, they're not a core part of what that algorithm is trying to accomplish
<DasSkelett[m]> In reality it's very unlikely that Balsa will have version 0.25 at release. It will probably be 1.x. If there's another alpha/beta weekend or early access, it'll also likely be another version.
<VITAS[m]> so the problem is non numbers?
<VITAS[m]> how a bout a flag for demo, beta, release?
<DasSkelett[m]> Users should be able to realize that 0.x means alpha/beta/pre-relase whatsoever, and >1.x means full release.
<VITAS[m]> i want to go by official version numbers
<DasSkelett[m]> Sure
<VITAS[m]> and we cant be sure that either ksp2 (if that applies) and balsa use consitant numbers with what we think
<VITAS[m]> so we need a way to deal with either prefixes or sufixes to version numbers or flags to compensate for inconstant version numbers
<VITAS[m]> btw what happens if theres a 0.25b
<VITAS[m]> ?
<DasSkelett[m]> `0.25b` is valid for the library.
<VITAS[m]> and dots are too?
<DasSkelett[m]> Dots where?
<VITAS[m]> 0.25.b
<VITAS[m]> i find it unlikely someone would name versions that way. so if its to much work to have a flag any other way lets do 0.25.d for demo
<VITAS[m]> and nothing for full evrsions
<DasSkelett[m]> But didn't you want to keep the official version number?
<HebaruSan[m]> The version names can stay as they are, we will handle them in code
<VITAS[m]> yes i would love to
<VITAS[m]> but i feel you dont want to flag versions
<VITAS[m]> and i want at least flag demos
<DasSkelett[m]> Yeah so what? Go with 0.25 then, it's the official version number.
<DasSkelett[m]> Why? Do you expect them to go backwards again or what?
<VITAS[m]> no i expect full versions for 0.25 to exist
<HebaruSan[m]> Hey, <span class="d-mention d-user">VITAS</span>, don't worry about the versions. The code will handle whatever they are.
<VITAS[m]> ok so i can keep it the way its now?
<VITAS[m]> i took away from the cheatlog that its a problem if i prefix demo- to version numbers
<DasSkelett[m]> Yeah okay, let's keep it as is, and in the unlikely event someone already creates a mod for Balsa it just won't get a penalty over time. Probably won't matter much.
<VITAS[m]> someone did but we dont allow it
<VITAS[m]> because it contains dlls from the demo
<VITAS[m]> and balsa is only the current case :)
<VITAS[m]> i still hope ksp2 open modding will be a thing
<HebaruSan[m]> Does Take Two have a closed modding framework?
<VITAS[m]> dont know but i guess
<VITAS[m]> i see no point in discussing every game that i could ever add to sd now :)
DasSkelett[m] has quit [Quit: Client limit exceeded: 6]
HebaruSan[m] has quit [Quit: Client limit exceeded: 6]
DasSkelett[m] has joined #spacedock
<DasSkelett[m]> Oups, got a little carried away with Grafana... At least I have shiny colourful traffic graphs now, even displaying the correct values now 😄
<VITAS[m]> bling
<VITAS[m]> i made beef jerky today
HebaruSan[m] has joined #spacedock
<HebaruSan[m]> Ooh, we should have a label for PRs with alembic in them
uovo has quit [Read error: Connection reset by peer]
uovo has joined #spacedock
Darklight[m] has joined #spacedock
<Darklight[m]> I had a horrible thought when sleeping, you may want to keep things merged into alpha rather than making a bazillion PRs because of alembic upgrades
<Darklight[m]> Unless alembic has a merge feature too
<HebaruSan[m]> I just rebase as they get merged
<HebaruSan[m]> Seems OK so far
<HebaruSan[m]> What about, "Type: Data Migration" ?
<HebaruSan[m]> Or just "Type: Alembic Stuff"
<DasSkelett[m]> Alembic theoretically has a merge feature, where you can have multiple scripts with the same down revision and create a merge revision (basically Git Lite), but I'd prefer to not use it if possible, because that sounds like a big opportunity to introduce even more migration bugs and problems.
<HebaruSan[m]> Yeah it's easy enough to keep everything linear
<Darklight[m]> If you keep it merged then yeah :P
<VITAS[m]> so i can expect db schmea changes on the next prod update?
<DasSkelett[m]> Regarding label: "Type: Alembic Stuff" probably fits better than "Type: Data Migration", because Alembic also does schema migration (you could argue it's even its primary use case).
<DasSkelett[m]> Yes <span class="d-mention d-user">VITAS</span>
<VITAS[m]> k
<DasSkelett[m]> <span class="d-mention d-user">Darklight</span> what do you mean with keep it merged? No reverts?
<Darklight[m]> Lasttime I looked you had weeks worth of work dangling in PR branches
<DasSkelett[m]> Hm, I don't think I understand :/
<DasSkelett[m]> Ohh, do you mean just merging stuff into alpha without awaiting review? I doubt you could convince <span class="d-mention d-user">HebaruSan</span> or me to do that.
<Darklight[m]> I think I will just leave you too it, but I feel like it is bad to leave PRs lying around gathering dust is all, especially if your upgrade window is far away
<DasSkelett[m]> That's why I'm reviewing them as of this moment 🙂
<Darklight[m]> PRs are good for "I am unsure" about this, but if its a simple change or definitely going in, well fixing commits are free :P
<HebaruSan[m]> Seems like a solution in search of a problem to me so far
<HebaruSan[m]> The beauty of PRs is exactly that you can work up a change and then let it sit there for a while till someone has a chance to look at it
<DasSkelett[m]> PRs are a great opportunity to actually test the changes you/others are about to merge. I for one rarely klook
<DasSkelett[m]> * look at code changes after they have been merged.
<HebaruSan[m]> Yeah and if commits are just rolling in willy-nilly, how would you even determine an appropriate cut-off point for what still needs review?
<Darklight[m]> I'll leave you to it, I don't work on spacedock as you can see, any any approach is fine if it works, but I can't shake the feeling that things need to actually live in alpha and messed with to see where things break
<VITAS[m]> i dont want complications like we had on the last 2 updates
<HebaruSan[m]> Well the docker stuff is pretty great for that actually. Local copy of the server running a particular branch, pretty easy to take a change through its paces.
<VITAS[m]> i vote for whatever prevents them
<VITAS[m]> Darklight loves docker
<Darklight[m]> hiss
<VITAS[m]> i will take another dive into kubernetes with ansible once i get a free computer to set up a test env
<HebaruSan[m]> Not interested in taking sides in some new vi-vs-emacs war, just like being able to start the web site with changes and play with it
<VITAS[m]> as you can see fromt he docker files: we did play with docker back when both SD and docker where new
<HebaruSan[m]> Whatever gets that done is fine b yme
<HebaruSan[m]> Whatever gets that done is fine by me
<VITAS[m]> i say: if you HebaruSan want to do docker i welcome you to make docker files
<HebaruSan[m]> There's already docker files. `./start-server.sh` runs them.
<DasSkelett[m]> I think the current Dockerfiles in the repo come from AllisTauri, and they work like a charm.
<VITAS[m]> no
<VITAS[m]> they are fromt homasp
<VITAS[m]> thomasp
<VITAS[m]> and me
<VITAS[m]> originaly
<VITAS[m]> AllisTauri might have modernized them
<DasSkelett[m]> The current
<VITAS[m]> i havent looked at them in a long time
<VITAS[m]> i dont know how well docker runs inside lxc containers
<VITAS[m]> theres docker on the physical host but i dont want to usie it to test code
<HebaruSan[m]> Aaaaanyway, point is, we're not flying blind on these feature branches. Everything's getting tried out on a running server, via a service that apparently some people don't like.
<VITAS[m]> problem being: how to get testers?
<HebaruSan[m]> Frankly, I don't think you can. There's nothing to offer them.
<HebaruSan[m]> At most, a little bit of glory if they report a bug they found on beta, which might get fixed someday?
<VITAS[m]> yes its sad that i need to offer them something apart of supporting a core community service
<VITAS[m]> we should also talk about the problems we had during the last update and how to avoid similar ones
<VITAS[m]> ive to go to bed now but you might want to give it a look
<HebaruSan[m]> If I had to stake my life on finding beta testers, I think I'd go with approaching specific individuals known to participate and rely on SpaceDock, like certain modders
<HebaruSan[m]> Just a blanket, "Hey everybody come help out" is going to suffer from the tendency to think that somebody else will do it
<HebaruSan[m]> But if the SD admin comes to you and says, "We'd like YOUR help specifically", that's a different animal
<HebaruSan[m]> And if they're good, you only need like 3-5 of them
<VITAS[m]> youre right
<VITAS[m]> its like when you want to control crowds or need help: address individuals
<VITAS[m]> its past midnight
<VITAS[m]> so good night
<VITAS[m]> and thank you
<DasSkelett[m]> Good night 👋
<HebaruSan[m]> Nighty night
<DasSkelett[m]> <span class="d-mention d-user">HebaruSan</span> I've left you a small PR with mostly visual changes for the modpack branch: https://github.com/HebaruSan/SpaceDock/pull/1
<DasSkelett[m]> Otherwise it looks really solid.
<HebaruSan[m]> I think the if/newline thing was something I did initially and then reverted because it resulted in labels and values without spaces in between, i.e.
<HebaruSan[m]> `Public:Yes` vs `Public: Yes`
<HebaruSan[m]> Can you check that on your branch?
<DasSkelett[m]> Oh interesting, now that you say it, I see it, yes.
<HebaruSan[m]> Yeah, once I saw that I decided to leave it as-is
<HebaruSan[m]> Maybe there's a better way
<DasSkelett[m]> Found one! Put a `&nbsp;` at the end of the span lines
<HebaruSan[m]> I guess I didn't try putting a space in the label's <span>, maybe that would do it
<DasSkelett[m]> Would probably work too
<DasSkelett[m]> Yes it does 🎉 gonna push it to my PR branch
<DasSkelett[m]> Apparently all users are able to see unpublished/locked mods in mod lists. I think we should hide them in the default, public view, but still include them in the edit view (maybe with a note that it is unpublished, so that the pack author knows why it doesn't appear?).
<Darklight[m]> Hrmmm... they never used to show up I think, at least on the gameshort page
<DasSkelett[m]> It's related to PR # 293
<DasSkelett[m]> (although it looks like that problem existed before)
* DasSkelett[m] uploaded an image: unknown.png (78KB) < https://matrix.52k.de/_matrix/media/r0/download/52k.de/opfHAyvnCCICtyZVGVIuduYy >