VITAS changed the topic of #spacedock to: Problems?: https://github.com/KSP-SpaceDock/SpaceDock/issues | Matrix/Riot Chat: https://im.52k.de +spacedock:52k.de Feel free to ask for help, we only bite a little bit! | If you want to help, please check https://github.com/KSP-SpaceDock/SpaceDock-Backend/issues/5 :) | <VITAS> inet users have the attentionspan of a squirrel...
VITAS[m] has joined #spacedock
<VITAS[m]> so am i a good bot? :D
<VITAS[m]> betw yay to that webhook fix
DasSkelett[m] has joined #spacedock
<DasSkelett[m]> Yes you are!
<VITAS[m]> :D
<VITAS[m]> thats so fun
<VITAS[m]> but he knows im not a bot i think
<DasSkelett[m]> Well I hope so
<VITAS[m]> i think people takint the bot tag serious even when im writing normaly tells you something about those people
<VITAS[m]> they belive the lable
DasSkelett[m] has quit [Quit: Idle timeout reached: 10800s]
DasSkelett[m] has joined #spacedock
<DasSkelett[m]> I'm thinking about adding some sort of notice to the mods page, that is shown when an author hits the "Add mod to CKAN" button, linking to https://github.com/KSP-CKAN/NetKAN/pulls, asking to leave a comment with additional information like install instructions, recommendations, dependencies and so on.
<DasSkelett[m]> Because right now someone not knowing the innerworkings of CKAN and how/where new mods are added might be confused about what actually happens, and has a hard time finding the place to check the indexing progress of the mod.
<VITAS[m]> feel free
<VITAS[m]> now that alpha and beta are cleaned up youre welcome to add new stuff to alpha
<DasSkelett[m]> First I have to review some PRs that HebaruSan is showering me with over at CKAN 😛
<VITAS[m]> k
DasSkelett[m] has quit [Quit: Idle timeout reached: 10800s]
Darklight[m] has joined #spacedock
<Darklight[m]> o/
<Darklight[m]> VITAS had it pinned in the thingy
DasSkelett[m] has joined #spacedock
<DasSkelett[m]> <span class="d-mention d-user">HebaruSan</span> we've got a problem with SpaceDock on production. You can't download newly uploaded mods.
<Darklight[m]> I just fixed prod data, but we need to stop the full path getting into the database
<DasSkelett[m]> This is what you get redirected to when clicking a download button, and that link is obviously a 404.
<Darklight[m]> 14223 is probably the first thing uploaded since we updated spacedock
<VITAS[m]> ok now i understand y your problem is that new since yesterday?
HebaruSan[m] has joined #spacedock
<HebaruSan[m]> Is this something we changed, and can I even access production?
<VITAS[m]> a fix could be to just subdtract the storage path from the db string
<Darklight[m]> It affects everything atm because prod got updated to beta, and beta was merged from alpha, all branches are the same atm
<VITAS[m]> no you cant only darklight can
<Darklight[m]> Also, vitas might have some dangling commits in master still that aren't in alpha, it doesn't FF merge
<Darklight[m]> But that is a different problem
<DasSkelett[m]> I think the problem is here, `_save_mod_zipball` returns an absolute path, and we save it in the DB:
<VITAS[m]> can we roll prod back
<VITAS[m]> ?
<VITAS[m]> to yesterdays code
<HebaruSan[m]> So it's a <span class="d-mention d-user">AllisTauri</span> change
<VITAS[m]> till we fixed it?
<Darklight[m]> Can we just change directory to storage and run that code?
<VITAS[m]> test it on alpha or beta first
<VITAS[m]> then on prod
<VITAS[m]> and i would scedule another downtime for that
<Darklight[m]> file_path = os.path.join(storage_path, filename) oh nevermind
<Darklight[m]> that thing should return a relative path
<VITAS[m]> to give us time i would like to roll back prod to the old code till thats done
<Darklight[m]> Or maybe not 🤔
<VITAS[m]> are you listening?
<DasSkelett[m]> I think this line should return `storage_base` + `filename`:
<VITAS[m]> ok seems like im a ghost now :/
<Darklight[m]> What does it return in master? probably a good thing to compare against
<Darklight[m]> Old code
<Darklight[m]> there in ModVersion()```os.path.join(base_path, filename)```
<DasSkelett[m]> Yes, exactly. This is it
<Darklight[m]> which I assume is now storage_path
<VITAS[m]> ok ill see my self out
<DasSkelett[m]> No, base path is now `storage_base`, it's the username + id + mod name. filename is modname + version
<Darklight[m]> Ah yeah sorry I'm blind
<Darklight[m]> Do I need to do this commit or can someone else push it 😛
<VITAS[m]> yes you are
<VITAS[m]> you cant even read what im writing :/
<Darklight[m]> Obviously into alpha where we can test
<VITAS[m]> thank you even partialy
<DasSkelett[m]> Yes, I'll do it.
<HebaruSan[m]> What about cleanup, is there bad data in the db now?
<Darklight[m]> Kk thankyou, I have access to prod, I think this is high enough priority to warrant a quick 30s downtime, I can get it updated that quickly
<Darklight[m]> Yeah, I've already done it manually with psql
<Darklight[m]> There was only about 10 mods
<Darklight[m]> It's a bit pants shitty but we have a db backup from yesterday so all g
<Darklight[m]> Should learn to use that rollback commit feature for manual commands...
VITAS[m] has quit [Network ban]
im52kde has quit [Network ban]
HebaruSan[m] has quit [Network ban]
Darklight[m] has quit [Network ban]
VITAS[m] has joined #spacedock
im52kde has joined #spacedock
<VITAS[m]> NO QUICK DOWNTIMES!!!!!
<VITAS[m]> at least post on the forums beforehand
<VITAS[m]> and after youve finished
<VITAS[m]> dont just take the site offlien. it will look like an accident
<VITAS[m]> thank you
<VITAS[m]> ok sould i set the downtime for in 1h?
<VITAS[m]> i still think we should roll back the code to yesterday
<VITAS[m]> and test it without rushing
<VITAS[m]> ok ill advertise in 1h for 30min?
<DasSkelett[m]> PR is up
<VITAS[m]> i want announcements and have just made one
<VITAS[m]> Darklight can do his walk and DasSkelett can make sure with some testing on alpha that it fixes the problem without new sideeffects
<VITAS[m]> downtime starts in 1h for 30min
<VITAS[m]> everybody ok with that? :)
<VITAS[m]> np
<VITAS[m]> go
<DasSkelett[m]> Okay I'd merge the change to alpha now, to test it.
<DasSkelett[m]> Depending on whether or not our auto-update triggers work currently, someone needs to ssh into alpha to pull it (since I'm not in reach of my PC with SSH access to SD).
<DasSkelett[m]> Darn it. What was that again, it only worked via IPv6? Or it only worked via IPv4?
<DasSkelett[m]> Let's try it!
<DasSkelett[m]> Okay PR is merged, can you see in the repository settings whether the webhook call succeeded <span class="d-mention d-user">HebaruSan</span> ?
<VITAS[m]> test
DasSkelett[m] has quit [Remote host closed the connection]
im52kde has quit [Remote host closed the connection]
VITAS[m] has quit [Remote host closed the connection]
im52kde has joined #spacedock
VITAS[m] has joined #spacedock
<VITAS[m]> uhm
<VITAS[m]> why isnt what im writing not showing up?
<VITAS[m]> can you read this?
<VITAS[m]> i had to go to discord
<VITAS[m]> strange
<VITAS[m]> i can still read you guys
<VITAS[m]> in matrix
<VITAS[m]> so maybe
<VITAS[m]> i restarted the im vm but no change
<VITAS[m]> yes
<VITAS[m]> anyways
<VITAS[m]> i dont like patchin prod without testing
<VITAS[m]> i would like to roll back prod to yesterdays code if possible
<VITAS[m]> and fix this on alpha and beta without haste
<VITAS[m]> am i reading correctly that both of you are having problems connecting to alpha and beta?
<VITAS[m]> ok you can always try ssh from beta to alpha
<VITAS[m]> ill check alpha the hook onno
<VITAS[m]> try sd6.52k
<VITAS[m]> or its lan ip
<VITAS[m]> ok ill check
<VITAS[m]> beta webalpha hook worked beta is showing warning
<VITAS[m]> sd6.52k:21352
<VITAS[m]> sd6.srv.52k.de.:21352
<VITAS[m]> yes
<VITAS[m]> what else do you need?
<VITAS[m]> you dont do ports like this in ssh
VITAS[m] has quit [Quit: Client limit exceeded: 6]
Darklight[m] has joined #spacedock
<Darklight[m]> I'm back
VITAS[m] has joined #spacedock
<VITAS[m]> i can only say the hook has checkmark
<Darklight[m]> VITAS: I can update prod in under 30seconds 🙂
<Darklight[m]> But when people give me the signal
<VITAS[m]> can you gurantee itll work after that?
<Darklight[m]> Possibly test everything first 🤔
<Darklight[m]> I can easily reset to the previous commit if something is bad
<VITAS[m]> darklight go and test it with the guys on alpha
<VITAS[m]> i fear we might unearth new problems
<VITAS[m]> i would prefer to fix this and test it while running yesterdays code on prod
<Darklight[m]> Will do
<Darklight[m]> Why don't we make it autoupdate though 🤔
<Darklight[m]> It hasn't updated
<VITAS[m]> we can start stoping prod at in 24min
<VITAS[m]> -at
<VITAS[m]> beta is returning 403 errors
<Darklight[m]> Ok, just updated
<Darklight[m]> I mean
<Darklight[m]> I just stopped alpha, updated, restarted
<Darklight[m]> Err restart seems to be hanging a bit..
<Darklight[m]> Probably the requirements fetch
<Darklight[m]> There it goes
VITAS[m] has quit [Quit: Client limit exceeded: 6]
<Darklight[m]> Not a valid zip file for me too
<Darklight[m]> Maybe return a tuple?
<Darklight[m]> Full path and relative
<Darklight[m]> Or use outs, I don't know how python does crap
<Darklight[m]> I guess I should look into the autoupdater...
VITAS[m] has joined #spacedock
<VITAS[m]> yes but maybe after we fixed prod?
<Darklight[m]> Kk
<Darklight[m]> Also, master has dangling commits that aren't in alpha and beta, I'm not sure where the diverged though
<Darklight[m]> Because it does not --ff merge
<VITAS[m]> all i can say is that alpha webhook has checkmark and beta has warnign with 403 error
<Darklight[m]> Alpha didn't actually update though
<VITAS[m]> so can we roll prod back to the old code without problem?
<VITAS[m]> i want to give us time to figure everything out without time limit
<Darklight[m]> Yeah but we should be right
<Darklight[m]> git reflog is your friend on that one
<VITAS[m]> if youre sure you can pull this off you can fix the current code
<Darklight[m]> It shows you checkouts/commits/resets etc
<Darklight[m]> It's a very tiny issue, it's just saving the full path in the database
<Darklight[m]> Like, it breaks spacedock, but not in a destructive way
<Darklight[m]> I've already manually fixed up the 10 uploads
<Darklight[m]> (I'll check again for new ones after we fixed in prod)
<VITAS[m]> all i want is prod to work properly in 30min
<Darklight[m]> The update will be very fast, people should not notice
<VITAS[m]> we will do it your way in 6min
<VITAS[m]> :)
<Darklight[m]> We need to verify the fix first 😛
<VITAS[m]> as i said: once downtime is over we need a working prod
<VITAS[m]> preferebly without the current bug :)
<VITAS[m]> good work
<VITAS[m]> just in time
<Darklight[m]> I'll update alpha
<Darklight[m]> Oh you actually do PR's
<Darklight[m]> I thought it was in alpha already
<Darklight[m]> I honestly don't acknowledge githubs existance and just use them as a git remote
<VITAS[m]> maintanance window on pro starts now for 30min
<Darklight[m]> Updating now
<Darklight[m]> Starting..
<Darklight[m]> Wow this takes foooorever
<Darklight[m]> It's probably the pip line
<Darklight[m]> There it goes
<VITAS[m]> or i capped the cpu to much
<Darklight[m]> The requested URL /content/godarklight_6/Test_mod/Test_mod-v1.0.zip was not found on this server.
<Darklight[m]> Getting closer.
<Darklight[m]> It starts at the username though
<Darklight[m]> Err one sec
<Darklight[m]> DB looks correct now
<Darklight[m]> Ah that's good news then
<Darklight[m]> That means it actually would work in prod as is, but yeah still a tiny bug that needs fixing 😛
<Darklight[m]> Do we want to update prod now or not?, everything else seems ok
<VITAS[m]> yes
<VITAS[m]> go
<VITAS[m]> i understand it wont make things worse
<Darklight[m]> Kk, going for it now
<VITAS[m]> and my placeholder site works when sd is down
<VITAS[m]> yay for that
<Darklight[m]> Coming back up
<Darklight[m]> Also, I just hard reset to alpha, I forgot we should probably have merged in beta 😛
<VITAS[m]> yes
<Darklight[m]> I'll quickly do that 🙂
<VITAS[m]> can we test if prod is now fixed?
<Darklight[m]> I think skelett has a testing mod
<VITAS[m]> thx
Darklight[m] has quit [Quit: Client limit exceeded: 6]
<VITAS[m]> kewl
<VITAS[m]> thank you guys :)
Darklight[m] has joined #spacedock
<Darklight[m]> I just checked the db, we are good
<Darklight[m]> I'll see why alpha and beta don't autoupdate now, it looks like sudoers is wrong for a start
<VITAS[m]> now youre welcome to fix autoupdating on alpha :)
<VITAS[m]> ill post on the forums then that were finished?
<Darklight[m]> For the update command, yes
<Darklight[m]> But the update command is kinda wrong too 😛
<VITAS[m]> ill go catch some sleep now thanks again and night
<Darklight[m]> Having it update on start is probably the way to go, then I can just stop/start it
<Darklight[m]> I can't do restart because apparently restart doesn't restart systemd submodules, stop and start work normally though
<Darklight[m]> I'll figure it out in my qemu first
<Darklight[m]> There is already existant code for dealing with /hook, I'll see if I can fix it and then merge it into beta which should signal it properly and get it to autoupdate if I've done everything correctly
<Darklight[m]> Oh I just saw the systemd stuff there
<Darklight[m]> Resetting up my vm so that it matches the file structure of real spacedock..
<Darklight[m]> I had my git in the home directory
<Darklight[m]> Pushing a change, gunicorn wasn't in requirements and /usr/bin/systemctl exists which stopped alpha from updating
<Darklight[m]> beta's key was wrong so I've fixed that