HebaruSan has quit [Ping timeout: 190 seconds]
HebaruSan has joined #spacedock
<HebaruSan> Testing 1 2 3
<VITAS> HebaruSan, i did a thing. Maybe sudo is working now?
<VITAS> ill dump the current apache config in your home folder on sd2 so you know how production does its thing.
<VITAS> ATS simply slabs http/2,SSL and all the other good things on it and relays sd1.52k:80
<VITAS> so if you want to replicate just do the same on sd2 &sd6
<VITAS> sd2.52k:80 is allready mapped to sd2.52k.de and sd6 does the same
<VITAS> file is in your home dir
<HebaruSan> Yes, sudo works now, thanks!
<VITAS> cool
<VITAS> and was so simple to fix :P
<HebaruSan> config.ini has the following. Does it work?
<HebaruSan> # Configure these to determine who can send GitHub-style hook notifications to the site
<HebaruSan> # to configure an automatic redeployment. Defaults to trusting GitHub and localhost.
<HebaruSan> hook_ips = 185.199.108.0/22,140.82.112.0/20,192.30.252.0/22,127.0.0.1
<HebaruSan> hook_repository=KSP-SpaceDock/SpaceDock
<HebaruSan> hook_branch=master
<HebaruSan> restart_command=sudo systemctl restart spacedock.target
<HebaruSan> This looks like a good way to handle the auto restarts when code changes are committed
<VITAS> there you go :)
<VITAS> i havent tried it
<VITAS> its years that we had a real dev setup
<VITAS> and i havent coded python since then
<VITAS> (i only learned it to do spacedock anyways)
<VITAS> so youre the person to know :D
<VITAS> at least of us two
<VITAS> im mainly a webdev (mostly php,js,....) and sys integrator.
<VITAS> i can build you a datacenter if you wish :P
<VITAS> ah and i did computer & mediasience as well as atrs school so i can analyze the crap out of your website and then paint you one. :>
<Lartza> There are certainly things to restart things when github things happen :P
<Lartza> I went through a couple before finding one that suited me though
<VITAS> hi and yes
<Lartza> Hello :)
<VITAS> youre welcome to help HebaruSan
<VITAS> you know the site a bit :)
<Lartza> I ended up using https://github.com/olipo186/Git-Auto-Deploy but even that had some shortcomings iirc
<Lartza> Could have just been messing with permissions...
<Lartza> Since it clones as one user and I would run the daemon as another user
<Lartza> Also who has the right to run systemctl restart for the affected daemons
<Lartza> Didn't want root to clone stuff and neither to run the daemon so
<Lartza> Oh yeah it still clones as the GAD user but the daemon user then has rights to the files that are not on git like logfile and config
<VITAS> darklight did some systemd stuff to production where you can do that service spacedock restart
<VITAS> the dev systems have nothing yet so thats an option for the willing
<VITAS> Lartza, you could get acc to those dev machines and help HebaruSan set them up...
<Lartza> Not sure I really have time :/ Procastination is a bitch :P
<VITAS> your call
<VITAS> i cant imagine your lazyness so i dont try
<VITAS> :P
<VITAS> i need an ssh key if you like to
<HebaruSan> Can I get a login on the reverse proxy server? Not sure how that works, but I think everything else depends on getting that right first
<VITAS> no sorry
<VITAS> its not serving spacedock alone
<VITAS> if you tell me the settings you need i can do them
<HebaruSan> Fixing redirects --> Merging code back to GitHub --> Creating branches for staging and testing --> Bringing the servers live
<VITAS> for me its more like: setting up dev machines -> marging code-> fixing redirects (like any other bug)
<VITAS> because it also exists in the github code
<HebaruSan> Unless it's not a code problem, which is what that ProxyPassReverse setting suggests to me
<VITAS> i say the dev server setup part first because then you can test what it iven does not speculate on what it might not :)
<HebaruSan> Anyway, the new servers need code to run, and it should be based on what's working on production
<VITAS> thats the code i dumped on sd2
<VITAS> that exactly whats on production right now
<HebaruSan> Right, which can't be automated with GitHub triggered restarts because that code isn't on GitHub
<VITAS> then i dont know where it is
<VITAS> maybe darklight knows?
<VITAS> or thomas
<HebaruSan> Maybe you could copy the config from the reverse proxy to my home dir, like you did with sd1? Then at least I could check whether it looks like the problem is what I'm saying
<VITAS> its a large amount of files
<VITAS> isnt all you need a bunch of settings?
<VITAS> you could name them and ill check if they set the way you want them to
<VITAS> thats why i copied the docs link
<VITAS> because few people know how to config ATS or that it even exists
<HebaruSan> Sorry, "that code isn't on GitHub" was specifically referring to the two commits that we're trying to clean up and merge, which include hard coding the host name on redirects.
<HebaruSan> The code for triggering automatic restarts from GitHub is there and looks like it might work fine.
<VITAS> ah yes
<HebaruSan> I'm trying to remove that hard coding, and all signs point to the reverse proxy
<VITAS> i remember changing one bit in multiple functions because the way it was written was depreached
<HebaruSan> BTW how will the reverse proxy work for sd2 and sd6? sd1 corresponds to spacedock.info, will there be spacedock2.info or something?
<VITAS> and i did the hardcoding
<VITAS> as i wrote a bunch of times now: sd2.52k.de -> sd2.52k
<VITAS> sd6.52k.de -> sd6.52k
<VITAS> "52k" (without .de) is the internal Domain
<HebaruSan> Ahh, got it, thanks
<VITAS> thats allready setup
<VITAS> you only need to have a webserver on port 80
<VITAS> (rev proxy does the https)
<HebaruSan> Right, but i need to know how to test it. Didn't realize the distinction you were making with the ".de" at the end previously
<VITAS> k
<VITAS> so now you can monitor logs on sd2 while doing requests on sd2.52k.de
<HebaruSan> We can agree that having sd2.52k.de redirect to spacedock.info doesn't make sense though? So the code has to change before we bring it live?
<VITAS> and im happy to tell you specific settinsg on the rev proxy if you ask me
<VITAS> i leave production as is until we have a better version tested
<VITAS> thats why sd2&6 exist
<VITAS> all that changes are domain names
<VITAS> they both use the same rev proxy with the same settings
<HebaruSan> OK let's start over. sd2 has code on it right now, right?
<HebaruSan> And that code contains hardcoded redirects to spacedock.info
<VITAS> for me the redirection problem is "just" another bug and something im happy to fix when we have the infrastructure up and running to test, debug and commit changes
<HebaruSan> So if we bring sd2 live now with the code it has, it will redirect over to the production server when we're trying to test it
<VITAS> yes
<VITAS> but you could remove them or change them if you like
<HebaruSan> So we can't bring sd2 live with the code it has now
<VITAS> we can
<HebaruSan> But it won't work
<HebaruSan> So there's no point
<VITAS> those redirects only cause redirects after you submited post requests
<VITAS> the requests are saved and processed before that
<VITAS> so you can register and use all the functions
<VITAS> all that breaks is that it now keeps redirecting you to spacedock info or sd2.52k (without de) after each form submit and after processing said form
<VITAS> i think you would need to set up the site to even fix that because you would need to play arround with solutions.
<VITAS> the alternative is to simply write a patch that you can manualy add to the code before merging it to github that uses the domain val from config ini and prefixes every redirect in the code with that val
<VITAS> i didnt do that because i had seconds to get it to work
<HebaruSan> OK, I'm going to read up on ATS. Hopefully I'll have a clearer picture of what settings might be relevant soon...
<VITAS> in my eyes todo: make db tables a thing (db acc is in config.ini),setup apache, setup virtual env and stuff to run gunicorn, get it running, debug redirect, setup systemd/hooks/...,merge code, do same on sd6
<VITAS> bonus: jenkins, whatever needed
<VITAS> then: attract devs to fix bugs and commit code
<VITAS> autodeploy dev/master to sd6
<VITAS> create/use stable branch to auto deploy to sd2
<VITAS> for (beta)testing
<VITAS> not stable but staging/beta
<HebaruSan> OK, so the existing 'dev' branch is for sd6? Or is that undetermined?
<VITAS> then have stable branch for everything that passed beta/staging to be deployed to sd1 (production) manualy
<VITAS> githubs setup are the reminents of the old dev process years ago
<VITAS> noone used it in i think 2 years?
<HebaruSan> OK, so we'll be resetting everything from master then
<VITAS> so its up to us to create something new/working
<VITAS> youre free to remodel github as you see fit
<VITAS> i want to have something where others can easily come in and help with coding
<VITAS> :)
<VITAS> im hopeing that you and micha could set something like that up and maybe even maintain it. while we hopefully find others to commit code
<HebaruSan> Yup, definitely. Would you imagine that contributor pull requests would target the testing branch and go to sd6 when merged? That would require someone to later merge the commit to sd2 and then sd1. Does that sound like an acceptable work flow?
<VITAS> (youre welcome to do that too)
<VITAS> i think every new line of code should (later) go to sd6
<VITAS> because thats unstable/experimental
<HebaruSan> Cool, that makes sense. Then I guess we would periodically put together pull requests to bring changes from testing --> staging, and then later staging --> master?
<VITAS> sd2 is for versions where we want to find the last problems with new code
<HebaruSan> I'm picturing all of this being driven from GitHub
<VITAS> before putting it into production
<VITAS> yes
<VITAS> or : dev->beta->production
<VITAS> sd6->sd2->sd1
<VITAS> i dont want to automate code deployment on sd1/production but we can have scripts that i can trigger.
<VITAS> but that comes last
<VITAS> ah you where asking about the naming.
<VITAS> im ok with whatever. for as long as the names are clear :)
<VITAS> we can even alias the systems to subdomains e.g. testing.spacedock.info for sd6.52k.de
<HebaruSan> That would be nice!
<VITAS> i can do that in a minute if you tell me your final naming
<VITAS> is it a good idea to have info like that on github (e.g. use the wiki)
<VITAS> ?
<HebaruSan> Frankly I don't know what's best for documentation at this point. I have a local markdown document where I'm tracking my notes and tasks.
<HebaruSan> But something shared would be better. A wiki might be too public and not dynamic enough. I don't know.
<VITAS> pad.52k.de exists
<VITAS> but the wiki would be the more obvious place
<VITAS> your call
<VITAS> :)
<HebaruSan> OK, I'll dump my document into a wiki page. No point holding out for a perfect solution when "good enough" is in reach
<VITAS> k
<VITAS> what aliases should sd6 and sd2 have?
<HebaruSan> I'm fine with "testing" and "staging", but I wouldn't insist on them
<HebaruSan> If you like "alpha" and "beta" better, I think people would understand those as well
<VITAS> and i ive no preferences at all so sd6->testing and sd2->staging?
<HebaruSan> Yeah, let's go with that
<VITAS> i would name them like the branch on github
<HebaruSan> Agreed
<VITAS> staging isnt a good branch name
<HebaruSan> Oh good point
<VITAS> unstable,beta ?
<HebaruSan> (I would suggest using Debian's convention, but I never found it particularly clear)
<VITAS> and production can be stable if we want to rename that
<VITAS> or we leave it because sd1 doesnt need an alias
<VITAS> as i said: your call. my job is to provide you with everything you need. :)
<HebaruSan> How about alpha and beta. At least I would be able to remember which of those was which.
<VITAS> ok
<VITAS> ill set those up
<HebaruSan> :)
<HebaruSan> Do you mind us installing packages on sd2? Such as git?
<VITAS> no
<VITAS> its your host
<VITAS> you setting it up so you can configure it as you need it to be
<VITAS> just be aware that it isnt a vm but an lxc container
<VITAS> added the subdomains
<VITAS> now we have to wait till the dns replicates
<VITAS> (rev. dns is also setup)
<VITAS> not dns but proxy
<HebaruSan> Created the wiki page for notes and tasks: https://github.com/KSP-SpaceDock/SpaceDock/wiki/Migration-notes-and-tasks
<VITAS> cool thanks
<VITAS> progress :D
<VITAS> you cant imagine how happy iam that its not me alone anymore
<HebaruSan> OK, dev branch is now renamed to dev-old to reduce confusion for new developers
<VITAS> top
<VITAS> you could even make ansible scripts for setting up spacedock if you know how to do it :)
<HebaruSan> OK, can you check whether this is set? It causes the original Host header to be passed to the real server, and flask's documentation suggests this would be used by redirect() if set: https://docs.trafficserver.apache.org/en/7.1.x/admin-guide/files/records.config.en.html#proxy-config-url-remap-pristine-host-hdr
<VITAS> is set to 0
<VITAS> want it to 1?
<HebaruSan> Yes please
<VITAS> done
<HebaruSan> So I have to edit a mod to reproduce the redirect problem in prod, right?
<VITAS> yes
<VITAS> after submitting the edit
<VITAS> it should redirect you wrong
<VITAS> looks good
<HebaruSan> woohoo emoji reaction
<VITAS> but the real test will be when the hardcoded spacedock.info is removed
<VITAS> huh?
<HebaruSan> Which we can try on sd2 safely
<HebaruSan> Once it's set up :)
<VITAS> yes
<VITAS> no messing with da production
<HebaruSan> Sorry, when I say "emoji reaction", imagine we're on Discord or Slack and I've clicked that reaction
<VITAS> -g
<VITAS> if $somone wants to maintain a discord thingy im happy to set up a bridge like i did with #dmp
<VITAS> so you can chat in either
<VITAS> i dont use discord (except for one channel)
<HebaruSan> Nah the only thing I really miss on IRC is the scrollback from when I'm not logged on
<VITAS> ive a BNC
<VITAS> that does that
<VITAS> you can have an acc on it if you like
<VITAS> i find the pasting of files usefull in modern IMs
<VITAS> but that can also lead to people just pasting walls of anim gifs and other crap
<VITAS> i also use "matrix"
<VITAS> its another try by the opensource community to have a go at a idiot proof decentralized IM
<VITAS> you can check it out on https://im.52k.de
<VITAS> that reminds me: if yoiu need any resources or accounts for CKAN related stuff im happy to give you that too
<HebaruSan> Appreciate that. I think we're good at the moment, but I'll keep it in mind.
<HebaruSan> Please let me know if this is/isn't how you envisioned doing the catch-up: https://github.com/KSP-SpaceDock/SpaceDock/pull/198
<Xinayder> what's going on?
<HebaruSan> We're trying to set up some testing servers with automated pull/restart triggers
<Xinayder> seeing that most people involved lost interest on spacedock, the remaining members should decide if it's worth spending more time into it, so we can test/fix the go branch (it separated backend from frontend) or keep using the old sircmpwn code
<Xinayder> (myself included :P)
<HebaruSan> Well at the moment it seems that VITAS is the one remaining voting member :)
<Xinayder> using the old code is better because it's already live, however the frontend is somewhat broken
<HebaruSan> Personally I just want a pathway to get #195 into production
<Xinayder> what's that?
<HebaruSan> File upload date in the API https://github.com/KSP-SpaceDock/SpaceDock/pull/195/files
<Xinayder> if we have a little commitment towards improving spacedock we can do whatever we like. last I checked it was abandoned, surviving from vitas' server life support :p
<Xinayder> the good thing about the go branch is that it works, but updating it to latest go is going to be hard because I suppose we don't know go
<Xinayder> on the other hand we can separate frontend from backend with the python code and just redo it by splitting it up
<HebaruSan> I think the idea is to create the infrastructure to facilitate whatever sort of stuff people want to work on
<HebaruSan> A branch in git can hold Go code just as easily as Python
<VITAS> Xinayder, i said multiple times: for as long as there isnt a fully working proven alternative the current code is where its at
<VITAS> and thats what i want to bugfix
<VITAS> (or have people bugfix if they want to)
<VITAS> btw if you and thomas realy wanted your go experiment could be in production by now.
<VITAS> HebaruSan, chaning that value messed up some other websites on that system. they all now know the external host :D
<HebaruSan> Hmm, I was afraid of something like that. There' s no way to make it per-host?
<VITAS> its ok. ill just have to change some config values in places
<VITAS> there might be a way to set it in the remap.conf rules
<VITAS> i will figure that out if i need to
<VITAS> you solved the redirect problem. thats whats important :)
<HebaruSan> Good, as long as we're not back to square one on that issue
<HebaruSan> Not sure how far back you've scrolled, but does this look reasonable to you? https://github.com/KSP-SpaceDock/SpaceDock/pull/198