VITAS changed the topic of #spacedock to: Problems?: https://github.com/KSP-SpaceDock/SpaceDock/issues | Matrix/Riot Chat: https://im.52k.de +spacedock:52k.de Feel free to ask for help, we only bite a little bit! | If you want to help, please check https://github.com/KSP-SpaceDock/SpaceDock-Backend/issues/5 :) | <VITAS> inet users have the attentionspan of a squirrel...
Darklight[m] has joined #spacedock
<Darklight[m]> Letsencrypt is supposed to renew way before that...
DasSkelett[m] has quit [Quit: Idle timeout reached: 10800s]
Shuudoushi has quit [Remote host closed the connection]
Darklight[m] has quit [Quit: Idle timeout reached: 10800s]
VITAS[m] has joined #spacedock
<VITAS[m]> good point
<VITAS[m]> ill do that later today
<VITAS[m]> cert updated
<VITAS[m]> remind me on the end of novemeber :D
<VITAS[m]> -e
<VITAS[m]> autoupdate gets stuck and i have to manualy hit it multiple times
Darklight[m] has joined #spacedock
<Darklight[m]> I have a crontab that reloads apache every week, certbot updates it monthly? I think so worksforme.jpg
<VITAS[m]> not-that-simple.jpg
<Darklight[m]> Not for you :P
<VITAS[m]> yes :D
<VITAS[m]> the problem is that the dns auth of letsencrypt sometimes failes
<VITAS[m]> so ive to run it multiple times till all domains are good
<Darklight[m]> I keep forgetting you don't have the primary DNS NS like I do
<VITAS[m]> i do but i host it myself
<Darklight[m]> So do I
<VITAS[m]> i dont know what the problem is but it doesnt propagate like it should
<Darklight[m]> DNS propigates instantly for new domains
<VITAS[m]> not always that is
<VITAS[m]> my secondary ns it with my domain provider
<VITAS[m]> i think thats the problem
<VITAS[m]> they use a cluster of servers
<Darklight[m]> Only thing I can think of is your domain provider running slaves that don't properly get the dns update thingy
<VITAS[m]> and they dont get it all the time
<VITAS[m]> yes
DasSkelett[m] has joined #spacedock
<DasSkelett[m]> What do you have set as wait time for certbot? Maybe try increasing that?
<Darklight[m]> If it annoyed me enough I would remove their NS records and don't use them
<VITAS[m]> so i would have to write a script that finds out if the cert is out of date and then retries ntil it gets a new cert
<VITAS[m]> like 10 minutes
<Darklight[m]> They don't provide any extra reliability because your server will still "not found"
<VITAS[m]> i started low and now am at that point
<Darklight[m]> certbot does that hourly I think
<VITAS[m]> i use dehydrated
<Darklight[m]> I think they use a systemd timer rather than a cron
<DasSkelett[m]> 10 minutes should be enough, yeah.
<Darklight[m]> Well given that they update only happens like 1 month before it is due and the certs are 3 months I think?
<VITAS[m]> dehydrated uses powerdns api to add entries
<VITAS[m]> then
<VITAS[m]> ive a second script that uses ansible to distribute the new cert and restart the services
<VITAS[m]> because of the failures to gen a cert AND the distribution/service restarting
<Darklight[m]> Doesn't a bind reload send notified if you have secondaries set up? or is their servers retarded and acting like primaries?
<VITAS[m]> i cant use normal methods
<Darklight[m]> *edit:* ~~Doesn't a bind reload send notified if you have secondaries set up? or is their servers retarded and acting like primaries?~~ -> Doesn't a bind reload send notifies if you have secondaries set up? or is their servers retarded and acting like primaries?
<VITAS[m]> no bind
<VITAS[m]> powerdns
<VITAS[m]> i dont know why or how letsencrypt doesnt ask my ns
<VITAS[m]> all i need is a script that detects if the cert can be renewed, runs the getcert script until it gets a good cert
<VITAS[m]> then runs the distribution script
<VITAS[m]> so i think ive to parse the getcertscripts output and act uppon it
<Darklight[m]> Is there any reason you are delegating to their servers?
<VITAS[m]> i want a secondary ns in case i screw up
<Darklight[m]> Maybe NXDOMAIN sticks around too long
<VITAS[m]> we can try to automated it once i can renew the cert again because till then the output isnt the same
<Darklight[m]> certbot has ``--dns-rfc2136-propagation-seconds`` but I have no idea about your one, you can increase the delay
<VITAS[m]> i did that
<VITAS[m]> to 10min
<VITAS[m]> did improve a little bit but not fix it
<Darklight[m]> I am genuinely curious, if you wireshark when your bind reloads, does it send out dnsnotify followed by the slaves doing an AXFR? or is it using some non-dns wanky protocol?
<VITAS[m]> no bind
<VITAS[m]> and no reload as such
<VITAS[m]> powerdns and its api
<VITAS[m]> and yes im doing axfr
<RockyTV> speaking of LE, I need to figure out a way for my letsencrypt docker to notify my nginx-proxy docker to restart as soon as the certificate changes
<RockyTV> the certificate renewal docker runs once a day or so
<VITAS[m]> docker \o/
<VITAS[m]> RockyTV (IRC): ansible
<VITAS[m]> is the answer to everything automated
<Darklight[m]> RockyTV certbot has a post run script I believe
<Darklight[m]> Although you have a month to restart it, just issue a reload weekly and you will be fine ;)
<Darklight[m]> I pointed my apache where certbot actually keeps the certs so don't need to do any movey changey replacey
<VITAS[m]> i personaly want to reload/restart servcices only if they need to not weekly
<VITAS[m]> so ive that ansible script that first copies the new cert to the machine/container and then does what needs to be done to make it active
<VITAS[m]> i can also add more services to that playbook so its easy to host stuff like mailservers and what not
<DasSkelett[m]> Yeah it's not that easy, Docker containers can't restart each other by default. You'd either have to mount the docker socket in the certbot container, or hack some internal webhook that runs a script inside the nginx container issuing `nginx -s reload`
<VITAS[m]> i would have an ansible script run on the host and check for a new file in a folder thats mapped as storage to the lets encrypt container
<DasSkelett[m]> (in answer to @RockyTV)
<VITAS[m]> if theres a new file it can restart all related containers and those should map the same foolder as storage for certs
<VITAS[m]> bonus: you can have ansible do other maintance tasks as well like deploy containers or youir whole setup
<RockyTV> I use docker-compose
<VITAS[m]> think of ansible like "check if everything is like i defined if not make it so"
<RockyTV> I think I can just do a systemd timer unit to run `nginx -s reload` once a week or so on my nginx-proxy container?
<VITAS[m]> sure
<VITAS[m]> or ask someone to check it for you
<VITAS[m]> or do a cronjob
<VITAS[m]> :)
<RockyTV> root cronjob :P
<VITAS[m]> whatever floats your boat
<VITAS[m]> i like ansible because i dont have to care how its done
<VITAS[m]> its done
<VITAS[m]> thats what matters to me
<VITAS[m]> i only deliver the blueprint
Darklight[m] has quit [Quit: Idle timeout reached: 10800s]
DasSkelett[m] has quit [Quit: Idle timeout reached: 10800s]
VITAS[m] has quit [Quit: Idle timeout reached: 10800s]