VITAS[m] has quit [Quit: Idle timeout reached: 10800s]
AllisTauri[m] has quit [Quit: Idle timeout reached: 10800s]
aradapilot has quit [Ping timeout: 190 seconds]
aradapilot has joined #spacedock
aradapilot has quit [Ping timeout: 190 seconds]
VITAS[m] has joined #spacedock
<VITAS[m]>
AllisTauri: just be aware that its harder to convince me of switching deploment methods AND webservers at the same time :)
<VITAS[m]>
RockyTV: acc to alpha and beta is just for observing and we dont need round the clook monitoring on dev systems.
AllisTauri[m] has joined #spacedock
<AllisTauri[m]>
@vitas:52k.de: not trying to; I'm trying to make it seamless as possible for the current production environment. And no one forces you to use docker for development =P
<VITAS[m]>
no you want me to want it :D
<AllisTauri[m]>
Does it matter what web server shows you the site locally while you change the code?
<VITAS[m]>
so you could use apache webserver then?
<AllisTauri[m]>
=^_^'=
<AllisTauri[m]>
I could, but that's additional work. With nginx container I say: "static here, the rest goes here, pass original host:port along for redirects".
<AllisTauri[m]>
With apache container it's like: oh, hell, where do I put this config so that it's used like I what to? And I need to add http_proxy.so somewhere?
<VITAS[m]>
youre basicly saying you know ngnix as well as i do apache webserver
<AllisTauri[m]>
Not at all
<AllisTauri[m]>
And that's the point
<AllisTauri[m]>
I don't know both, but **the container** with nginx is made so that I don't need to; while there's no good little *"containers** dedicated to apache.
<AllisTauri[m]>
<AllisTauri[m] "I don't know both, but **the con"> But you can also compare site configs in my PR in `frontend/configs`
<VITAS[m]>
ok thats an argument
<VITAS[m]>
im just worried that i will have to maintain all of this myself again at some point
<AllisTauri[m]>
There's another argument: if we go the kuber way, they have ready to use load-balancer based on nginx; and by load-balancing I mean automatic backend and DB instance scaling depending on traffic and response times.
<VITAS[m]>
since im running all off this on one physical machien i dont need that.
<VITAS[m]>
if i had more time i would certenly look at kubernets again
<VITAS[m]>
i know docker well enough
<AllisTauri[m]>
Not true)
<AllisTauri[m]>
We're using flask, so the only means to parallelize things is to spawn instances
<VITAS[m]>
yes
<VITAS[m]>
and there are enough instances to fill most of the cores
<VITAS[m]>
and keep in mind that only 13% of all requests are handled by the webserver and even fewer by flask
<VITAS[m]>
i also doubt that we will more than 100x the visitors
<AllisTauri[m]>
And it's better to base their number on demand than on guesswork. But again, that's only applicable to kuber and swarm
<AllisTauri[m]>
True, but maybe it would work better work less instances of the backend and more of postgres?
<VITAS[m]>
i base the number on max capability of the hardware
<AllisTauri[m]>
Where goes the rest of the requests?
<VITAS[m]>
if i would have some difuse cloud to work with and would spread my processes across machiens i would agree with you
<VITAS[m]>
rev proxy
<VITAS[m]>
ATS
<VITAS[m]>
the thing between all containers and the interwebs
<VITAS[m]>
it also does the https thing and all the others stuff
<AllisTauri[m]>
Not sure i got it; rev proxy is a proxy; where go the requests from there? Other services?
<VITAS[m]>
its the reverse of a proxy
<VITAS[m]>
so not in front of the client and the internet but in front of the server
<VITAS[m]>
its the same mode apache webserver is currently working in (see the config)
<VITAS[m]>
btw thats the reason i say webserver and not only apache
<VITAS[m]>
because that software shares no code with the webserver offered by apache
<AllisTauri[m]>
Don't I know it)
<AllisTauri[m]>
But that doesn't answer the question: where go the requests that the proxy received? Does it serves 77% cache?
<VITAS[m]>
yes
<VITAS[m]>
87%
<VITAS[m]>
last time i checked at least
<AllisTauri[m]>
Anyway, I'm not proposing to switch to kuber; I just say that if for some reason we decide to try it, we'll face nginx anyway
<AllisTauri[m]>
Cool!
<VITAS[m]>
and im very interrested in your results
<AllisTauri[m]>
BTW, HebaruSan , want to help you with autodeploy. I'll look at the backend code and make a script that updates systemd units before restarting everything.
<AllisTauri[m]>
*hm... Mobile app has lost autocompletion of usernames
DasSkelett[m] has joined #spacedock
<DasSkelett[m]>
AllisTauri Can you rebase your branch on the current alpha? You can't ff-pull it right now, I think because it's missing the merge commit of your previous PR.
<AllisTauri[m]>
Ok, so the actual git pull and restart command should go to celery, and the restart subprocess should be daemonized, so as not to be killed by the restart itself.
<AllisTauri[m]>
@vitas:52k.de: can I have the right to assign issues to myself on github? Need a way to organise the tasks.
AllisTauri[m]1 has joined #spacedock
<AllisTauri[m]1>
DasSkelett why want you do that?
<VITAS[m]>
sure AllisTauri
<DasSkelett[m]>
Well, I first created a new local branch, then tried to pull yours from upstream. But it's not necessary, it works if you go the other way round, first checkout, then create local branch.
<DasSkelett[m]>
I get the following error trying to execute `./start-server.sh`:
<DasSkelett[m]>
```
<DasSkelett[m]>
...(truncated)
<AllisTauri[m]>
*boy, the Discord Bridge is slow
<VITAS[m]>
irc to matrix can be slow
<VITAS[m]>
discord bridge depends on discords willingness to accept messages
<AllisTauri[m]>
I mean the other way, from discord to here
<VITAS[m]>
is it always that slow?
<AllisTauri[m]>
Here is the matrix? Where's irc then?
<AllisTauri[m]>
Yes, as far as I can tell.
<VITAS[m]>
on espernet
<VITAS[m]>
#spacedock
<DasSkelett>
Right now, Discord seems to be completely broken. No messages in, no messages out.
<AllisTauri[m]>
X_x
<AllisTauri[m]>
Security by obscurity
<VITAS[m]>
hmm
<AllisTauri[m]>
In matrix we see messages from discord now. But with considerable amount
<AllisTauri[m]>
Of lag
<DasSkelett>
Uhmm, so how long does it usually take for the db to come online?
* VITAS[m]
will take a look later. i encurage everyoen to use matrix :)
<DasSkelett>
Yeah no, I'm not setting up a third communication app ^^
<AllisTauri[m]>
@dasskelett:52k.de: for new it was quick enough that I rarely saw this message at all
<AllisTauri[m]>
For me
<DasSkelett>
Maybe once Konversation ships matrix support :)
<AllisTauri[m]>
Heck, why riot doesn't have message editing?
<VITAS[m]>
i understand. you could use the website: https://im.52k.de if you want
<VITAS[m]>
it should
<VITAS[m]>
i dont belive in taking back things you transmitted
<AllisTauri[m]>
I use it from desktop, it's very usable.
<VITAS[m]>
web and desktop is the same
<VITAS[m]>
riot is an web app except for iphone
<DasSkelett>
That's weird. Doesn't seem to come online at all for me, even after waiting a few minutes
<AllisTauri[m]>
I believe in stupid phone autocorrection that doesn't know what I'm trying to say
<AllisTauri[m]>
@dasskelett:52k.de: wired indeed
<AllisTauri[m]>
@vitas:52k.de: see that? ^^^
<VITAS[m]>
what?
<VITAS[m]>
* what??
<VITAS[m]>
* what?
<AllisTauri[m]>
Wired instead of weird)
<VITAS[m]>
ah
<VITAS[m]>
i was able to edit my text
<VITAS[m]>
* i was able to edit my text. like this.
<VITAS[m]>
doenst work on discord or irc ofcause
<AllisTauri[m]>
I can do it in web app, but not on phone
<VITAS[m]>
thats strange
<VITAS[m]>
maybe the phone ver is older
<VITAS[m]>
the windows ver can
* VITAS[m]
has to pack now
<VITAS[m]>
later
<AllisTauri[m]>
Nevermind that
<AllisTauri[m]>
Later!
<AllisTauri[m]>
*wait! GitHub rights for issues? ^_^'
<AllisTauri[m]>
@dasskelett:52k.de: does it spool the "errors"once per second?
<DasSkelett>
Very much, yes. Sometimes with breaks
<AllisTauri[m]>
Ah, I see the problem! Chicken and egg: there's no spacedock database yet. Need to check the connection to `postgres` database instead
uovo is now known as egg
<egg>
chicken and me
<DasSkelett>
:D
<AllisTauri[m]>
Thanks for the test)
<DasSkelett>
No prob
AllisTauri[m]1 has quit [Quit: Idle timeout reached: 10800s]
DasSkelett[m] has quit [Quit: Idle timeout reached: 10800s]
VITAS[m] has quit [Quit: Idle timeout reached: 10800s]
AllisTauri[m] has quit [Quit: Idle timeout reached: 10800s]
aradapilot has joined #spacedock
aradapilot has quit [Remote host closed the connection]
aradapilot has joined #spacedock
aradapilot has quit [Remote host closed the connection]
aradapilot has joined #spacedock
Webchat556 has joined #spacedock
Webchat556 has quit [Client Quit]
Llamato has joined #spacedock
VITAS[m] has joined #spacedock
<VITAS[m]>
theres no chicken and him problem
<VITAS[m]>
the egg was before the chicken becaause once there where egg laying dinausaurs and at some point one lay an egg that spawnd somethign we could call chicken
<Llamato>
?
Llamato5 has joined #spacedock
<VITAS[m]>
!
<DasSkelett>
I would argue the dinosaurs gradually evolved to chicken. So it's hard to pin a point where you could say "NOW it is a chicken". But eggs per se existed before, yes
Llamato5 has quit [Client Quit]
<VITAS[m]>
and aat some point the first chicken was born from it
Llamato has left #spacedock [#spacedock]
<VITAS[m]>
therfore a chicken couldnt have layed it but soemthing else
<VITAS[m]>
most likely a dinosauer we wouldnt call chicken
Llamato has joined #spacedock
<DasSkelett>
Yeah
<VITAS[m]>
so the egg was before the chicken even the one that the first chicken came out off
<VITAS[m]>
paradox solved
<VITAS[m]>
:)
<DasSkelett>
So, AllisTauri, you just have to lay an egg in the docker container to create a database! Or something
<VITAS[m]>
how are your vue studdiys going?
<VITAS[m]>
-i
<DasSkelett>
Good I think. Right now a bit on a hold, since I've been busy last week, and next week are exams, but it's progressing.
<VITAS[m]>
good to hear
<VITAS[m]>
:)
<DasSkelett>
And right now I'm trying to fight my way through really messy code of CKAN and refactor it on the way.
<VITAS[m]>
messy? i thought theirs would be better than SpaceDocks
<DasSkelett>
I think it's better in overall, but we also have some of those parts...
<VITAS[m]>
if you come across others you can drag into one or the other Project theyre most welcome :)
<DasSkelett>
Sure
<VITAS[m]>
Many thanks for your efforts :)
<DasSkelett>
There's my roommate who has a lot of free time, but I could hardly convince him to even buy KSP. Nor does he have experience in C# or web dev. But once I come across someone.
<VITAS[m]>
what a strange person: Free time and no coding? :P
<DasSkelett>
Indeed. Well, he's working in the SAP team of a small company once every two weeks. Of his own free will. So I think he's lost.
<VITAS[m]>
SAP so he must be mad
<DasSkelett>
Yeah I don't like SAP
* VITAS[m]
has some SPA certificates
<VITAS[m]>
it only deepend my dislike
<DasSkelett>
It's horrible they have such a monopoly
<VITAS[m]>
and the worst UX ive seen since some invoice management software 10 years ago
<DasSkelett>
Really. Like, did they stop doing any UX, or even UI improvements 10 years ago? At least it looks and feels like it.
<VITAS[m]>
lets hope SPaceDock wont be the same because noone improves its UI :)
<DasSkelett>
:D
<DasSkelett>
But even if, there's a big difference, the one is a small website hold together by mostly only one guy in his free time, the other one is software by a multi-billion euro company which does nothing else.
<VITAS[m]>
correct
<VITAS[m]>
but no reason for SpaceDock not to be a usuabale and feature rich software.
<VITAS[m]>
leading by example
<VITAS[m]>
i still wish the "free time" part could be full time
<DasSkelett>
Understandable
HebaruSan[m] has joined #spacedock
<HebaruSan[m]>
Oh. Docker is just Plan 9.
AllisTauri[m] has joined #spacedock
<AllisTauri[m]>
<irc_DasSkele "So, AllisTauri, you just have to"> Done. Pushed two commits to that extent; you can re-test it.
DasSkelett[m] has joined #spacedock
<DasSkelett[m]>
Hey, messages are going out again from Discord. But not in yet...
<VITAS[m]>
i blame discord then
<VITAS[m]>
i didnt change anything
<VITAS[m]>
cant find the option to give you the desired rights on github :/
<VITAS[m]>
AllisTauri:
<AllisTauri[m]>
Ok, so the new docker setup is in alpha, thanks VITAS . Am I to try to deploy it later tonight?
<AllisTauri[m]>
For some reason I have them now; or at least I have them in web UI.
<AllisTauri[m]>
Grabbed several issues I think I know how to solve
<VITAS[m]>
you can on your stuff. we cant on alpha or beta because its a container in itself and might not work
<AllisTauri[m]>
I don't, don't don't intend to start docker on alpha/beta 🤣
<VITAS[m]>
so where do you?
<AllisTauri[m]>
I mean to deploy this revision normally, as you always do; because aside from docker env it introduced frontend building with npm/
<AllisTauri[m]>
* I mean to deploy this revision normally, as you always do; because aside from docker env it introduced frontend building with npm.
<VITAS[m]>
ithat im all for
<DasSkelett[m]>
HebaruSan you might want to check out the IRC or matrix, the bridge to Discord is not working currently
<DasSkelett>
Uhhhhm... you already merged it VITAS?
<VITAS[m]>
yes
<VITAS[m]>
blindly clicking :P
<DasSkelett>
Oh wow. That was fast.
<AllisTauri[m]>
*if something's wrong I'll fix it separately in alpha itself, I suppose
<VITAS[m]>
yes thats whaat its for
<DasSkelett>
Yeah so, it's still not working on my system, even after removing all the images and rebuilding them...
<VITAS[m]>
it should reflect the "Unstable" branch
<VITAS[m]>
or alpha as we called it i think
<DasSkelett>
Yeah but, if it's not working at all? Or was it working for you Vitas?
<VITAS[m]>
i hope we only merge stuff that was localy tested (at some point)
<AllisTauri[m]>
Hm... DasSkelett (IRC) I've checked it with clean setup, with different db name and all. Could you share the logs again? Pastebin maybe?
<HebaruSa->
Maybe we should talk about code review expectations
<DasSkelett>
^^
<VITAS[m]>
alpha is for: testing additions and changes to the code by as meny devs as possible to make it ready to go into public enduser testing in beta
<VITAS[m]>
thats my idea
<VITAS[m]>
many
<HebaruSa->
If some of us expect that a pull request will already be working at submit, and others expect that it may be a rough sketch in need of collaboration, then we're likely to end up with some chaos
<AllisTauri[m]>
*now I'm testing it with another workstation
<VITAS[m]>
i always expect local testing by the producer of the commit before commiting
<VITAS[m]>
what procedures have to be done to call it "tested" is another matter
<HebaruSa->
OK, that should be written down somewhere. DasSkelett, are you able to edit the wiki yet?
<VITAS[m]>
maybe unit tests?
<AllisTauri[m]>
and if the later is true (sketch in need) we should tag the MR with `WIP`, I think
<VITAS[m]>
different specified setups?
<DasSkelett>
Yeah but especially for Docker related stuff, there's a high chance that during local testing, the submitter has a different environment.
<VITAS[m]>
isnt the idea of docker to unify the enviroment?
<AllisTauri[m]>
Docker is the thing to isolate from environment, no?
<DasSkelett>
But only once the dockerfile is complete, static and untouched
<VITAS[m]>
i leave it to you to define procedures you guys are happy with and as HebaruSa- (IRC) said: write it down so newcommers can read up on it.
<VITAS[m]>
(its you who has to work with and in this enviromentso you should form it)
<DasSkelett>
Like during setting up the docker compose, the submitter already has built the database image on a previous step. And it's working for him to connect to it later on, but for others where it didn't exist already, there's something different going on
<AllisTauri[m]>
So on clean machine it took DB container 39 seconds to start postgres
<VITAS[m]>
btw if you guys need more hosts to do what you want to do just tell me :)
<AllisTauri[m]>
DasSkelett (IRC): here's what I've got
<AllisTauri[m]>
```
<AllisTauri[m]>
...(truncated)
<DasSkelett>
HebaruSan: no, no editing rights yet
<AllisTauri[m]>
After re-testing on pristine system found some bugs in database initialization routine and fixed them in alpha
<AllisTauri[m]>
But what DasSkelett (IRC) reports is normal. 40 seconds and more of waiting for the first start
<DasSkelett>
So I'm 6 minutes in and it still throws the error. I'll try you new commits now.
<AllisTauri[m]>
New commits shouldn't make a difference with this, I'm afraid. Is your use in docker group?
<AllisTauri[m]>
Another thought: you can start the process, then open another terminal in the same directory and run `docker-compose logs` to see what happens in running containers
<AllisTauri[m]>
Yes, i see: your db container exits as soon as it's started for some reason
<AllisTauri[m]>
Need to check it's logs.
<DasSkelett>
Some permission thinngy going on I see
<AllisTauri[m]>
The `/var/lib/postgres/data` is mounted outside the container as `./spacedock-db` for persistence; so docker should be able to change it's permissions, because you run it
<AllisTauri[m]>
Weird
<DasSkelett>
Yeah I just checked, the folder is 777 and belongs to my user
<AllisTauri[m]>
And that's probably what PG tries to fix: the 777
<AllisTauri[m]>
But why can't it?
<DasSkelett>
but to what? 770? 775?
<AllisTauri[m]>
Google digests it may be selinux problem. In that case adding :z suffix to volume definition in docker-compose may help
<HebaruSa->
What Ubuntu package do I have to install to start up dockerd? "docker build" is complaining about not being able to connect.
<DasSkelett>
Permission denied?
<HebaruSa->
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.39/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile.netkan&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&session=1o76xvgqzcpkjc0pwwjexzm6e&shms
<AllisTauri[m]>
Cool, I've made corresponding commit just now directly on github
<AllisTauri[m]>
HebaruSa-: almost
<AllisTauri[m]>
`docker run -it --rm ruby:latest bash`
<AllisTauri[m]>
So it's interactive, is started if not already running and is removed as soon as you exit
<AllisTauri[m]>
exec will work only on already running container
<HebaruSa->
It's still just running the entrypoint
<HebaruSa->
OK, adding --entrypoint="" worked.
<DasSkelett>
Yeah the current ckan docker images are a bit weird. I'm working on redoing them (using multistage Dockerfiles to reduce the size)
<HebaruSa->
After exiting, "docker image rm" says it's still running. "docker container ls" says nothing is running.
<HebaruSa->
Ahh, "ls -a" find it.
<VITAS[m]>
i also find it an interresting opportunity that you are working on CKAN AND SD at the same time. you can optimize the dataflow between them.
<HebaruSa->
Heh, I was requested to review docker-related PRs in both projects within a few hours of each other.
<DasSkelett>
:D
<HebaruSa->
Knowing nothing about docker, this was something of a challenge.
<VITAS[m]>
now youre getting a tutorial at the same time :)
<HebaruSa->
Error response from daemon: conflict: unable to remove repository reference "kspckan/inflator:latest" (must force) - container b5def2f78d07 is using its referenced image bf17cd9d89e1
<HebaruSa->
$ docker container kill b5def2f78d07
<HebaruSa->
Error response from daemon: Cannot kill container: b5def2f78d07: Container b5def2f78d0787ae555d195755ee14e35a6e4ba47912287a6cd8ae81a775ac76 is not running
<HebaruSa->
How do I get rid of this thing so I can try it again from scratch
<AllisTauri[m]>
Mine was more about changes to normal deployment)
<HebaruSa->
"docker container prune" fixed it.
<DasSkelett>
docker container rm b5def2f78d07 might have sufficed too
<DasSkelett>
Uhm AllisTauri, bad news... it only worked accidentally, because I have typoed the path to pgdata...
<AllisTauri[m]>
I'll soon be at home and check it myself
<DasSkelett>
I'll play around myself in the meantime
<AllisTauri[m]>
Meanwhile you can pull alpha and retry
HebaruSan[m] has quit [Quit: Idle timeout reached: 10800s]
DasSkelett[m] has quit [Quit: Idle timeout reached: 10800s]
<VITAS[m]>
A bunch new ideas/issues :)
Guest77 has joined #spacedock
<Guest77>
hi guys
<Guest77>
I'm trying to use the API to update a KSP mod
AllisTauri[m] has quit [Quit: Idle timeout reached: 10800s]
<DasSkelett>
Yeah you also need `-b "$OUTPUT_PATH/cookies"` in the curl command for the update
<Guest77>
Yes that is working
<Guest77>
Thanks a lot
<DasSkelett>
You're welcome. You have a second issue?
<Guest77>
Well sometimes i was getting a "bad gateway" error instead of the "not logged", but it seems the root cause was that the cookie wasn't read.
<Guest77>
it seems to work consistently now
<DasSkelett>
That's good. If it starts appearing more often again, come here again, I think this would be something VITAS (or someone other network savy) should have a look at.
<Guest77>
Allright, will do
<Guest77>
Thank for the help
<DasSkelett>
No problem
<Guest77>
lol, actually just realized that we talked earlier on the CKAN Discord
<Guest77>
I'm "Got " there
<DasSkelett>
Yeah I thought so, it said Kerbalism in your script :D
<DasSkelett>
But it's hard to deduce from Guest77 to Got ;)
<Guest77>
Indeed ;p
VITAS[m] has quit [Quit: Idle timeout reached: 10800s]