Thanks for the continued interest. Let me provide some friendly corrections:
(A) We do not need defaults inside the Docker container. This is already provided by the config/ tree in the main application. Any config setting not supplied in /etc/vespene/settings.d already has a default. (I worry that some folks COULD be trying to containerize Vespene before they understand the codebase and deployment needs - maybe not - but ... this should be something everyone is aware of).
(B) "We need an ability to redefine any of config parameters during Docker image startup" - to do this, you can ship anything you want in /etc/vespene/settings.d that loads any parameter setting from the environment. So this is also already possible.
(C) "and it's bad because any configuration file error" - it's not, because if you ever want to load anything in the environment, the code can do it! JSON can't :) So this argument seems to counteract "B".
This is a long standing Django convention, it's pretty darn useful, and while I would agree some unfamiliar to Django might not like it, it's not going to change - and is easy enough to work around. You may just have to tolerate it :)
I will reiterate what points I think we need here:
We need an easy way to ship whatever configuration the user wants in the Dockerfile. This is a pre-build step that creates some files in /etc, and then produces that file. This is mainly for PLUGIN configuration and things that won't need to change at runtime, or can be easily replaced by changing the image.
It seems fine to have the name of the worker queue to be served in the worker image from an environment variable, which would avoid having lots of different worker images. Although in reality, hardly any settings.d content will be the same between sites, (triggers especially) and this is the kind of stuff that doesn't make sense to shove into the environment because it's structured data.
It seems deeply insecure to pass the secrets.py stuff around in the environment, or worse, bake them into any public image shared in Docker hub. As such, I think there is still a pre-build step that creates PERSONAL dockerfiles (for private registries) and people can easily customize their own docker files without having to write a lot of Docker.
If the idea of a pre-build step that produces docker files and directories to load in during the "Docker build" step is too much, It may be easier to think of the Docker file work as constructing EXAMPLES, coupled with a README that says how to produce them. For instance, the user will need a common secrets file and to make some choices about plugins specific to their deployment of Vespene.
I know Docker has a lot of fans, but I think this underscores some things - it sucks for stateful configuration, and Vespene does have a lot of state (configuration). It's not insurmountable, but I want to make sure all thinking about Docker images isn't about establishing a demo environment, it's about establishing a repeatable, upgradable, production environment.
- the python configs aren't changing away from python (sorry!) - but I don't think this hurts anything. JSON/YAML syntax errors are still JSON/YAML syntax errors
- the python configs CAN load environment variables - write code that loads them and drop them into /etc/vespene/settings.d/*.py and you can overide anything!
- we need an example repo that shows how to build a production grade config complete with custom worker images for both the web nodes and worker nodes, allowing heavy user customization of the config and site-specific secrets
- parameters like secrets.py and the database password should not be passed around in the environment, as that seems VERY unsafe. Having that kind of stuff in the images would probably reflect badly on the project.
- this content is probably not suitable for docker hub, but I'm quite open to an easy build step (Makefile target) etc that produces a site specific set of images so users don't have to do all the work of writing their own configurations
- If people want to see this in tree, I STILL want the docker files to use as much of the setup/* scripts as possible, to avoid dual maintainance. This doesn't have to be the configuration steps at all, but I mean for things like installing apt packages, so the package list is in ONE place in the main repo. I am willing to bend on this a little, but I think attempts haven't been made to make the config scripts responsive to environment variables such that the docker builds can call them and reuse partial functionality. That would be interesting and keeps the Docker content easier to maintain and not a "fork" as it were of the install instructions. I agree it's not going to be 100% common, but things like the package list and repo setup and so on wouldn't need to be repeated. (Clearly one wouldn't run the systemd parts)
If work goes down those lines, I'll support it as best I can. I still think it would be easiest to let it evolve out of tree and then we can merge it back in, in something like a "docker" subdirectory, if we get it into a reasonable state.
I say "we" here despite clearly not doing the work, but I'm mostly trying to suggest what we need for security and configurability in production, and looking towards what it would take to adopt this in tree as something I would be comfortable with maintaining.
As while volunteer efforts to help maintain something do exist, in practice, any code you take is kind of like taking on a free puppy, so I need to make sure those bases are covered.
I'm still excited about this effort and I want to see it successful.
Mostly make sense?
Thank you all once again for the excitement about this - it is really cool to see.
Questions? Any points/concerns maybe not understood?