Using docker-compose.yml to deploy citus cluster. Works great!
Now, in preparation for a larger scale deployment I would like to mount the data directories of the workers to local folders. This would allow me to direct worker data volumes easily to different physical drives/storage.
I started by adding a line into the docker-compose.yml file extending the "volumes:" specification so that it reads:
...
volumes:
- healthcheck-volume:/healthcheck
- /mnt/local:/var/lib/postgresql/data
manager:
...
For a cluster with a single worker node this obviously works like a charm. However, when starting a cluster with two nodes, the second node mounts the identical local folder and both worker nodes read and write onto the same files - disaster happens!
I was hoping to specify a local subfolder named like the worker. Really, just any way to provide distinct folders to every worker. Reading through the docker-compose.yml specification I could not find a way to do so. E.g. I tried various ways to reference a container name or retrieve a runtime enumeration of worker nodes. But to no avail. E.g.
...
volumes:
- healthcheck-volume:/healthcheck
- /mnt/local/{services.worker.container_name}:/var/lib/postgresql/data
manager:
...
I am relatively new to yml in general and docker-compose.yml in particular. Would you know a way to accomplish my goal?
Using docker-compose.yml to deploy citus cluster. Works great!
Now, in preparation for a larger scale deployment I would like to mount the data directories of the workers to local folders. This would allow me to direct worker data volumes easily to different physical drives/storage.
I started by adding a line into the docker-compose.yml file extending the "volumes:" specification so that it reads:
...
volumes:
- healthcheck-volume:/healthcheck
- /mnt/local:/var/lib/postgresql/data
manager:
...
For a cluster with a single worker node this obviously works like a charm. However, when starting a cluster with two nodes, the second node mounts the identical local folder and both worker nodes read and write onto the same files - disaster happens!
I was hoping to specify a local subfolder named like the worker. Really, just any way to provide distinct folders to every worker. Reading through the docker-compose.yml specification I could not find a way to do so. E.g. I tried various ways to reference a container name or retrieve a runtime enumeration of worker nodes. But to no avail. E.g.
...
volumes:
- healthcheck-volume:/healthcheck
- /mnt/local/{services.worker.container_name}:/var/lib/postgresql/data
manager:
...
I am relatively new to yml in general and docker-compose.yml in particular. Would you know a way to accomplish my goal?