Add Docker Compose setup for local development - #383
Conversation
probot-autolabeler
Bot
added
chore
documentation
Updated the command in the stories_webapp service to use 'npm run develop'.
Updated the volume and working directory paths to use the WORKSPACE_ variable.
dduportal
left a comment
dduportal
left a comment
There was a problem hiding this comment.
Thanks, it's getting better. But you clearly have not tested this change: the pipeline would be broken.
I've let comment, can you please fix and test a bit please?
| # // Added the below to fix permissions issue with the cache | ||
| GATSBY_CACHE_DIR = "${WORKSPACE_}/.gatsby-cache" | ||
| GATSBY_INTERNAL_CACHE_DIR = "${WORKSPACE_}/.cache" |
There was a problem hiding this comment.
This will break the CI. Is it needed with Docker Compose?
There was a problem hiding this comment.
It is not needed in Docker. I initially thought it was important to fix permissions in the CI, so I kept it in Docker. However, after testing, I found a problem that may occur if the user runs it on the host OS using the default user with UID=1000. They will face an error related to cache permissions, because the Jenkins user used in Docker Compose has UID=1001. As a result, the bind-mounted data will have 1000 ownership, and Jenkins will not have permission to write to these files.
Therefore, I should add this to the documentation. For example:
Before running docker compose up, make sure that the user you are using has UID=1001. If not, use the command below.
// for linux and wsl
sudo chown -R 1001:1001 .
or create and use a user with UID=1001
There was a problem hiding this comment.
That is not a safe neither a portable solution:
- UID is different on macOS with Docker Desktop
- UID is different on Linux with Docker CE
Since there are no reason to share the cache dir with the host, WDYT about setting the env. var in Docker Compose only, to a container-internal directory.
There was a problem hiding this comment.
The issue is that Gatsby is ignoring GATSBY_CACHE_DIR or GATSBY_INTERNAL_CACHE_DIR and still trying to create .cache inside the workspace directory (APP_WORKING_DIR), where the jenkins (UID=1001) user has no write permission because the workspace directory need root ownership to have write access in APP_WORKING_DIR
I recently reach to solution:
- add another service with root user to fix permission for
.cashandpublice.g
services:
fix-permissions:
image: busybox
user: root
working_dir: /app
volumes:
- .:/app
command: >
sh -c "
if ! [ -d /app/public ]; then
mkdir -p /app/public
fi
if ! [ -d /app/.cache ]; then
mkdir -p /app/.cache
fi
if ! id -u jenkins > /dev/null 2>&1; then
adduser -u 1001 -D jenkins
fi
chown -R jenkins:jenkins /app/public /app/.cache
"There was a problem hiding this comment.
I recently reach to solution:
- add another service with root user to fix permission for
.cashandpublice.gservices: fix-permissions: image: busybox user: root working_dir: /app volumes: - .:/app command: > sh -c " if ! [ -d /app/public ]; then mkdir -p /app/public fi if ! [ -d /app/.cache ]; then mkdir -p /app/.cache fi if ! id -u jenkins > /dev/null 2>&1; then adduser -u 1001 -D jenkins fi chown -R jenkins:jenkins /app/public /app/.cache "
The new version of this recommended solution replace chown with chmod to avoid create user on host with UID 1001
fix-permissions:
image: busybox
user: root
working_dir: /app
volumes:
- .:/app
command: >
sh -c "
dirs='public .cache node_modules'
for d in $$dirs; do
mkdir -p $$d
chmod -R 777 $$d
done
chmod 666 package-lock.json
"There was a problem hiding this comment.
What is the behavior if you keep sharing the current dir to /app, the 2 nested bind mount from volumes but remove the src mount?
Removing the ./src mount leads to a static build, so the container needs to be rebuilt again to show the effect of the code change. This remove unable the advantage of hot reload feature.
I am pleased to share with you a demonstration to show the behavior if src is removed: https://1drv.ms/v/c/6458a979282b4a55/IQCZ85M2QP7BSYuzT17LDGXyAdJfNMGfmK7U4C4ef9mfCq8?e=rTlINr
There was a problem hiding this comment.
Oh I see. Any reason to mount ./ to $APP_WORKING_DIR/workspace instead of `$APP_WORKING_DIR then? It would solve your issue.
There was a problem hiding this comment.
Yeah exactly, Mounting ./ to a subdirectory instead off APP_WORKING_DIR will solve the issue
There was a problem hiding this comment.
Good, then let's roll with mounting ./ to APP_WORKING_DIR and specify the GASTBY_* environment variables to another directories inside the container filesystem
There was a problem hiding this comment.
Good, then let's roll with mounting
./toAPP_WORKING_DIRand specify theGASTBY_*environment variables to another directories inside the container filesystem
Done, this video shows that the GASTBY_* environment variables are ignored and .cache is still used in APP_WORKING_DIR
Video.Project.3.mp4
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
Co-authored-by: Damien Duportal <damien.duportal@gmail.com>
… permissions and environment handling
|
Hello @Eng-Omar-Hussein can you fix the problem related to the Jenkinsfile conflict? Don't forget to update the environment variables. |
dduportal
left a comment
dduportal
left a comment
There was a problem hiding this comment.
Can you update this PR to resolve conflicts please?
ALso, can you fix the .env technique so it works on contributors machines? The .env file should be split into 2 explicit developement.env and ci.jenkins.io.env files which may or may not have same values.
Then you can use docker-compose features (see at least https://docs.docker.com/compose/how-tos/environment-variables/set-environment-variables/) to properly set the values
…or improved environment configuration
…ent configuration
dduportal
left a comment
dduportal
left a comment
There was a problem hiding this comment.
This PR is slowly drifting and need attention. A few mandatory elements (in the current state):
- Unless there is a strong reason, I suggest to have the default
.envfile for values common between pipeline and dev's environment - Keep the ci.jenkins.io 's pipeline specific env. variables in the pipeline's
environment {}block
The reason for this is that we are trying to fix permission errors, and on the other hand, we might lose the hot reload feature.
Why don't we just keep one .env file for both and try changing the values that the compose file needs like we did in the previous PR? |
|
Available Solutions
|
Have you tried using https://docs.docker.com/compose/how-tos/file-watch/ for the "watch/change/reload" to avoid bind mounts?
I don't think so: it's a common "challenge" with any web development (in any languages) and Docker Compose stack. The
Nope, won't work on many configurations (enforced Docker CE, user namsepaced Docker daemons, ci.jenkins.io, etc.)
It could help but it still involve messing up with permissions of the host.
Could be an idea If the image is built as part of the |
|
… improved performance and consistency
The code with the |
Thanks:') I'm glad it looks cleaner 🎊 So far, the "live reload" feature has performed well in my tests; changes are detected quickly, and I haven't noticed any significant slowdowns, even during routine development work. |
4 participants
Description
This PR Adds a Docker Compose-based local development environment as an alternative to the existing Node.js + npm workflow. The existing workflow is unchanged — this sits alongside it as a second supported path for contributors.
Fixes
Fixes #354
Submitter checklist
Additional Context
When use Docker Compose
When avoid Docker Compose