Skip to content

Add Docker Compose setup for local development#383

Open
Eng-Omar-Hussein wants to merge 18 commits into
jenkins-infra:mainfrom
Eng-Omar-Hussein:compose
Open

Add Docker Compose setup for local development#383
Eng-Omar-Hussein wants to merge 18 commits into
jenkins-infra:mainfrom
Eng-Omar-Hussein:compose

Conversation

@Eng-Omar-Hussein

@Eng-Omar-Hussein Eng-Omar-Hussein commented Mar 30, 2026

Copy link
Copy Markdown

Description

This PR Adds a Docker Compose-based local development environment as an alternative to the existing Node.js + npm workflow. The existing workflow is unchanged — this sits alongside it as a second supported path for contributors.

Fixes

Fixes #354

Submitter checklist

  • Descriptive PR title and meaningful summary
  • Changes align with project conventions
  • No unrelated changes included
  • Documentation updated (if needed)

Additional Context

When use Docker Compose

  • Need isolated development environment
  • No Node version manager installed (nvm, asdf, etc.)
  • Need use the same environment as the CI (same base OS, same versions, env. variables, etc.) to decrease the gap (all you need in one image)
  • Used docker before (Familiar with docker)

When avoid Docker Compose

  • Need maximum performance, single service only
  • Have bind-mount issues (considerations-and-constraints)
  • Already have a version manager and correct Node version
  • when Docker can't be used (Not familiar with docker)
  • when you want to be closer to CI/production

@probot-autolabeler probot-autolabeler Bot added chore documentation Improvements or additions to documentation labels Mar 30, 2026
Updated the command in the stories_webapp service to use 'npm run develop'.
Updated the volume and working directory paths to use the WORKSPACE_ variable.

@dduportal dduportal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, it's getting better. But you clearly have not tested this change: the pipeline would be broken.

I've let comment, can you please fix and test a bit please?

Comment thread .env Outdated
Comment thread .env
Comment thread .env Outdated
Comment on lines +7 to +9
# // Added the below to fix permissions issue with the cache
GATSBY_CACHE_DIR = "${WORKSPACE_}/.gatsby-cache"
GATSBY_INTERNAL_CACHE_DIR = "${WORKSPACE_}/.cache"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will break the CI. Is it needed with Docker Compose?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not needed in Docker. I initially thought it was important to fix permissions in the CI, so I kept it in Docker. However, after testing, I found a problem that may occur if the user runs it on the host OS using the default user with UID=1000. They will face an error related to cache permissions, because the Jenkins user used in Docker Compose has UID=1001. As a result, the bind-mounted data will have 1000 ownership, and Jenkins will not have permission to write to these files.

Therefore, I should add this to the documentation. For example:

Before running docker compose up, make sure that the user you are using has UID=1001. If not, use the command below.

// for linux and wsl 
sudo chown -R 1001:1001 .

or create and use a user with UID=1001

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is not a safe neither a portable solution:

  • UID is different on macOS with Docker Desktop
  • UID is different on Linux with Docker CE

Since there are no reason to share the cache dir with the host, WDYT about setting the env. var in Docker Compose only, to a container-internal directory.

@Eng-Omar-Hussein Eng-Omar-Hussein Apr 10, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is that Gatsby is ignoring GATSBY_CACHE_DIR or GATSBY_INTERNAL_CACHE_DIR and still trying to create .cache inside the workspace directory (APP_WORKING_DIR), where the jenkins (UID=1001) user has no write permission because the workspace directory need root ownership to have write access in APP_WORKING_DIR
image

I recently reach to solution:

  • add another service with root user to fix permission for .cash and public e.g
services:
  fix-permissions:
    image: busybox
    user: root
    working_dir: /app
    volumes:
      - .:/app
    command: >
      sh -c "
        if ! [ -d /app/public ]; then
          mkdir -p /app/public
        fi
        if ! [ -d /app/.cache ]; then
          mkdir -p /app/.cache
        fi
        if ! id -u jenkins > /dev/null 2>&1; then
          adduser -u 1001 -D jenkins
        fi
        chown -R jenkins:jenkins /app/public /app/.cache
      "

@Eng-Omar-Hussein Eng-Omar-Hussein Apr 11, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recently reach to solution:

  • add another service with root user to fix permission for .cash and public e.g
services:
  fix-permissions:
    image: busybox
    user: root
    working_dir: /app
    volumes:
      - .:/app
    command: >
      sh -c "
        if ! [ -d /app/public ]; then
          mkdir -p /app/public
        fi
        if ! [ -d /app/.cache ]; then
          mkdir -p /app/.cache
        fi
        if ! id -u jenkins > /dev/null 2>&1; then
          adduser -u 1001 -D jenkins
        fi
        chown -R jenkins:jenkins /app/public /app/.cache
      "

The new version of this recommended solution replace chown with chmod to avoid create user on host with UID 1001

  fix-permissions:
    image: busybox
    user: root
    working_dir: /app
    volumes:
      - .:/app

    command: >
      sh -c "
        dirs='public .cache node_modules'
        for d in $$dirs; do
          mkdir -p $$d
          chmod -R 777 $$d
        done
        chmod 666 package-lock.json
      "

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the behavior if you keep sharing the current dir to /app, the 2 nested bind mount from volumes but remove the src mount?

Removing the ./src mount leads to a static build, so the container needs to be rebuilt again to show the effect of the code change. This remove unable the advantage of hot reload feature.

I am pleased to share with you a demonstration to show the behavior if src is removed: https://1drv.ms/v/c/6458a979282b4a55/IQCZ85M2QP7BSYuzT17LDGXyAdJfNMGfmK7U4C4ef9mfCq8?e=rTlINr

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see. Any reason to mount ./ to $APP_WORKING_DIR/workspace instead of `$APP_WORKING_DIR then? It would solve your issue.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah exactly, Mounting ./ to a subdirectory instead off APP_WORKING_DIR will solve the issue

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good, then let's roll with mounting ./ to APP_WORKING_DIR and specify the GASTBY_* environment variables to another directories inside the container filesystem

@Eng-Omar-Hussein Eng-Omar-Hussein Apr 15, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good, then let's roll with mounting ./ to APP_WORKING_DIR and specify the GASTBY_* environment variables to another directories inside the container filesystem

Done, this video shows that the GASTBY_* environment variables are ignored and .cache is still used in APP_WORKING_DIR

Video.Project.3.mp4

Comment thread docker-compose.yml Outdated
Comment thread Jenkinsfile Outdated
Comment thread Jenkinsfile Outdated
Comment thread Jenkinsfile Outdated
Comment thread Jenkinsfile Outdated
Comment thread Jenkinsfile
Comment thread Jenkinsfile Outdated
@lemeurherve

This comment was marked as resolved.

@Eng-Omar-Hussein

This comment was marked as resolved.

Co-authored-by: Damien Duportal <damien.duportal@gmail.com>
…e setup by removing unnecessary permissions service
@krisstern

Copy link
Copy Markdown
Member

@dduportal what's your take on this? Should we close this as unplanned?

@dduportal

Copy link
Copy Markdown
Contributor

@dduportal what's your take on this? Should we close this as unplanned?

@krisstern the planficiation work is on the issue first: #354.

I believe it should be implemented. I've just been out of time to review and move forward. Is there any emergency or event that should push us to close or to accelerate the review?

@krisstern

Copy link
Copy Markdown
Member

@dduportal this project is planned to undering some revamping work for GSoC 2026 and so this may need to be reworked anyways, so would like to check with you if we should keep it open for future reference or close it if we will not go ahead with it

@dduportal

Copy link
Copy Markdown
Contributor

@dduportal this project is planned to undering some revamping work for GSoC 2026 and so this may need to be reworked anyways, so would like to check with you if we should keep it open for future reference or close it if we will not go ahead with it

I assume the production website won't be changed until the end of the GSoC project so this tooling is still interesting to have right?

What is the timeline to "update" the current website by the result of the GSoC project?

@krisstern

Copy link
Copy Markdown
Member

Yes, that's correct. But I am not sure if the contributor of this PR would want to continue work on this since he has not been selected as a GSoC participant. Also that I am closing most of the PR's that are definitely not going ahead so I am asking now, as we will examine remaining issues and pull requests and check if we will need to include these in the current GSoC project.

@dduportal

Copy link
Copy Markdown
Contributor

Yes, that's correct. But I am not sure if the contributor of this PR would want to continue work on this since he has not been selected as a GSoC participant.

OK make sense, thanks for the explanation. It's a good point to ask the contributor what they think about continuing.

Ping @Eng-Omar-Hussein how do you see your contribution?

Also that I am closing most of the PR's that are definitely not going ahead so I am asking now, as we will examine remaining issues and pull requests and check if we will need to include these in the current GSoC project.

@krisstern there is an associated issue that lives with this PR. I would not want to see someone else implementing this issue since @Eng-Omar-Hussein did a due diligence on this

@Eng-Omar-Hussein

Copy link
Copy Markdown
Author

Yes, that's correct. But I am not sure if the contributor of this PR would want to continue work on this since he has not been selected as a GSoC participant.

OK make sense, thanks for the explanation. It's a good point to ask the contributor what they think about continuing.

Ping @Eng-Omar-Hussein how do you see your contribution?

I'm still interested in finishing this PR and seeing it through.

Thanks @dduportal for the support!

@krisstern

Copy link
Copy Markdown
Member

Also would like to add we will be migrating to use Vite.js instead of Gatsby, so after this PR is merged we expect there will be significant changes to the work done in tihs PR later by one of the GSoC 2026 contributors

@krisstern

Copy link
Copy Markdown
Member

@Eng-Omar-Hussein please resolve the merge conflicts in this PR

@dduportal

Copy link
Copy Markdown
Contributor

Also would like to add we will be migrating to use Vite.js instead of Gatsby, so after this PR is merged we expect there will be significant changes to the work done in tihs PR later by one of the GSoC 2026 contributors

I doubt it, the goal of this PR is to provide an efficient abstraction layer with docker. There will be minor changes of course (the content of package json, the port etc.) but all of these are nodejs apps and work the same

@krisstern krisstern requested a review from dduportal May 13, 2026 15:30
@Eng-Omar-Hussein

Copy link
Copy Markdown
Author

Changes in the commits 7b4bc81
1- Update jenkins agent image version to 2.102.0 (latest)
2- Remove asdf

Ref. jenkins-infra/helpdesk#5080

3- Replace npm install with npm ci

Ref. https://docs.npmjs.com/cli/v8/commands/npm-ci

@krisstern

Copy link
Copy Markdown
Member

Also would like to add we will be migrating to use Vite.js instead of Gatsby, so after this PR is merged we expect there will be significant changes to the work done in tihs PR later by one of the GSoC 2026 contributors

I doubt it, the goal of this PR is to provide an efficient abstraction layer with docker. There will be minor changes of course (the content of package json, the port etc.) but all of these are nodejs apps and work the same

Yeah, agreed

@krisstern

Copy link
Copy Markdown
Member

By the end of the project all the Gatsby-specific config in .env will be gone

@dduportal

Copy link
Copy Markdown
Contributor

By the end of the project all the Gatsby-specific config in .env will be gone

Yep, that is also a minor change but good catch !!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve local development setup

4 participants