Skip to content

Honor .gitignore when packaging job context - #279

Open
GargiGupta-io wants to merge 4 commits into
keras-team:mainfrom
GargiGupta-io:fix-262-git-aware-packaging
Open

Honor .gitignore when packaging job context#279
GargiGupta-io wants to merge 4 commits into
keras-team:mainfrom
GargiGupta-io:fix-262-git-aware-packaging

Conversation

@GargiGupta-io

@GargiGupta-io GargiGupta-io commented Jul 19, 2026

Copy link
Copy Markdown

What I found

I was working on packaging job contexts and realized the packager wasn't actually
respecting .gitignore files. It would include files that should've been ignored,
which is a problem when you're trying to keep your job context clean and not upload
stuff you explicitly told Git to ignore.

Why this matters

When you're packaging a job with sensitive files or build artifacts, you expect
.gitignore to actually work. Right now it doesn't, so you end up shipping files
you never intended to include.

How I fixed it

Instead of trying to manually parse and apply ignore patterns (which is error-prone),
I switched to using git ls-files when Git is available. Git already knows how to
handle .gitignore properly, so why reinvent that wheel? If Git isn't available,
it still falls back to the old directory traversal approach.

Testing

Tested it with a few different .gitignore patterns to make sure it works:

  • Simple patterns like *.tmp and build/
  • More complex patterns with negations
  • Verified the fallback works when Git isn't available

All the existing tests pass too, so no regressions.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the packager utility to support packaging Git-tracked and non-ignored untracked files using git ls-files, falling back to a standard directory walk if Git is not available. The feedback focuses on improving robustness, performance, and cross-platform compatibility. Key suggestions include broadening exception handling from FileNotFoundError to OSError in subprocess execution, normalizing base_dir to an absolute path to prevent exclusion check failures, optimizing the exclusion helper to return early when no paths are excluded, and using posixpath.join to ensure ZIP archive paths use forward slashes on all operating systems.

Comment thread kinetic/utils/packager.py Outdated
Comment thread kinetic/utils/packager.py
Comment thread kinetic/utils/packager.py
Comment thread kinetic/utils/packager.py Outdated
Comment thread kinetic/utils/packager.py
@GargiGupta-io
GargiGupta-io marked this pull request as ready for review July 22, 2026 18:43
@GargiGupta-io

Copy link
Copy Markdown
Author

This PR has been waiting for 11 days. Could a maintainer review when you have time? All checks are passing.

@GargiGupta-io

Copy link
Copy Markdown
Author

Just resolved the merge conflicts with main! PR is now ready for review. 👍

@jeffcarp

jeffcarp commented Aug 5, 2026

Copy link
Copy Markdown
Member

Thanks for the contribution - can you verify this is still compatible post #284?

@GargiGupta-io

Copy link
Copy Markdown
Author

PR #284 needs the plan_json parameter in zip_working_dir(). I've added it back to maintain compatibility while keeping your git-aware improvements. All set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants