Skip to content

CLAUDE.md

Latest commit

 

History

History
88 lines (57 loc) · 4.3 KB

CLAUDE.md

File metadata and controls

88 lines (57 loc) · 4.3 KB
Raw

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Summary

pulp_hugging_face is a Pulp platform plugin that provides pull-through caching for Hugging Face Hub content (models, datasets, spaces). It acts as a transparent caching proxy — requests for HF content are fetched from upstream on first access and served from cache thereafter. The plugin injects HF Hub-compatible headers so the huggingface_hub CLI works seamlessly against a Pulp instance.

Build and Development Commands

# Install in development mode
pip install -e .

# Lint
pip install -r lint_requirements.txt
black --check --diff .
flake8

# Auto-format
black .

# Run unit tests (no running Pulp instance needed)
pip install -r unittest_requirements.txt
pytest pulp_hugging_face/tests/unit/ -v

# Run a single test
pytest pulp_hugging_face/tests/unit/test_foo.py::test_bar -v

# Functional tests (requires a running Pulp instance in containers)
pip install -r functest_requirements.txt
pytest pulp_hugging_face/tests/functional/ -v

# Check package manifest
check-manifest

Code Style

  • black formatter with line length 100, targeting Python 3.9–3.12
  • flake8 with max line length 100; migrations and __init__.py (F401) are excluded
  • Migrations directory is excluded from both black and flake8

Architecture

Plugin Registration

Entry point in pyproject.toml registers pulp_hugging_face as a pulpcore.plugin. The Django app config lives in pulp_hugging_face/app/__init__.py with app label hugging_face.

Core Models (pulp_hugging_face/app/models.py)

All models extend pulpcore base classes and use TYPE = "hugging-face":

  • HuggingFaceContent → Content — cached files keyed by (repo_id, repo_type, relative_path, revision, _pulp_domain). The init_from_artifact_and_relative_path() classmethod parses HF URL patterns to populate fields during pull-through caching.
  • HuggingFaceRemote → Remote — stores hf_hub_url and hf_token. Implements get_remote_artifact_url(), get_remote_artifact_content_type(), and get_downloader() for the pull-through caching protocol.
  • HuggingFaceRepository → Repository
  • HuggingFaceDistribution → Distribution — the key integration point. Its content_handler property returns the custom handler function, and content_headers_for() injects HF-compatible response headers (X-Repo-Commit, X-Linked-ETag, ETag, Content-Disposition, etc.).
  • HuggingFacePublication → Publication (skeleton)

Pull-Through Caching Flow

  1. Request hits Distribution's content_handler (handler.py)
  2. Handler matches HF URL patterns ({repo_id}/resolve|blob/{revision}/{filename}, with optional models/|datasets/|spaces/ prefix)
  3. If artifact exists in cache → returns ArtifactResponse with HF headers
  4. If not cached → Remote's get_remote_artifact_url() builds upstream URL, get_downloader() fetches with HF auth headers, Content.init_from_artifact_and_relative_path() creates the content record
  5. content_headers_for() adds HF-compatible headers using deterministic SHA1 hashes for commit/ETag values

URL Routing (urls.py, views.py)

  • File downloads: /pulp/content/{base_path}/{repo_id}/resolve/{revision}/{filename} → 307 redirect to resolve-cache endpoint
  • API proxy: /pulp/content/{base_path}/api/{path} → forwarded to upstream HF Hub
  • Resolve-cache: /pulp/api/v3/content/huggingface/resolve-cache/models/{repo_id}/{revision}/{filename}/ → redirects to Pulp content handler

ViewSets (viewsets.py)

Standard Pulp CRUD viewsets at endpoint hugging-face. Repository viewset has a sync action; Publication viewset has a create/publish action. Both dispatch async tasks.

Tasks (tasks/)

synchronizing.py and publishing.py contain skeleton task implementations.

Plugin Template

Many CI/config files are auto-generated by pulp/plugin_template (marked WARNING: DO NOT EDIT!). To regenerate: ./plugin-template --github pulp_hugging_face. Configuration is in template_config.yml.

Changelog

Uses towncrier. Add fragments to CHANGES/ named {issue_number}.{type}.md where type is one of: feature, bugfix, doc, removal, deprecation, misc.