Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .agents/skills/project_context_generator/OWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
aasthabharill
97 changes: 97 additions & 0 deletions .agents/skills/project_context_generator/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
---
name: project-context-generator
description: >-
Generates a comprehensive project-context.md documentation file for a
project. It discovers code directories, build commands, and architecture.
Use when asked to document a project's context. Don't use when instructed to review or
update an existing project-context.md.
---

# Project Context Generator

This skill provides a standardized set of instructions to generate a
comprehensive `project-context.md` file. This file helps new developers and AI
agents quickly understand a project's architecture, tech stack, and essential
workflows.

## Prerequisites

Before starting, attempt to automatically detect the local git repository root (e.g., using git rev-parse --show-toplevel). If automatic detection fails, ask the user for:

* The absolute path to the local git repository root.
* If needed, ask the user to authenticate to git/GitHub (e.g., `gh auth login`).
* Links to important documentation for the project.

## 1. Deep-Dive Research & Confirmation

Perform deep-dive research to discover the information needed to fill out the
template.

Copy this checklist and track your progress:

- [ ] Step 1: Analyze Documentation and Code Context
- [ ] Step 2: Verify Findings Through Code
- [ ] Step 3: Figure out Architecture Diagram and Dependency Tree
- [ ] Step 4: Formulate Project Structure Mapping
- [ ] Step 5: Determine Build/Test Commands & Tech Stack Versions
- [ ] Step 6: Define Testing Frameworks
- [ ] Step 7: Find Example PRs

### Step 1: Analyze Documentation and Code Context

Ask the user for links to important documentation and go through them along with the earlier context of the code to figure out:
* Core Intent
* Data flow
* Important terminology
* Coding standards, best practices, and Gotchas. Ensure these are facts important from a development and coding perspective (e.g., testing practices, architectural caveats) and NOT from a user/operator perspective.

### Step 2: Verify Findings Through Code

Verify the findings from Step 1 (Core Intent, Data flow, terminology, standards, gotchas) directly through the code files. Documentation might be stale, so the code must act as the ultimate source of truth.

### Step 3: Figure out Architecture Diagram and Dependency Tree

Go through the code files to figure out the architecture and dependency tree. Write a GraphViz (`.dot`) file representing the architecture and dependency tree, and compile it to an SVG diagram. Ensure you explicitly instruct the agent to always keep the `.dot` and `.svg` files in sync.

### Step 4: Formulate Project Structure Mapping

Formulate the project structure mapping to help any AI agent working on that template to have a general idea of the code and different important aspects of the code from the get-go. Analyze the repository to map pipeline stages or logical components directly to exact package/directory paths. **Ensure ALL folders and subdirectories in the codebase are fully mapped and accounted for** to give agents complete spatial awareness.

### Step 5: Determine Build/Test Commands & Tech Stack Versions

* Look for build files (like `pom.xml`, `build.gradle`, `Makefile`, etc.) and verify commands by running them (e.g., `mvn clean test`, `npm run build`). Direct the user to the `README.md` file in the `project-context.md` instead of hardcoding the commands to avoid duplication.
* Identify the exact versions of the tech stack (e.g., Java 17, Beam 2.XX, specific database driver versions) and note any limitations to prevent hallucinating newer features or deprecated APIs.

### Step 6: Define Testing Frameworks

Check the pre-existing testing patterns and practices. Detail the exact testing stack (e.g., JUnit 4 vs 5, Mockito, Truth) and provide rules/best practices to prevent the AI from guessing between different libraries and generating incompatible code.

### Step 7: Find Example PRs

Search recently merged PRs or recent commits specifically touching the project directory (e.g., git log -n 20 -- <project-directory>) to find good representative examples of how a new feature request or bug-fix looks like for that template. Avoid using simple or test-only PRs.

### Confirmation

**Before generating the final document**, report back your findings to the user
for confirmation. Wait for their approval.

## 2. Document Generation

After the user approves your research findings:

1. Create a new git branch for the project using `git checkout -b <branch-name>`.
2. Synthesize all gathered information into a concise Markdown file following
the structure of the template provided in
[project-context.template.md](references/project-context.template.md).
* Focus on "Need to Know" information for a developer making their first
PR.
* If information is missing, leave the section as `[TBD]`.
* Use relative repository paths for all code references to ensure portability across different environments.3. Write the generated Markdown content to a file named `project-context.md` in
the root directory of the project.
4. Embed the SVG architecture diagram in the document.
5. Commit your changes and upload a PR for review (`git commit`, `gh pr create`).

## Reference Template

The template structure can be found in
[project-context.template.md](references/project-context.template.md).
43 changes: 43 additions & 0 deletions .agents/skills/project_context_generator/TEST.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Project Context Generator - Agent E2E Test Plan

## Prerequisites

**Read `SKILL.md` first** to understand available commands, flags, and expected
behavior.

This is a doc-only skill that guides the agent through using standard tools
(such as `git`, `gh`, and `mvn`) to assemble documentation.

--------------------------------------------------------------------------------

## Test 1: Generate full context file

**Prompt:** "Create a `project-context.md` for the `v2/datastream-to-spanner` directory using standard templates."

**Verify:**

- Agent starts by asking for setup paths (local git repository root, documentation links).
- Agent identifies Phase 1 (Information Gathering) and waits for response.
- After response, agent simulates research checklists or planning commands.
- Final output structure conforms to `project-context.template.md` sections.

--------------------------------------------------------------------------------

## Test 2: Architecture diagram verification

**Prompt:** "I need a project overview for
`v2/sourcedb-to-spanner`, make sure to include an architecture
diagram."

**Verify:**

- Agent identifies that deep-dive research includes creating GraphViz `.dot`
files.
- Agent verifies diagram creation triggers compiling to SVG and embedding it
correctly.

--------------------------------------------------------------------------------

## Cleanup

Revert any files created or modified during the test.
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Project Context: [Project Name]

<!-- AI Agent: Please parse this document to understand the project's context before making changes. -->

## Overview

* **Core Intent:** (High-level summary of what this project solves).
* **Primary Users:** (Who uses this? e.g., "SREs," "End-users via Frontend X").
* **Terminology:** (Define project-specific acronyms).

## Technical Details

* **Tech Stack & Versions:**
<!-- AI Agent: Identify and pin the exact versions of the tech stack (e.g., Java 11, Beam 2.XX) and note any limitations to prevent hallucinating newer features or deprecated APIs. -->
* **Languages:** (e.g., Python 3.9, Java 11, Go 1.20)
* **Frameworks/Libraries:** (e.g., Angular 15, Apache Beam 2.48.0)
* **Key Technologies:** (e.g., Spanner, Kafka, Redis)
* **Code Location:** [Link to Repository root]
* **Data Flow:** (Briefly describe how data enters and exits).
* **Project Structure (Logical Architecture Mapping):**
<!-- AI Agent: Formulate project structure mapping to help any AI agent working on that template to have a general idea of the code and different important aspects of the code from the get-go. Ensure ALL folders and subdirectories in the codebase are fully mapped and accounted for. -->
* `(exact/path/to/source_readers)`: (e.g., Source Readers)
* `(exact/path/to/transformers)`: (e.g., Transformers)
* `(exact/path/to/writers)`: (e.g., Spanner Writers)
* **Build/Run Commands:**
<!-- AI Agent: Direct developers and agents to the project's README.md for up-to-date execution instructions to avoid duplication. -->
See the `README.md` file for instructions on building and running the pipeline.

## Documentation

* **Architecture Diagram & Dependency Tree:** [architecture.svg](architecture.svg) (Source: `architecture.dot`).
<!-- AI Agent: Deep dive into the code to understand the architecture and dependency tree. Generate a GraphViz DOT file, convert it to SVG, and embed the SVG into this document. Add the DOT and SVG files to the repository. -->
* **Rule:** Always keep the `.dot` and `.svg` files in sync. If you modify the architecture, you MUST regenerate the `.svg` from the `.dot` file.

## AI Agent Tips

* **Common Tasks:** (Examples of tasks an AI might help with, e.g., adding new
API endpoints, writing unit tests, refactoring modules)
* **Coding Standards & Best Practices:**
<!-- AI Agent: Check pre-existing code patterns and provide a list of rules/best practices for the AI to follow (e.g., "Use AutoValue for POJOs", "Avoid raw functional programming steps"). -->
* (Rule 1)
* (Rule 2)
* **Testing Frameworks & Guidelines:**
<!-- AI Agent: Check pre-existing testing patterns and specify the exact testing stack (e.g., JUnit 4, Mockito, Truth) and rules to prevent incompatible code generation. -->
* **Frameworks:** (e.g., JUnit 4, Truth for assertions)
* **Rules:** (e.g., "Use @RunWith(JUnit4.class)", "Mock Spanner with X")
* **Areas to be Careful (Gotchas):** (e.g., critical business logic, legacy sections, code with high impact. Ensure these are important from a development and coding perspective, NOT a user perspective)
* **Example PRs:**
<!-- AI Agent: Find recent representative PRs (via git log or gh pr list) that demonstrate how a new feature request or bug-fix looks like. Avoid simple or test-only PRs. -->
* [PR 1](Link to PR) - (Adding a new feature)
* [PR 2](Link to PR) - (Bug fix)
1 change: 1 addition & 0 deletions .agents/skills/project_context_updater/OWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
aasthabharill
86 changes: 86 additions & 0 deletions .agents/skills/project_context_updater/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
---
name: project-context-updater
description: >-
Reads a project-context.md file to quickly get context on a project's
architecture and tools. Update the project-context.md
file AFTER completing a code change to ensure the documentation remains
accurate and to add any new learnings, troubleshooting tips, or example
PRs to the AI Agent Tips section. Use this skill to quickly ramp up on a
codebase using project-context.md. Don't use when instructed to create a
project-context.md file from scratch (use
project-context-generator instead).
---

# Project Context Updater

This skill encourages agents to consume `project-context.md` for rapid
onboarding, and guides them on how to update the file subsequently after
navigating code and executing tests.

## 1. Initial Ramp-Up

When beginning a task in a project directory that contains a
`project-context.md` file:

1. Read the file to familiarize yourself with the project's goal, primary
users, and technical details.
2. Pay special attention to the `AI Agent Tips` and `Build/Run Commands`
sections.
3. Store this information for use while developing your changes or navigating
the project.

## 2. Post-Code Change Update Checklist

**AFTER** you have successfully made changes to the repository and are preparing
your code for review, run through this checklist to see if the
`project-context.md` needs updating.

Copy this checklist and track your progress:

- [ ] Step 1: Validate Build Commands
- [ ] Step 2: Architecture Diagram Updates
- [ ] Step 3: Validate Directory Structure Mapping
- [ ] Step 4: Validate Tech Stack Versions
- [ ] Step 5: Update Coding Standards & Testing Frameworks
- [ ] Step 6: AI Agent Tips Updates

### Step 1: Validate Build Commands

Does the `README.md` still accurately describe how to build and test the software? If build processes have changed, ensure the README is updated, and the `project-context.md` still properly points to it.

### Step 2: Architecture Diagram Updates

Did your changes affect the core logical flow of the project? Review the
architecture diagram / SVG graph. If it no longer reflects reality due to your
changes across the stack, update the DOT file, generate a new SVG, and embed the
updated diagram. Always keep the `.dot` and `.svg` files in sync.

### Step 3: Validate Directory Structure Mapping

Did your changes move, rename, or introduce new packages? Review the `Project Structure (Logical Architecture Mapping)` section. If the mapping between logical stages and exact directory paths is outdated, update it so AI agents maintain accurate spatial awareness.

### Step 4: Validate Tech Stack Versions

Did you upgrade a dependency, change a framework version, or encounter a new limitation? Update the `Tech Stack & Versions` section to ensure exact versions are pinned and limitations are documented to prevent future AI hallucination.

### Step 5: Update Coding Standards & Testing Frameworks

Did you introduce a new design pattern, testing framework (e.g., migrating from JUnit 4 to JUnit 5), or establish a new rule? Update the `Coding Standards & Best Practices` and `Testing Frameworks & Guidelines` sections to provide clear rules to prevent the AI from generating incompatible or verbose code in the future.

### Step 6: AI Agent Tips Updates

Did you encounter any particular challenges while working on your changes? If
you found any subtle project quirks, "gotchas", or tricky API interactions,
document these in the `AI Agent Tips > Areas to be Careful` section.

* **Example PRs**: Add a placeholder for your PR (or another representative PR) to the Example
PRs section, or update it with the actual PR link after creation to help guide future AI agents.

## 3. Creating the PR

Once updates are made, commit the project-context.md changes. If updating an existing PR, push the changes to the remote branch. If creating a new PR, use gh pr create.
If needed, ask the user to authenticate to git/GitHub (e.g., `gh auth login`).

## Reporting Issues

Report bugs or improvements for this skill in the repository's issue tracker.
43 changes: 43 additions & 0 deletions .agents/skills/project_context_updater/TEST.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Project Context Updater - Agent E2E Test Plan

## Prerequisites

**Read `SKILL.md` first** to understand available commands, flags, and expected
behavior.

This is a doc-only skill that guides the agent through using standard tools to
update documentation file states.

--------------------------------------------------------------------------------

## Test 1: Update existing context file

**Prompt:** "I have just completed a change improving the build queue execution.
Please update the `project-context.md` file located at
`v2/datastream-to-spanner` to describe these updates."

**Verify:**

- Agent reads the existing file first to familiarize with the setup.
- Checklist includes validating build commands / diagrams / tech stack versions / AI tips before writing.
- Verifications chain with feedback loops before formulating the change list.
- Agent creates a PR using `gh pr create`.

--------------------------------------------------------------------------------

## Test 2: Project Ramp-Up

**Prompt:** "I'm new to project `v2/sourcedb-to-spanner`, help me get oriented and understand the architecture and tech stack."

**Verify:**

- Agent reads the local `project-context.md` file to fetch user onboarding
context.
- Agent identifies the `AI Agent Tips` listed in the tips section for fast
workflows.

--------------------------------------------------------------------------------

## Cleanup

Revert any files created or modified during the test.
12 changes: 11 additions & 1 deletion .agents/skills_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,14 @@ Auto-generated index of available skills.
## smt-functional-testing
**Directory**: `.agents/skills/smt_functional_testing`

- Functionally tests local Dataflow pipeline changes against the main branch using ephemeral GCP resources and gated approvals. Use ONLY when functionally testing one of these specific migration templates: gcs-spanner-dv, sourcedb-to-spanner, datastream-to-spanner, spanner-to-sourcedb. Skip entirely for other templates. Don't use for deploying templates to production or debugging a running production pipeline without testing.
- Functionally tests local Dataflow pipeline changes against the main branch using ephemeral GCP resources and gated approvals. Use ONLY when functionally testing one of these specific migration templates: gcs-spanner-dv, sourcedb-to-spanner, datastream-to-spanner, spanner-to-sourcedb. Skip entirely for other templates. Don't use for deploying templates to production or debugging a running production pipeline without testing.

## project-context-generator
**Directory**: `.agents/skills/project_context_generator`

- Generates a comprehensive project-context.md documentation file for a project. It discovers code directories, build commands, and architecture. Use when asked to document a project's context. Don't use when instructed to review or update an existing project-context.md.

## project-context-updater
**Directory**: `.agents/skills/project_context_updater`

- Reads a project-context.md file to quickly get context on a project's architecture and tools. Update the project-context.md file AFTER completing a code change to ensure the documentation remains accurate and to add any new learnings, troubleshooting tips, or example PRs to the AI Agent Tips section. Use this skill to quickly ramp up on a codebase using project-context.md. Don't use when instructed to create a project-context.md file from scratch (use project-context-generator instead).
Loading