Skip to content

Add agent skill for F´ integration testing#62

Open
devin-ai-integration[bot] wants to merge 1 commit into
develfrom
devin/1779321673-integration-testing-skill
Open

Add agent skill for F´ integration testing#62
devin-ai-integration[bot] wants to merge 1 commit into
develfrom
devin/1779321673-integration-testing-skill

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented May 21, 2026

Copy link
Copy Markdown
Related Issue(s) N/A
Has Unit Tests (y/n) n
Documentation Included (y/n) y
Generative AI was used in this contribution (y/n) AI

Change Description

Adds a comprehensive agent skill (SKILL.md) at .agents/skills/integration-testing/SKILL.md that teaches AI coding agents how to author F´ integration test (InT) scripts using the GDS Python testing framework (fprime_gds.common.testing_fw).

The skill covers:

  • Pytest setup and the fprime_test_api fixture (session + function scoped)
  • Core API methods: sending commands (send_command, send_and_assert_command, send_and_assert_event, send_and_assert_telemetry)
  • Event search/assert: await_event, assert_event, await_event_sequence, assert_event_count
  • Telemetry search/assert: await_telemetry, assert_telemetry, await_telemetry_count, assert_telemetry_sequence
  • File operations: uplink_file, uplink_file_and_await_completion, uplink_sequence_and_await_completion
  • Sequence execution: compiling .seq files with fprime-seqgen and running them via CS_RUN
  • Predicates module: comparison, set, logic, and data object predicates with composition patterns
  • Search scope: start/timeout semantics, NOW marker, TimeType usage
  • Sub-histories: filtered event/telemetry sub-histories for targeted assertions
  • Assert helpers: test_assert, predicate_assert, expect mode for soft checks
  • Deployment configuration: get_mnemonic() for portable component name resolution
  • Anti-patterns: zero-count with timeout, no-scope searches, timestamp in sequences, predicate inversion gotchas
  • Complete working examples and a 10-point checklist for test authors

Rationale

AI agents working on F´ projects need structured guidance to write correct integration tests. This skill provides the complete API reference, patterns, and anti-patterns derived from the actual fprime-gds testing framework source code and existing Ref app integration tests. It enables agents to produce idiomatic, correct InT scripts without having to rediscover the API each session.

Testing/Review Recommendations

  • Verify the skill content aligns with the current fprime_gds.common.testing_fw.api.IntegrationTestAPI interface
  • Check that the examples are consistent with existing tests in TestDeploymentsProject/Ref/test/int/
  • This is a documentation-only change (a SKILL.md file); no code paths are affected

Future Work

  • Could add coverage for Data Products (DP) testing patterns
  • Could expand with deployment-specific skill variants for different projects

AI Usage (see policy)

This skill was authored by an AI agent (Devin) by analyzing the fprime-gds testing framework source code (common/testing_fw/api.py, predicates.py, pytest_integration.py), the GDS test API guide documentation (docs/user-manual/gds/gds-test-api-guide.md), and existing integration test examples across the fprime repository.

IAMAI

Link to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/1d703e38f8e84373b8df69ff9a9a5075


Open in Devin Review

Adds a comprehensive SKILL.md that teaches AI agents how to author
integration test scripts using the GDS Python testing framework
(fprime_gds.common.testing_fw). Covers:

- pytest setup and the fprime_test_api fixture
- Core API methods (commands, events, telemetry, file ops, sequences)
- Predicates module (comparison, set, logic, data object predicates)
- Search scope (start/timeout), sub-histories, TimeType operations
- Assert helpers, deployment configuration, anti-patterns
- Complete working examples and a checklist for test authors

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Author
Original prompt from thomas.boyer.chammard

Looking across both fprime and fprime-gds, including documentation of the testing_fw in GDS and whatever you can find about integration testing: please create an agent skill for integration testing in F´, for authoring InT scripts with the Python testing framework.

@devin-ai-integration

Copy link
Copy Markdown
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant