Add F Prime unit testing skill#145
Conversation
Add a Devin skill (.devin/skills/fprime-unit-testing/SKILL.md) that provides step-by-step instructions for writing F Prime component unit tests. Covers: - Scaffold generation via fprime-util impl --ut - Tester / TestMain / GTestBase pattern - Helper function design for clean, readable tests - Rules-based testing with STest (AbstractState, TestState, Rules, Scenarios) - CMakeLists.txt registration with UT_AUTO_HELPERS - Quality checklist for coverage completeness Co-Authored-By: michael.d.starch <michael.d.starch@jpl.nasa.gov>
Original prompt from michael.d.starch
|
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
Co-Authored-By: michael.d.starch <michael.d.starch@jpl.nasa.gov>
Coverage report — base
|
| Module | Line | Function | Branch |
|---|---|---|---|
CFDP/Checksum |
71.15 | 57.14 | 44.44 |
Drv/AsyncByteStreamBufferAdapter |
100.00 | 100.00 | 100.00 |
Drv/ByteStreamBufferAdapter |
100.00 | 100.00 | 80.00 |
Drv/Ip |
43.88 | 57.38 | 20.94 |
Drv/TcpClient |
75.00 | 100.00 | 31.03 |
Drv/TcpServer |
84.72 | 100.00 | 44.26 |
Drv/Udp |
68.09 | 90.91 | 28.12 |
Fw/Buffer |
81.25 | 89.47 | 43.75 |
Fw/DataStructures |
98.30 | 97.14 | 57.26 |
Fw/Dp |
94.83 | 96.67 | 47.92 |
Fw/FilePacket |
75.24 | 89.06 | 41.43 |
Fw/Log |
71.43 | 72.41 | 34.48 |
Fw/Logger |
100.00 | 100.00 | 62.50 |
Fw/SerializableFile |
90.00 | 100.00 | 44.44 |
Fw/Time |
88.62 | 85.48 | 50.00 |
Fw/Tlm |
53.76 | 60.00 | 24.29 |
Fw/Types |
54.63 | 57.49 | 29.48 |
Os |
18.88 | 20.59 | 12.70 |
Os/Generic |
89.10 | 96.30 | 48.02 |
Os/Generic/Types |
92.45 | 91.67 | 60.94 |
Os/Posix |
60.75 | 83.72 | 39.74 |
Svc/ActiveRateGroup |
100.00 | 100.00 | 60.87 |
Svc/ActiveTextLogger |
79.05 | 90.00 | 53.85 |
Svc/AssertFatalAdapter |
94.55 | 100.00 | 59.09 |
Svc/BufferAccumulator |
88.00 | 94.12 | 54.79 |
Svc/BufferLogger |
92.62 | 86.96 | 57.46 |
Svc/BufferManager |
99.05 | 100.00 | 55.93 |
Svc/BufferRepeater |
91.67 | 100.00 | 55.56 |
Svc/ChronoTime |
100.00 | 100.00 | 50.00 |
Svc/CmdDispatcher |
96.97 | 91.67 | 52.60 |
Svc/CmdSequencer |
93.89 | 97.12 | 56.64 |
Svc/CmdSplitter |
100.00 | 100.00 | 54.55 |
Svc/ComLogger |
97.37 | 91.67 | 59.15 |
Svc/ComSplitter |
100.00 | 100.00 | 66.67 |
Svc/ComStub |
98.51 | 100.00 | 59.76 |
Svc/DpCatalog |
78.00 | 100.00 | 42.92 |
Svc/DpManager |
97.44 | 100.00 | 52.27 |
Svc/DpWriter |
97.58 | 90.00 | 62.07 |
Svc/FileDownlink |
84.15 | 90.91 | 45.29 |
Svc/FileManager |
88.41 | 93.33 | 50.45 |
Svc/FileUplink |
92.78 | 96.55 | 54.78 |
Svc/FileWorker |
92.33 | 100.00 | 45.11 |
Svc/FprimeDeframer |
100.00 | 100.00 | 58.04 |
Svc/FprimeFramer |
100.00 | 100.00 | 42.19 |
Svc/FprimeRouter |
88.89 | 100.00 | 48.65 |
Svc/FpySequencer |
86.41 | 99.02 | 51.67 |
Svc/GenericHub |
100.00 | 100.00 | 50.29 |
Svc/Health |
100.00 | 100.00 | 60.00 |
Svc/LinuxTimer |
97.06 | 100.00 | 47.06 |
Svc/OsTime |
70.00 | 83.33 | 37.80 |
Svc/PassiveRateGroup |
100.00 | 100.00 | 52.50 |
Svc/PolyDb |
100.00 | 100.00 | 42.50 |
Svc/PosixTime |
100.00 | 100.00 | 50.00 |
Svc/PrmDb |
92.20 | 90.00 | 51.29 |
Svc/RateGroupDriver |
100.00 | 100.00 | 62.50 |
Svc/SeqDispatcher |
73.08 | 80.00 | 49.18 |
Svc/StaticMemory |
100.00 | 100.00 | 55.00 |
Svc/SystemResources |
98.63 | 100.00 | 55.00 |
Svc/TlmChan |
82.40 | 85.71 | 48.57 |
Svc/TlmPacketizer |
92.54 | 100.00 | 54.30 |
Svc/Version |
96.10 | 100.00 | 54.91 |
Utils |
19.93 | 26.44 | 14.53 |
Utils/Types |
92.11 | 95.83 | 58.45 |
Modules without UTs
CFDP/Checksum/GTest, Drv/ByteStreamDriverModel, Drv/Interfaces, Drv/LinuxGpioDriver, Drv/LinuxI2cDriver, Drv/LinuxSpiDriver, Drv/LinuxUartDriver, Drv/Ports, Drv/Ports/DataTypes, FppTestProject/FppTest/interfaces, FppTestProject/FppTest/topology/async, FppTestProject/FppTest/topology/components/Comp, FppTestProject/FppTest/topology/components/Framework, FppTestProject/FppTest/topology/components/Receiver, FppTestProject/FppTest/topology/components/Sender, FppTestProject/FppTest/topology/guarded, FppTestProject/FppTest/topology/ports, FppTestProject/FppTest/topology/sync, FppTestProject/FppTest/topology/top_ports, FppTestProject/FppTest/topology/types, Fw/Cmd, Fw/Com, Fw/Comp, Fw/FilePacket/GTest, Fw/Fpy, Fw/Interfaces, Fw/Obj, Fw/Port, Fw/Ports/CompletionStatus, Fw/Ports/Ready, Fw/Ports/Signal, Fw/Ports/SuccessCondition, Fw/Prm, Fw/SerializableFile/test/TestSerializable, Fw/Sm, Fw/Test, Fw/Types/GTest, Os/Models, Svc/Cycle, Svc/DpPorts, Svc/Fatal, Svc/FatalHandler, Svc/FileDownlinkPorts, Svc/FprimeProtocol, Svc/Interfaces, Svc/PassiveConsoleTextLogger, Svc/Ping, Svc/PolyIf, Svc/Ports/CommsPorts, Svc/Ports/FilePorts, Svc/Ports/OsTimeEpoch, Svc/Ports/TlmPacketizerPorts, Svc/Ports/VersionPorts, Svc/Sched, Svc/Seq, Svc/Subtopologies/CdhCore, Svc/Subtopologies/ComCcsds, Svc/Subtopologies/ComFprime, Svc/Subtopologies/ComLoggerTee, Svc/Subtopologies/DataProducts, Svc/Subtopologies/FileHandling, Svc/Types/TlmPacketizerTypes, Svc/WatchDog, TestDeploymentsProject/Ref/PingReceiver, TestDeploymentsProject/Ref/RecvBuffApp, TestDeploymentsProject/Ref/SendBuffApp, TestDeploymentsProject/Ref/Top, TestDeploymentsProject/Ref/TypeDemo, Utils/Hash, cmake/test/data/TestDeployment/TestBuildAutocoder, cmake/test/data/TestDeployment/TestChainedAutocoder, cmake/test/data/TestDeployment/TestHeaderAutocoder, cmake/test/data/TestDeployment/TestTargetAutocoder, cmake/test/data/test-fprime-library/TestLibrary/TestComponent, cmake/test/data/test-fprime-library2/TestLibrary2/TestComponent
| @@ -0,0 +1,441 @@ | |||
| --- | |||
There was a problem hiding this comment.
This should be put with the other skills in the .github folder
| ### Pattern: assertion helpers | ||
|
|
||
| ```cpp | ||
| void <Component>Tester::assertTelemetryIdle() { |
There was a problem hiding this comment.
When meaningful and used in other places.
There was a problem hiding this comment.
Otherwise explain chunks used once with a terse comment describing the change.
|
|
||
| ### 5.6 — Random scenario | ||
|
|
||
| Wire all rules into a bounded random walk: |
There was a problem hiding this comment.
You should also write basic unit tests using a static set of rule invocations....not just rely on random testing.
- Move from .devin/skills/ to .github/agents/_shared/skills/fprime-unit-testing.skill.md - Add unit-testing section to copilot-instructions.md so other AIs discover the skill - Trim skill from 441 to ~300 lines while preserving all sections - Exclude .github/agents/ from spell-check Co-Authored-By: michael.d.starch <michael.d.starch@jpl.nasa.gov>
devin-ai-integration
Bot
changed the title
Replace hallucinated macro patterns (TEST_STATE_DEF_RULE, RULES_DEF_RULE, AbstractState, etc.) with the official approach: - FW_RBT_DEFINE_RULE macro from TestUtils/RuleBasedTesting.hpp - Shadow state as Tester member (not separate AbstractState/TestState) - fprime-util new --rule-based-test scaffolding - Link to docs/how-to/rule-based-testing.md Based on the official guide at nasa/fprime/docs/how-to/rule-based-testing.md and the Svc/Ccsds/ApidManager exemplar. Co-Authored-By: michael.d.starch <michael.d.starch@jpl.nasa.gov>
Replace detailed scaffold/file-layout/shadow-state/rules/scenarios code examples with a brief orientation (decision table, core constructs, build note) and a pointer to docs/how-to/rule-based-testing.md. Skills should not duplicate canonical documentation — they provide the 'when/why' and defer the 'how' to the docs. Co-Authored-By: michael.d.starch <michael.d.starch@jpl.nasa.gov>
New §4 instructs skill authors to search for pre-existing docs and link to them rather than duplicating procedural content. Skills should provide when/why + brief orientation + link; the canonical doc owns the full how-to. Duplicated content drifts and risks hallucination. Co-Authored-By: michael.d.starch <michael.d.starch@jpl.nasa.gov>
Expand docs/user-manual/overview/unit-testing.md with sections that were missing: scaffold generation (fprime-util impl --ut), Tester class structure, TestMain with COMMENT/REQUIREMENT macros, helper function patterns, CMakeLists.txt registration, clearHistory/ doDispatch patterns, port invocation, and a link to the rules-based testing guide. Fix escaped underscores in code blocks and add cpp language specifiers. Trim the skill from 242 to 108 lines — it now provides workflow order, key patterns to enforce, the rules-based decision table, and the quality checklist, deferring all implementation detail to the guide. Co-Authored-By: michael.d.starch <michael.d.starch@jpl.nasa.gov>
Updated the documentation to reflect changes in file naming conventions and clarify the generation of test framework classes.
2 participants
Change Description
Adds
.github/agents/_shared/skills/fprime-unit-testing.skill.md— a skill providing step-by-step instructions for writing F Prime component unit tests.Sections: prerequisites, scaffold generation (
fprime-util impl --ut), Tester / GTestBase, TestMain, helper functions, rules-based testing with STest, CMakeLists registration, build/run, quality checklist.Also updates
.github/copilot-instructions.mdto reference the skill so other AI agents discover it when writing unit tests.Rationale
Provides structured, high-quality guidance for F Prime unit testing. Promotes consistent use of helper functions, rules-based testing for stateful components, and comprehensive coverage.
Testing/Review Recommendations
Svc/DpManager/test/ut/,Svc/ComSplitter/test/ut/).copilot-instructions.mdaddition is appropriate for cross-agent discovery.Future Work
AI Usage (see policy)
AI (Devin) authored this skill by analyzing existing F Prime unit test exemplars (
Svc/DpManager,Svc/ComSplitter,Svc/BufferManager) and thefprime-util impl --utimplementation infprime-tools.IAMAI
Link to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/75aec353513246c3bcea5888ec653f35