Skip to content
Draft
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
445df74
Ensure that python scenarios pass with current models
LarryOsterman Jun 26, 2026
a71fe6c
Updated python skills; added skills links
LarryOsterman Jun 29, 2026
b103ad9
Added initial evaluations for all python skills
LarryOsterman Jun 30, 2026
c38e89a
Updated vally evaluations and verified functionality
LarryOsterman Jun 30, 2026
4441d58
Added vally experiments for all the python skills
LarryOsterman Jul 1, 2026
234315e
Updated skill generator to improve compactness of skills
LarryOsterman Jul 1, 2026
2c5195e
Update rust evaluations to give tools more time to run; significant i…
LarryOsterman Jul 7, 2026
95a59b4
Updated node version to current
LarryOsterman Jul 7, 2026
34655a4
Added rust toolchain install for rust evals
LarryOsterman Jul 7, 2026
adc50d0
CI pipeline and build fixes
LarryOsterman Jul 7, 2026
b053b67
Neuter vally evaluations for now
LarryOsterman Jul 7, 2026
2800214
CI pipeline possible fix
LarryOsterman Jul 9, 2026
d972fc7
CI pipeline possible fix
LarryOsterman Jul 9, 2026
cb7adf9
Just try out copilot cli before doing expensive stuff
LarryOsterman Jul 9, 2026
c3c0bd6
Install copilot
LarryOsterman Jul 9, 2026
6f35944
And set the environment variable
LarryOsterman Jul 9, 2026
ff4da64
Updated evaluations
LarryOsterman Jul 10, 2026
5bdc963
Restored older ealuations
LarryOsterman Jul 10, 2026
3335acd
Moved back to agentic workflow; update to vally 0.7.0
LarryOsterman Jul 10, 2026
9fc75a9
Moved back to agentic workflow; update to vally 0.7.0
LarryOsterman Jul 10, 2026
8e39594
enable copilot-requests permission
LarryOsterman Jul 10, 2026
1883683
enable copilot-requests permission again
LarryOsterman Jul 10, 2026
7286a99
Fixed lockfile
LarryOsterman Jul 10, 2026
0cf1bb0
Added lockfile explicitly
LarryOsterman Jul 10, 2026
3ca00fb
CI fixes
LarryOsterman Jul 10, 2026
4772426
For ce update to prompt
LarryOsterman Jul 10, 2026
8c45247
Try defining a token for vally to use
LarryOsterman Jul 10, 2026
a312f96
Added back evaluation workflow separate from agentic workflow
LarryOsterman Jul 10, 2026
dc4a46b
Try pull_request_target
LarryOsterman Jul 10, 2026
d79c98f
Oops, wrong evaluation
LarryOsterman Jul 10, 2026
0397ebe
Renamed workflow
LarryOsterman Jul 10, 2026
5bf1c9f
Renamed workflow 2
LarryOsterman Jul 10, 2026
dc7d0a4
Removed agentic workflows for evaluation
LarryOsterman Jul 10, 2026
5e60981
Updated pnpm dependencies
LarryOsterman Jul 10, 2026
fb0bd8a
Potential fix for pull request finding 'Unused variable, import, func…
LarryOsterman Jul 10, 2026
2666345
Enabled copilot in CI and updated vally to 0.7.0
LarryOsterman Jul 9, 2026
fb96170
Updated vally version
LarryOsterman Jul 10, 2026
eaf5cf2
Merge branch 'larryo/python_skills' of https://github.com/microsoft/s…
LarryOsterman Jul 10, 2026
0d3ec2d
Use different model for graders and evals
LarryOsterman Jul 11, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
5 changes: 5 additions & 0 deletions .github/aw/actions-lock.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@
"version": "v8",
"sha": "ed597411d8f924073f98dfc5c65a23a2325f34cd"
},
"github/gh-aw-actions/setup@v0.81.6": {
"repo": "github/gh-aw-actions/setup",
"version": "v0.81.6",
"sha": "ba6380cc6e5be5d21677bebe04d52fb48e3abec7"
},
"githubnext/gh-aw/actions/setup@v0.36.0": {
"repo": "githubnext/gh-aw/actions/setup",
"version": "v0.36.0",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -347,7 +347,7 @@ if __name__ == "__main__":

## Best Practices

1. **This SDK is async-first** — use `async def` handlers and `async with` throughout.
1. **This SDK is async-first — use `async def` handlers and `async with` throughout.**
2. **Always use context managers for clients and async credentials.** Wrap every client in `with Client(...) as client:` (sync) or `async with Client(...) as client:` (async). For async `DefaultAzureCredential` from `azure.identity.aio`, also use `async with credential:` so tokens and transports are cleaned up.

## Reference Files
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -240,3 +240,10 @@ request = AnalyzeTextOptions(
7. **Log analysis results** for audit and improvement
8. **Consider 8-severity mode** for finer-grained control
9. **Pre-moderate AI outputs** before showing to users

## Reference Files

| File | Contents |
|------|----------|
| [references/capabilities.md](references/capabilities.md) | Additional non-hero capabilities, operation-group coverage, and production checklists. |
| [references/non-hero-scenarios.md](references/non-hero-scenarios.md) | Dedicated non-hero examples for secondary/advanced scenarios. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# azure-ai-contentsafety-py capability coverage

**SDK/package**: `azure-ai-contentsafety`

This index maps hero scenarios in `SKILL.md` and links non-hero scenarios documented in dedicated reference files.

## Hero scenarios covered in SKILL.md

- `Analyze Text`
- `Analyze Image`
- `Text Blocklist Management`
- `Severity Levels`

## Non-hero scenarios

- `Harm Categories`: | Category | Description |
See: [`non-hero-scenarios.md#harm-categories`](non-hero-scenarios.md#harm-categories)
- `Severity Scale`: | Level | Text Range | Image Range | Meaning |
See: [`non-hero-scenarios.md#severity-scale`](non-hero-scenarios.md#severity-scale)
- `Client Types`: | Client | Purpose |
See: [`non-hero-scenarios.md#client-types`](non-hero-scenarios.md#client-types)

## Related deep-dive references

- [`non-hero-scenarios.md`](non-hero-scenarios.md): Dedicated non-hero examples and implementation notes.

## API breadth checklist

- Verify client/auth mode for the environment before coding.
- Confirm operation-group/method names against current Microsoft Learn API reference.
- For Python SDKs with both sync and async clients, document both forms without a blanket preference.
- Include cleanup/delete paths for created resources in examples.
- Prefer idempotent create/update operations where available.
- Validate paging/LRO/error-handling patterns for production paths.
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# azure-ai-contentsafety-py non-hero scenarios

These scenarios are intentionally separate from hero flows in `SKILL.md`.
They cover secondary/advanced patterns typically used after the primary end-to-end path is working.

## Harm Categories

| Category | Description |
|----------|-------------|
| `Hate` | Attacks based on identity (race, religion, gender, etc.) |
| `Sexual` | Sexual content, relationships, anatomy |
| `Violence` | Physical harm, weapons, injury |
| `SelfHarm` | Self-injury, suicide, eating disorders |

## Severity Scale

| Level | Text Range | Image Range | Meaning |
|-------|------------|-------------|---------|
| 0 | Safe | Safe | No harmful content |
| 2 | Low | Low | Mild references |
| 4 | Medium | Medium | Moderate content |
| 6 | High | High | Severe content |

## Client Types

| Client | Purpose |
|--------|---------|
| `ContentSafetyClient` | Analyze text and images |
| `BlocklistClient` | Manage custom blocklists |
Original file line number Diff line number Diff line change
Expand Up @@ -292,3 +292,10 @@ from azure.ai.contentunderstanding.models import (
7. **Use async client** for high-throughput scenarios with `azure.identity.aio` credentials
8. **Handle long-running operations** — video/audio analysis can take minutes
9. **Use URL sources** when possible to avoid upload overhead

## Reference Files

| File | Contents |
|------|----------|
| [references/capabilities.md](references/capabilities.md) | Additional non-hero capabilities, operation-group coverage, and production checklists. |
| [references/non-hero-scenarios.md](references/non-hero-scenarios.md) | Dedicated non-hero examples for secondary/advanced scenarios. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# azure-ai-contentunderstanding-py capability coverage

**SDK/package**: `azure-ai-contentunderstanding`

This index maps hero scenarios in `SKILL.md` and links non-hero scenarios documented in dedicated reference files.

## Hero scenarios covered in SKILL.md

- `Core Workflow`
- `Prebuilt Analyzers`
- `Analyze Document`
- `Access Document Content Details`

## Non-hero scenarios

- `Analyze Image`: Dedicated example and implementation notes.
See: [`non-hero-scenarios.md#analyze-image`](non-hero-scenarios.md#analyze-image)
- `Analyze Video`: Dedicated example and implementation notes.
See: [`non-hero-scenarios.md#analyze-video`](non-hero-scenarios.md#analyze-video)
- `Analyze Audio`: Dedicated example and implementation notes.
See: [`non-hero-scenarios.md#analyze-audio`](non-hero-scenarios.md#analyze-audio)
- `Custom Analyzers`: Create custom analyzers with field schemas for specialized extraction:
See: [`non-hero-scenarios.md#custom-analyzers`](non-hero-scenarios.md#custom-analyzers)
- `Analyzer Management`: Dedicated example and implementation notes.
See: [`non-hero-scenarios.md#analyzer-management`](non-hero-scenarios.md#analyzer-management)
- `Async Client`: Dedicated example and implementation notes.
See: [`non-hero-scenarios.md#async-client`](non-hero-scenarios.md#async-client)
- `Content Types`: | Class | For | Provides |
See: [`non-hero-scenarios.md#content-types`](non-hero-scenarios.md#content-types)
- `Model Imports`: Dedicated example and implementation notes.
See: [`non-hero-scenarios.md#model-imports`](non-hero-scenarios.md#model-imports)
- `Client Types`: | Client | Purpose |
See: [`non-hero-scenarios.md#client-types`](non-hero-scenarios.md#client-types)

## Related deep-dive references

- [`non-hero-scenarios.md`](non-hero-scenarios.md): Dedicated non-hero examples and implementation notes.

## API breadth checklist

- Verify client/auth mode for the environment before coding.
- Confirm operation-group/method names against current Microsoft Learn API reference.
- For Python SDKs with both sync and async clients, document both forms without a blanket preference.
- Include cleanup/delete paths for created resources in examples.
- Prefer idempotent create/update operations where available.
- Validate paging/LRO/error-handling patterns for production paths.
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
# azure-ai-contentunderstanding-py non-hero scenarios

These scenarios are intentionally separate from hero flows in `SKILL.md`.
They cover secondary/advanced patterns typically used after the primary end-to-end path is working.

## Analyze Image

```python
from azure.ai.contentunderstanding.models import AnalyzeInput

poller = client.begin_analyze(
analyzer_id="prebuilt-imageSearch",
inputs=[AnalyzeInput(url="https://example.com/image.jpg")]
)
result = poller.result()
content = result.contents[0]
print(content.markdown)
```

## Analyze Video

```python
from azure.ai.contentunderstanding.models import AnalyzeInput

poller = client.begin_analyze(
analyzer_id="prebuilt-videoSearch",
inputs=[AnalyzeInput(url="https://example.com/video.mp4")]
)

result = poller.result()

# Access video content (AudioVisualContent)
content = result.contents[0]

# Get transcript phrases with timing
for phrase in content.transcript_phrases:
print(f"[{phrase.start_time} - {phrase.end_time}]: {phrase.text}")

# Get key frames (for video)
for frame in content.key_frames:
print(f"Frame at {frame.time}: {frame.description}")
```

## Analyze Audio

```python
from azure.ai.contentunderstanding.models import AnalyzeInput

poller = client.begin_analyze(
analyzer_id="prebuilt-audioSearch",
inputs=[AnalyzeInput(url="https://example.com/audio.mp3")]
)

result = poller.result()

# Access audio transcript
content = result.contents[0]
for phrase in content.transcript_phrases:
print(f"[{phrase.start_time}] {phrase.text}")
```

## Custom Analyzers

Create custom analyzers with field schemas for specialized extraction:

```python
# Create custom analyzer
analyzer = client.create_analyzer(
analyzer_id="my-invoice-analyzer",
analyzer={
"description": "Custom invoice analyzer",
"base_analyzer_id": "prebuilt-documentSearch",
"field_schema": {
"fields": {
"vendor_name": {"type": "string"},
"invoice_total": {"type": "number"},
"line_items": {
"type": "array",
"items": {
"type": "object",
"properties": {
"description": {"type": "string"},
"amount": {"type": "number"}
}
}
}
}
}
}
)

# Use custom analyzer
from azure.ai.contentunderstanding.models import AnalyzeInput

poller = client.begin_analyze(
analyzer_id="my-invoice-analyzer",
inputs=[AnalyzeInput(url="https://example.com/invoice.pdf")]
)

result = poller.result()

# Access extracted fields
print(result.fields["vendor_name"])
print(result.fields["invoice_total"])
```

## Analyzer Management

```python
# List all analyzers
analyzers = client.list_analyzers()
for analyzer in analyzers:
print(f"{analyzer.analyzer_id}: {analyzer.description}")

# Get specific analyzer
analyzer = client.get_analyzer("prebuilt-documentSearch")

# Delete custom analyzer
client.delete_analyzer("my-custom-analyzer")
```

## Async Client

```python
import asyncio
import os
from azure.ai.contentunderstanding.aio import ContentUnderstandingClient
from azure.ai.contentunderstanding.models import AnalyzeInput
from azure.identity.aio import DefaultAzureCredential

async def analyze_document():
endpoint = os.environ["CONTENTUNDERSTANDING_ENDPOINT"]
async with DefaultAzureCredential() as credential:
async with ContentUnderstandingClient(
endpoint=endpoint,
credential=credential
) as client:
poller = await client.begin_analyze(
analyzer_id="prebuilt-documentSearch",
inputs=[AnalyzeInput(url="https://example.com/doc.pdf")]
)
result = await poller.result()
content = result.contents[0]
return content.markdown

asyncio.run(analyze_document())
```

## Content Types

| Class | For | Provides |
|-------|-----|----------|
| `DocumentContent` | PDF, images, Office docs | Pages, tables, figures, paragraphs |
| `AudioVisualContent` | Audio, video files | Transcript phrases, timing, key frames |

Both derive from `MediaContent` which provides basic info and markdown representation.

## Model Imports

```python
from azure.ai.contentunderstanding.models import (
AnalyzeInput,
AnalyzeResult,
MediaContentKind,
DocumentContent,
AudioVisualContent,
)
```

## Client Types

| Client | Purpose |
|--------|---------|
| `ContentUnderstandingClient` | Sync client for all operations |
| `ContentUnderstandingClient` (aio) | Async client for all operations |
Original file line number Diff line number Diff line change
Expand Up @@ -96,4 +96,11 @@ with ConversationAnalysisClient(endpoint, credential) as client:
}
)

print(f"Top intent: {result['result']['prediction']['topIntent']}")
print(f"Top intent: {result['result']['prediction']['topIntent']}")

## Reference Files

| File | Contents |
|------|----------|
| [references/capabilities.md](references/capabilities.md) | Additional non-hero capabilities, operation-group coverage, and production checklists. |
| [references/non-hero-scenarios.md](references/non-hero-scenarios.md) | Dedicated non-hero examples for secondary/advanced scenarios. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# azure-ai-language-conversations-py capability coverage

**SDK/package**: `azure-ai-language-conversations`

This index maps hero scenarios in `SKILL.md` and links non-hero scenarios documented in dedicated reference files.

## Hero scenarios covered in SKILL.md

- `Core workflow`

## Non-hero scenarios

- `Operational hardening`: Use this section for retries, timeouts, pagination, and cleanup patterns specific to this SDK.
See: [`non-hero-scenarios.md#operational-hardening`](non-hero-scenarios.md#operational-hardening)

## Related deep-dive references

- [`non-hero-scenarios.md`](non-hero-scenarios.md): Dedicated non-hero examples and implementation notes.

## API breadth checklist

- Verify client/auth mode for the environment before coding.
- Confirm operation-group/method names against current Microsoft Learn API reference.
- For Python SDKs with both sync and async clients, document both forms without a blanket preference.
- Include cleanup/delete paths for created resources in examples.
- Prefer idempotent create/update operations where available.
- Validate paging/LRO/error-handling patterns for production paths.
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# azure-ai-language-conversations-py non-hero scenarios

These scenarios are intentionally separate from hero flows in `SKILL.md`.
They cover secondary/advanced patterns typically used after the primary end-to-end path is working.

## Operational hardening

Use this section for retries, timeouts, pagination, and cleanup patterns specific to this SDK.
Original file line number Diff line number Diff line change
Expand Up @@ -299,3 +299,10 @@ print(f"Default: {default_ds.name}")
7. **Register models** after successful training jobs
8. **Use pipelines** for multi-step workflows
9. **Tag resources** for organization and cost tracking

## Reference Files

| File | Contents |
|------|----------|
| [references/capabilities.md](references/capabilities.md) | Additional non-hero capabilities, operation-group coverage, and production checklists. |
| [references/non-hero-scenarios.md](references/non-hero-scenarios.md) | Dedicated non-hero examples for secondary/advanced scenarios. |
Loading
Loading