Skip to content

Add Data Storage in Cloud Native AI whitepaper#2124

Open
xing-yang wants to merge 9 commits into
cncf:mainfrom
xing-yang:ai_storage
Open

Add Data Storage in Cloud Native AI whitepaper#2124
xing-yang wants to merge 9 commits into
cncf:mainfrom
xing-yang:ai_storage

Conversation

@xing-yang

Copy link
Copy Markdown
Contributor

No description provided.

@xing-yang xing-yang requested a review from a team as a code owner April 19, 2026 21:03
@github-actions github-actions Bot added needs-triage Indicates an issue or PR that has not been triaged yet (has a 'triage/foo' label applied) needs-kind Indicates an issue or PR that is missing an issue type or kind (a kind/foo label) labels Apr 21, 2026
@github-actions github-actions Bot added the needs-group Indicates an issue or PR that has not been assigned a group (toc or tag/foo label applied) label Apr 21, 2026
@xing-yang xing-yang changed the title WIP: Add Data Storage in Cloud Native AI whitepaper Add Data Storage in Cloud Native AI whitepaper Apr 26, 2026
@github-actions github-actions Bot requested review from GenPage and kashifest April 30, 2026 14:35
@xing-yang xing-yang force-pushed the ai_storage branch 2 times, most recently from 866338c to 80c854a Compare June 3, 2026 02:32
@GenPage GenPage added kind/initiative An initiative or an item related to imitative processes tag/infrastructure TAG Infrastructure and removed needs-group Indicates an issue or PR that has not been assigned a group (toc or tag/foo label applied) needs-kind Indicates an issue or PR that is missing an issue type or kind (a kind/foo label) labels Jun 3, 2026
@github-project-automation github-project-automation Bot moved this to status/new in Initiatives Jun 3, 2026
@GenPage GenPage moved this from status/new to status/in-progress in Initiatives Jun 3, 2026
@GenPage GenPage moved this to In Progress in TAG Infrastructure Jun 3, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new “Data on Kubernetes - Data Analytics and AI/ML Workloads” document set under the Infrastructure tag, introducing topic pages that describe storage patterns and related infrastructure considerations across the AI lifecycle.

Changes:

  • Added a top-level overview page that frames the whitepaper section and links to topic pages.
  • Added topic pages covering storage patterns for training, inference, and AI agents.
  • Added topic pages describing related data infrastructure areas (pipelines, databases, caching/locality, storage types, network management, data mesh/fabric, and HW/SW co-design).

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 17 comments.

Show a summary per file
File Description
tags/tag-infrastructure/data-storage-in-cloud-native-ai/overview.md Introduces the section overview and links out to topic pages.
tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/training.md Describes training storage usage patterns and references.
tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/inference.md Describes inference storage usage patterns and references.
tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/ai-agent.md Describes AI agent workflow storage patterns.
tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/block-file-object.md Summarizes block/file/object storage and FUSE CSI drivers.
tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/data-cache-locality.md Discusses caching and data locality strategies for AI/data workloads.
tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/data-lake-houses.md Summarizes warehouses, lakes, and lakehouses.
tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/data-mesh-and-fabric.md Introduces data mesh and data fabric concepts and references.
tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/data-pipelines.md Introduces modern data pipeline patterns and example infrastructure.
tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/databases.md Summarizes analytic, federated, and vector database categories.
tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/network-management.md Highlights networking considerations for storage and caches in Kubernetes.
tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/hardware-software-co-design.md Discusses HW/SW co-design trends and a reference link.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tags/tag-infrastructure/data-storage-in-cloud-native-ai/overview.md
Comment thread tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/training.md Outdated
Comment thread tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/training.md Outdated
Comment thread tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/data-pipelines.md Outdated
Comment thread tags/tag-infrastructure/data-storage-in-cloud-native-ai/overview.md Outdated
Comment thread tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/ai-agent.md Outdated

@angellk angellk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @xing-yang -- Overall, looks fantastic. Please update the missing reference links and acronym first use spell out for DAG.

Comment thread tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/databases.md Outdated
Comment thread tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/ai-agent.md Outdated
Comment thread tags/tag-infrastructure/data-storage-in-cloud-native-ai/overview.md
@xing-yang xing-yang force-pushed the ai_storage branch 3 times, most recently from 6a9599d to 3d66390 Compare June 9, 2026 01:29
@xing-yang

Copy link
Copy Markdown
Contributor Author

Hi @angellk, thanks for reviewing! I've addressed your comments.

Comment thread tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/data-pipelines.md Outdated
Comment thread tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/data-pipelines.md Outdated
Comment thread tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/training.md Outdated
xing-yang and others added 7 commits June 8, 2026 22:48
Signed-off-by: xing-yang <xingyang105@gmail.com>
Added comprehensive reference links to both inference.md and training.md files in the AI storage documentation. The references include relevant tools, frameworks, and platforms for model inference and training workflows in cloud-native environments.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Signed-off-by: alexagriffith <agriffith96@gmail.com>
Signed-off-by: xing-yang <xingyang105@gmail.com>
Added comprehensive reference links to both inference.md and training.md files in the AI storage documentation. The references include relevant tools, frameworks, and platforms for model inference and training workflows in cloud-native environments.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Signed-off-by: alexagriffith <agriffith96@gmail.com>
Signed-off-by: xing-yang <xingyang105@gmail.com>
Signed-off-by: xing-yang <xingyang105@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Karena Angell <kangell@redhat.com>
Signed-off-by: xing-yang <xingyang105@gmail.com>
Signed-off-by: xing-yang <xingyang105@gmail.com>
Co-authored-by: Karena Angell <kangell@redhat.com>
Signed-off-by: Xing Yang <xingyang105@gmail.com>
Signed-off-by: xing-yang <xingyang105@gmail.com>
Signed-off-by: xing-yang <xingyang105@gmail.com>

@angellk angellk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor additional nits that I missed. looks great @xing-yang !

Comment thread tags/tag-infrastructure/data-storage-in-cloud-native-ai/topics/databases.md Outdated
Co-authored-by: Karena Angell <kangell@redhat.com>
Signed-off-by: Xing Yang <xingyang105@gmail.com>
@xing-yang

Copy link
Copy Markdown
Contributor Author

Thanks @angellk!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/initiative An initiative or an item related to imitative processes needs-triage Indicates an issue or PR that has not been triaged yet (has a 'triage/foo' label applied) tag/infrastructure TAG Infrastructure

Projects

Status: New
Status: status/in-progress
Status: No status
Status: In Progress
Status: No status

Development

Successfully merging this pull request may close these issues.

6 participants