This is a Python 3 based pipeline that scrapes daily Treasury bill rates and provides interactive visualization through a web application.
Note: The web application runs entirely in your browser using Shinylive. Initial loading may take a minute on first access.
- Yield curve
- Yield spread
- Break-even implied forward yield
This project automates the collection, processing, and analysis of U.S. Treasury bill data from treasury.gov. It provides break-even yield calculations and interactive visualizations through a Shinylive for Python web application. The project architecture can be visualized as follows:
Diagram created with Excalidraw
- 🔄 Data Ingestion: Automated scraping using
requestsandpolars - 🗄️ Storage: AWS S3 Table Buckets with the Apache Iceberg open table format
- 📊 Analytics: AWS Athena for SQL-based data processing
- ⚙️ Orchestration: GitHub Actions for CI/CD and scheduled data updates
- 🏗️ Infrastructure: Terraform for AWS resource management
- 🌐 Web App: Shiny for Python with interactive visualizations
- 🚀 Deployment: GitHub Pages for static web hosting
This project uses uv to manage its dependencies (non-package mode). The required python version in pyproject.toml is >=3.12. For local development, ensure that a Python 3.12.x interpreter is searchable.
There are at least three simple ways to set up this project.
uv sync --frozen --all-groups --managed-pythonSee documentions for frozen, managed-python, and all-groups.
conda search python | grep " 3\.\(12\)\."
conda create --name tbills_scraper -y python=3.12
conda activate tbills_scraper
uv sync --frozen --all-groups --no-managed-python# List available Python versions
pyenv install --list | grep " 3\.\(12\)\."
# As an example, install Python 3.12.8
pyenv install 3.12.8
pyenv local 3.12.8
uv sync --frozen --all-groups --no-managed-python