This repository contains the full pipeline and experimental framework developed for the research project presented at XLVIII Reunião Anual da Sociedade Astronômica Brasileira (RASAB). You can check out the presentation slides in this Drive.
The project integrates Topological Data Analysis (TDA) and Self-Supervised Learning (SSL) to investigate the unsupervised clustering of galaxies from the Galaxy Zoo 2 (GZ2) dataset. The objective is to evaluate whether topological features (via persistence homology) combined with learned SSL embeddings can reveal meaningful astrophysical structures, enabling interpretable morphological groupings without supervision.
- Self-Supervised Learning (SSL): representation learning without labeled data using contrastive paradigms (SimCLR-based).
- Persistence Diagrams (PDs) and Persistence Images (PIs): topological summaries capturing structural information of galaxy morphology.
- Unsupervised Clustering: methods (K-means, DBSCAN) applied to SSL and TDA features to identify morphologically coherent galaxy groups.
- Evaluation Metrics: Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and Purity against GZ2 majority labels.
The repository follows a clean modular organization, reflecting different stages of the pipeline:
TDA-SSL_Galaxy_Morphology-XLVIII_RASAB/
│
├── configs/ # Configuration files for experiments
│ ├── examples/ # Example configuration templates
│ ├── profiles/ # Experiment and run profiles
│ ├── sweeps/ # Hyperparameter sweep configurations
│ ├── config/ # Central configuration YAMLs or JSONs
│ ├── schema/ # Config structure schemas
│ └── RASAB_TDA_SSL-configs.pdf
│
├── data/ # Data directory (not versioned)
│ ├── images_gz2/ # GZ2 galaxy image set (local only)
│ ├── gz2sample.* # CSV/FITS samples and catalogs
│ ├── zoo2Main*.fits # Galaxy Zoo metadata tables
│ └── README # Notes on dataset access
│
├── notebooks/ # Jupyter notebooks for analysis and visualization
│ └── ...
│
├── results/ # Experiment outputs (not tracked by git)
│ ├── clustering/ # Clustering metrics and results
│ ├── robustness/ # Robustness test outputs
│ ├── ssl/ # SSL feature embeddings
│ ├── tda/ # Persistence diagrams & images
│ ├── figures/ # Final figures for papers & posters
│ └── k_sweep/ # K-range clustering experiments
│
├── src/ # Core source code (modular pipeline)
│ ├── clustering/ # K-means, DBSCAN, evaluation metrics
│ ├── data/ # Data loading and preprocessing utilities
│ ├── experiments/ # End-to-end experiment orchestration scripts
│ ├── ssl/ # SimCLR training and embedding extraction
│ ├── tda/ # Topological computations (PDs, PIs)
│ ├── tools/ # Auxiliary scripts for consistency & fixing
│ └── visualize/ # Visualization utilities (t-SNE, UMAP, PI plots)
│
└── README.md # You are here
Environment setup (Conda recommended):
conda create -n tda-ssl python=3.10
conda activate tda-ssl
pip install -r requirements.txtAlternatively, if using an environment file:
conda env create -f environment.yml
conda activate tda-sslInstallation setup:
git clone https://github.com/GabrielWendell/TDA-SSL_Galaxy_Morphology-XLVIII_RASABpython -m src/data/prepare_gz2.py \
--config config/config.yaml \
--profile config/profile/dev.yaml
python -m src/data/inspect_stats.py- Compute & Validade Cubical Persistence Images
$(H_{0},H_{1})$
# Validate pipeline on 100 - 200 images
python -m src/tda/compute_pis.py \
--config config/config.yaml \
--profile configs/profiles/dev.yaml \
--limit 200 --stratified --tag dev
python -m src/tda/validate_pis.py \
--features results/tda/pi/pi_dev_200.npy \
--manifest results/tda/pi/pi_dev_200_manifest.csv
# For the full dataset
python -m src/tda/run_full_pi.py \
--config configs/config.yaml \
--profile configs/profiles/dev.yaml \
--batch-size 1024 --tag full \- PCA Reduction & Variance Analysis
python -m src/tda/pca_reduce.py \
--features results/tda/pi/pi_full_13965.npy \
--manifest results/tda/pi/pi_full_13965_manifest.csv \
--outdir results/tda/pca \
--n-components 512 --batch-size 2048- Baseline Clustering with KMeans and DBSCAN
# KMeans (k = 5, as per 5 GZ2 classes)
python -m src/clustering/baseline_cluster.py \
--features results/tda/pi/pi_full_13965.npy \
--manifest results/tda/pi/pi_full_13965_manifest.csv \
--outdir results/clustering/baselines \
--k 5
# DBSCAN (try a small eps sweep)
python -m src/clustering/baseline_cluster.py \
--features results/tda/pi/pi_full_13965.npy \
--manifest results/tda/pi/pi_full_13965_manifest.csv \
--outdir results/clustering/baselines \
--dbscan-grid "0.8,1.0,1.2,1.5,2.0" \
--dbscan-min-samples 10- SimCLR Training on PCA Features
python -m src/ssl/simclr_train.py \
--features results/tda/pca/pi_full_13965_pca512.npy \
--manifest results/tda/pi/pi_full_13965_manifest.csv \
--outdir results/ssl/simclr \
--epochs 50 --batch-size 1024 --proj-dim 128 --temp 0.2- Parameter Tuning & 128-D Embeddings
# Projector-native
python -m src/ssl/export_embeddings.py \
--features results/tda/pca/pi_full_13965_pca512.npy \
--model results/ssl/simclr/simclr_model.pt \
--outdir results/ssl/emb128
# Force PCA→128 (if projector ≠ 128)
python -m src/ssl/export_embeddings.py \
--features results/tda/pca/pi_full_13965_pca512.npy \
--model results/ssl/simclr/simclr_model.pt \
--outdir results/ssl/emb128 \
--force-pca- SSL Embedding Clustering
python -m src/clustering/cluster_embeddings.py \
--embeddings results/ssl/emb128/embeddings_128.npy \
--manifest results/tda/pi/pi_full_13965_manifest.csv \
--outdir results/clustering/ssl \
--k-grid 3,4,5,6,7,8,9,10 \
--dbscan-grid 0.8,1.0,1.2,1.5,2.0 \
--dbscan-min-samples 10- t-SNE & UMAP Visualization of Emeddings
# Full set (can take a few minutes); set --sample for faster drafts
python -m src/visualize.tsne_umap_w_legends.py \
--embeddings results/ssl/emb128/embeddings_128.npy \
--manifest results/tda/pi/pi_full_13965_manifest.csv \
--outdir results/figs/tsne_umap_full \
--perplexity 35 --umap-ne 40 --umap-min-dist 0.05
# Quick 6k stratified sample + overlay KMeans(k = 10)
python -m src/visualize/tsne_umap_w_legends.py \
--embeddings results/ssl/emb128/embeddings_128.npy \
--manifest results/tda/pi/pi_full_13965_manifest.csv \
--outdir results/figs/tsne_umap_sample \
--sample 6000 --kmeans-k 10python -m src/experiments/extended_experiments_v2.py \
--manifest results/tda/pi/pi_full_13965_manifest.csv \
--images-root data/images_gz2/imgs \
--pi-pca-feats results/tda/pca/pi_full_13965_pca512.npy \
--ssl-pi-feats results/ssl/emb128/embeddings_128.npy \
--tda-diagrams results/tda/pd/pd_full_13965.npy \
--outdir results/extended_exp \
--pl-bins 128 --pl-layers 3 --pl-chunk 2000 --bc-bins 128 --njobs 4- Raw Image Embeddings
# If you have CUDA and want mixed precision: add --fp16.
# Models available: resnet18 (D = 512), resnet50 (D = 2048), efficientnet_b0 (D = 1280).
python raw_image_embeddings.py \
--images-root data/images_gz2/imgs\^
--manifest results/tda/pi/pi_full_13965_manifest.csv ^
--outdir results/ssl/raw ^
--model resnet50 \
--batch-size 128 \
--num-workers 0- SSL Control Experiment
# Repeat the all the procedures for ResNet18 and EfficientNet-B0
python src/tools/utils/fix_memmap_to_npy_patch.py \
--memmap results/ssl/raw/emb_resnet50_13965x2048.npy \
--runlog results/ssl/raw/emb_resnet50_run.json \
--out results/ssl/raw/emb_resnet50_13965x2048_fixed.npy
# Run the Control Experiment
python src/experiments/ssl_control_experiment.py \
--embeddings results/ssl/raw/emb_resnet50_13965x2048_fixed.npy \
--manifest results/ssl/raw/emb_resnet50_13965x2048_manifest.csv \
--labels-manifest results/tda/pi/pi_full_13965_manifest.csv \
--outdir results/ssl_control/resnet50 \
--k-grid 3,4,5,6,7,8,9,10 \
--dbscan-grid 0.8,1.0,1.2,1.5,2.0 \
--dbscan-min-samples 10- K-Sweep Evaluation
# Build a merged manifest (one-time per embedding set)
python prep_labels_for_k_sweep.py \
--emb-manifest results/ssl/raw/emb_resnet18_13965x512_manifest.csv \
--label-manifest results/tda/pi/pi_full_13965_manifest.csv \
--out results/ssl/raw/emb_resnet18_13965x512_manifest_with_labels.csv
# Run your K-sweep using the merged manifest
python src/clustering/eval_k_sweep.py \
--features results/ssl/raw/emb_resnet18_13965x512_fixed.npy \
--manifest results/ssl/raw/emb_resnet18_13965x512_manifest_with_labels.csv \
--outdir results/k_sweep/resnet18 \
--k-grid 3,4,5,6,7,8,9,10 \
--standardize --pca2d- Robustness Tests: Random Subsampling & Noise Injection
# Feature-level robustness (fast)
python src/experiments/robustness_run.py \
--features ./results/ssl/emb_resnet18_13965x512_fixed.npy \
--manifest ./results/ssl/raw/emb_resnet18_13965x512_manifest.csv \
--labels-manifest ./results/tda/pi/pi_full_13965_manifest.csv \
--id-col image_path --standardize \
--k-grid 3,4,5,6,7,8,9,10 \
--fractions 0.1,0.25,0.5,0.75,1.0 \
--sigmas 0.0,0.01,0.05,0.1 \
--outdir ./results/robustness/resnet18
python src/tools/plots/poster_panels.py --run-dir ./results/robustness/resnet18
# Image-noise + new embeddings (slower)
python src/experiments/image_noise_embeddings.py \
--manifest ./results/ssl/raw/emb_resnet18_13965x512_manifest.csv \
--image-root ./data/gz2 \
--backbone resnet18 --sigma 0.05 --jpeg-quality 50 \
--out-features ./results/ssl/raw/emb_resnet18_noisy.npy \
--out-manifest ./results/ssl/raw/emb_resnet18_noisy_manifest.csv
python src/experiments/robustness_run.py \
--features ./results/ssl/raw/emb_resnet18_noisy.npy \
--manifest ./results/ssl/raw/emb_resnet18_noisy_manifest.csv \
--labels-manifest ./results/tda/pi/pi_full_13965_manifest.csv \
--id-col image_path --standardize \
--k-grid 3,4,5,6,7,8,9,10 \
--outdir ./results/robustness/resnet18_noisy- PIs Montages
python plot_pi_gallery_auto.py \
--pi-fixed ./results/tda/pi/pi_full_13965_fixed.npy \
--manifest ./results/tda/pi/pi_full_13965_manifest.csv \
--group-col majority_morphology \
--per-class 20 --cols 10 \
--outdir ./figures/pi_montage- SSL Embeddings: generated using SimCLR with ResNet50, ResNet18, and EfficientNet backbones.
- TDA Features: extracted via persistent homology on grayscale intensity landscapes.
-
Clustering Evaluation: purity and NMI peaks at
$K=5$ , reflecting dominant morphological groups. - Visualization: 2D embeddings via t-SNE and UMAP highlight clear separation between spiral, smooth, and irregular galaxies.
This repository does not include raw Galaxy Zoo 2 data due to licensing and storage constraints. Users must obtain the dataset directly from:
Ensure images are stored in the following structure:
data/images_gz2/imgs/<galaxy_id>.jpg
- Python ≥ 3.9
- PyTorch ≥ 1.13
- scikit-learn ≥ 1.3
- numpy, pandas, matplotlib, seaborn
- gudhi or ripser (for persistence diagrams)
- umap-learn, tqdm, pillow
All final plots (e.g., persistence image montages, confusion-like cluster alignments, t-SNE/UMAP maps) are stored under:
results/figures/
Examples:
tsne_2d_labels.pngumap_2d_labels.pngconfusion_k5_ssl.pngpi_mean_by_class.pngpi_montage_by_class.png
Developed as part of the Topological Machine Learning in Astrophysics initiative (UFRN Physics Graduate Program). Presented at XLVIII RASAB (2025) in the Oral Communication category.
Special thanks to:
- The Galaxy Zoo collaboration for the public dataset.
- RASAB Organizing Committee (2025) for presentation support.
- UFRN PPGF for institutional support.
If you use this repository, please cite:
Wendell, G. (2025). TDA-SSL Galaxy Morphology — XLVIII RASAB 2025. GitHub: GabrielWendell/TDA-SSL_Galaxy_Morphology-XLVIII_RASAB
📦 GitHub Repository: GabrielWendell/TDA-SSL_Galaxy_Morphology-XLVIII_RASAB
Author: Gabriel Wendell Celestino Rocha
Institution: Universidade Federal do Rio Grande do Norte (UFRN) — Programa de Pós-Graduação em Física
Contact: gabrielwendell@fisica.ufrn.br