Skip to content

Research

Both papers in full, with authors, venues, findings, figures and the numbers behind them, followed by the research roles they came from.

Poster, ICML 2026 FoGen WorkshopMay 2026

Interpreting Latent CoT Reasoning as Dynamical Systems

ICML 2026 Workshop on Foundations of Deep Generative Models: Understanding Memorization, Generalization, and Reasoning

  • Sabari Iyyappan Duraipandian (equal contribution),
  • Shreya Sanjay Boyane (equal contribution),
  • Manju Nagesh (equal contribution),
  • Jerome Francis,
  • Archana Vaidheeswaran,
  • Kevin Zhu

Equal contribution

San Jose State University, Worcester Polytechnic Institute, George Mason University, Algoverse AI Research

Role
Co-first author and lead pipeline engineer (Algoverse AI Research Fellowship)
Status
Accepted, poster
Review
Reviewer rating 6/6, confidence 3/3

Latent chain-of-thought trajectories form structured dynamical systems: CODI behaves as a stable attractor, COCONUT as an unstable expanding system, and Sim-CoT supervision tightens both without changing the underlying dynamics.

Abstract

Recent latent reasoning methods such as CODI and COCONUT face a fundamental interpretability problem: they carry multiple superimposed candidate traces in hidden space at each step, which obscures how reasoning evolves. We model the sequence of latent tokens as a trajectory in representation space and apply dynamical-systems analysis to characterize reasoning evolution. Using quantitative metrics (step-to-step change, direction consistency, Lyapunov sensitivity) and qualitative projections (UMAP, DMD, PHATE), we show that latent CoT exhibits structured, non-random dynamics with two distinct stability classes.

Findings

  1. CODI behaves as a stable attractor: latent states converge into a bounded region, and shorter, simpler paths correlate with correct answers.

  2. COCONUT behaves as an unstable expanding system: latent states spread outward across steps, and more latent work correlates with correctness.

  3. Sim-CoT supervision tightens both behaviours without changing the underlying dynamics.

  4. COCONUT shows three-attractor compression with a mid-chain direction-consistency hinge at step 3, while CODI behaves as a near period-2 oscillator with 85% of samples carrying at least one unstable DMD mode (41% for COCONUT). Signatures hold across GSM8K and MATH.

Figures

CODI latent trajectories projected onto dominant DMD modes, forming tight clusters
CODI trajectories projected onto dominant DMD modes contract toward a bounded region.
Three-component PHATE embedding of COCONUT latent trajectories colored by step
COCONUT trajectories in a 3D PHATE embedding spread outward across steps.
Step-to-step displacement between consecutive latent states
Step-to-step change between consecutive latent states.
Relative divergence under Gaussian input noise at three sigma levels
Perturbation divergence under Gaussian noise at sigma 0.01, 0.1 and 1.0.

Submitted, DASHSys 2026May 2026

Toward Fairness-Aware Human Feedback for Table Unionability

DASHSys 2026, regular paper track

  • Nina Klimenkova (equal contribution),
  • Shreya Sanjay Boyane (equal contribution),
  • Roee Shraga

Equal contribution

Worcester Polytechnic Institute

Role
Co-first author, research assistant under Dr. Roee Shraga (BSF Grant 2024022)
Status
Submitted

First adaptation of the FairEM framework from entity matching to annotator-level fairness in table union search, showing that human feedback should be treated as conditionally reliable.

Abstract

Table union search is a foundational primitive in modern data lakes, and human feedback is increasingly used to supervise it. We adapt the FairEM framework (Shahbazi et al., VLDB 2023) from entity matching to annotator-level fairness and ask whether feedback quality varies systematically across demographic groups. On a 480-row dataset of 60 annotators across 8 questions and 4 survey versions, we find statistically significant disparities that survive multiple-testing correction and five orthogonal cleaning strategies.

Findings

  1. 6 of 10 FairEM metrics flag unfairness against non-native English annotators.

  2. 25% versus 44% accuracy on hard questions (Fisher exact p = 0.031).

  3. The non-native, lower-education, young triple intersection persists at p_BH = 0.037 after Benjamini-Hochberg FDR correction.

  4. Robust across 5 of 5 orthogonal data-cleaning strategies; behavioral disparities exceed demographic ones.

Research roles

  1. Latent chain-of-thought trajectories as dynamical systems

    Organization
    Algoverse AI Research Fellowship
    Role
    AI Research Fellow, lead pipeline engineer
    Advisor
    Jerome Francis
    Dates
    Feb 2026 to May 2026

    Selected for the competitive 12-week Algoverse fellowship; modeled latent chain-of-thought reasoning in LLMs as discrete-time dynamical systems and reproduced two latent-reasoning training methods end to end on GPT-2.

    • Applied Dynamic Mode Decomposition, PHATE, UMAP, t-SNE and PCA to 768-dimensional latent trajectories across 8,792 GSM8K and 500 MATH samples, producing 22 quantitative plots per run.
    • Reproduced CODI self-distillation (parallel teacher-student with a 3-loss objective) and the COCONUT 4-stage latent curriculum as LoRA adapters (rank 128, alpha 32) on stock GPT-2 with PyTorch, Hugging Face Transformers and PEFT; recovered 99.84% of test samples with a positional overflow guard.
    • Engineered the reliability layer: a multi-stage Bash and Python orchestrator with CUDA health polling that stabilized 14+ hour training runs, HDF5 resume-from-cache saving about 90 minutes per recovery, a GPU-resident Kraskov KSG-1 mutual-information estimator, and a safetensors-level checkpoint repair that preserved trained embeddings byte for byte.
    • Built a verification harness that passes 28 of 28 reproducible numeric claims on the canonical run.
    Project details

  2. FairPrep: demographic fairness for table union search

    Organization
    Worcester Polytechnic Institute
    Role
    Research Assistant
    Advisor
    Dr. Roee Shraga
    Grant
    BSF Grant 2024022
    Dates
    Jan 2025 to May 2026

    First adaptation of the FairEM framework to annotator-level fairness in table union search, with a complete statistical methodology stack.

    • Built 9 source modules across 5 experimental phases on a 480-row dataset (60 annotators x 8 questions x 4 survey versions).
    • Designed the statistical stack: 10 FairEM confusion-matrix metrics, permutation testing (10,000 shuffles), bootstrap confidence intervals (10,000 stratified resamples), Fisher exact tests, Benjamini-Hochberg FDR correction, and dual-track logistic regression (scikit-learn for interpretation, statsmodels for inference).
    • Surfaced statistically significant bias against non-native English annotators on hard questions (25% versus 44% accuracy, p = 0.031), robust across 5 of 5 cleaning strategies.
    Project details

  3. Where does physics live in vision encoders?

    Organization
    Independent research
    Role
    Lead researcher
    Dates
    Mar 2026 to May 2026

    Spatial probing of physical reasoning across the vision encoder, projection and LLM stages of three vision-language models, plus a factorial LoRA ablation to find the leverage point for physical intuition.

    • Trained linear and MLP probes on per-patch activations at four pipeline stages of Qwen2.5-VL-7B, InternVL 2.5-8B and LLaVA-OneVision-7B using Physion++ ground-truth physical properties.
    • Produced physics saliency maps and measured information loss across the projection bottleneck.
    • Ran a five-condition factorial LoRA ablation (encoder, projection, LLM, encoder plus projection, full model) evaluated on PhysBench, GRASP Level 2 and ConservationBench.
    Project details