Research
My research focuses on machine learning and visual understanding, with an emphasis on models that can learn from multimodal data, adapt to changing environments, and understand dynamic scenes. We develop methods, benchmarks, and systems spanning language models, video understanding, 3D perception, and real-world applications.
Below are selected research directions and representative papers. For the full publication list, see Publications or Google Scholar.
Language Models and Multimodal Intelligence
We study how language and multimodal models learn, adapt, and reason across text, images, and video.
- Language model learning and reasoning. Developing methods for improving language models through on-policy self-distillation.
- Adaptive and explainable video understanding. Adapting vision-language models to video anomaly detection, with an emphasis on generalization, uncertainty, and interpretable predictions.
- Multimodal evaluation and alignment. Building tools for evaluating unified multimodal models and reducing the gap between visual and language representations.
- Time series and recommendation. Learning adaptive representations and parameter generation methods for forecasting and sequential recommendation.
3D Perception and Object-Centric Learning
We develop representations of scene geometry, motion, and objects to understand dynamic environments.
- Spatiotemporal point cloud processing. Modeling scene dynamics, scene flow, shape correspondence, and efficient learning from 3D point clouds.
- Object-centric representations and world models. Learning structured scene representations that separate objects and support scene understanding and generation.
Trustworthy AI and Real-World Visual Understanding
We investigate the reliability of learning systems and apply visual intelligence to transportation, text, and everyday scenes.
- Security and robustness. Understanding adversarial vulnerabilities in language models, multimodal perception, and scene text recognition.
- Intelligent transportation. Developing benchmarks, perception systems, and learning-based traffic signal control for real-world traffic analysis.
- Scene text and scene understanding. Detecting and recognizing text in natural images, and understanding activities in egocentric video.
Code, Datasets, and Demos
- COPRA: adaptive vision-language models for video anomaly detection.
- TorchUMM: a unified multimodal model codebase for evaluation, analysis, and post-training.
- Ego-Deliver: an egocentric video dataset and benchmark.
- EfficientMORL: object-centric learning. Video demo.
- SparsePipe: distributed training for 3D point clouds.
- SSTD and CTPN: scene text detection. SSTD video demo.
- Real-time video processing: a traffic monitoring demonstration for pedestrian safety.
Funding and Grants
- PI. National Science Foundation (NSF), “Toward Human-Centered, Generalizable, and Verifiable Automated Video Surveillance: A Verbalized Vision-Language Model Paradigm,” awarded 2026. Announcement
- Argonne Leadership Computing Facility (ALCF), Director’s Discretionary (DD) allocation award, 2025.
- National Artificial Intelligence Research Resource (NAIRR) Pilot allocation, NAIRR260121, 2026.
- National Artificial Intelligence Research Resource (NAIRR) Start-Up allocation, NAIRR250417, 2025.
- Lambda Research Grant Program, cloud computing credits, 2025.
- Oak Ridge Associated Universities (ORAU), Ralph E. Powe Junior Faculty Enhancement Award, 2025. Announcement
