Ye Zhu

Researcher in CV/ML.


Postdoc Researcher

Ph.D. (she/her)

Princeton University

I am Ye Zhu (竺 烨 in Chinese), currently a postdoc research associate in the Computer Science Department at Princeton University, working with Prof. Olga Russakovsky at the VisualAI lab. My main research focuses on Multimodality Generation, with interdisciplinary research interests in Machine Learning for Physics.

Prior to the postdoc, I received my Ph.D. in Computer Science from Illinois Institute of Technology, USA; and my M.S. and B.S. from Shanghai Jiao Tong University (SJTU), China. I completed my partial education within the French engineering system, which included three years of intensive mathematics and physics training as an undergraduate. Subsequently, I pursued studies at École Polytechnique (X), France and obtained my French Engineering Diploma (Diplôme d’Ingénieur in French).

I serve as a regular reviewer for NeurIPS, ICLR, ICML, CVPR, ECCV, ICCV, and SIGGRAPH. I also hold French language diplomas in both DALF and TCF C1, and was a part-time French translator for the European scientific magazine Science & Vie for 8 years.


Feb 25, 2024 I am recently giving a themed talk on the Tuning-free Paradigm for Versatile Applications with Diffusion Models at academia and industrial labs.
Feb 15, 2024 I am co-organizing the first Responsible Generative AI (ReGenAI) workshop at CVPR 24!
Jan 15, 2024 Our Cyclic One-Way Diffusion for Text-Vision-Conditioned Generation (COW) paper accepted to ICLR 2024.
Sep 30, 2023 Our BoundaryDiffusion paper accepted to NeurIPS 2023.
Sep 2, 2023 Finish my Ph.D defense and join Princeton CS as a postdoc researcher.

selected publications

  1. boundarydiffusion.png
    Boundary Guided Learning-Free Semantic Control with Diffusion Models
    Ye Zhu, Yu Wu, Zhiwei Deng, Olga Russakovsky, and Yan Yan
    In Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), 2023
  2. cdcd.png
    Discrete Contrastive Diffusion for Cross-Modal Music and Image Generation
    Ye Zhu, Yu Wu, Kyle Olszewski, Jian Ren, Sergey Tulyakov, and Yan Yan
    In The Eleventh International Conference on Learning Representations (ICLR), 2023
  3. diffdensity.jpg
    Denoising Diffusion Probabilistic Models to Predict the Density of Molecular Clouds
    Duo Xu, Jonathan C Tan, Chia-Jung Hsu, and Ye Zhu
    The Astrophysical Journal, 2023
  4. Vision+ X: A Survey on Multimodal Learning in the Light of Data
    Ye Zhu, Yu Wu, Nicu Sebe, and Yan Yan
    arXiv preprint arXiv:2210.02884, 2022
  5. d2m.png
    Quantized GAN for Complex Music Generation from Dance Videos
    Ye Zhu, Kyle Olszewski, Yu Wu, Panos Achlioptas, Menglei Chai, Yan Yan, and Sergey Tulyakov
    In European Conference on Computer Vision (ECCV), 2022
  6. Saying the Unseen: Video Descriptions via Dialog Agents
    Ye Zhu, Yu Wu, Yi Yang, and Yan Yan
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022
  7. Describing unseen videos via multi-modal cooperative dialog agents
    Ye Zhu, Yu Wu, Yi Yang, and Yan Yan
    In European Conference on Computer Vision (ECCV), 2020