Skip to content
Postdoctoral Fellow
·
Wellcome Sanger Institute
Junior Research Fellow
·
Wolfson College, Cambridge
AI for Biology
·
Genomics
·
Biomolecules
I develop scalable, data-driven AI models that learn from large-scale biological data to decode the language of life and uncover novel fundamental principles of biological systems.
About Me
I am a Postdoctoral Researcher at the Wellcome Sanger Institute in Cambridge, UK, working with
Dr. Mo Lotfollahi
on developing scalable, generalizable foundation models for large-scale biological data. Since January 2026, I have also been a Junior Research Fellow at Wolfson College, University of Cambridge.
My work spans sequence modeling (proteins, genomes) and structural modeling (3D molecular structures), with the goal of advancing integrative understanding across biological modalities.
During my Ph.D., I conducted research on foundation models for molecular modeling, which formed the core of my doctoral thesis titled Research on Molecular Modeling Based on Pre-trained Models , and this work was recognized as the Winner of the ACM Beijing Doctoral Dissertation Award .
Academic Background
Junior Research Fellow
(Jan 2026 – Present)
Postdoctoral Fellow , Wellcome Sanger Institute
(Sep 2025 – Present)
Ph.D. in Computer Science , Peking University
(Aug 2020 – Jun 2025)
B.Eng. in Computer Science , Harbin Institute of Technology
(Aug 2016 – Jun 2020)
College: The Honors School of HIT (Top 10 graduates )
Industry Experience
Research Intern , AIR in Tsinghua University
(Aug 2022 – Nov 2024)
Research Topic: AI for Protein and Drug Molecules Modeling
Research Intern , Tencent AI Lab
(Aug 2021 – Aug 2022)
Research Topic: Non-autoregressive Text Generation
Research Intern , Baidu Research
(Aug 2019 – May 2020)
Mentor:
Dr. Mingming Sun
Research Topic: Information Extraction
Research Highlights
Multi-scale foundation models for single-cell gene regulation and unified molecular modeling.
Selected Publications
* equal contribution · See a full list of papers on
Google Scholar
.
AI for Science
MUSS: A Multi-scale and Sequence-based Model for Single-cell Gene Regulation. NeurIPS 2026.
Kangjie Zheng * , Amirhossein Vahidi* , Liying Jin* , Joseph Clarke, Vijaya Baskar MS, Connor Rogerson, Mengji Zhang, Muzlifah Haniffa, Anshul Kundaje, Mohammad Lotfollahi.
CrossMol: cross-modal mask-predict pre-training for 3D molecular data. Bioinformatics · 2026.
Kangjie Zheng , Junwei Yang, Siyu Long, Wei Ju, Wei-Ying Ma, Hao Zhou, Ming Zhang.
ESM All-Atom: Multi-Scale Protein Language Model for Unified Molecular Modeling. ICML 2024.
Kangjie Zheng * , Siyu Long* , Tianyu Lu, Junwei Yang, Xinyu Dai, Ming Zhang, Zaiqing Nie, Wei-Ying Ma, Hao Zhou.
SMI-Editor: Edit-based SMILES Language Model with Fragment-level Supervision. ICLR 2025.
Kangjie Zheng , Siyue Liang, Junwei Yang, Bin Feng, Zequn Liu, Wei Ju, Zhiping Xiao, Ming Zhang.
Mol-AE: Auto-Encoder Based Molecular Representation Learning With 3D Cloze Test Objective. ICML 2024.
Junwei Yang* , Kangjie Zheng * , Siyu Long, Zaiqing Nie, Ming Zhang, Xinyu Dai, Wei-Ying Ma, Hao Zhou.
Language Modeling
ExLM: Rethinking the Impact of [MASK] Tokens in Masked Language Models. ICML 2025.
Kangjie Zheng , Junwei Yang, Siyue Liang, Bin Feng, Zequn Liu, Wei Ju, Zhiping Xiao, Ming Zhang.
Towards a Unified Training for Levenshtein Transformer. ICASSP 2023.
Kangjie Zheng , Longyue Wang, Zhihao Wang, Binqi Chen, Ming Zhang, Zhaopeng Tu.
A Decoding Algorithm for Length-Control Summarization Based on Directed Acyclic Transformers. EMNLP Findings 2024.
Chenyang Huang, Hao Zhou, Cameron Jen, Kangjie Zheng , Osmar Zaiane, Lili Mou.
Gloss Matters: Unlocking the Potential of Non-Autoregressive Sign Language Translation. ACM MM 2025.
Zhihao Wang, Shiyu Liu, Zhiwei He, Kangjie Zheng , Liangying Shao, Junfeng Yao, Jinsong Su.
Community Contributions
Academic Service
Program Committee / Reviewer
Conference on Neural Information Processing Systems (NeurIPS’25 ) — Reviewer
International Conference on Learning Representations (ICLR’24, ’25 ) — Reviewer
International Conference on Machine Learning (ICML’24, ’25 ) — Reviewer
AAAI Conference on Artificial Intelligence (AAAI’25 ) — Reviewer
ACL Rolling Review (ACL ARR ) — Reviewer
Abstract ×
MUSS: A Multi-scale and Sequence-based Model for Single-cell Gene Regulation
NeurIPS 2026
Kangjie Zheng * , Amirhossein Vahidi* , Liying Jin* , Joseph Clarke, Vijaya Baskar MS, Connor Rogerson, Mengji Zhang, Muzlifah Haniffa, Anshul Kundaje, Mohammad Lotfollahi.
* Equal contribution.
Modeling gene regulation in single cells requires jointly representing DNA sequence, chromatin accessibility, and gene expression across regulatory scales. Sequence-based models capture DNA-level regulatory grammar but often lack cell-level resolution, whereas many single-cell omics models represent genes and peaks as fixed-vocabulary tokens without their underlying sequence.
Here, we present MUSS, a sequence-based multi-scale model for single-cell gene regulation that preserves both DNA-level and cell-level resolution. MUSS derives peak- and gene-level representations from DNA using a frozen pre-trained sequence encoder and integrates them through peak-peak self-attention and distance-biased peak-gene interactions. Peak masked language modeling followed by training on paired scATAC-seq and scRNA-seq aligns these representations with cell-specific regulatory states.
Across gene expression prediction, enhancer-gene linking, motif-guided regulatory prioritization, and zero-shot cell clustering, MUSS achieves strong performance against sequence-based and single-cell omics baselines. It further predicts expression for genes and peaks unseen during task training. These results support sequence-grounded, multi-scale representations for decoding cell-type-specific regulatory programs. Code, datasets, and model checkpoints are available at https://github.com/Lotfollahi-lab/MUSS_neurips26 .
Abstract source: arXiv:2611.08086 · v1
Abstract ×
CrossMol: cross-modal mask-predict pre-training for 3D molecular data
Bioinformatics · 2026
Kangjie Zheng , Junwei Yang, Siyu Long, Wei Ju, Wei-Ying Ma, Hao Zhou, Ming Zhang.
Motivation Self-supervised pre-training models for molecular data have demonstrated notable results across many downstream tasks. The inherent multimodal properties of molecules have also motivated efforts to capture information from different modalities. However, current multimodal molecular pre-training models usually treat these modalities as equal and independent, despite differences in their information content. Three-dimensional (3D) molecular structures generally contain finer-grained information than the simplified molecular-input line-entry system (SMILES), which primarily captures higher-level semantic information, such as molecular topology.
Results We designed a cross-modal mask-predict pre-training model, CrossMol, to capture semantic associations between modalities with unequal information volumes. The model completes missing 3D structure using higher-level semantic information from another modality, such as SMILES. This allows it to learn cross-modal associations and better understand fine-grained 3D structural information. We also introduce a reweighted distance prediction loss to improve the modelling of short-range structural information. Experiments show that CrossMol achieves large performance gains on multiple downstream molecular tasks, attaining state-of-the-art results.
Availability and implementation The source code and training data are available at https://github.com/zhengkangjie/crossmol . The code is archived at https://doi.org/10.5281/zenodo.19653204 .
Abstract source: Bioinformatics
Abstract ×
ESM All-Atom: Multi-Scale Protein Language Model for Unified Molecular Modeling
ICML 2024
Kangjie Zheng * , Siyu Long* , Tianyu Lu, Junwei Yang, Xinyu Dai, Ming Zhang, Zaiqing Nie, Wei-Ying Ma, Hao Zhou.
* Equal contribution.
Protein language models have demonstrated significant potential in the field of protein engineering. However, current protein language models primarily operate at the residue scale, which limits their ability to provide information at the atom level. This limitation prevents us from fully exploiting the capabilities of protein language models for applications involving both proteins and small molecules. In this paper, we propose ESM-AA (ESM All-Atom), a novel approach that enables atom-scale and residue-scale unified molecular modeling. ESM-AA achieves this by pre-training on multi-scale code-switch protein sequences and utilizing a multi-scale position encoding to capture relationships among residues and atoms. Experimental results indicate that ESM-AA surpasses previous methods in protein-molecule tasks, demonstrating the full utilization of protein language models. Further investigations reveal that through unified molecular modeling, ESM-AA not only gains molecular knowledge but also retains its understanding of proteins.
Abstract source: ICML 2024 · PMLR
Abstract ×
SMI-Editor: Edit-based SMILES Language Model with Fragment-level Supervision
ICLR 2025
Kangjie Zheng , Siyue Liang, Junwei Yang, Bin Feng, Zequn Liu, Wei Ju, Zhiping Xiao, Ming Zhang.
SMILES, a crucial textual representation of molecular structures, has garnered significant attention as a foundation for pre-trained language models (LMs). However, most existing pre-trained SMILES LMs focus solely on the single-token level supervision during pre-training, failing to fully leverage the substructural information of molecules. This limitation makes the pre-training task overly simplistic, preventing the models from capturing richer molecular semantic information. Moreover, during pre-training, these SMILES LMs only process corrupted SMILES inputs, never encountering any valid SMILES, which leads to a train-inference mismatch. To address these challenges, we propose SMI-Editor, a novel edit-based pre-trained SMILES LM. SMI-Editor disrupts substructures within a molecule at random and feeds the resulting SMILES back into the model, which then attempts to restore the original SMILES through an editing process. This approach not only introduces fragment-level training signals, but also enables the use of valid SMILES as inputs, allowing the model to learn how to reconstruct complete molecules from these incomplete structures. As a result, the model demonstrates improved scalability and an enhanced ability to capture fragment-level molecular information. Experimental results show that SMI-Editor achieves state-of-the-art performance across multiple downstream molecular tasks, and even outperforming several 3D molecular representation models.
Abstract source: ICLR 2025 Proceedings
Abstract ×
Mol-AE: Auto-Encoder Based Molecular Representation Learning With 3D Cloze Test Objective
ICML 2024
Junwei Yang* , Kangjie Zheng * , Siyu Long, Zaiqing Nie, Ming Zhang, Xinyu Dai, Wei-Ying Ma, Hao Zhou.
* Equal contribution.
3D molecular representation learning has gained tremendous interest and achieved promising performance in various downstream tasks. A series of recent approaches follow a prevalent framework: an encoder-only model coupled with a coordinate denoising objective. However, through a series of analytical experiments, we prove that the encoder-only model with coordinate denoising objective exhibits inconsistency between pre-training and downstream objectives, as well as issues with disrupted atomic identifiers. To address these two issues, we propose Mol-AE for molecular representation learning, an auto-encoder model using positional encoding as atomic identifiers. We also propose a new training objective named 3D Cloze Test to make the model learn better atom spatial relationships from real molecular substructures. Empirical results demonstrate that Mol-AE achieves a large margin performance gain compared to the current state-of-the-art 3D molecular modeling approach.
Abstract source: ICML 2024 · PMLR
Abstract ×
ExLM: Rethinking the Impact of [MASK] Tokens in Masked Language Models
ICML 2025
Kangjie Zheng , Junwei Yang, Siyue Liang, Bin Feng, Zequn Liu, Wei Ju, Zhiping Xiao, Ming Zhang.
Masked Language Models (MLMs) have achieved remarkable success in many self-supervised representation learning tasks. MLMs are trained by randomly masking portions of the input sequences with [MASK] tokens and learning to reconstruct the original content based on the remaining context. This paper explores the impact of [MASK] tokens on MLMs.
Analytical studies show that masking tokens can introduce the corrupted semantics problem, wherein the corrupted context may convey multiple, ambiguous meanings. This problem is also a key factor affecting the performance of MLMs on downstream tasks. Based on these findings, we propose a novel enhanced-context MLM, ExLM. Our approach expands [MASK] tokens in the input context and models the dependencies between these expanded states.
This enhancement increases context capacity and enables the model to capture richer semantic information, effectively mitigating the corrupted semantics problem during pre-training. Experimental results demonstrate that ExLM achieves significant performance improvements in both text modeling and SMILES modeling tasks. Further analysis confirms that ExLM enriches semantic representations through context enhancement, and effectively reduces the semantic multimodality commonly observed in MLMs.
Abstract source: PMLR · ICML 2025
Abstract ×
Towards a Unified Training for Levenshtein Transformer
ICASSP 2023
Kangjie Zheng , Longyue Wang, Zhihao Wang, Binqi Chen, Ming Zhang, Zhaopeng Tu.
Levenshtein Transformer (LevT) is a widely-used text-editing model, which generates a sequence based on editing operations (deletion and insertion) in a non-autoregressive manner. However, it is challenging to train the key refinement components of LevT due to training-inference discrepancy. By carefully designing experiments, our work reveals that the deletion module is under-trained while the insertion module is over-trained due to the imbalance training signals for the two refinement modules. Based on these observations, we further propose a dual learning approach that can remedy the imbalance training by feeding an initial input to both refinement modules, which is consistent with the process in inference. Experimental results on three representative NLP tasks demonstrate the effectiveness and universality of the proposed approach.
Abstract source: ICASSP 2023 · Author poster · IEEE Xplore
Abstract ×
A Decoding Algorithm for Length-Control Summarization Based on Directed Acyclic Transformers
EMNLP Findings 2024
Chenyang Huang, Hao Zhou, Cameron Jen, Kangjie Zheng , Osmar Zaiane, Lili Mou.
Length-control summarization aims to condense long texts into a short one within a certain length limit. Previous approaches often use autoregressive (AR) models and treat the length requirement as a soft constraint, which may not always be satisfied. In this study, we propose a novel length-control decoding algorithm based on the Directed Acyclic Transformer (DAT). Our approach allows for multiple plausible sequence fragments and predicts a path to connect them. In addition, we propose a Sequence Maximum a Posteriori (SeqMAP) decoding algorithm that marginalizes different possible paths and finds the most probable summary satisfying the length budget. Our algorithm is based on beam search, which further facilitates a reranker for performance improvement. Experimental results on the Gigaword and DUC2004 datasets demonstrate our state-of-the-art performance for length-control summarization.
Abstract source: ACL Anthology · Findings of EMNLP 2024 (PDF)
Abstract ×
Gloss Matters: Unlocking the Potential of Non-Autoregressive Sign Language Translation
ACM MM 2025
Zhihao Wang, Shiyu Liu, Zhiwei He, Kangjie Zheng , Liangying Shao, Junfeng Yao, Jinsong Su.
While non-autoregressive sign language translation (NASLT) has the advantage in inference speed, the translation quality of NASLT models lags significantly behind that of the state-of-the-art (SOTA) autoregressive sign language translation (ASLT) models. To bridge the quality gap, we exploit glosses to unlock the potential of NASLT models. Concretely, we propose Gloss-enhanced Levenshtein Transformer (GLevT) for sign language translation (SLT), which takes glosses as initial sequences for editing into texts. In particular, to alleviate the inconsistency between training and inference of GLevT, which is introduced by glosses, we propose a dual-centric learning policy and a keyframe-based gloss replacement method for training, further improving the translation quality of GLevT. Experiments on CSL-Daily demonstrate that GLevT outperforms other NASLT models by approximately 4 points in BLEU and ROUGE scores, while achieving performance comparable to the SOTA ASLT models with a 3.46~5.26× inference speed-up. Furthermore, we extend GLevT to gloss-free SLT, achieving performance comparable to SOTA large models, despite having only 49M parameters. We release code at https://github.com/XMUDeepLIT/GLevT .
Abstract source: ACM Multimedia 2025
