← All publications

Journal · 2026

AI for Healthcare

Cross-Modal Progressive Modeling for Neuro-Visual Representation Learning

Yueming Sun, Jiyao Pu, Kaili Sun, Zeyu Fu, Haoran Duan, Yang Long

Neurocomputing

CFT-NET and progressive visual-semantic adaptation framework for aligning EEG signals with visual representations.

Abstract

Overview

Neural decoding from scalp signals requires models that respect spatial, temporal, and spectral structure while leveraging strong visual priors. In this paper, we introduce CFT-NET for disentangled neural visual representation together with a progressive visual–semantic adaptation (PVSA) framework that aligns EEG embeddings to pretrained visual backbones under a contrastive objective followed by pairwise matching. CFT-NET integrates Frequency-Separated Weights (FSW), Spatial-Context Aggregation (SCA), and Adaptive Temporal Filtering (ATF) to explicitly extract spectral, spatial, and temporal factors. PVSA consists of an instance-guided visual encoder and a visual-guided semantic decoder linked by cross attention, enabling fine-grained neuro–image interaction. On THINGS-EEG and THINGS-MEG dataset, the approach consistently outperforms state-of-the-art baselines in both subject-dependent and subject-independent zero-shot classification. By aligning model architecture with visual cognition principles and coupling it to strong visual priors, our methods narrows the gap between neural activity and visual cognition, advancing more efficient neural decoding network.

Citation

BibTeX

@article{Sun_2026, title={Cross-modal progressive modeling for neuro-visual representation learning}, volume={683}, ISSN={0925-2312}, url={http://dx.doi.org/10.1016/j.neucom.2026.133450}, DOI={10.1016/j.neucom.2026.133450}, journal={Neurocomputing}, publisher={Elsevier BV}, author={Sun, Yueming and Pu, Jiyao and Sun, Kaili and Fu, Zeyu and Duan, Haoran and Long, Yang}, year={2026}, month=June, pages={133450} }