CoLA: Cross-Modal Low-rank Adaptation for Multimodal Downstream Tasks
📄 CoLA: Cross-Modal Low-rank Adaptation for Multimodal Downstream Tasks #音视频理解 #参数高效微调 #LoRA #多模态模型 8.3/10 | 创新 1.5/2 | 严谨 1.2/1.5 | 实验 1.3/1.5 | 清晰 0.7/1 | 影响 1/1.5 | 开源 1.2/1.5 | 复现 0.4/0.5 | 工程 1/1.5 🔥 8.3/10 | 前25% | #音视频理解 | #参数高效微调 | #LoRA #多模态模型 | arxiv 👥 作者与机构 第一作者:Wish Suharitdamrong(Surrey Institute for People-Centred AI, University of Surrey; Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey) 通讯作者:Wish Suharitdamrong(ws00372@surrey.ac.uk) 作者列表:Wish Suharitdamrong(Surrey Institute for People-Centred AI, University of Surrey; CVSSP, University of Surrey)、Tony Alex(Surrey Institute for People-Centred AI, University of Surrey; CVSSP, University of Surrey)、Muhammad Awais(Surrey Institute for People-Centred AI, University of Surrey; CVSSP, University of Surrey)、Sara Atito(Surrey Institute for People-Centred AI, University of Surrey; CVSSP, University of Surrey) 💡 毒舌点评 CoLA 将 LoRA 的低秩分解巧妙扩展为双路径结构,为双编码器多模态适配提供了一条简洁的跨模态融合范式;视觉‑语言与音频‑视觉两组任务上的实验也较为扎实,并首次实现了基于 PEFT 的多任务视觉定位。然而,该方法本质上仍是对 LoRA 的线性外推,理论分析仅停留在秩和线性跨度层面,未能给出更深的表征交互机制;且跨模态路径在推理时不可合并带来的开销,在资源敏感场景中会成为硬伤。此外,损失函数完全缺失,复现存在实质性缺口。 ...