Aligning source visual and target language domains for unpaired video captioning

Journal article

Training supervised video captioning model requires coupled video-caption pairs. However, for many targeted languages, sufficient paired data are not available. To this end, we introduce the unpaired video captioning task aiming to train models without coupled video-caption pairs in target language. To solve the task, a natural choice is to employ a two-step pipeline system: first utilizing video-to-pivot captioning model to generate captions in pivot language and then utilizing pivot-to-targ...

Expand abstract

Authors: Liu, F; Wu, X; You, C; Ge, S; Zou, Y

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: IEEE Publisher's website
• Journal: IEEE Transactions on Pattern Analysis and Machine Intelligence Journal website
• Volume: 44
• Issue: 12
• Pages: 9255-9268
• Publication date: 2022-12-02
• Acceptance date: 2021-11-16
• DOI: 10.1109/tpami.2021.3132229
• EISSN: 1939-3539
• ISSN: 0162-8828
• Pmid: 34855588
• Language: English
• Keywords: taining; adversarial training; pipelines; data models; task analysis; pseudo supervised training; visualization; feature extraction; video captioning; decoding; unpaired video captioning; pipeline system; FFR