Towards AI-assisted Video Editing: Generating Shorts from Long Videos
Hao-Wen (Herman) Dong
University of Michigan
Add to Calendar
2025-09-12 16:00:00
2025-09-12 17:00:00
America/New_York
Towards AI-assisted Video Editing: Generating Shorts from Long Videos
Abstract:Video editing is a crucial yet laborious step for video production that sometimes takes more time than filming the content. Oftentimes, video creators first shoot a pool of raw video materials and later edit them into a cohesive story, sometimes in various formats and durations for different platforms. While current video generation models can generate compelling short videos, they fall short in inserting real, filmed materials into its outputs. In this talk, I will introduce our two recent projects that explore new machine learning models that can cut, select, and rearrange a long video into a short video. In the first TeaserGen project, we proposed a narration-centered teaser generation system that can effectively compress >30-min documentaries into 3-min teasers leveraging pretrained LLMs and language-vision models. In the second REGen project, we proposed a retrieval-embedded generation framework that allows an LLM to quote multimodal resources while maintaining a coherent narrative. I will conclude by discussing future work towards next-generation video editing interfaces that can be integrated into the existing creative workflows of video creators using multimodal LLMs and RAG.Bio:Hao-Wen (Herman) Dong is an Assistant Professor in the Department of Performing Arts Technology at the University of Michigan. Herman’s research aims to augment human creativity with machine learning. He develops human-centered generative AI technology that can be integrated into the professional creative workflow, with a focus on music, audio and video content creation. His long-term goal is to lower the barrier of entry for content creation and democratize professional content creation for everyone. Herman received his PhD degree in Computer Science from University of California San Diego, where he worked with Julian McAuley and Taylor Berg-Kirkpatrick. His research has been recognized by the UCSD CSE Doctoral Award for Excellence in Research, KAUST Rising Stars in AI, UChicago and UCSD Rising Stars in Data Science, ICASSP Rising Stars in Signal Processing, and UCSD GPSA Interdisciplinary Research Award.
TBD