Project Description

You are visiting the webpage of the SLIC (Semantically Linked Instructional Content ) project. This project aims to assist students and scholars efficiently browse and seek segments of interest in educational videos of lectures and talks. In particular, it focuses on lectures that use slides, where the content of the slides file gives valuable hints as to how to break the video into meaningful parts (segments), and how to enable students to access these segments. In this way, a student who is seeking a specific topic in the video of a lecture(s) can first find the relevant slide(s), and start watching the video only from the segment(s) where this slide was used. Using similar ideas, the system can also improve significantly the understandability of the video, improve its quality, and increase the overall effectiveness of the learning process.

Algorithmic Overview

The demo contains several talks. In each frame of the videos, the system will find the slide that is most likely to appear in the frame. This required the use of non-trivial stochastic models and pattern matching, since slides may be similar, but in some of the frames, only part or none of the slide is shown, slides sometimes appear very blurry in the videos etc. In the demo you can also see the effect of segmenting the videos based on slides, versus the effect of segmenting the video based on changes in the flow of images only (shot-boundary detection). Fonts are generally clearly read.

Collaborations

The project is the result of a joint work of academia and industry. In particular, it is resulted of a collaboration between the Computer Science Department, KUAT and the MIS department in the Eller College of Management, all at the University of Arizona, and Almaden IBM Research Center.

Publications

Qiyam Tung, Ranjini Swaminathan, Alon Efrat, and Kobus Barnard. "Expanding the point -- Automatic Enlargement of Presentation Video Elements." Proc. of ACM Multimedia, Scottsdale, AZ, November 2011.

Quanfu Fan, Kobus Barnard, Arnon Amir, Alon Efrat. "Robust Spatiotemporal Matching of Electronic Slides to Presentation Videos." IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 8, August 2011 1

Alon Efrat, Arnon Amir, Kobus Barnard and Quanfu Fan. "Cross-modality Indexing, Browsing and Search of Distance Learning Media on the Web in eBook - Internet Multimedia Search and Mining (edited by Xian-Sheng Hua, Marcel Worring and Tat-Seng Chua).

Ranjini Swaminathan, Michael Thompson, Sandiway Fong, Alon Efrat, Arnon Amir, Kobus Barnard, "Improving and aligning speech with presentation slides," ICPR 2010.

Quanfu Fan, Kobus Barnard, Arnon Amir, and Alon Efrat, "Accurate Alignment of Presentation Slides with Educational Video," IEEE International Conference on Multimedia & Expo (ICME), 2009.    [PDF]

Andrew Winslow, Qiyam Tung, Quanfu Fan, Juhani Torkkola, Ranjini Swaminathan, Kobus Barnard, Arnon Amir, Alon Efrat, and Chris Gniady, "Studying On The Move - Enriched Presentation Video For Mobile Devices," 2nd IEEE Workshop on Mobile Video Delivery (MoViD), in conjunction with INFOCOM, April 24, 2009.    [PDF]

Quanfu Fan, Kobus Barnard, Arnon Amir, Alon Efrat, and Ming Lin. "Matching Slides To Presentation Videos Using SIFT and Scene Background Matching", 8th ACM SIGMM International Workshop on Multimedia Information Retrieval (MIR) , Santa Barbara, October 26-27. 2006. [PDF]

Quanfu Fan, Arnon Amir, Kobus Barnard, Ranjini Swaminathan, Alon Efrat, "Temporal modeling of slide change in presentation videos," International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2007 [PDF].

Acknowlegements

We wish to thank the Arizona Center for Information Science and Technology (ACIST), and NSF CAREER grant 0348000. We also want to thank IBM Research for the CueVideo Toolkit.

Personnel (in alphabetic order)

Arnon Amir (IBM) *, Kobus Barnard *, Joe Chitwood (KUAT), Alon Efrat *, Quanfu Fan, Sandiway Fong, Steven Gregory, Yekaterina Kharitonova *, Vivek Kumar, Ming Lin, Daniel Mathis, Adam McFarlin, Ranjini Swaminathan, Mohan Tanniru, Michael Thompson, Juhani Torkkola, Qiyam Tung *, Gabriel Wilson, Andrew Winslow, Steve Zhou

* Currently actively involved in the project.