The University of Otago and NHNZ Moving Images have signed a research agreement that could revolutionise production footage logging practices and footage archives around the globe.
The project gives the University access to NHNZ Moving Images’ archive of 200,000 hours of footage to investigate image searching without the need for shot logs.
The end result could be revolutionary technology giving archivists and footage researchers the ability to find a video clip based on its similarity to a reference clip, rather than the shot listed description manually and subjectively loaded against the clip.
“The University’s Information Science researchers were on the hunt for a massive video database for research. And we have this huge library right here in Dunedin,” says Caroline Cook, Manager of NHNZ’s Moving Images.
Ms Cook says the University’s research is leading edge and, if it becomes a reality, will revolutionise production practices and footage archives around the world.
“In our library, across all genres including natural history, culture, science and adventure, there are some 200,000 hours of footage collected over 30 years of film-making. That amounts to about 12 million individual shots. If we could get a system which can search for recognisable images that haven’t been shot listed, this would help footage archivists immensely.
“The approach is similar to that law enforcement agencies use to automatically recognise people in crowds. With current technology we can grab location and time-in, time-out, so we can capture quite a lot of data, but the challenge is that descriptions are subjective, who decides what the focus of the shot is? This will eliminate that subjectivity and allow us to search footage for all manner of subjects – be it a tree, an elephant or an insect.”
“Other uses for this technology in the archive industry would be, for example, helping us with our 2000 hours of beta SP, a lot of which has never been logged. It’s a dilemma: Do we transfer that to digital tape? But then you have to eyeball every single shot. If we could automate the process of determining the focus of each shot, it would cut out the dilemma of whether or not to convert the tape.”
Dr. Jeremiah Deng, the University of Otago researcher leading the project, is excited by the opportunity.
“Machine learning techniques are quickly becoming mature enough to tackle more challenging tasks than what existing solutions such as Google Image Search can do. We are experimenting with new data-mining solutions that can more effectively extract semantic descriptions from video clips. The NHNZ archive is a gold mine for us and we hope to collaborate with other researchers around the world working in similar areas,” says Dr Deng.
It is anticipated this project will bring NHNZ Moving Images into the already close relationship between NHNZ and the University of Otago. NHNZ and the researchers will also cooperate to take their work from the lab to a “real world” setting, because, as Ms Cook says, “the commercial potential is huge”.
The research team will begin their mission by taking delivery of a large dataset of wildlife clips for summarization and object/action recognition experiments. They are starting by focusing on the most popular wildlife characters including penguin, dolphin, elephant, zebra, lion, and polar bear.