Hi
Ganesh,
Thanks for the note. I
apologize for the delayed response. I have been working on conference deadlines
and travel and just could not get back to you
earlier.
Yesterday I received
your proposed set of year one activities (Enabling Content-based
Searches in Video Streams using MPEG-7) from Matthew.
We believe that
interfacing to our system, flowing to MPEG-7, would be pretty straightforward.
We output information on our annotations in XML. I have attached an email and
the associated files that outline what we are doing. The XML has evolved a
little from this information, but it should give you an idea of the info.
I am stalled on the
following points, though:
-
We don’t
do real time processing of video streams (at full frame rate and resolution,
anyway) at the present time. I am not sure that is required for the demo in
November (which is working from Tape gathered in September). If it is required
then our AVED system is not going to be ready to do that this
year.
-
We don’t
do HDTV at the current time. All of our HD video has been down-sized to
digitalBetaCam (NTSC) before we input the data. We can commit to *trying* HDTV native input by September, but
it means we need to get hardware integrated into our system at MBARI to input
the HDTV streams and we need to allocate the time. It is not on our project
roadmap for this year at this time. Again, I don’t know if that is required for
the demo, or just nice to have.
-
Finally,
and probably most importantly, we do not know what it means to “detect
organisms” in the busy scenes coming off of Endeavor. In the one that I have
seen, like these
http://www.amnh.org/nationalcenter/expeditions/blacksmokers/gallery.html
there are often more animals than a human could count. That is not the
application space of our system, which is trying to deal with sparseb scenes
(where it is expense to have a human look through a lot of “nothing” to get to
the interesting events). So for me, the Real Time and HDTV aspects that you lead
with are moot if the video is so busy that our system just can’t handle it in
any case.
So, I am eager to get
observatory video, and try it in our system. We would be delighted to work with
you to integrate our stuff to your MPEG-7 encoder and your search engine work. I
am guarded about successfully processing the video that is going to be collected
at Endeavor until we have a much better idea of what that is going to look
like.
Duane
From: Ganesh
Gopalan [mailto:gopalang@coas.oregonstate.edu]
Sent: Thursday, May 26, 2005 5:18
PM
To:
duane@mbari.org
Cc: 'Mark
Abbott'
Subject: Real-time
detection and classification of organisms demo for SuperComputing
'05
Duane,
It was good talking to
you at the Skamania meeting earlier this month.
With regard to a
teleconference with Matthew Arrott today, a scenario that was discussed for the
demo in November
was that your AVED
system would be coupled with the live HD feed from the bottom of the Pacific to
detect organisms in real time, and we thought we’d encode the annotations in
MPEG-7 (which is basically XML) and provide a retrospective search engine
(web-based or otherwise) for these creatures, where one could type in the name
of a creature, the depth and/or latitude/longitude and times and get a set of
matching frames that contain the organisms.
Would it be possible
for you to give us an overview of your current system (or pointers to white
papers) to give us an idea of the interface required to enable the MPEG-7
encoding?
At some point we were
also wondering if we could have access to processed video samples with
annotations in whatever format available.
Thanks,
Ganesh.