?

Log in

No account? Create an account
lazyweb: image processing - brad's life [entries|archive|friends|userinfo]
Brad Fitzpatrick

[ website | bradfitz.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

lazyweb: image processing [Oct. 9th, 2005|06:05 pm]
Brad Fitzpatrick
[Tags|, ]

Dear Lazyweb,

Given a couple hundred JPEGs, what's the most efficient way, using Free tools, to one of:

-- find the brightest file

or:

-- find the file with the most overall contrast (not all white or all black, but a good mix of both?)

Background: got video thumbnailing working in FotoBilder, but sometimes people's videos start with some black frames at the beginning. Or bright white (presumably?) while their phone/camera CCDs adjust to outdoor sunlight and the user aims at their actual target.

So I'd like to extract the first 20-100 frames and pick the most "interesting" one, for some definition of interesting. But definitely not solid black or white.
LinkReply

Comments:
[User Picture]From: nothings
2005-10-10 01:18 am (UTC)
Simplest solution: take whichever file is the largest (or, ideally, maximum ratio of filesize to pixel dimension, but if these are thumbnails of fixed size, you're all set). All-one-color files will code smaller; indeed, the same file with contrast increased becomes larger for a fixed jpeg quality setting.

Back to the original question, efficient in terms of your effort, or efficient in terms of server usage?

If you want quality, and the efficiency issue is the CPU use, because jpgs encode the "DC" component of each image block separately from the rest of the data, in terms of the second thing it's fast to extract a 1/8th scale b&w image. The standard jpeg library everyone uses, the Independent Jpeg Group's, supports this in the low-level API. So make sure whatever you end up using makes use of that ability.
(Reply) (Thread)
[User Picture]From: brad
2005-10-10 01:31 am (UTC)
Simplest solution: take whichever file is the largest

Whoa, that's so retroactively-obvious that I'm embarassed that I didn't think of it.

And it works great! Just tried it on a couple videos, and it always picked a suitable frame.
(Reply) (Parent) (Thread)
[User Picture]From: jwz
2005-10-10 01:32 am (UTC)
That's what I was going to say; you really want the "noisiest" image, or possibly the first image over some threshold of noise, and "how well does it compress" is a really good indicator of that.

I'll bet if you look at a bunch of the kind of cases you're trying to detect, there will be a big obvious discontinuity in file size from which you can pick your noise floor.
(Reply) (Parent) (Thread)
[User Picture]From: endquote
2005-10-10 03:34 am (UTC)
Seems like a good way to get a default, but from a user perspective it might be better to pull thumbnails at every ten seconds or something, and then let the user choose one to use as a preview image.
(Reply) (Parent) (Thread)
[User Picture]From: brad
2005-10-10 05:11 am (UTC)
maybe later
(Reply) (Parent) (Thread)
[User Picture]From: kvance
2005-10-10 01:31 am (UTC)
I was thinking of a really gross hack involving imagemagick -convolve with a kernel that would average most of the image into the first pixel of a PGM that it dumps to stdout so you can grep for it. . . and then I read the comments.

Measuring the file size sounds much more efficient :P
(Reply) (Thread)
[User Picture]From: newscane
2005-10-10 05:12 am (UTC)
In case you aren't aware, Gallery 2.0 now does video thumbnailing. I don't know if they do any checking to make sure it's a good thumbnail (haven't looked at it all that closely yet), but I know it's something that's been implemented. They're using ffmpeg (http://ffmpeg.sourceforge.net/index.php).
(Reply) (Thread)
[User Picture]From: brad
2005-10-10 05:31 am (UTC)
Well, getting the frames with either ffmpeg or mplayer wasn't the hard part. Actually none of it was that hard, in retrospect.
(Reply) (Parent) (Thread)
[User Picture]From: newscane
2005-10-10 05:32 am (UTC)
Ah, okay. Well, do you know if they are doing anything to get the best frame for a thumbnail?
(Reply) (Parent) (Thread)
[User Picture]From: brad
2005-10-10 05:34 am (UTC)
I don't follow Gallery.
(Reply) (Parent) (Thread)
[User Picture]From: taral
2005-10-10 09:44 pm (UTC)
Hm, looks like the above comments will get you the noisiest frame. If you want something more sophisticated, hit me up. :)
(Reply) (Thread)
[User Picture]From: brad
2005-10-10 10:14 pm (UTC)
Not willing to share with the rest of the Internets? :-)
(Reply) (Parent) (Thread)
[User Picture]From: taral
2005-10-10 10:48 pm (UTC)
No, but I have ideas, and ideas != implementation without a goodly bit of thinking. In my state, I'm trying to avoid excess thinking. So if the current solution is good enough, then I'll just shelve the ideas until a future time.
(Reply) (Parent) (Thread)