We're Not There...Yet
Or...How I Stopped Worrying and
Learned to Deal With Cinematography for Multimedia
by Art Adams, SOC
It's Saturday night, and you've decided to
stay home with your significant other. The smell of hot buttered
popcorn wafts through the dimly lit house as you settle on the
couch, ready for a quiet evening of cuddling in front of the
television. Earlier in the evening you spent a half hour in the
video store searching for a couple of special movies that would
set the mood for romance. And now the time has come. The VCR
beckons quietly. "Feed me," it says. "Feed me and I will set
your heart afire."
"Sorry," you mutter, instead turning on your
personal computer and inserting the first CD-ROM of the evening.
You are so cool. One step ahead of the crowd, you've forsaken
your VCR and TV for the next step in human evolution: digital
playback using your top-of-the-line multimedia computer and high
resolution monitor. Once in the video store you strode past the
VHS section with scorn, laughed lightly while passing the laser
disc display, and headed straight to the Compact Disc department
and now you are about to realize the fruits of your decision.
You hit the "Enter" key on the keyboard, cuddle up with your
babe and anxiously await the first high definition image of the
evening to appear on the screen.
Your significant other stirs anxiously, moves
closer, and whispers in your ear. The words penetrate like daggers,
slashing straight to your ego after trodding on your heart and
stomping your soul.
"Is that it?"
IS THAT IT? Hey, this is state of the art technology!
So what if the picture is only three inches tall by four inches
wide? So what if the music sounds like you're listening to the
Boston Philharmonic through a pair of cans connected by rotting
twine? So what? This is state of the art technology!!!
And it's not quite good enough. Yet.
Someday you'll be able to request movies on
demand through the Internet, talk to your friends on a full color
videophone or tap into a live concert on the next continent and
hear the swish of the conductor's baton over the string section.
But not yet. Maybe in a year, but not yet.
Multimedia is currently overrated. For now
you can watch video recorded on a CD-ROM but the image appears
in a small window in the center of your computer screen and it
plays back at 15 frames-per-second, if you're lucky. The technology
is still fascinating and when you consider that just a couple
of years ago video on CD-ROM was naught but a hallucinogenic
cloud in a demented programmer's mind, you can easily appreciate
how far we've come in a short period of time. But we're not there
yet. Full-color, full-sized, full-speed video on a disk is still
a dream.
The biggest problem is storage. A video signal
contains an enormous amount of information; each frame consists
of 525 lines of picture information, information that changes
at the rate of 30 times per second. This information has almost
always been stored on a form of magnetic tape: first on huge
reels of fullcoat two inches across, then on one-inch tape and
eventually on half-inch or even 8mm tape. Each step required
a major advance in technology to place constantly refreshing
analog video information into smaller and smaller packages.
Now consider the difficulties of placing all
that information on a CD-ROM. In a 320x240 pixel image (approximately
3 inches by 4 inches on an SVGA monitor) there are 89,600 pixels
to be accounted for in each frame. At 30 frames-per-second, one
second's worth of video requires the reconstruction of 2,688,000
pixels. At full SVA 800x600 pixel resolution, the number quadruples.
The primary solution to digital video storage
is video compression. It's a simple idea. Certain parts of a
television picture don't change from frame to frame, so why store
the information necessary to redraw the complete frame? If the
only difference between frame #151 and #152 is that the actor's
head moved a little, then we need only store enough new information
to reproduce that small movement.
If 90% of the shot remains the same, there's
no need (and no room) to store 100% of the next frame and every
frame thereafter. We need only store the change between one frame
and the next. The obvious result is that, for the most part,
the camera does not move in CD-ROM productions. Less movement
means more compression and if the camera pans then everything
in the shot will move. Compression goes out the window and begins
its long, slow fall to the hard pavement below.
Video compression alone is often not enough.
The typical CD-ROM needs to store not only video information
but also data files containing non-moving graphics, sound effects,
music, and the software necessary for interaction with the user
and video playback. As a result, video clips are often stored
as low-resolution images intended to play back in a small on-screen
window at a reduced frame rate.
The reduced image size saves an enormous amount
of storage space, and by playing back every second or third frame
instead of every frame the storage space is halved again. Many
CD-ROM games contain video clips that are designed to play in
a small 320x240 pixel window at 15 frames-per-second. The resulting
visual effect is much like watching a circa-1920 silent movie,
except in color and without the organist.
All this will probably change within a year
or two but for now storage limitations are a major factor that
directly affect how footage intended for CD-ROM release is shot.
The director of photography on such a project must be acutely
aware of the limitations of video compression. His or her decisions
on the set will have a direct impact on the final product and
the consequences of a bad decision may not become apparent until
the last stages of post production when selected video sequences
are compressed to previously unknown file sizes. If the footage
doesn't fit on a CD-ROM, the director of photography may be the
first person on the list to receive a frantic, angry early morning
phone call. The following guidelines should significantly reduce
the odds of forced early retirement from the multimedia industry.
DON'T MOVE THE CAMERA
Less movement in the frame means greater compression.
Obviously actors must move but be sure to discuss shot sizes
with the director and the video compression company in advance.
A plethora of closeups incorporating even subtle head movement
may spell trouble later on. The more the image moves, the less
it will compress. If the camera moves, nothing will compress.
If forced into a situation where the camera
must move, the only suitable solution may be to use a background
that is not finely detailed. (See "Fine Detail" below.)
FIND OUT HOW BIG THE FINAL IMAGE WILL
BE ONSCREEN
This may affect your framing. A beautiful wide
landscape shot may not read well in a one-and-a-half by one inch
window.
BE AWARE OF EXPOSURE CHANGES IN THE
SHOT
Outdoor shooting can be problematic. A series
of small clouds passing in front of the sun may result in exposure
changes that are negligible to the eye but will result in a significant
loss of video compression.
FINE DETAIL CAN BE A PROBLEM
A finely detailed background will not compress
very well. In order to reduce the amount of data used to store
the bandwidth inherent in an NTSC signal, information is often
recorded symbolically rather than pixel-by-pixel. The compression
data utilized to recreate a solid blue background may be very
short ("draw blue from here to here") whereas fine detail forces
the compression software to store a literal map of color and
brightness changes throughout portions of the image, resulting
in little or no compression. Of course while a blank blue wall
will compress extremely well it isn't much to look at. A happy
medium must be struck. Be aware that actors with frizzy hair,
checkered shirts and acne may be a problem. In some cases, slight
non-haloing diffusion (e.g. Soft FX) might help. If shooting
on video, make sure the detail circuit isn't turned up too high.
When in doubt, call the post house and consult
with your compression guru for guidelines. Depending on the software
used for compression, you may find that you can get away with
quite a lot of detail or very, very little.
BRUSH UP ON YOUR BLUE SCREEN SKILLS
An enormous amount of multimedia photograpy
takes place in front of a blue or green screen for later compositing
with computer-generated footage. I suggest talking to your post
production house early on to find what compositing tools they
will be using. Many multimedia shoots are fast-paced due to short
schedules and the enormous length of interactive game scripts.
Knowing the software tolerances necessary for a successful blue
screen matte can be very handy. Many different software programs
are currently in favor for digital blue screen compositing and
each one has slightly different demands in the areas of screen
illumination, saturation and foreground image sharpness.
Multimedia shoots create unique problems. I
was recently involved in two shoots that presented completely
different photographic and logistical problems.
One project involved a ten minute multimedia
introduction to a new piece of software written by Apple Computers.
The mediums for distribution included VHS cassette, CD-ROM and
the Internet (via the World Wide Web). From the start the script
had to be tailored to the "lowest common denominator," in this
case the World Wide Web. The typical viewer would access the
Apple web site and download the file to their computer for viewing.
The difficulty lies in the speed of the average Internet connection.
Corporate users often have very fast access to the Internet,
allowing them to download a megabyte of information in about
ten seconds. Users who connect over phone lines connect at a
much slower rate, and the same megabyte of information may take
between ten and forty minutes to download.
The solution was twofold: First, the program
would be modular so that interested parties could download only
the module or modules that answered their specific questions
about the new software product. Second, the Internet version
of the video would play in a small 160x120 pixel (about one-and-a-half
by one inch) window and would serve as a teaser for the more
convenient CD-ROM or VHS versions of the program. Each module
would be about one megabyte in length, making the size acceptable
for download to those with slower Internet connections.
The script called for compositing an Apple
spokesperson into a computer screen for an "inside-out" tour
of the new product. The opening of the video took place in an
office set with two monitors plainly visible. Corporate multimedia
productions often include computer monitors which refresh at
different rates from 60hz NTSC video. In our case we had a Silicon
Graphics monitor and a Sun Microsystems monitor in the shot at
the same time, both refreshing at a different rate in the 70-80hz
range. This made the elimination of roll bars impossible. Our
solution was to attach Ultimatte blue fabric to the faces of
the monitors and insert still frame screen shots onto each monitor
in post. (See Frame
1)
The main portion of the shoot called for the
spokeswoman to interact with various objects on the screen in
which she would be composited. (See Frame
2) A videoswitcher was used to preview her position on the
screen in relation to the objects. In some shots she would appear
in a window inside the main computer screen. The switcher was
used as an aid in placing blue-painted sheets of foamcore so
that the actress would appear to be blocked in by the edges of
the window. (See Frame
3)
The last problem was that of lighting. Our
actress needed special treatment, but diffusion was out of the
question. Ultimatte is very good at dealing with soft edges but
placing a slightly diffused actress directly on top of a razor
sharp computer-generated screen was not an artistically viable
solution. In addition, while light diffusion may help video compression
by lessening detail, heavier diffusion can decrease compression
due to the softening of otherwise hard, easily defined edges.
A soft obie light directly under the camera lens was employed
to smooth facial features without affecting the blue screen behind
the subject.
Every single shot was a lock-off. Not once
did the camera move during a take.
In many ways video compression is uncharted
territory. At one point it was determined mathematically that
we could store a high resolution (640x480) 30 frames-per-second
version of this video program on a single CD-ROM. While this
turned out to be possible, the large amount of data to be processed
continuously by the microprocessor during playback proved prohibitive
on many platforms. The Macintosh Power PC 8100 was more than
capable of dealing with the high data flow necessary to create
a 30 fps high-resolution on-screen video image. Playing the same
file on a Macintosh Quadra 600 proved impossible as the older
microprocessor simply didn't have the speed to process the information
necessary to draw each frame fast enough.
Location blue screen shoots are becoming very
common. I was recently hired to shoot medium closeup interviews
of corporate executives who were to be interviewed at the Seybold
Convention in San Francisco. Agfa intended to produce a CD-ROM
containing pithy quotes relating to the future of the electronic
prepress industry. We were relegated to a back room at the convention
center and expected to make do with a limited grip and lighting
package and a 6'x6' blue screen.
The biggest problem was in trying to maintain
a good frame around the subject without moving the camera during
the shot. Immediate camera corrections were out of the question.
In the end, rather than lock-off the camera as required for good
compression I left the pan lock off. If the subject leaned out
of the shot I would have to interrupt the proceedings and ask
for a retake. Otherwise my corrections had to be made during
the few seconds when the interviewer was posing a question, a
period I knew would be cut from the final product. At that time
I was able to quickly nudge the camera left or right to achieve
a more acceptable framing.Later on the convention floor we shot
frenetic "MTV" style background shots to be composited behind
the interviewee. These were shot in video with a high-speed electronic
shutter. The resulting strobe effect was not only artistically
desirable but made for better video compression as hard edges
tend to compress better than soft motion blurs.
Multimedia is here to stay, but it is still
in its infancy. When next you see jerky low-resolution video
playback on a computer monitor, take heart; two years ago this
was not possible. In two more years who knows where we'll be?
Meantime revel in the challenges of video compression and multimedia.
It's a strange new world and those who help define it are destined
to work for a long, long time.
|
