Twelve year old kids manage to put webcams (is that a real word?) up on the Net every day. So, you'd figure a grown man with over a quarter-century of programming experience could get one up and running in a matter of minutes, right?
Wrong.
It took me countless hours of intense, head-scratching, cuss-word-slinging programming to get mine up and running to DumbSoutherner standards. Here, share my pain...
I'm using a Creative WebCam NX camera to capture the images you see on our Webcam page. It's a small, auto-exposure, manual focus camera with a 352 x 288 pixel CMOS image sensor. When my wife gave me this camera on Christmas Eve of 2003, I figured I'd write all the software to control it. However, I soon discovered the camera came packaged with a program called "Creative WebCam Monitor". One of this buggy little program's features is the ability to periodically capture an image, time stamp it, then upload the image file to a web server using FTP. That took care of a lot of the work to be done, so I decided to use it. Not completely trusting this software with my web server root password, I created a new webcam FTP account on my server (with its own password), then allowed the program to upload images to this account. After some minor finagling, I got the program to reliably transfer images my XP box at home to my remote Apache server.
Next came the task of presenting these images on a web page. Since I was not at all impressed with the free software available to perform this task, I decided to write it myself. I began with several design goals in mind: First, I wanted images to update smoothly, without all the ugly image flickering you see on most all webcam sites. Next, I wanted the images to automatically update even if visitors used a browser that did not support Java or JavaScript. Finally, I wanted something completely unique - the ability to vary the rate at which images update in users' browsers by simply varying the rate at which I uploaded images from my XP box to the server. I'd never seen this done before. Such a feature could be invaluable. If I noticed my server bandwidth usage nearing the point where I might incur surcharges, I could simply slow the rate at which images are uploaded to the server. In turn, the image refresh rate for all visitors would then automatically slow down without me having to do a thing! If my bandwidth usage remained reasonable, I could increase refresh rates in a similar manner, all without having to change a thing in my code or XHTML documents! Cool, huh?
These design goals turned out to be pretty dang lofty. I could think of only one way to pull off all this magic. I wrote a Perl CGI script to run on my server. This script monitored the timestamp of the webcam image file. When it changed (due to a new image being uploaded to the server), the script then monitored the file size until it achieved a steady state. (This, by the way, was a necessity. It takes about 2.5 seconds for an image file to upload via my slow-ass, rural, dial-up connection. You don't want to go grabbing the image before it's there in its entirety.) Once the script saw that a new image was ready, it handled the image as MIME "multipart/x-mixed-replace" content, and pushed it to the browser. All I had to include in my XHTML document was a simple <img> tag pointing to the script. I thought this was all brilliant. I was proud of myself! There was only one little problem:
It didn't work.
Now, I'm not a Perl expert by any stretch of the imagination. But, I can usually get a script up and running after a little head scratching. I ended up debugging this script for hours. I could NOT get it to work! Finally, I did a few searches on the Internet to see if anyone else had ever encountered a similar problem. I eventually located a description of various MIME types. There was a footnote for the "multipart/x-mixed-replace" definition. It said, "Not supported by Microsoft Internet Explorer".
By chance, I'd been testing my script using Internet Explorer. I shut down Explorer and fired up a copy of Netscape Navigator. Everything instantly began working flawlessly. Webcam images refreshed so smoothly it was almost like watching television! No flickering. No partial images. Everything performed exactly as I'd planned. It was a beautiful thing to behold. If you've got a copy of Navigator, you can see it for yourself. (Note, however, this Netscape-only viewer refreshes much more slowly now than it did when I first wrote it. This is due to changes in the file upload method I'm now forced to employ.)
As it turns out, Netscape has supported multipart/x-mixed-replace content for nearly a decade. Microsoft's Internet Explorer still does not. Since the vast majority of visitors to my website use Internet Explorer, I was left with no choice but to scrap everything and start over.
Have I mentioned lately how much I hate Microsoft?
So, I threw away the pretty Perl script I'd worked so hard on, and scrapped the idea of using server push. Instead, I'd have to use client pull. I finally settled on using JavaScript embedded in my XHTML to pull webcam image from my server to the users' browsers.
You can find thousands of examples of such scripts on the Internet. However, they all seem to work basically the same way. They sleep for some period of time, then attempt to suck in the next webcam image to be displayed. This method works reasonably well if the visitor has a fast Internet connection. But if they've got a slow, dial-up connection, the place where the image is supposed be displayed goes blank, then slowly fills with a new image each time it's refreshed. All this flickering annoys the hell out of me. So, I wrote my own script to handle things a bit differently.
My script employs image caching. While one image is being displayed, the script is busy in the background retrieving the next image to be displayed. So, when time comes to refresh the screen, the next image to be displayed already resides in memory. This allow the script to swap images virtually instantaneously. This makes for much smoother image transitions, eliminating all the annoying flickering.
I also built one other handy feature into my JavaScript - the ability to count the number of images retrieved from the server. After a visitor has viewed some prescribed number of webcam images, the script can present a message requiring the visitor to click on the image in order to continue viewing. This prevents visitors from opening my webcam page, then going on vacation for two weeks while their computer continues to pound on my server, racking up bandwidth charges. Slick, right?
Well, it was slick in theory, but not so slick in practice.
In hurriedly writing this script in hopes of getting it up and running by New Year's Eve, I'd completely forgotten about the problem of the slow Internet connection I use to upload image files to the server. At the time, I was transferring images to my server at a rate of one every 10 seconds. I'd forgotten that around 2.5 seconds of this 10 second window are eaten up by my slow file transfers. So, if my JavaScript, running in a visitor's browser, attempted to fetch a new image during this 2.5 second time period, it would receive an incomplete image. You can do the math. About one quarter of all the images displayed by my script were incomplete. It looked like crap.
By this point, some of the fun of webcam script writing was beginning to wear off. But, it was New Year's Eve, and I wanted out-of-town friends to be able to view the festivities at our house. So, I went live with my crappy script on December 31, 2003, with plans to fix the problem the next day.
As it turned out, I didn't feel much like programming on January 1, 2004. My mind had told me to party like a 21-year-old on New Year's Eve. The following day my body reminded me of my actual age. It suggested I begin acting my age, or suffer the consequences.
A couple days later (after I fully recovered), I woke up one morning dreaming of a solution to the problem of the broken images images served up by my JavaScript. The idea that came to me in my sleep turned out to be a pretty good one. Here's how I solved the problem:
It's possible to configure the Creative WebCam Monitor to use multiple files names for the images it uploads to the server, rotating through these file names in a round-robin fashion. For example, it can be configured to use three different file names. The first image captured is stored in a file named webcam01.jpg, the second in webcam02.jpg, and the third in webcam03.jpg. When it comes time to store the forth image, the software reuses the first file name of webcam01.jpg. The fifth image is stored in webcam02.jpg, and so on. That's precisely how I set up the software. It now rotates through three file names as it uploads image files to the server.
Next, I wrote a Perl CGI script to examine these webcam image files on the server. It checks their timestamps, then decides which file should be sent to the visitor's browser. It always selects the file with the second freshest timestamp. Here's my reasoning: The file with the freshest timestamp may still be in the process of being uploaded by the Monitor. The file with the oldest timestamp will be next in line to be overwritten by the Monitor, and may be corrupted before the user's browser has a chance to grab it in its entirety. That leaves the file with the second freshest (or, middle) timestamp. It should be complete, and should remain untouched by the Monitor for a minimum of 10 seconds. That should give any visitor's browser plenty of time to retrieve the image before the Monitor begins overwriting it.
Modifying my JavaScript to make use of this Perl script was easy. Rather than fetching an image directly from the server, I told it to fetch images from the Perl script. This Perl script, in turn, always selects a nice, complete image for caching, then presentation.
Problem solved. Finally. It only took me about a week longer to pull it off than it should have. Where's a 12-year-old kid when you need one?
Direct comments, questions, compliments, and death threats to fred@DumbSoutherner.com.
Copyright © 1996-2004 Fred D. Bean
All rights reserved.
