__   ___  ___       _______       ___  _______   
|/"| /  ")|"  |     /"     "|     |"  ||   _  "\  
(: |/   / ||  |    (: ______)     ||  |(. |_)  :) 
|    __/  |:  |     \/    |       |:  ||:     \/  
(// _  \   \  |___  // ___)_   ___|  / (|  _  \\  
|: | \  \ ( \_|:  \(:      "| /  :|_/ )|: |_)  :) 
(__|  \__) \_______)\_______)(_______/ (_______/  
			
			

Viewing Columbo at a Distance

Inspired by Taylor Arnold and Lauren Tilton’s Distant Viewing: Computational Exploration of Digital Images , Frank Fischer's network analysis of Curb, and the Columbo Screenshots BlueSky bot, I decided to take a look at the colors of Columbo.

Firstly, I wanted to be able to find the Columbo frame with the average color closest to a given color. In order to do this, I first got my hands on a low-resolution (360p) rip of the entirety of Columbo. Then, using the cv2 Python wrapper for OpenCV, I iterated through every frame and recorded each frame’s average red, green, and blue value (deleting the black bars at the top and bottom of the frames when necessary) , along with the episode number, season number, and frame count. The resulting Pandas DataFrame was 10,782,016 rows long, each row representing a frame, and the resulting CSV weighed in at 1gb. Reasonably hefty.

This process took about eight hours (I challenge my dear friend/mortal enemy Siddha Mavuram to do it faster). But once it was done, I was pretty close to being able to color match Columbo frames. All I had to do next was find a loss function. I settled on summed squared error, i.e. finding the difference between the frames’ RGB values and the desired RGB values, squaring them, and summing them. Then all you need to do is find the frame with the lowest loss.

For example, suppose I want to find the frame with the color #3a5133 (a dark green). All I need to do is calculate this loss for the ~11,000,000 frames (with Pandas NumPy this actually takes no time at all), then sort the frames by ascending loss and take the first. In this case, the frame with the lowest loss is this one:

A nice bunch of bell peppers with a very attractive color.

Using this color matching, you can produce color palettes of frames, like the ones below.

Four of the seven colors from the Rainbow pallete come from the same episode. S02E07 The Most Dangerous Match begins with a dream sequence with a shifting, rainbow strobe effect, meaning that if you’re looking for highly saturated, monocolor Columbo frames, its your best bet nine times out of ten.

To see just how crazy this strobe effect is in terms of color value saturation, take a look at the first five minutes of this episode compared to the first five minutes of the episodes that come directly before and after.

Cool as these frames are, they are outliers, and I got to wondering what the general color distribution of Columbo looks like. To take a look at it, I took a sample of 1,000 frames and visualized each of them as a point of that color in 3D space, with their RGB values as xyz coordinates. This is that general distribution:

As you can see, the xyz coordinates tend to increase and decrease together, clustering around the center diagonal of the cube, and producing colors that are, on average, pretty gray, although we get some colorful scallion-style divergence at the top. The average color (visualized below) is a dark gray.

But the outliers really are the most fun, and I wanted to see them in this format too. To this end, I added a boolean column (“outlier”) to the combined data frame, which was true when a given frame had a zscore of three or higher, and false otherwise. Sampling from the outliers produces this visualization:

And sampling 500 from each (outlier and non-outlier) gives a broad (if quantitatively inaccurate) account of the colors of Columbo:

In general, it seems the average color palette of Columbo is muted and rather dark. The figure below illustrates the average RGB values per episode throughout the series.

The colors seem to be generally the same over the course of the series, although season 8 is significantly darker than the rest. In 1989, after a 12 year hiatus, ABC releases new episodes (the first 7 seasons ran on NBC). Here, the format of the show switches from 4:3 to 16:9 (at least in my rip). The first episode of season 8 (Columbo Goes to the Guillotine) is just about as dark as the show gets, and we don't return to normal until season 9. It seems the ABC execs wanted a change of tone (let's call it "Columbo goes dark"), but reverted to the old brightness in the following seasons.

I was also curious to see how colors change over the course of the episode. Across episodes, I took the average color of each frame, and came up with the below graph. The divot at around ten minutes is interesting, but I don’t know how to explain it. My only guess is that around 10 minutes, the murders usually take place… An avenue for future research.

Perhaps we learned a little bit more about Columbo? Maybe, maybe not. Anyway, as a finale, here is Columbo made out of frames from Columbo: