Min - Plus
/In the early days of working at R&H, bandwidth was limited and the need to minimize the amount of data being transferred was critical. Most of the data being sent across the pipe were high resolution images and some quicktime files.
The typical workflow back then would result in the Source or Background(BG) images being scanned in Los Angeles. These 2K (2048x1556) images would then be transferred over to the Mumbai office to be worked upon.
Most of the work that was done to these images involved painting or altering portions of the image to remove Film dust or maybe to remove objects that were not meant to be part of the shot. This process was called Background Preparation (BgPrep) .
The modified clean plate would then be uploaded back to Los Angeles for review or delivery to further stages in the pipeline.
We learned very quickly that this shuffling of images back and forth between the locations was only increasing as we grew the number of people or as the amount of work increased. But bandwidth was still a premium and we needed a way to efficiently transfer this data.
A cheap trick that we came up with to reduce the amount of data was to employ a method of only sending the "difference" pixels back to Los Angeles and then running a post process to create the final image.
When you look at the workflow, you have a source image which is called the Background image(BG), the artist paints over it modifying a set of pixels and then sends the entire modified image back called the Painted image. What if we could send only the pixels that the artist had actually changed and then run a process that could produce the resultant image that is mathematically similar to the Painted image.
Using simple compositing operations we can perform a arithmetic subtraction of the Painted image from the BG which results in a Difference image. You then transfer this extremely small difference image and the then perform an arithmetic addition of the BG with the Difference image to recreate the Painted image.
I have whipped up a quick example along with some python code that explains all of it.
Things that you will for this example:
- A background image
- A linux machine/vm (I used Centos 6.3 VM running on my Macbook)
- Python (Most Linux distributions have this preinstalled)
- Python Image Library (Most Linux distributions have this preinstalled)
- OpenEXR packages (OpenEXR, Ilmbase, Pyilmbase & openexr_viewer)
- OpenEXR has a bunch of dependencies that should be easily installed using the package manager
- Python OpenEXR bindings
Its important to note that you cannot use Lossy compression formats like JPEG for any of the images as you are likely to loose information and your resultant image will be incorrect
For this example i started with a Background Image (BG). This is a 8bit TIF file
(This is a photo i had taken while driving through Sonora Pass)
I created a copy of the BG, to which i made some modifications to create my Painted image
(Created a copy of the truck to the screen left)
Next i wrote up a quick Python script that loads the BG and the Painted image and performs a subtraction pixel by pixel between the two images. Below is the min.py script
min.py
I need to explain why i chose to use an EXR as an output file and not just another TIF file. The Python Image Library (PIL) does not allow you to output signed int/float TIF files. If you read the documentation it seems to allow that, but that is a single signed int or float per pixel. I needed to store 3 signed int or floats per pixel, each representing 'R', 'G' and 'B' channels. (For this example im ignoring the Alpha/Matte channel)
The PIL library only allowed Integer (0-255) values to be stored in the 'RGBA' format, this does not work for the Difference image as i needed to store negative values.
The Difference image is not very viewable, as the negative values are misrepresented or clamped. If you were to open up the EXR file in an exr viewer it would look something like this. Notice that most of the image is black other for the pixels that i changed. Using a Run Length Encoding scheme to compress the pixels, your resultant file size is significantly smaller (117KB).
We can transfer this Difference image very quickly and when you want to recover the Painted image you can run the addition operation back on the BG. Below is the plus.py script that performs just that
plus.py
And that's about it, if you load load the Final.tif file, it should look like this
Finally if you want to ensure that the images are identical i wrote up a quick pixel comparison script, which compares the Painted image to the Final image.