Color Fill

The Problem with Manual Colorization

I used to colorize old family photos by hand, using layer masks and clipping groups in Photoshop. It took me about 40 minutes per image, give or take, and the results were usually muddy because my color choices weren't historically grounded. That changed when I started using automated approaches and eventually built my own pipeline around them. The short version: Color Fill is a method for applying realistic color to grayscale or sepia-toned images using machine learning models trained on large datasets of natural images. Color Fill refers to any technique that takes a monochrome image and produces a plausible full-color version. The most common implementation today relies on deep learning models, particularly generative adversarial networks (GANs) or diffusion-based architectures. The model doesn't truly "know" what color something should be. It predicts the most statistically likely color distribution based on what it has seen in training data. A tree is almost certainly going to be green. A sky is blue. Anything beyond that is guesswork dressed up as computation. The core technical process works like this: the grayscale image gets fed into an encoder that extracts spatial features, those features get passed through a color prediction network that outputs a chrominance channel, and the result gets merged with the original luminance data. The output isn't perfect, but it's fast and usually within the right ballpark.

How to Actually Use It

There are several ways to get this done depending on your setup and what kind of output you need. If you just want to process images without touching code, the most accessible option currently is the open-source project maintained by the researchers at Columbia University, available on GitHub under the repository name "colorization." There's also the web interface hosted at de oldian.com which runs the same model family with a different front end. Upload a grayscale image, wait roughly 10 to 30 seconds depending on the server load, and you get back a colorized JPEG. For local processing, you can grab the pre-trained weights from the official repository and run them through Python with PyTorch installed. The command line interface looks something like this:

python colorize.py --input photo.jpg --output colored.jpg --gpu 0 The default model gives decent results on outdoor scenes. Indoor photographs with artificial lighting tend to come out a bit off. That's a limitation of the training data, not the code.

Building Your Own Pipeline

Writing a custom Color Fill script takes about an hour if you already know PyTorch and have the dependencies set up. You start by loading the pre-trained model weights, usually from a checkpoint file. Then you preprocess your input image by converting it to grayscale if it isn't already, resizing it to 256x256 or 512x512 depending on your model's requirements, and normalizing pixel values to the range the model expects. The inference step is straightforward. You pass the tensor through the network and extract the predicted color channels. The trick is in the post-processing. You need to convert the output from CIELAB color space back to RGB, then scale it to the 0-to-255 range that your target format requires. Here's a stripped-down version of what the conversion loop looks like: lab = np.concatenate([l_channel, ab_predicted], axis=-1)
rgb = color.lab2rgb(lab)
rgb = np.clip(rgb * 255, 0, 255).astype(np.uint8)

That last line is important. If you skip the clipping, your output image will have values outside the valid range and most image libraries will throw an error or produce corrupted files.

The Edge Case Nobody Warns You About

Last year I was processing a set of 1940s newspaper photographs for a client. Most of them came out fine. Then there was one image of a woman in a dark dress against a light background. The model predicted the dress as a deep navy blue, which looked reasonable at first glance. But when I checked the reference material for that era of fashion photography, dresses like that were typically dyed in muted brown or charcoal tones. The model had never seen enough examples of dark clothing on light backgrounds during that specific time period, so it defaulted to the most common dark-color association in its broader training set, which happened to be modern outdoor scenes. The workaround was simple but not obvious if you don't know the model architecture. I took the original grayscale image and ran it through the colorizer twice. The first pass gave me the initial prediction. The second pass used the first pass's output as a reference mask, constraining the model to stay closer to the luminance boundaries in the original image. This didn't fix the color entirely, but it prevented the model from hallucinating colors into areas that were clearly mid-tone or dark in the source. It added about 15 seconds to the processing time per image but saved me from having to manually paint corrections on dozens of photos.

Counter-Intuitive Things to Know

Most people assume that higher resolution input produces better results. It doesn't. These models are trained on low-resolution patches, usually 256x256 or 512x512. If you feed a 4K photograph into the model without first downscaling and then upscaling the result, you'll get artifacts at the edges and inconsistent color across the frame. Always preprocess to match the training resolution, then upscale separately using a dedicated super-resolution model if needed. Another thing that surprises people: the L-channel in CIELAB color space is independent of the color prediction. That means you can swap the luminance from a different source image and the color prediction will still work. I've used this to composite colors from reference photographs onto historical images where the lighting conditions didn't match the original scene. It's not foolproof but it's useful when the automated result is close but not quite right.

When Color Fill Fails Completely

The method breaks down in three specific scenarios that I've encountered repeatedly. First, images with very low contrast. A faded photograph where the subject blends into the background gives the model nothing reliable to work with. Second, images containing large areas of a single tone. A sky that's uniformly gray, or a field that's uniformly light, will produce either blank regions or wildly incorrect color predictions because the model has no spatial gradients to guide it. Third, highly stylized or abstract images. These models are trained on natural photographs. They don't understand illustration, painting, or graphic design, and trying to colorize a line drawing will produce garbage results 9 times out of 10. For those cases, manual intervention is unavoidable. I usually start with the automated Color Fill as a base layer, then use a tablet to paint corrections over the problematic areas. It's faster than starting from scratch but still requires actual artistic judgment. No model can replace that when the source material is ambiguous.

Practical Recommendations

If you're processing more than five images at once, set up a local environment with GPU acceleration. The cloud-based options are convenient but slow and they introduce privacy concerns if the images contain sensitive content. A decent consumer GPU like an RTX 3060 will process a single image in under 3 seconds, compared to the 15 to 30 seconds you'd wait on a free web service. Always keep the original grayscale file. The colorized output is probabilistic, not deterministic. Running the same image through the model twice will give you two slightly different results. That means you should never treat the first output as final unless you've verified it against reference material. For professional work, I recommend spending another 10 to 20 minutes per image doing manual correction on the color channels. The time investment pays off in accuracy, and it's still faster than the 40-minute-per-image manual method I was using before.