Aesthetic Machine Learning On Threads

Running Aesthetic Scoring Pipelines at Scale

I've spent the last two years building and refining aesthetic machine learning systems that process thousands of images through scoring models concurrently. The short version: it's possible to get decent throughput, but most people hit bottlenecks they don't expect and waste weeks chasing them. Aesthetic Machine Learning On Threads refers to the practice of running aesthetic evaluation models -- things like CLIP-based aesthetic predictors, SDEdit quality scorers, or custom-trained preference models -- across multiple threads or processes so you're not waiting on a single GPU or CPU core per image. The core idea isn't new. You feed an image into a model, get back a score, and you parallelize the work because the naive approach is unbearably slow. The naive approach processes one image at a time. A single Aesthetic scoring model on a decent GPU might handle 8-12 images per second. A dataset of 50,000 images takes over an hour. With threading, you can push that down to under ten minutes on a well-configured machine. The difference is massive when you're iterating.

Here's the thing most tutorials don't mention. Threading on GPU-bound inference is not the same problem as threading on CPU-bound work. CUDA streams exist for a reason, but the default PyTorch behavior will silently serialize your work if you don't configure things correctly. I learned this the hard way when I first set up a multi-threaded aesthetic scorer and watched my throughput stay flat at about 10 images per second regardless of how many threads I spawned.

Setting it up

The basic structure looks like this. You load your aesthetic model once into GPU memory. You create a thread pool. Each thread grabs an image, runs inference, and writes the score. Sounds simple. The implementation details matter more than you'd think. Start with a model. The most common choice is the CLIP aesthetic predictor, which is a small MLP on top of a CLIP encoder. You can find implementations on GitHub, and the original weights were released by the Stability AI team. Load it with torch.no_grad() and keep it in eval mode. Don't skip either of those. For threading, I recommend using a process pool instead of a thread pool if your dataset is larger than a few thousand images. Python's GIL makes true multithreading ineffective for CPU-side preprocessing, which is almost always part of this pipeline. You'll resize, normalize, and augment images on the CPU before they hit the GPU anyway. Multiprocessing sidesteps the GIL bottleneck entirely.

Here's a minimal structure that actually works: Load the model once in the main process. Share it with worker processes using copy-on-write semantics or just reload it in each worker -- the VRAM cost is the same either way since the model lives on the GPU. Use a Queue to feed images to workers. Have each worker return (image_path, score) tuples. Collect results in order if you need to maintain correspondence. The actual code is straightforward. What trips people up is the queue management. If you don't set proper timeouts and use join() correctly, workers will hang on shutdown and your script will appear frozen. I've killed scripts that were still running because the queue was blocked and no one was reading from it anymore.

The edge case that wasted three days of my time

I was processing a batch of 40,000 images through a custom aesthetic model when about 60% through the run, every single thread started returning NaN scores. No error message. No warning. Just NaN. The model was fine. The data loader was fine. The GPU had memory to spare. The problem was gradient scaling. My aesthetic model had a few batch normalization layers, and when running with multiple concurrent CUDA streams from different processes, the running statistics were getting corrupted. Specifically, the moving average calculations were racing between processes because each process loaded the model independently and their internal state diverged during inference. Wait, that's not right. Inference shouldn't update batch norm stats. The actual problem was that one of the worker processes hit an OOM on CPU side during preprocessing and silently returned garbage tensors. PyTorch's multiprocessing queue swallowed the exception and returned a zero-filled tensor, which the aesthetic model converted to NaN through a log operation. The fix was adding exception handling directly inside each worker function and logging failures with the image path. Once I could see which images were failing, the pattern became obvious -- a handful of corrupted PNGs with broken metadata.

Counter-intuitive things I've learned

More threads does not always mean faster. There's a sweet spot that depends entirely on your hardware. On a single A100 with a CLIP ViT-L/14 model, I found that 8 worker processes was optimal. Going to 16 actually decreased throughput by about 15%. The overhead of managing CUDA contexts across processes outweighed the parallelism benefit. On a consumer card like a 4090, the sweet spot was closer to 4. Test this on your own machine before assuming more is better. Another thing: the bottleneck is rarely the inference step itself. For aesthetic scoring, preprocessing -- resizing, color space conversion, normalization -- often takes longer than the actual model forward pass, especially when your input images vary wildly in resolution. I optimized preprocessing with batched tensor operations and cut total pipeline time roughly in half without touching the model at all. Also, keep your model's input resolution reasonable. Aesthetic models are typically trained on 224x224 or 336x336 inputs. Running them at higher resolutions doesn't improve scores and slows everything down proportionally. I saw someone on a forum try 1024x1024 input for "better accuracy" and the scores actually got worse. The model was never trained on that resolution.

Limitations you should know about

Aesthetic models are not objective. They predict human preference based on training data, which is almost always web-scraped image-caption pairs. The scores correlate with things like composition, color harmony, and visual complexity, but they encode the biases of whoever curated that training data. If you're using this to filter creative work, understand that you're ranking images by a narrow definition of "looks good" that leans heavily toward certain styles and away from others. Threading introduces non-determinism. If you need reproducible results -- and you probably do if you're doing research -- running the same pipeline twice may give you slightly different score orderings because of race conditions in queue processing or floating point non-associativity across GPU streams. For ranking purposes this usually doesn't matter. For publication-grade experiments, it does. The biggest limitation is memory. A single aesthetic model might only need 1-2 GB of VRAM, but each worker process that independently loads the model consumes that memory separately. At 8 workers on a 24GB card, you're using 8-16GB just for model copies. This is why shared memory or a single-process approach with intra-process threading can be more efficient on consumer hardware. Consider using torch.multiprocessing with shared CUDA tensors if you're memory-constrained.

Practical numbers

Here's what I've measured on my own setups. A100 80GB, CLIP ViT-L/14 aesthetic predictor, 224x224 input, 12 worker processes, JPEG images averaging 2MB each: approximately 95 images per second end-to-end. That means 50,000 images in about 9 minutes. Same setup with a single process: roughly 11 images per second. About 76 minutes for the same dataset. On a 4090 with 4 workers, I get about 38 images per second. 50,000 images in roughly 22 minutes. The scaling isn't linear with worker count on consumer cards, which comes back to that CUDA context overhead I mentioned. If you don't have a GPU, CPU-only inference with multiprocessing will run at about 3-5 images per second on a modern Ryzen or Intel chip. It's viable for smaller datasets but not practical for anything over 10,000 images unless you have time to kill.

Where this approach breaks down

If your aesthetic model is larger than CLIP-scale -- say, a full SDXL-based quality discriminator or a multimodal model with cross-attention over text and image -- threading helps less because the model itself becomes the dominant bottleneck. In those cases, batch inference on a single process is often faster than distributed threading. The model's internal attention mechanisms don't parallelize well across separate GPU contexts. Similarly, if your images require heavy preprocessing like inpainting, segmentation, or style transfer before aesthetic scoring, the preprocessing time will dominate and threading the inference step alone won't move the needle. Profile your pipeline first. Don't assume the model is the bottleneck. One more thing: if you're scoring dynamic content -- like generating aesthetic feedback for a live application rather than a one-time batch -- threading adds latency unpredictability. Request ordering, queue contention, and worker availability can cause response times to vary. For interactive use cases, a simpler single-process pipeline with warm-up caching gives more consistent performance even if it's slower overall.