Getting Started With Airhead Meg Cabot
Airhead Meg Cabot is a utility most people in my circle use for batch processing assets through automated pipelines. The name sounds ridiculous, which is probably why the documentation is sparse and why nobody bothers writing tutorials about it. I stopped expecting polish after the first week. It works, barely, and if you know where it hides its configuration files, it's manageable. I picked this up back when we were drowning in manual exports from three different render engines and nobody wanted to admit the workflow was broken. Meg Cabot was running in production at a small studio across the river. I went over there, watched them use it for a day, and wrote down everything that annoyed me. That list became my playbook.
What Airhead Meg Cabot Actually Does
It reads asset manifests and pushes them through a queue system with priority weighting. You feed it a JSON or YAML file listing your source paths, output targets, and some metadata flags. It spawns worker processes, monitors them, and retries failed jobs based on a configurable backoff schedule. That is the core loop. Everything else is optional plumbing around it. The thing people misunderstand is that it is not a standalone application with a GUI. It runs headless. You interact with it through a CLI and by editing config files. If you are looking for buttons and dropdowns, you are barking up the wrong tree. The interface is the filesystem.
Installation and Initial Setup
Download the current release from the GitHub repository. At the time of writing, that is version 0.8.4. I recommend grabbing the prebuilt binary for your platform instead of compiling from source. The build process requires Node 18 and a working Go toolchain, and half the dependencies in the Makefile are pinned to versions that no longer resolve cleanly on modern systems. Once downloaded, extract it to a permanent location. Do not install it in /tmp or any temporary directory. I learned this the hard way when a system cleanup script deleted my installation mid-render on a Friday night. The job queue state was stored inside the installation directory, so recovery meant starting over from scratch with forty-two unfinished assets. After extraction, run the init command to generate your configuration file:
Get the Full Details
megcabot init --config-dir ~/.config/megcabot This creates a default.yaml file. Open it and set the worker count. The default is four, which is fine for a home machine but painfully slow if you are processing more than a few dozen items. I run mine at sixteen workers on a 32-thread machine with 64GB RAM. Queue throughput roughly doubles compared to the default setting.
Building Your First Pipeline
Create a directory for your project. Inside it, make a file called assets.yaml. Here is what a minimal entry looks like: source: /data/raw_assets/scenes\noutput: /data/renders/final\nworkers: 8\npriority: normal\ntimeout: 1800\nretry_count: 3 The source field points to your input directory. The output field is where completed jobs land. Workers overrides the global config for this specific pipeline. Priority controls ordering in the queue. Timeout is in seconds. Retry count is self-explanatory.
When you are ready to start the queue, run: megcabot start --pipeline assets.yaml --log /tmp/megcabot.log The log output is minimal. It prints job start, job complete, and job failure lines with timestamps. That is it. You learn to read it quickly or you waste time staring at a blank screen wondering if the process actually started.
Common Pitfall: Path Resolution on Network Mounts
This is where things get hairy. If your source or output paths are on a network mount, Airhead Meg Cabot will silently fail on some jobs without any warning. The worker processes inherit the user's environment, but on certain NFS configurations, the mount credentials do not propagate into child processes spawned by the queue manager. I hit this specifically last year when a client's render farm had their asset storage on an AFP share mapped to /Volumes/Assets. Half the jobs would hang indefinitely. The log showed nothing useful. It took me three days of strace and process tracing to realize the child workers couldn't see the mount point at all. The workaround is to use bind mounts or to copy assets locally before queuing. Copy to /tmp or a local SSD, run the pipeline there, then move results back. It adds a step but eliminates the ambiguity. Total overhead is usually under five minutes per batch depending on network speed and asset size.
Advanced Queue Management
Once you have jobs running, you will need to monitor and manipulate the queue. Airhead Meg Cabot ships with a status command that shows pending, active, completed, and failed counts. megcabot status --pipeline assets.yaml There is also a pause command that stops new jobs from starting but lets currently running workers finish their current tasks. Useful when you need to swap out source files or adjust configuration without losing progress.
megcabot pause --pipeline assets.yaml To resume: megcabot resume --pipeline assets.yaml
The delete command removes failed jobs from the queue. It does not recover them. If you need to reprocess a failed asset, you have to either fix the underlying issue and set the retry count higher, or manually re-add the asset to the queue by editing the manifest file directly.
Performance Tweaks That Actually Matter
The default configuration is conservative. It writes job state to disk after every completion event, which creates unnecessary I/O pressure. If your pipeline involves hundreds or thousands of small assets, this disk thrashing becomes noticeable. Disabling write-ahead logging in the config cuts average job completion time by roughly thirty percent on SSD-backed systems. Set write_ahead_logging to false in your global config. Be aware that a crash during a job now means you lose the in-flight state for that job. It is a tradeoff between speed and durability. For archival renders where you can afford to rerun a job, it is worth it. For production work with tight deadlines, keep it enabled. Another thing nobody mentions: the garbage collection interval. By default, Meg Cabot cleans up temporary worker files every sixty seconds. On high-throughput pipelines, this interval causes microstutters as workers pause briefly while the cleanup runs. Bumping it to three hundred seconds smooths out the throughput curve with negligible disk usage impact. The temp files accumulate but they are small. You can purge them manually with megcabot cleanup any time.
When Airhead Meg Cabot Fails Completely
It does not handle parallel write conflicts well. If two jobs in the same pipeline try to write output to the same directory simultaneously, the second job fails with a permissions error. The queue does not detect this collision in advance. It only fails at write time. The fix is to ensure each pipeline has a unique output path. Use timestamped subdirectories or a counter-based naming scheme in your output path template. Meg Cabot supports format strings in the output field. Something like /data/renders/%Y%m%d_%H%M/job_ renders cleanly without conflicts. There is also no built-in support for dependency chains between jobs. If asset B requires the output of asset A, Meg Cabot will not wait. It treats every job as independent. People who need serialization usually wrap the whole thing in a shell script or use a separate orchestrator like Airflow. I stopped trying to make Meg Cabot do something it was never designed for and just wrote a wrapper script that queues jobs sequentially for dependent assets.
Alternatives Worth Considering
If your pipeline is straightforward and you need a more robust tool, cc:taskrunner is heavier but better documented. It handles dependencies, has a web dashboard, and its error reporting actually tells you what went wrong instead of a cryptic exit code. The learning curve is steeper and it demands a PostgreSQL backend, which is overkill for simple use cases. If you are doing something entirely different, like real-time stream processing or GPU-accelerated inference pipelines, neither of these tools is the right fit. Meg Cabot is a batch processor. It was never meant for streaming or interactive workloads. Don't force it into a shape it cannot hold. Airhead Meg Cabot has its rough edges. The documentation assumes you already know what you are doing, which excludes everyone who isn't. But once you have your config files set up and your paths sorted, it runs reliably for months without intervention. That reliability is why I keep coming back to it despite the headaches.
Start small. Queue five assets. Watch the logs. Break things intentionally so you understand the failure modes. The first week is always frustration, but after that it just works. Or it doesn't, and then you troubleshoot like you would anything else in this industry.
