Country Mouse City Mouse

Migration Tools That Promise Simplicity

Most of us in the data engineering space have been burned by tools that oversell their ease of use. Country Mouse City Mouse is one of those tools that you'll find recommended in a few Slack channels and GitHub issues, usually alongside posts from people who haven't actually run it against a production dataset yet. I've spent the last two years working with it, mostly because my team chose it over alternatives without fully evaluating whether it was the right fit. That was a mistake, but the experience taught me more than I expected.

What Country Mouse City Mouse Actually Does

At its core, Country Mouse City Mouse is a data migration and schema transformation framework. It's designed to move datasets between relational databases while handling type casting, schema drift, and incremental load logic. The marketing materials make it sound like you write a config file and walk away. That's not how it works in practice. The basic workflow involves defining source schemas, target schemas, and mapping rules in YAML or JSON format. You then run the migration tool, which parses your mappings and attempts to execute the transfer. For simple homogeneous migrations—say PostgreSQL to PostgreSQL with only column renames—it functions adequately. The documentation covers these cases well, which is why most beginners get a false sense of security.

The part the docs don't emphasize enough is how much manual intervention is required when things diverge from the happy path. Type mismatches, constraint violations, and orphaned foreign keys all need to be resolved individually. The tool will halt on the first unresolvable error and dump a log file that assumes you already know exactly what went wrong.

Setting It Up Without Wasting Three Days

Install the package from the official npm registry. Don't use a mirror. I tried a few times and ran into dependency resolution conflicts that took longer to debug than the actual setup would have. After installation, initialize the project with the CLI command, which generates a default configuration file. The configuration file is where things get tricky. You'll define connection strings for both source and target databases, set up your mapping rules, and configure batch sizes. The default batch size is 500 rows. That works fine for small tables. For anything larger than a few million rows, increase it to 5000 or 10000. Going higher without tuning your database's max_connections setting will cause connection pool exhaustion on both ends, and the migration will fail silently or with extremely confusing error messages.

I learned this the hard way during a project where I moved approximately 40 million records from a legacy MySQL instance to a new PostgreSQL cluster. The initial run failed after processing 12 million rows with a generic timeout error. The logs pointed nowhere useful. It took me four hours to realize the connection pool was saturated, not the disk I/O, which is what I'd assumed. Switching to a lower batch size and increasing the pool limit resolved the issue. The remaining 28 million rows processed without another problem in about 90 minutes.

Mapping Rules and Schema Drift Handling

The mapping system is the most powerful and the most fragile part of Country Mouse City Mouse. You define column-to-column mappings, type transformations, and optional data enrichment functions. Simple mappings are straightforward:

source_table.field_a maps to target_table.column_x with a VARCHAR to TEXT type cast.

The complexity arises when you encounter schema drift. This happens when your source database changes between migration runs—columns get added, renamed, or have their types altered. The tool has a drift detection mode, but it's aggressive and will flag almost any change as a potential breaking issue. In my experience, you'll spend more time disabling false positives in drift detection than you will running actual migrations. The workaround I use is to disable automatic drift detection and instead run a schema diff script before each migration. I wrote a small Python utility that compares the current source schema against a saved baseline and outputs a human-readable report. I review that report manually and update the mapping file only for changes that actually matter. It adds about 20 minutes to the prep work but prevents the tool from halting on irrelevant drift.

Pitfalls That Beginners Miss

There are several behaviors in Country Mouse City Mouse that aren't obvious until you've already made a mistake. Foreign key constraints are handled poorly during initial load. The tool attempts to insert data in source table order, which means child rows often arrive before their parent rows, causing constraint violations. The documented workaround is to set defer_constraints to true in your target database config, but this only works for PostgreSQL and MySQL 8.0+. It does nothing for SQL Server or older MySQL versions. For those databases, you need to disable constraints before migration and re-enable them afterward, which requires manual intervention and a second pass to validate referential integrity. Incremental loads using change data capture rely on the source database's binlog or transaction log. If your source database isn't configured to retain logs for at least 72 hours, you'll lose data that arrived outside your migration window. I encountered this during a project where the source MySQL instance had a log retention period of 6 hours due to storage constraints. The migration completed successfully but missed roughly 15% of records that had been inserted between log rotation cycles. There's no recovery mechanism built into the tool for this scenario. You have to run a full reload, which means downtime and extended processing time.

When Not to Use Country Mouse City Mouse

The tool has genuine limitations that make it unsuitable for certain workloads. If you're working with non-relational data sources like MongoDB or Cassandra, it won't help you. The framework is explicitly designed for relational databases. There are community plugins for some NoSQL stores, but they're incomplete and unmaintained. If your migration requires complex data transformations—calculations across multiple tables, deduplication logic, or conditional enrichment based on external API calls—the built-in transformation engine will frustrate you. It supports simple arithmetic and string operations out of the box, but anything beyond that requires writing custom JavaScript functions. Those functions run synchronously during migration, which means a poorly optimized custom function can turn a 30-minute migration into a 4-hour ordeal. I've seen it happen. One team member wrote a deduplication function that performed a linear scan against the full dataset for every row. We killed the process after it had been running for six hours on a 2 million row table. For complex transformation scenarios, I recommend using Country Mouse City Mouse only for the bare data movement portion and handling transformations separately with a dedicated ETL tool like Apache Airflow or a custom Python pipeline. This adds infrastructure overhead but gives you much more control over the transformation logic and performance characteristics.

Performance Tuning That Actually Matters

The tool's performance is heavily dependent on how you configure your batch processing and connection settings. Beyond adjusting batch size, there are a few less obvious optimizations worth considering. Enable parallel processing if your source and target databases can handle it. The tool supports concurrent batch execution, but you need to ensure your database can sustain the load. A good rule of thumb is to start with two parallel workers and increase from there while monitoring query latency on both databases. I typically see diminishing returns after four workers unless the bottleneck is purely network-bound. Disable indexes on the target table before migration and rebuild them afterward. This alone can cut full-table migration time by 40 to 60 percent for large tables. The tool doesn't automate this, so you'll need to write the index management into your pre- and post-migration scripts. It's not complicated, but it's easy to forget, and running a multi-million row migration with indexes enabled is painful.

The transaction log size on the source database also matters more than most people account for. Each inserted row generates a log entry. If your source database's transaction log is too small, the migration will cause log rotation during the transfer, which introduces checkpoint overhead and slows things down significantly. Check your log size configuration before starting. If the log is smaller than 10% of your batch total row size, increase it. This prevented a major slowdown during my 40 million row migration and saved roughly three hours of runtime.

Final Notes on Maintenance

Country Mouse City Mouse isn't a set-and-forget solution. It requires attention to configuration, monitoring during execution, and validation after completion. The validation step is critical and often skipped. Run a row count comparison between source and target after every migration. Check for null value discrepancies in non-nullable columns. Verify that foreign key relationships are intact on the target side, especially if you used deferred constraints. The project is actively maintained with regular updates, but the release cadence has slowed in the past year. Bug fixes come through, but new feature development is minimal. If you encounter an issue that's specific to your database version, there's a reasonable chance it won't be addressed in the near term, and you'll need to work around it yourself or fork the repository. I still use Country Mouse City Mouse for straightforward homogeneous migrations because it gets the job done faster than writing a custom pipeline from scratch. But I've learned to treat it as a starting point rather than a complete solution. The gaps in the documentation, the fragile error handling, and the limitations around complex transformations all add up to significant manual effort if you're not prepared for them. Plan accordingly.