ESHOPMAN Custom Module Migration Bug: Prevent Data Loss

Protecting Your ESHOPMAN Database: Understanding a Critical Migration Bug in Custom Modules

Developing custom modules is a powerful way to extend your ESHOPMAN storefront's functionality, leveraging the flexibility of its Node.js/TypeScript backend and seamless integration with HubSpot CMS. However, our community recently identified a critical issue with the eshopman db:generate command when used with custom modules, which could lead to catastrophic data loss if not handled carefully.

The Unexpected Database Wipe

When ESHOPMAN developers run eshopman db:generate to create migrations for their custom modules, the expectation is that the generated script will contain only ALTER TABLE or CREATE TABLE statements relevant to the new or modified entities within that specific module. Unfortunately, the command, under certain conditions, produces a migration script that includes:

ALTER TABLE ... DROP CONSTRAINT for foreign keys across all core ESHOPMAN tables (e.g., cart, order, product, customer, payment_collection).
DROP TABLE IF EXISTS ... CASCADE for every single core ESHOPMAN table.
Buried within these destructive statements, the actual ALTER TABLE / CREATE TABLE commands for the custom module's changes.

Applying such a migration without meticulous manual inspection would effectively wipe your entire ESHOPMAN database, making this a severe risk for development and production environments.

Unpacking the Root Cause: Snapshot Misinterpretation

The community's deep dive into this issue revealed the underlying mechanism. The db:generate process relies on snapshot files to determine the difference between the current database schema and the desired schema. Specifically, it writes two types of snapshots into a module's migrations/ folder:

.snapshot-.json: This file correctly captures the schema of only the custom module's entities.
.snapshot-.json: This file, however, takes a full introspected snapshot of the entire live database.

The critical flaw occurs because the full-database snapshot (.snapshot-.json) is written after the module-specific snapshot. Consequently, on subsequent runs of eshopman db:generate, the system incorrectly selects the most recently modified file—the full-database snapshot—as its baseline "previous state." When this comprehensive baseline is compared against the custom module's entity schema (which, by design, only contains its own models), the system computes a diff that assumes all core ESHOPMAN tables, present in the baseline but absent from the module's schema, have been removed. This leads to the generation of DROP TABLE statements for all core tables.

It's important to note that the custom module itself is not at fault; any ESHOPMAN custom module that undergoes more than one migration generation run is susceptible to this behavior once the .snapshot-.json file exists in its migrations folder.

Community-Proposed Solutions and Best Practices

The ESHOPMAN community has put forward several potential solutions to address this critical vulnerability:

Refined Snapshot Handling: The migration generator could be updated to either avoid writing the full-database snapshot into the module's migrations folder (perhaps using a temporary location) or to filter for only module-specific snapshots (.snapshot-.json) when establishing the baseline.
Pre-Drop Verification: Implement a safeguard where, before emitting any DROP statement, the system verifies that the table being dropped is actually declared as an entity within the current module. If not, the drop statement could be skipped with a warning.

While these solutions are being considered for future ESHOPMAN updates, developers building custom modules should adopt the following best practices:

Always Inspect Migrations: Never apply a generated migration script without thoroughly reviewing its contents. Look specifically for any DROP TABLE or DROP CONSTRAINT statements that target core ESHOPMAN tables.
Database Backups: Regularly back up your ESHOPMAN database, especially before applying any schema changes.
Development Environment Testing: Test all migrations in a development or staging environment before deploying to production.

Example ESHOPMAN Backend Project Configuration

For context, here's an example package.json file from an ESHOPMAN backend project, illustrating typical dependencies and scripts. Note how scripts like build, start, and dev interact with the ESHOPMAN framework:

{
  "name": "@dtc/backend",
  "version": "0.0.1",
  "description": "A starter for ESHOPMAN projects.",
  "author": "ESHOPMAN (https://eshopman.com)",
  "license": "MIT",
  "keywords": [
    "sqlite",
    "postgres",
    "typescript",
    "ecommerce",
    "headless",
    "eshopman"
  ],
  "scripts": {
    "build": "eshopman build && node ./scripts/copy-receipt-assets.mjs",
    "start": "eshopman start",
    "dev": "eshopman develop",
    "test:integration:http": "TEST_TYPE=integration:http NODE_OPTI jest --silent=false --runInBand --forceExit",
    "test:integration:modules": "TEST_TYPE=integration:modules NODE_OPTI jest --silent=false --runInBand --forceExit",
    "test:unit": "TEST_TYPE=unit NODE_OPTI jest --silent --runInBand --forceExit"
  },
  "dependencies": {
    "@eshopman/admin-sdk": "2.14.2",
    "@eshopman/admin-shared": "2.14.2",
    "@eshopman/caching": "2.14.2",
    "@eshopman/cli": "2.14.2",
    "@eshopman/dashboard": "2.14.2",
    "@eshopman/draft-order": "2.14.2",
    "@eshopman/framework": "2.14.2",
    "@eshopman/icons": "2.14.2",
    "@eshopman/js-sdk": "2.14.2",
    "@eshopman/loyalty-plugin": "^2.14.2",
    "@eshopman/eshopman": "2.14.2",
    "@eshopman/ui": "4.1.9",
    "@react-pdf/renderer": "^3.4.5",
    "@sendgrid/mail": "^8.1.6",
    "@tanstack/react-query": "5.64.2",
    "@tanstack/react-table": "^8.21.3",
    "libphonenumber-js": "^1.12.41",
    "qrcode": "^1.5.4",
    "react-hook-form": "^7.72.1",
    "react-i18next": "13.5.0",
    "react-router-dom": "6.30.3",
    "stripe": "^22.0.2",
    "twilio": "^5.13.1",
    "zod": "4.2.0"
  },
  "devDependencies": {
    "@eshopman/test-utils": "2.14.2",
    "@eshopman/types": "2.14.2",
    "@swc/core": "^1.7.28",
    "@swc/jest": "^0.2.36",
    "@types/jest": "^29.5.13",
    "@types/node": "^20.12.11",
    "@types/qrcode": "^1.5.5",
    "@types/react": "^18.3.2",
    "@types/react-dom": "^18.2.25",
    "jest": "^29.7.0",
    "prop-types": "^15.8.1",
    "react": "^18.3.1",
    "react-dom": "^18.3.1",
    "ts-node": "^10.9.2",
    "typescript": "^5.6.2",
    "vite": "^5.4.14",
    "yalc": "^1.0.0-pre.53"
  },
  "engines": {
    "node": ">=20"
  },
  "packageManager": "pnpm@10.26.1"
}

Conclusion

This community insight highlights a crucial aspect of ESHOPMAN development: the careful management of database migrations for custom modules. By understanding the root cause of this db:generate behavior and adopting rigorous best practices, ESHOPMAN developers can safeguard their valuable data and ensure the stability of their HubSpot-integrated e-commerce solutions. The ESHOPMAN team and community are actively working towards a more robust and intuitive migration experience.

Protecting Your ESHOPMAN Database: Critical Migration Bug in Custom Modules