feat(drizzle): add soft property to select fields to avoid database enum limitations

[stevenceuppens]

What?

Adds a soft property to select fields that stores values as text instead of database enums, solving PostgreSQL enum limitations for large option sets while providing benefits across all supported databases.

Why?

PostgreSQL: When using select fields with large option sets (e.g., country lists with 200+ options), PostgreSQL enum creation fails during
migrations due to database limitations. This forces developers to either:

• Use workarounds like custom text fields with select UI components
• Limit their option sets artificially
• Face deployment failures in production

SQLite: Enum constraints add validation overhead at the database level

MongoDB: No impact (already stores as text in documents, but harmless)

The soft property provides a clean, built-in solution that maintains the full select field experience while optimizing storage across database types.

How?

Implementation:

• Added soft?: boolean property to SelectField type definition
• Modified Drizzle schema generation to store soft select values as varchar/text instead of enum
• Supports both single and hasMany soft select fields
• Maintains identical UI/UX - users see no difference in the admin interface
• Preserves all existing select field features (filterOptions, validation, etc.)

Database-Specific Benefits:

• PostgreSQL: ⭐⭐⭐⭐⭐ Solves critical enum limitations for large option sets
• SQLite: ⭐⭐ Minor performance optimization by removing enum constraints
• MongoDB: ⭐ No change (already stores as text, but harmless)

Usage:

// Before: Can cause PostgreSQL enum limitations
{
  name: 'country',
  type: 'select',
  options: countries, // 200+ countries may cause migration issues
}

// After: Uses soft select to optimize across all databases
{
  name: 'country',
  type: 'select',
  soft: true,        // 🎉 NEW PROPERTY
  options: countries, // Stored as text, no enum limitations
}

Code Changes:

• Types: Added soft property to SelectField and SelectFieldClient interfaces
• Schema: Modified traverseFields.ts to generate text columns for soft selects
• Tests: Added comprehensive e2e tests for both single and multi-select soft fields
• Validation: No changes needed - existing validation works with both storage types

Benefits:
✅ Solves PostgreSQL enum limitations for large option sets
✅ Provides SQLite performance optimization
✅ Zero breaking changes - existing select fields continue working
✅ Identical UI/UX experience across all databases
✅ Clean, intuitive API design
✅ Full feature compatibility

Primary use case: Fixes deployment failures when using select fields with large option sets like countries, languages, or other extensive lists on PostgreSQL databases.

Secondary benefit: Provides minor performance improvements for SQLite databases and maintains compatibility across Payload's entire database ecosystem.

Key updates:

• Multi-database perspective: Explains impact on PostgreSQL, SQLite, and MongoDB
• Benefit hierarchy: Shows PostgreSQL as primary beneficiary, SQLite as secondary
• Universal solution: Emphasizes it works optimally across all databases
• Clear use cases: Primary (PostgreSQL fixes) and secondary (SQLite optimization)
• Maintains focus: Still solves your original problem while showing broader value

[stevenceuppens]

Example: Migration failure when schema contains multiple enum fields with large value sets
This demonstrates the issue that occurs when a Payload schema includes multiple enum fields containing large value sets (such as country codes). Countries are appropriately implemented as enums since they represent a stable, predefined list that rarely changes - using relations would be unnecessary overhead for static data.
The migration generator creates a single massive SQL statement that attempts to create all enum types and tables in one query, which exceeds database query size limits and causes deployment failures.

In this example:

• Multiple country enum types are created with 195+ country codes each
• The same country enum is duplicated across different collections (customers, locations, organizations)
• The resulting migration file contains hundreds of lines of SQL in a single query
• Deployment fails with "Failed query" error due to the enormous query size

[10:49:19.070] @ea/app-web:migrate: [16:49:19] [31mERROR[39m: [36mError running migration 20250712_164735 Failed query: 
[10:49:19.070] @ea/app-web:migrate:    CREATE TYPE "public"."_locales" AS ENUM('nl', 'en');
[10:49:19.070] @ea/app-web:migrate:   CREATE TYPE "public"."enum_customers_billing_type" AS ENUM('private', 'company');
[10:49:19.070] @ea/app-web:migrate:   CREATE TYPE "public"."enum_customers_address_country" AS ENUM('af', 'al', 'dz', 'ad', 'ao', 'ag', 'ar', 'am', 'au', 'at', 'az', 'bs', 'bh', 'bd', 'bb', 'by', 'be', 'bz', 'bj', 'bt', 'bo', 'ba', 'bw', 'br', 'bn', 'bg', 'bf', 'bi', 'cv', 'kh', 'cm', 'ca', 'cf', 'td', 'cl', 'cn', 'co', 'km', 'cg', 'cd', 'cr', 'hr', 'cu', 'cy', 'cz', 'dk', 'dj', 'dm', 'do', 'ec', 'eg', 'sv', 'gq', 'er', 'ee', 'sz', 'et', 'fj', 'fi', 'fr', 'ga', 'gm', 'ge', 'de', 'gh', 'gr', 'gd', 'gt', 'gn', 'gw', 'gy', 'ht', 'hn', 'hu', 'is', 'in', 'id', 'ir', 'iq', 'ie', 'il', 'it', 'ci', 'jm', 'jp', 'jo', 'kz', 'ke', 'ki', 'kp', 'kr', 'kw', 'kg', 'la', 'lv', 'lb', 'ls', 'lr', 'ly', 'li', 'lt', 'lu', 'mg', 'mw', 'my', 'mv', 'ml', 'mt', 'mh', 'mr', 'mu', 'mx', 'fm', 'md', 'mc', 'mn', 'me', 'ma', 'mz', 'mm', 'na', 'nr', 'np', 'nl', 'nz', 'ni', 'ne', 'ng', 'mk', 'no', 'om', 'pk', 'pw', 'ps', 'pa', 'pg', 'py', 'pe', 'ph', 'pl', 'pt', 'qa', 'ro', 'ru', 'rw', 'kn', 'lc', 'vc', 'ws', 'sm', 'st', 'sa', 'sn', 'rs', 'sc', 'sl', 'sg', 'sk', 'si', 'sb', 'so', 'za', 'ss', 'es', 'lk', 'sd', 'sr', 'se', 'ch', 'sy', 'tw', 'tj', 'tz', 'th', 'tl', 'tg', 'to', 'tt', 'tn', 'tr', 'tm', 'tv', 'ug', 'ua', 'ae', 'gb', 'us', 'uy', 'uz', 'vu', 'va', 've', 'vn', 'ye', 'zm', 'zw');
[10:49:19.070] @ea/app-web:migrate:   CREATE TYPE "public"."enum_locations_address_country" AS ENUM('af', 'al', 'dz', 'ad', 'ao', 'ag', 'ar', 'am', 'au', 'at', 'az', 'bs', 'bh', 'bd', 'bb', 'by', 'be', 'bz', 'bj', 'bt', 'bo', 'ba', 'bw', 'br', 'bn', 'bg', 'bf', 'bi', 'cv', 'kh', 'cm', 'ca', 'cf', 'td', 'cl', 'cn', 'co', 'km', 'cg', 'cd', 'cr', 'hr', 'cu', 'cy', 'cz', 'dk', 'dj', 'dm', 'do', 'ec', 'eg', 'sv', 'gq', 'er', 'ee', 'sz', 'et', 'fj', 'fi', 'fr', 'ga', 'gm', 'ge', 'de', 'gh', 'gr', 'gd', 'gt', 'gn', 'gw', 'gy', 'ht', 'hn', 'hu', 'is', 'in', 'id', 'ir', 'iq', 'ie', 'il', 'it', 'ci', 'jm', 'jp', 'jo', 'kz', 'ke', 'ki', 'kp', 'kr', 'kw', 'kg', 'la', 'lv', 'lb', 'ls', 'lr', 'ly', 'li', 'lt', 'lu', 'mg', 'mw', 'my', 'mv', 'ml', 'mt', 'mh', 'mr', 'mu', 'mx', 'fm', 'md', 'mc', 'mn', 'me', 'ma', 'mz', 'mm', 'na', 'nr', 'np', 'nl', 'nz', 'ni', 'ne', 'ng', 'mk', 'no', 'om', 'pk', 'pw', 'ps', 'pa', 'pg', 'py', 'pe', 'ph', 'pl', 'pt', 'qa', 'ro', 'ru', 'rw', 'kn', 'lc', 'vc', 'ws', 'sm', 'st', 'sa', 'sn', 'rs', 'sc', 'sl', 'sg', 'sk', 'si', 'sb', 'so', 'za', 'ss', 'es', 'lk', 'sd', 'sr', 'se', 'ch', 'sy', 'tw', 'tj', 'tz', 'th', 'tl', 'tg', 'to', 'tt', 'tn', 'tr', 'tm', 'tv', 'ug', 'ua', 'ae', 'gb', 'us', 'uy', 'uz', 'vu', 'va', 've', 'vn', 'ye', 'zm', 'zw');
[10:49:19.071] @ea/app-web:migrate:   CREATE TYPE "public"."enum_organizations_country" AS ENUM('af', 'al', 'dz', 'ad', 'ao', 'ag', 'ar', 'am', 'au', 'at', 'az', 'bs', 'bh', 'bd', 'bb', 'by', 'be', 'bz', 'bj', 'bt', 'bo', 'ba', 'bw', 'br', 'bn', 'bg', 'bf', 'bi', 'cv', 'kh', 'cm', 'ca', 'cf', 'td', 'cl', 'cn', 'co', 'km', 'cg', 'cd', 'cr', 'hr', 'cu', 'cy', 'cz', 'dk', 'dj', 'dm', 'do', 'ec', 'eg', 'sv', 'gq', 'er', 'ee', 'sz', 'et', 'fj', 'fi', 'fr', 'ga', 'gm', 'ge', 'de', 'gh', 'gr', 'gd', 'gt', 'gn', 'gw', 'gy', 'ht', 'hn', 'hu', 'is', 'in', 'id', 'ir', 'iq', 'ie', 'il', 'it', 'ci', 'jm', 'jp', 'jo', 'kz', 'ke', 'ki', 'kp', 'kr', 'kw', 'kg', 'la', 'lv', 'lb', 'ls', 'lr', 'ly', 'li', 'lt', 'lu', 'mg', 'mw', 'my', 'mv', 'ml', 'mt', 'mh', 'mr', 'mu', 'mx', 'fm', 'md', 'mc', 'mn', 'me', 'ma', 'mz', 'mm', 'na', 'nr', 'np', 'nl', 'nz', 'ni', 'ne', 'ng', 'mk', 'no', 'om', 'pk', 'pw', 'ps', 'pa', 'pg', 'py', 'pe', 'ph', 'pl', 'pt', 'qa', 'ro', 'ru', 'rw', 'kn', 'lc', 'vc', 'ws', 'sm', 'st', 'sa', 'sn', 'rs', 'sc', 'sl', 'sg', 'sk', 'si', 'sb', 'so', 'za', 'ss', 'es', 'lk', 'sd', 'sr', 'se', 'ch', 'sy', 'tw', 'tj', 'tz', 'th', 'tl', 'tg', 'to', 'tt', 'tn', 'tr', 'tm', 'tv', 'ug', 'ua', 'ae', 'gb', 'us', 'uy', 'uz', 'vu', 'va', 've', 'vn', 'ye', 'zm', 'zw');
   ... [hundreds more lines of CREATE TABLE statements] ...
[10:49:19.169] @ea/app-web:migrate:       "message": "Failed query: \n   CREATE TYPE \"public\".\"_locales\" AS ENUM('nl', 'en');\n  CREATE TYPE \"public\".\"enum_customers_billing_type\" AS ENUM('private', 'company');\n  CREATE TYPE \"public\".\"enum_customers_address_country\" AS ENUM('af', 'al', 'dz', 'ad', 'ao', 'ag', 'ar', 'am', 'au', 'at', 'az', 'bs', 'bh', 'bd', 'bb', 'by', 'be', 'bz', 'bj', 'bt', 'bo', 'ba', 'bw', 'br', 'bn', 'bg', 'bf', 'bi', 'cv', 'kh', 'cm', 'ca', 'cf', 'td', 'cl', 'cn', 'co', 'km', 'cg', 'cd', 'cr', 'hr', 'cu', 'cy', 'cz', 'dk', 'dj', 'dm', 'do', 'ec', 'eg', 'sv', 'gq', 'er', 'ee', 'sz', 'et', 'fj', 'fi', 'fr', 'ga', 'gm', 'ge', 'de', 'gh', 'gr', 'gd', 'gt', 'gn', 'gw', 'gy', 'ht', 'hn', 'hu', 'is', 'in', 'id', 'ir', 'iq', 'ie', 'il', 'it', 'ci', 'jm', 'jp', 'jo', 'kz', 'ke', 'ki', 'kp', 'kr', 'kw', 'kg', 'la', 'lv', 'lb', 'ls', 'lr', 'ly', 'li', 'lt', 'lu', 'mg', 'mw', 'my', 'mv', 'ml', 'mt', 'mh
[10:49:19.169] @ea/app-web:migrate: ERROR: command finished with error: command (/vercel/path0/packages/app-web) /yarn1/node_modules/yarn/bin/yarn run migrate exited (1)