Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 31 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Test data setup is slow. Every `Model.create!` or `FactoryBot.create` hits the d

## The Solution

FixtureKit caches database records as raw SQL INSERT statements. On first use, it executes your fixture definition, captures the resulting database state, and generates optimized batch INSERT statements. Subsequent loads replay these statements directlyno ORM overhead, no callbacks, just fast SQL.
FixtureKit caches database records as raw SQL INSERT statements. It executes your fixture definition once, captures the resulting database state, and generates optimized batch INSERT statements. Fixture loads then replay these statements directly: no ORM overhead, no callbacks, just fast SQL.

Combined with RSpec's transactional fixtures, each test runs in a transaction that rolls back—so cached data can be reused across tests without cleanup.

Expand All @@ -26,7 +26,7 @@ end

### 1. Define a Fixture

Create fixture files in `spec/fixture_kit/`. Use whatever method you prefer to create records—FixtureKit doesn't care.
Create fixture files in `spec/fixture_kit/` (or `test/fixture_kit/` for test-unit/minitest-style setups). Use whatever method you prefer to create records.

**Using ActiveRecord directly:**

Expand Down Expand Up @@ -117,6 +117,8 @@ RSpec.configure do |config|
end
```

When you call `fixture "name"` in an example group, FixtureKit registers that fixture with its runner.

## Configuration

```ruby
Expand All @@ -128,27 +130,35 @@ FixtureKit.configure do |config|
# Where cache files are stored (default: tmp/cache/fixture_kit)
config.cache_path = Rails.root.join("tmp/cache/fixture_kit").to_s

# Whether to regenerate caches on every run (default: true)
config.autogenerate = true
# Wrapper used to isolate generation work (default: FixtureKit::TestCase::Isolator)
# config.isolator = FixtureKit::TestCase::Isolator
# config.isolator = FixtureKit::RSpec::Isolator

# Optional: customize how pregeneration is wrapped.
# Default is FixtureKit::TestCase::Generator.
# config.generator = FixtureKit::TestCase::Generator
# Optional callback, called once when a fixture cache is first generated.
# Receives the fixture name as a String.
# config.on_cache = ->(fixture_name) { puts "cached #{fixture_name}" }
end
```

Custom generators should subclass `FixtureKit::Generator` and implement `#run`.
`#run` receives the pregeneration block and should execute it in whatever lifecycle you need.
Custom isolators should subclass `FixtureKit::Isolator` and implement `#run`.
`#run` receives the generation block and should execute it in whatever lifecycle you need.

By default, FixtureKit uses `FixtureKit::TestCase::Isolator`, which runs generation inside an internal `ActiveSupport::TestCase` and removes that harness case from minitest runnables.

### Autogenerate
When using `fixture_kit/rspec`, FixtureKit sets `FixtureKit::RSpec::Isolator`. It runs generation inside an internal RSpec example, and uses a null reporter so harness runs do not count toward suite example totals.

When `autogenerate` is `true` (the default), FixtureKit clears all caches at the start of each test run, then regenerates them on first use. Subsequent tests that use the same fixture reuse the cache from earlier in the run. This ensures your test data always matches your fixture definitions.
## Lifecycle

When `autogenerate` is `false`, FixtureKit pre-generates all fixture caches at suite start. This runs through the configured `generator`, and still rolls back database changes.
Fixture generation is managed by `FixtureKit::Runner`.

By default, FixtureKit uses `FixtureKit::TestCase::Generator`, which runs pregeneration inside an internal `ActiveSupport::TestCase` so setup/teardown hooks and transactional fixture behavior run as expected. The internal test case is removed from Minitest runnables, so it does not count toward suite totals.
With `fixture_kit/rspec`:

When using `fixture_kit/rspec`, FixtureKit sets `FixtureKit::RSpec::Generator` as the generator. This runs pregeneration inside an internal RSpec example so your normal `before`/`around`/`after` hooks apply. The internal example uses a null reporter, so it does not count toward suite example totals.
1. `fixture "name"` registers the fixture with the runner during spec file load.
2. In `before(:suite)`, runner `start`:
- clears `cache_path` (unless preserve-cache is enabled),
- generates caches for all already-registered fixtures.
3. If new spec files are loaded later (for example, queue-mode CI runners), newly registered fixtures are generated immediately because the runner has already started.
4. At example runtime, fixture mounting loads from cache.

### Preserving Cache Locally

Expand All @@ -158,19 +168,9 @@ If you want to skip cache clearing at suite start (e.g., to reuse caches across
FIXTURE_KIT_PRESERVE_CACHE=1 bundle exec rspec
```

This is useful when you're iterating on tests and your fixture definitions haven't changed.

### CI Setup

For CI, set `autogenerate` to `false`. FixtureKit will automatically generate any missing caches at suite start:
Truthy values are case-insensitive: `1`, `true`, `yes`.

```ruby
FixtureKit.configure do |config|
config.autogenerate = !ENV["CI"]
end
```

This means CI "just works" - no need to pre-generate caches or commit them to the repository. The first test run will generate all caches, and subsequent runs (if caches are preserved between builds) will reuse them.
This is useful when you're iterating on tests and your fixture definitions haven't changed.

## Nested Fixtures

Expand All @@ -189,11 +189,11 @@ fixture "teams/sales"

## How It Works

1. **First load (cache miss)**: FixtureKit executes your definition block, subscribes to `sql.active_record` notifications to track which tables received INSERTs, queries all records from those tables, and generates batch INSERT statements with conflict handling (`INSERT OR IGNORE` for SQLite, `ON CONFLICT DO NOTHING` for PostgreSQL, `INSERT IGNORE` for MySQL).
1. **Cache generation**: FixtureKit executes your definition block inside the configured isolator, subscribes to `sql.active_record` notifications to track inserted models, queries those model tables, and generates batch INSERT statements with conflict handling (`INSERT OR IGNORE` for SQLite, `ON CONFLICT DO NOTHING` for PostgreSQL, `INSERT IGNORE` for MySQL).

2. **Subsequent loads (cache hit)**: FixtureKit loads the cached JSON file and executes the raw SQL INSERT statements directly. No ORM instantiation, no callbacks—just fast SQL execution.
2. **Mounting**: FixtureKit loads the cached JSON file and executes the raw SQL INSERT statements directly. No ORM instantiation, no callbacks.

3. **In-memory caching**: Once a cache file is parsed, the data is stored in memory. Multiple tests using the same fixture within a single test run don't re-read or re-parse the JSON file.
3. **Repository build**: FixtureKit resolves exposed records by model + id and returns a `Repository` for method-based access.

4. **Transaction isolation**: RSpec's `use_transactional_fixtures` wraps each test in a transaction that rolls back, so data doesn't persist between tests.

Expand All @@ -216,7 +216,7 @@ Caches are stored as JSON files in `tmp/cache/fixture_kit/`:
```

- **records**: Maps model names to their INSERT statements. Using model names (not table names) allows FixtureKit to use the correct database connection for multi-database setups.
- **exposed**: Maps fixture accessor names to their model class and ID for querying after cache replay
- **exposed**: Maps fixture accessor names to their model class and ID for querying after cache replay.

## Cache Management

Expand All @@ -225,7 +225,7 @@ Delete the cache directory to force regeneration:
rm -rf tmp/cache/fixture_kit
```

Caches are automatically cleared at suite start when `autogenerate` is enabled, so manual clearing is rarely needed.
Caches are cleared at runner start unless `FIXTURE_KIT_PRESERVE_CACHE` is truthy.

## Multi-Database Support

Expand Down
7 changes: 4 additions & 3 deletions lib/fixture_kit.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,19 @@ class CacheMissingError < Error; end
class PregenerationError < Error; end
class FixtureDefinitionNotFound < Error; end
class ExposedRecordNotFound < Error; end
class RunnerAlreadyStartedError < Error; end

autoload :VERSION, File.expand_path("fixture_kit/version", __dir__)
autoload :Configuration, File.expand_path("fixture_kit/configuration", __dir__)
autoload :Singleton, File.expand_path("fixture_kit/singleton", __dir__)
autoload :Fixture, File.expand_path("fixture_kit/fixture", __dir__)
autoload :DefinitionContext, File.expand_path("fixture_kit/definition_context", __dir__)
autoload :Definition, File.expand_path("fixture_kit/definition", __dir__)
autoload :Registry, File.expand_path("fixture_kit/registry", __dir__)
autoload :Repository, File.expand_path("fixture_kit/repository", __dir__)
autoload :SqlCapture, File.expand_path("fixture_kit/sql_capture", __dir__)
autoload :SqlSubscriber, File.expand_path("fixture_kit/sql_subscriber", __dir__)
autoload :Cache, File.expand_path("fixture_kit/cache", __dir__)
autoload :Runner, File.expand_path("fixture_kit/runner", __dir__)
autoload :Generator, File.expand_path("fixture_kit/generator", __dir__)
autoload :Isolator, File.expand_path("fixture_kit/isolator", __dir__)
autoload :TestCase, File.expand_path("fixture_kit/test_case", __dir__)

extend Singleton
Expand Down
131 changes: 42 additions & 89 deletions lib/fixture_kit/cache.rb
Original file line number Diff line number Diff line change
Expand Up @@ -7,109 +7,54 @@

module FixtureKit
class Cache
# In-memory cache to avoid re-reading/parsing JSON for every test
@memory_cache = {}
attr_reader :fixture

class << self
attr_accessor :memory_cache

def clear_memory_cache(fixture_name = nil)
if fixture_name
@memory_cache.delete(fixture_name)
else
@memory_cache.clear
end
end

# Clear fixture cache (both memory and disk)
def clear(fixture_name = nil)
clear_memory_cache(fixture_name)

cache_path = FixtureKit.configuration.cache_path
if fixture_name
cache_file = File.join(cache_path, "#{fixture_name}.json")
FileUtils.rm_f(cache_file)
else
FileUtils.rm_rf(cache_path)
end
end

# Generate caches for all fixtures.
# Each fixture is generated in a transaction that rolls back, so no data persists.
def generate_all
Registry.load_definitions
Registry.fixtures.each { |fixture| generate(fixture.name) }
end

def generate(fixture_name)
clear(fixture_name)

FixtureKit.configuration.generator.run do
Runner.run(fixture_name, force: true)
end
end
def initialize(fixture, definition)
@fixture = fixture
@definition = definition
end

attr_reader :records, :exposed

def initialize(fixture_name)
@fixture_name = fixture_name
@records = {}
@exposed = {}
end

def cache_file_path
cache_path = FixtureKit.configuration.cache_path
File.join(cache_path, "#{@fixture_name}.json")
def path
File.join(FixtureKit.configuration.cache_path, "#{fixture.name}.json")
end

def exists?
# Check in-memory cache first, then disk
self.class.memory_cache.key?(@fixture_name) || File.exist?(cache_file_path)
@data || File.exist?(path)
end

def load
# Check in-memory cache first
if self.class.memory_cache.key?(@fixture_name)
data = self.class.memory_cache[@fixture_name]
@records = data.fetch("records")
@exposed = data.fetch("exposed")
return true
unless exists?
raise FixtureKit::CacheMissingError, "Cache does not exist for fixture '#{fixture.name}'"
end

# Fall back to disk
return false unless File.exist?(cache_file_path)

data = JSON.parse(File.read(cache_file_path))
@records = data.fetch("records")
@exposed = data.fetch("exposed")

# Store in memory for subsequent loads
self.class.memory_cache[@fixture_name] = data
@data ||= JSON.parse(File.read(path))
@data.fetch("records").each do |model_name, sql|
model = ActiveSupport::Inflector.constantize(model_name)
model.connection.execute(sql)
end

true
build_repository(@data.fetch("exposed"))
end

def save(models_with_connections:, exposed_mapping:)
@records = generate_statements(models_with_connections)
@exposed = exposed_mapping

FileUtils.mkdir_p(File.dirname(cache_file_path))

data = {
"records" => @records,
"exposed" => @exposed
}
def save
FixtureKit.configuration.isolator.run do
models = SqlSubscriber.capture do
@definition.evaluate
end

# Store in memory cache
self.class.memory_cache[@fixture_name] = data
@data = {
"records" => generate_statements(models),
"exposed" => build_exposed_mapping(@definition.exposed)
}
end

File.write(cache_file_path, JSON.pretty_generate(data))
FileUtils.mkdir_p(File.dirname(path))
File.write(path, JSON.pretty_generate(@data))
end

# Query exposed records from the database and return a Repository.
def build_repository
exposed_records = @exposed.each_with_object({}) do |(name, value), hash|
def build_repository(exposed)
exposed_records = exposed.each_with_object({}) do |(name, value), hash|
was_array = value.is_a?(Array)
records = Array.wrap(value).map { |record_info| find_exposed_record(record_info.fetch("model"), record_info.fetch("id"), name) }
hash[name.to_sym] = was_array ? records : records.first
Expand All @@ -125,27 +70,27 @@ def find_exposed_record(model_name, id, exposed_name)
model.find(id)
rescue ActiveRecord::RecordNotFound
raise FixtureKit::ExposedRecordNotFound,
"Could not find #{model_name} with id=#{id} for exposed record '#{exposed_name}' in fixture '#{@fixture_name}'"
"Could not find #{model_name} with id=#{id} for exposed record '#{exposed_name}' in fixture '#{@fixture.name}'"
end

def generate_statements(models_with_connections)
def generate_statements(models)
statements_by_model = {}

models_with_connections.each do |model, connection|
models.each do |model|
columns = model.column_names

rows = []
model.order(:id).find_each do |record|
row_values = columns.map do |col|
value = record.read_attribute_before_type_cast(col)
connection.quote(value)
model.connection.quote(value)
end
rows << "(#{row_values.join(", ")})"
end

next if rows.empty?

sql = build_insert_sql(model.table_name, columns, rows, connection)
sql = build_insert_sql(model.table_name, columns, rows, model.connection)
statements_by_model[model.name] = sql
end

Expand Down Expand Up @@ -175,5 +120,13 @@ def add_conflict_handling(sql, connection)
sql
end
end

def build_exposed_mapping(exposed)
exposed.each_with_object({}) do |(name, value), hash|
was_array = value.is_a?(Array)
records = Array.wrap(value).map { |record| { "model" => record.class.name, "id" => record.id } }
hash[name] = was_array ? records : records.first
end
end
end
end
8 changes: 4 additions & 4 deletions lib/fixture_kit/configuration.rb
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ module FixtureKit
class Configuration
attr_writer :fixture_path
attr_writer :cache_path
attr_accessor :generator
attr_accessor :autogenerate
attr_accessor :isolator
attr_accessor :on_cache

def initialize
@fixture_path = nil
@cache_path = nil
@generator = FixtureKit::TestCase::Generator
@autogenerate = true
@isolator = FixtureKit::TestCase::Isolator
@on_cache = nil
end

def fixture_path
Expand Down
Original file line number Diff line number Diff line change
@@ -1,23 +1,22 @@
# frozen_string_literal: true

module FixtureKit
class DefinitionContext
class Definition
attr_reader :exposed

def initialize
def initialize(&definition)
@definition = definition
@exposed = {}
end

def evaluate
instance_eval(&@definition)
end

def expose(**records)
records.each do |name, record|
name = name.to_sym

if @exposed.key?(name)
raise FixtureKit::DuplicateNameError, <<~ERROR
Duplicate expose name :#{name}

A record with this name has already been exposed in this fixture.
ERROR
raise FixtureKit::DuplicateNameError, "Name #{name} already exposed"
end

@exposed[name] = record
Expand Down
Loading