Skip to content

Commit ab8dcc5

Browse files
bsboddenbsbodden
committed
docs: add ephemeral indexes and index migration documentation
Add new documentation pages for dynamic indexing features: - ephemeral-indexes.adoc: Covers EphemeralIndexService for creating temporary indexes with TTL, including use cases for testing, analytics, and temporary data processing - index-migration.adoc: Covers IndexMigrationService with Blue-Green, Dual-Write, and In-Place migration strategies for zero-downtime index updates Also update multi-tenant-support.adoc to clarify that keyPrefix in @IndexingOptions affects only index configuration, not actual key storage by Spring Data Redis. Add navigation entries and update outline.md.
1 parent ee5cb39 commit ab8dcc5

File tree

6 files changed

+906
-6
lines changed

6 files changed

+906
-6
lines changed

‎docs/content/modules/ROOT/nav.adoc‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,8 @@
2828
* xref:index-annotations.adoc[Index Annotations]
2929
* xref:index-creation.adoc[Index Creation and Management]
3030
* xref:multi-tenant-support.adoc[Multi-Tenant Support]
31+
* xref:index-migration.adoc[Index Migration]
32+
* xref:ephemeral-indexes.adoc[Ephemeral Indexes]
3133
* xref:multilanguage.adoc[Multilanguage Support]
3234
3335
.Query Capabilities

‎docs/content/modules/ROOT/pages/ephemeral-indexes.adoc‎

Lines changed: 310 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,310 @@
1+
= Ephemeral Indexes
2+
:page-toclevels: 3
3+
:experimental:
4+
:source-highlighter: highlight.js
5+
6+
== Introduction to Ephemeral Indexes
7+
8+
Ephemeral indexes are temporary search indexes that automatically expire after a specified time-to-live (TTL). They are ideal for:
9+
10+
* **Batch processing**: Create temporary indexes for one-time data imports or exports
11+
* **Analytics workloads**: Build indexes for temporary analytical queries that don't need to persist
12+
* **Testing and development**: Create indexes for testing without leaving behind artifacts
13+
* **Temporary features**: Support time-limited features or campaigns
14+
15+
Redis OM Spring provides the `EphemeralIndexService` component to manage ephemeral indexes with automatic TTL-based cleanup.
16+
17+
== EphemeralIndexService
18+
19+
The `EphemeralIndexService` is a Spring-managed component that handles the creation, tracking, and automatic deletion of ephemeral indexes.
20+
21+
=== Basic Usage
22+
23+
Inject the service and create an ephemeral index:
24+
25+
[source,java]
26+
----
27+
@Service
28+
public class AnalyticsService {
29+
30+
@Autowired
31+
private EphemeralIndexService ephemeralIndexService;
32+
33+
public void runBatchAnalytics() {
34+
// Create an ephemeral index that expires in 30 minutes
35+
boolean created = ephemeralIndexService.createEphemeralIndex(
36+
Product.class,
37+
"temp_analytics_idx",
38+
Duration.ofMinutes(30)
39+
);
40+
41+
if (created) {
42+
// Use the temporary index for analytics queries
43+
performAnalyticsQueries();
44+
}
45+
}
46+
}
47+
----
48+
49+
=== Creating Ephemeral Indexes
50+
51+
The `createEphemeralIndex` method creates a temporary index with automatic cleanup:
52+
53+
[source,java]
54+
----
55+
@Autowired
56+
private EphemeralIndexService ephemeralIndexService;
57+
58+
public void createTemporaryIndex() {
59+
// Parameters:
60+
// 1. Entity class - defines the index schema
61+
// 2. Index name - unique name for this ephemeral index
62+
// 3. TTL - how long the index should exist
63+
64+
boolean success = ephemeralIndexService.createEphemeralIndex(
65+
Product.class,
66+
"temp_demo_idx",
67+
Duration.ofSeconds(60) // Auto-delete after 60 seconds
68+
);
69+
70+
if (success) {
71+
log.info("Ephemeral index created successfully");
72+
}
73+
}
74+
----
75+
76+
The service:
77+
78+
1. Creates the index using the entity class schema
79+
2. Schedules automatic deletion after the TTL expires
80+
3. Tracks the index in an internal registry
81+
4. Generates a key prefix in the format `{entityname}:ephemeral:`
82+
83+
=== Extending TTL
84+
85+
If you need more time, extend the TTL of an existing ephemeral index:
86+
87+
[source,java]
88+
----
89+
// Extend the TTL by another hour
90+
boolean extended = ephemeralIndexService.extendTTL(
91+
"temp_analytics_idx",
92+
Duration.ofHours(1)
93+
);
94+
95+
if (extended) {
96+
log.info("TTL extended successfully");
97+
} else {
98+
log.warn("Index not found or not ephemeral");
99+
}
100+
----
101+
102+
This cancels the current deletion schedule and creates a new one with the extended TTL.
103+
104+
=== Checking Ephemeral Status
105+
106+
Verify if an index is tracked as ephemeral:
107+
108+
[source,java]
109+
----
110+
boolean isEphemeral = ephemeralIndexService.isEphemeralIndex("temp_analytics_idx");
111+
112+
if (isEphemeral) {
113+
log.info("This is a temporary index");
114+
} else {
115+
log.info("This is a permanent index");
116+
}
117+
----
118+
119+
== Complete Example
120+
121+
Here's a complete example from the multi-tenant demo application showing ephemeral indexes in action:
122+
123+
[source,java]
124+
----
125+
@Component
126+
public class DemoRunner implements CommandLineRunner {
127+
128+
@Autowired(required = false)
129+
private EphemeralIndexService ephemeralIndexService;
130+
131+
@Override
132+
public void run(String... args) throws Exception {
133+
if (ephemeralIndexService != null) {
134+
demoEphemeralIndexes();
135+
}
136+
}
137+
138+
private void demoEphemeralIndexes() {
139+
log.info("Creating temporary indexes for batch processing...");
140+
141+
// Create an ephemeral index for demo
142+
String ephemeralIndexName = "temp_demo_idx";
143+
boolean created = ephemeralIndexService.createEphemeralIndex(
144+
Product.class,
145+
ephemeralIndexName,
146+
Duration.ofSeconds(60)
147+
);
148+
149+
if (created) {
150+
log.info("Created ephemeral index: {}", ephemeralIndexName);
151+
log.info("TTL: 60 seconds (will auto-delete)");
152+
log.info("Is ephemeral: {}",
153+
ephemeralIndexService.isEphemeralIndex(ephemeralIndexName));
154+
}
155+
}
156+
}
157+
----
158+
159+
== Use Cases
160+
161+
=== Batch Import Processing
162+
163+
Create temporary indexes for large data imports:
164+
165+
[source,java]
166+
----
167+
@Service
168+
public class DataImportService {
169+
170+
@Autowired
171+
private EphemeralIndexService ephemeralIndexService;
172+
173+
@Autowired
174+
private RediSearchIndexer indexer;
175+
176+
public void importProducts(List<Product> products, String batchId) {
177+
String tempIndexName = "import_" + batchId + "_idx";
178+
179+
// Create ephemeral index for this batch
180+
ephemeralIndexService.createEphemeralIndex(
181+
Product.class,
182+
tempIndexName,
183+
Duration.ofHours(2) // Keep for validation
184+
);
185+
186+
try {
187+
// Import products to the temporary index
188+
for (Product product : products) {
189+
// Process and validate...
190+
}
191+
192+
// If validation passes, migrate to permanent index
193+
migrateToProduction(tempIndexName);
194+
} catch (Exception e) {
195+
log.error("Import failed, ephemeral index will auto-cleanup");
196+
}
197+
}
198+
}
199+
----
200+
201+
=== Time-Limited Search Features
202+
203+
Support temporary promotional searches:
204+
205+
[source,java]
206+
----
207+
@Service
208+
public class PromotionService {
209+
210+
@Autowired
211+
private EphemeralIndexService ephemeralIndexService;
212+
213+
public void createBlackFridayIndex(Duration promotionDuration) {
214+
// Create a special promotional index
215+
boolean created = ephemeralIndexService.createEphemeralIndex(
216+
Product.class,
217+
"black_friday_deals_idx",
218+
promotionDuration
219+
);
220+
221+
if (created) {
222+
log.info("Black Friday index created, expires in: {}", promotionDuration);
223+
}
224+
}
225+
}
226+
----
227+
228+
=== Analytics with Automatic Cleanup
229+
230+
Run analytics queries without permanent storage:
231+
232+
[source,java]
233+
----
234+
@Service
235+
public class ReportingService {
236+
237+
@Autowired
238+
private EphemeralIndexService ephemeralIndexService;
239+
240+
public Report generateDailyReport() {
241+
String reportIndexName = "daily_report_" + LocalDate.now() + "_idx";
242+
243+
// Create index just for this report
244+
ephemeralIndexService.createEphemeralIndex(
245+
Transaction.class,
246+
reportIndexName,
247+
Duration.ofMinutes(30) // Cleanup after report generation
248+
);
249+
250+
try {
251+
return runReportQueries(reportIndexName);
252+
} finally {
253+
// Index will auto-cleanup, but we could also cleanup early if needed
254+
log.info("Report complete, index will auto-expire");
255+
}
256+
}
257+
}
258+
----
259+
260+
== Best Practices
261+
262+
=== TTL Selection
263+
264+
* **Short TTL (seconds to minutes)**: Testing, quick batch jobs
265+
* **Medium TTL (minutes to hours)**: Analytics, report generation
266+
* **Longer TTL (hours to days)**: Data validation, migration staging
267+
268+
=== Naming Conventions
269+
270+
Use descriptive names that indicate the temporary nature:
271+
272+
[source,java]
273+
----
274+
// Good naming patterns
275+
"temp_analytics_" + batchId + "_idx"
276+
"import_" + timestamp + "_idx"
277+
"staging_" + version + "_idx"
278+
"test_" + testName + "_idx"
279+
----
280+
281+
=== Error Handling
282+
283+
Always check the return value of `createEphemeralIndex`:
284+
285+
[source,java]
286+
----
287+
boolean created = ephemeralIndexService.createEphemeralIndex(
288+
entityClass, indexName, ttl
289+
);
290+
291+
if (!created) {
292+
log.error("Failed to create ephemeral index: {}", indexName);
293+
// Handle failure appropriately
294+
throw new IndexCreationException("Could not create temporary index");
295+
}
296+
----
297+
298+
=== Resource Management
299+
300+
The `EphemeralIndexService` implements `DisposableBean` and automatically:
301+
302+
* Cancels all scheduled deletion tasks on shutdown
303+
* Clears the ephemeral index tracking registry
304+
* Gracefully shuts down the scheduler
305+
306+
== Next Steps
307+
308+
* xref:index-migration.adoc[Index Migration]
309+
* xref:multi-tenant-support.adoc[Multi-Tenant Support]
310+
* xref:index-creation.adoc[Index Creation and Management]

‎docs/content/modules/ROOT/pages/index-creation.adoc‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -385,6 +385,9 @@ FT.DROPINDEX product-idx
385385

386386
== Next Steps
387387

388+
* xref:multi-tenant-support.adoc[Multi-Tenant Support]
389+
* xref:index-migration.adoc[Index Migration]
390+
* xref:ephemeral-indexes.adoc[Ephemeral Indexes]
388391
* xref:search.adoc[Redis Query Engine Integration]
389392
* xref:entity-streams.adoc[Entity Streams]
390393
* xref:vector-search.adoc[Vector Similarity Search]

0 commit comments

Comments
 (0)