Bulk export fails when payload offloading is configured #3874
Description
Describe the bug
[8/11/22, 17:55:29:709 UTC] 00000054 FHIRPersisten E Unexpected error while processing token value records.
java.lang.NullPointerException
at java.base/java.util.zip.InflaterInputStream.<init>(Unknown Source)
at java.base/java.util.zip.GZIPInputStream.<init>(Unknown Source)
at java.base/java.util.zip.GZIPInputStream.<init>(Unknown Source)
at org.linuxforhealth.fhir.persistence.jdbc.dao.impl.FetchResourcePayloadsDAO.run(FetchResourcePayloadsDAO.java:132)
at org.linuxforhealth.fhir.persistence.jdbc.impl.FHIRPersistenceJDBCImpl.fetchResourcePayloads(FHIRPersistenceJDBCImpl.java:3168)
at org.linuxforhealth.fhir.bulkdata.jbatch.export.fast.ResourcePayloadReader.readItem(ResourcePayloadReader.java:316)
at com.ibm.jbatch.container.artifact.proxy.ItemReaderProxy.readItem(ItemReaderProxy.java:77)
at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.readItem(ChunkStepControllerImpl.java:329)
at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.readAndProcess(ChunkStepControllerImpl.java:245)
at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.invokeChunk(ChunkStepControllerImpl.java:626)
at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.invokeCoreStep(ChunkStepControllerImpl.java:795)
at com.ibm.jbatch.container.controller.impl.BaseStepControllerImpl.execute(BaseStepControllerImpl.java:295)
at com.ibm.jbatch.container.controller.impl.ExecutionTransitioner.doExecutionLoop(ExecutionTransitioner.java:118)
at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.executeCoreTransitionLoop(WorkUnitThreadControllerImpl.java:96)
at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.executeWorkUnit(WorkUnitThreadControllerImpl.java:178)
at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl$AbstractControllerHelper.runExecutionOnThread(WorkUnitThreadControllerImpl.java:503)
at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.runExecutionOnThread(WorkUnitThreadControllerImpl.java:92)
at com.ibm.jbatch.container.util.BatchWorkUnit.run(BatchWorkUnit.java:113)
at com.ibm.ws.context.service.serializable.ContextualRunnable.run(ContextualRunnable.java:79)
at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:245)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
When I tried configuring it locally, I got this one (which makes more sense to me since fhir-persistence-blob is not a dependency of our fhir-bulkdata-webapp.
An error occurred while trying to load FHIR persistence factory class 'org.linuxforhealth.fhir.persistence.blob.FHIRPersistenceJDBCBlobFactory': java.lang.ClassNotFoundException: org.linuxforhealth.fhir.persistence.blob.FHIRPersistenceJDBCBlobFactory
Environment
main
To Reproduce
Steps to reproduce the behavior:
- start a fhir server that is configured with payload offloading (e.g. via
build/pre-integration-test-docker.sh azurite
) - add a resource
- invoke export
- note that the export fails and check the server log
Expected behavior
Either:
A. Bulk export should work irregardless of the payload offload configuration;
B. this should be clearly documented as a limitation of the current payload offload documentation
Additional context
At the very least, I think it should be possible to fall back to our legacy "search-based" export implementation.