java.lang.IndexOutOfBoundsException when parsing PPI data and starting offline HR recording

[orestesgaolin]

Platform on which you observed the bug:

• Android
• iOS
• Other
• Platform is not relevant for this bug

Device on which you observed the bug:

• Polar OH1
• Polar Verity Sense
• Polar H10
• Polar H9
• Other [360]
• Device is not relevant for this bug

Describe the bug

There are 2 cases where IndexOutOfBoundsException happens

Case 1 - offline recording

When starting offline recording the following exception is sometimes thrown from this slice call:

polar-ble-sdk/sources/Android/android-communications/library/src/main/java/com/polar/androidcommunications/api/ble/model/offlinerecording/OfflineRecordingData.kt

Line 288 in 71191b5

val data = decryptedData.slice(offset until packetSize + offset)

          Caused by java.lang.Exception: java.lang.IndexOutOfBoundsException: toIndex (11021) is greater than size (425).
       at com.polar.sdk.impl.BDBleApiImpl.handleError(BDBleApiImpl.kt:3317)
       at com.polar.sdk.impl.BDBleApiImpl.access$handleError(BDBleApiImpl.kt)
       at com.polar.sdk.impl.BDBleApiImpl$getOfflineRecord$1$1$3.apply(BDBleApiImpl.kt:825)
       at com.polar.sdk.impl.BDBleApiImpl$getOfflineRecord$1$1$3.apply(BDBleApiImpl.kt:825)
       at io.reactivex.rxjava3.internal.operators.single.SingleResumeNext$ResumeMainSingleObserver.onError(SingleResumeNext.java:73)
       at io.reactivex.rxjava3.internal.operators.single.SingleMap$MapSingleObserver.onError(SingleMap.java:70)
       at io.reactivex.rxjava3.internal.operators.single.SingleMap$MapSingleObserver.onError(SingleMap.java:70)
       at io.reactivex.rxjava3.internal.operators.single.SingleMap$MapSingleObserver.onSuccess(SingleMap.java:61)
       at io.reactivex.rxjava3.internal.observers.ResumeSingleObserver.onSuccess(ResumeSingleObserver.java:46)
       at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.onSuccess(SingleSubscribeOn.java:68)
       at io.reactivex.rxjava3.internal.operators.single.SingleDoFinally$DoFinallyObserver.onSuccess(SingleDoFinally.java:73)
       at io.reactivex.rxjava3.internal.operators.single.SingleDoOnSubscribe$DoOnSubscribeSingleObserver.onSuccess(SingleDoOnSubscribe.java:77)
       at io.reactivex.rxjava3.internal.operators.single.SingleCreate$Emitter.onSuccess(SingleCreate.java:68)
       at com.polar.androidcommunications.api.ble.model.gatt.client.psftp.BlePsFtpClient.lambda$request$0(BlePsFtpClient.java:243)
       at io.reactivex.rxjava3.internal.operators.single.SingleCreate.subscribeActual(SingleCreate.java:40)
       at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4855)
       at io.reactivex.rxjava3.internal.operators.single.SingleDoOnSubscribe.subscribeActual(SingleDoOnSubscribe.java:41)
       at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4855)
       at io.reactivex.rxjava3.internal.operators.single.SingleDoFinally.subscribeActual(SingleDoFinally.java:44)
       at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4855)
       at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
       at io.reactivex.rxjava3.core.Scheduler$DisposeTask.run(Scheduler.java:644)
       at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:65)
       at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java)
       at java.util.concurrent.FutureTask.run(FutureTask.java:264)
       at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:307)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
       at java.lang.Thread.run(Thread.java:1012)

          Caused by java.lang.IndexOutOfBoundsException: toIndex (11021) is greater than size (425).
       at kotlin.collections.ArraysKt__ArraysJVMKt.copyOfRangeToIndexCheck(ArraysJVM.kt:49)
       at kotlin.collections.ArraysKt___ArraysJvmKt.copyOfRange(_ArraysJvm.kt:1465)
       at kotlin.collections.ArraysKt___ArraysKt.slice(_Arrays.kt:4441)
       at com.polar.androidcommunications.api.ble.model.offlinerecording.OfflineRecordingData$Companion.parseData-ARK9YDc(OfflineRecordingData.kt:288)
       at com.polar.androidcommunications.api.ble.model.offlinerecording.OfflineRecordingData$Companion.parseDataFromOfflineFile-ARK9YDc(OfflineRecordingData.kt:70)
       at com.polar.androidcommunications.api.ble.model.offlinerecording.OfflineRecordingData$Companion.parseDataFromOfflineFile-ARK9YDc$default(OfflineRecordingData.kt:49)
       at com.polar.sdk.impl.BDBleApiImpl$getOfflineRecord$1$1$1.apply(BDBleApiImpl.kt:796)
       at com.polar.sdk.impl.BDBleApiImpl$getOfflineRecord$1$1$1.apply(BDBleApiImpl.kt:794)
       at io.reactivex.rxjava3.internal.operators.single.SingleMap$MapSingleObserver.onSuccess(SingleMap.java:58)
       at io.reactivex.rxjava3.internal.observers.ResumeSingleObserver.onSuccess(ResumeSingleObserver.java:46)
       at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.onSuccess(SingleSubscribeOn.java:68)
       at io.reactivex.rxjava3.internal.operators.single.SingleDoFinally$DoFinallyObserver.onSuccess(SingleDoFinally.java:73)
       at io.reactivex.rxjava3.internal.operators.single.SingleDoOnSubscribe$DoOnSubscribeSingleObserver.onSuccess(SingleDoOnSubscribe.java:77)
       at io.reactivex.rxjava3.internal.operators.single.SingleCreate$Emitter.onSuccess(SingleCreate.java:68)
       at com.polar.androidcommunications.api.ble.model.gatt.client.psftp.BlePsFtpClient.lambda$request$0(BlePsFtpClient.java:243)
       at io.reactivex.rxjava3.internal.operators.single.SingleCreate.subscribeActual(SingleCreate.java:40)
       at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4855)
       at io.reactivex.rxjava3.internal.operators.single.SingleDoOnSubscribe.subscribeActual(SingleDoOnSubscribe.java:41)
       at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4855)
       at io.reactivex.rxjava3.internal.operators.single.SingleDoFinally.subscribeActual(SingleDoFinally.java:44)
       at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4855)
       at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
       at io.reactivex.rxjava3.core.Scheduler$DisposeTask.run(Scheduler.java:644)
       at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:65)
       at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java)
       at java.util.concurrent.FutureTask.run(FutureTask.java:264)
       at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:307)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
       at java.lang.Thread.run(Thread.java:1012)   

This exception seems to be irrecoverable due to following, and this means the PPI stream stops:

          Non-fatal Exception: rl.d: The exception was not handled due to missing onError handler in the subscribe() method call. Further reading: https://github.com/ReactiveX/RxJava/wiki/Error-Handling | java.lang.Exception: java.lang.IndexOutOfBoundsException: toIndex (11021) is greater than size (425).
       at io.reactivex.rxjava3.internal.functions.Functions$OnErrorMissingConsumer.accept(Functions.java:718)
       at io.reactivex.rxjava3.internal.functions.Functions$OnErrorMissingConsumer.accept(Functions.java:715)
       at io.reactivex.rxjava3.internal.observers.ConsumerSingleObserver.onError(ConsumerSingleObserver.java:46)
       at io.reactivex.rxjava3.internal.operators.single.SingleDoFinally$DoFinallyObserver.onError(SingleDoFinally.java:79)
       at io.reactivex.rxjava3.internal.observers.ResumeSingleObserver.onError(ResumeSingleObserver.java:51)
       at io.reactivex.rxjava3.internal.disposables.EmptyDisposable.error(EmptyDisposable.java:78)
       at io.reactivex.rxjava3.internal.operators.single.SingleError.subscribeActual(SingleError.java:41)
       at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4855)
       at io.reactivex.rxjava3.internal.operators.single.SingleResumeNext$ResumeMainSingleObserver.onError(SingleResumeNext.java:80)
       at io.reactivex.rxjava3.internal.operators.single.SingleMap$MapSingleObserver.onError(SingleMap.java:70)
       at io.reactivex.rxjava3.internal.operators.single.SingleMap$MapSingleObserver.onError(SingleMap.java:70)
       at io.reactivex.rxjava3.internal.operators.single.SingleMap$MapSingleObserver.onSuccess(SingleMap.java:61)
       at io.reactivex.rxjava3.internal.observers.ResumeSingleObserver.onSuccess(ResumeSingleObserver.java:46)
       at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.onSuccess(SingleSubscribeOn.java:68)
       at io.reactivex.rxjava3.internal.operators.single.SingleDoFinally$DoFinallyObserver.onSuccess(SingleDoFinally.java:73)
       at io.reactivex.rxjava3.internal.operators.single.SingleDoOnSubscribe$DoOnSubscribeSingleObserver.onSuccess(SingleDoOnSubscribe.java:77)
       at io.reactivex.rxjava3.internal.operators.single.SingleCreate$Emitter.onSuccess(SingleCreate.java:68)
       at com.polar.androidcommunications.api.ble.model.gatt.client.psftp.BlePsFtpClient.lambda$request$0(BlePsFtpClient.java:243)
       at io.reactivex.rxjava3.internal.operators.single.SingleCreate.subscribeActual(SingleCreate.java:40)
       at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4855)
       at io.reactivex.rxjava3.internal.operators.single.SingleDoOnSubscribe.subscribeActual(SingleDoOnSubscribe.java:41)
       at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4855)
       at io.reactivex.rxjava3.internal.operators.single.SingleDoFinally.subscribeActual(SingleDoFinally.java:44)
       at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4855)
       at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
       at io.reactivex.rxjava3.core.Scheduler$DisposeTask.run(Scheduler.java:644)
       at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:65)
       at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java)
       at java.util.concurrent.FutureTask.run(FutureTask.java:264)
       at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:307)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
       at java.lang.Thread.run(Thread.java:1012)

Case 2 - starting PPI stream

The exception is similar but has a different stack trace:

polar-ble-sdk/sources/Android/android-communications/library/src/main/java/com/polar/androidcommunications/common/ble/TypeUtils.kt

Line 37 in 71191b5

return convertArrayToSignedInt(data.copyOfRange(offset, offset + length))

          Non-fatal Exception: java.lang.IndexOutOfBoundsException: toIndex (8) is greater than size (6).
       at kotlin.collections.ArraysKt__ArraysJVMKt.copyOfRangeToIndexCheck(ArraysJVM.kt:49)
       at kotlin.collections.ArraysKt___ArraysJvmKt.copyOfRange(_ArraysJvm.kt:1465)
       at com.polar.androidcommunications.common.ble.TypeUtils.convertArrayToSignedInt(TypeUtils.kt:37)
       at com.polar.androidcommunications.api.ble.model.gatt.client.pmd.PmdSetting.parsePmdSettingsData(PmdSetting.kt:48)
       at com.polar.androidcommunications.api.ble.model.gatt.client.pmd.PmdSetting.updateSelectedFromStartResponse(PmdSetting.kt:58)
       at com.polar.androidcommunications.api.ble.model.gatt.client.pmd.BlePMDClient$startMeasurement$1.accept(BlePMDClient.kt:357)
       at com.polar.androidcommunications.api.ble.model.gatt.client.pmd.BlePMDClient$startMeasurement$1.accept(BlePMDClient.kt:357)
       at io.reactivex.rxjava3.internal.operators.single.SingleDoOnSuccess$DoOnSuccess.onSuccess(SingleDoOnSuccess.java:54)
       at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.onSuccess(SingleSubscribeOn.java:68)
       at io.reactivex.rxjava3.internal.operators.single.SingleCreate$Emitter.onSuccess(SingleCreate.java:68)
       at com.polar.androidcommunications.api.ble.model.gatt.client.pmd.BlePMDClient.sendControlPointCommand$lambda$16(BlePMDClient.kt:268)
       at io.reactivex.rxjava3.internal.operators.single.SingleCreate.subscribeActual(SingleCreate.java:40)
       at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4855)
       at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
       at io.reactivex.rxjava3.core.Scheduler$DisposeTask.run(Scheduler.java:644)
       at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:65)
       at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
       at java.lang.Thread.run(Thread.java:919)
        

How to Reproduce

In case of PPI streaming we just call following API on some phones:

if (ppiDisposable != null) {
  ppiDisposable?.dispose()
}

var initialTimestamp: Instant? = null
ppiDisposable =
    bluetoothForegroundService
        .startPpiStreaming(deviceId)
        .subscribe(
            { data ->
              Logger.info(
                  TAG,
                  "Received ${data.samples.size} PPI samples")
              lastReceivedTimestamp = Instant.now()

              val totalDuration =
                  data.samples.sumOf { it.ppi } + 50 // Offset by 50ms to account for latency
              if (initialTimestamp == null) {
                initialTimestamp = Instant.now().minusMillis(totalDuration.toLong())
              }
              val startTimestamp = initialTimestamp!!
              var accumulatedMs = 0L

              data.samples.forEach {
                val sampleTime = startTimestamp.plusMillis(accumulatedMs)

                var ppiData =
                    PpiData(
                        it.ppi,
                        sampleTime,
                        deviceId,
                        it.errorEstimate,
                        it.blockerBit,
                        it.hr,
                        it.skinContactStatus,
                    )

                // posting ppi samples to event channel
                Handler(Looper.getMainLooper()).post { eventSink.success(ppiData.toString()) }
                accumulatedMs += it.ppi.toLong()
              }

              // Set the initialTimestamp for the next batch
              initialTimestamp = startTimestamp.plusMillis(accumulatedMs)
            },
            { error ->
              Logger.error(
                 TAG,
                  "Error occurred when streaming PPI data",
                  error)
            })

Expected behavior

The slice operation should not fail and PPI stream should continue, and the offline recording should be able to start

[orestesgaolin]

Edit: updated to reflect 2 distinct cases of the IndexOutOfBoundsException

[pth-pe-gh]

FLOW-61198

[abbjetmus]

Hi @orestesgaolin!

Not the answer to your question but I'm using Polar Verity Sense myself with Flutter and the polar flutter wrapper. I wonder how are you starting and reading HR and PPI recordings offline, because I can't get it to work? Stuck with his issue: #528

[orestesgaolin]

I wish I could help you @abbjetmus, but we don't use offline PPI recordings, only HR.

When it comes to settings we just pass empty value:

private val hrRecordingSettings: PolarSensorSetting by lazy {
  PolarSensorSetting(mapOf())
}
// and later
polarBleApi.startOfflineRecording(deviceId, feature, hrRecordingSettings, null)

[abbjetmus]

I wish I could help you @abbjetmus, but we don't use offline PPI recordings, only HR.

When it comes to settings we just pass empty value:

private val hrRecordingSettings: PolarSensorSetting by lazy {
  PolarSensorSetting(mapOf())
}
// and later
polarBleApi.startOfflineRecording(deviceId, feature, hrRecordingSettings, null)

Thank you so much, this is actually what I was needing. Both HR and PPI are computed/derived data from PPG so should be the same approach. I never tried passing an empty map to PolarSensorSettings I assinged null to settings.

                    if (!offlineRecordingStatus.contains(PolarDataType.acc)) {
                      var settingsAcc =
                          await polar.requestOfflineRecordingSettings(
                              identifier.value, PolarDataType.acc);

                      await polar.startOfflineRecording(
                          identifier.value, PolarDataType.acc,
                          settings: settingsAcc);
                    }

                    if (!offlineRecordingStatus.contains(PolarDataType.ppi)) {
                      await polar.startOfflineRecording(
                        identifier.value,
                        PolarDataType.ppi,
                        settings: PolarSensorSetting(<PolarSettingType, int>{}),
                      );
                    }```