compact: data corruption during downsapmle, test and fix.

CHANGELOG.md

@@ -70,6 +70,7 @@ We use *breaking :warning:* to mark changes that are not backward compatible (re
 - [#6592](https://github.com/thanos-io/thanos/pull/6592) Query Frontend: fix bugs in vertical sharding `without` and `union` function to allow more queries to be shardable.
 - [#6317](https://github.com/thanos-io/thanos/pull/6317) *: Fix internal label deduplication bug, by resorting store response set.
 - [#6189](https://github.com/thanos-io/thanos/pull/6189) Rule: Fix panic when calling API `/api/v1/rules?type=alert`.
+- [#6598](https://github.com/thanos-io/thanos/pull/6598) compact: fix data corruption with "invalid size" error during downsample
 
 ### Changed
 - [#6049](https://github.com/thanos-io/thanos/pull/6049) Compact: *breaking :warning:* Replace group with resolution in compact metrics to avoid cardinality explosion on compact metrics for large numbers of groups.

pkg/compact/downsample/downsample.go

@@ -372,8 +372,17 @@ func downsampleRawLoop(data []sample, resolution int64, numChunks int) []chunks.
 		for ; j < len(data) && data[j].t <= curW; j++ {
 		}
 
-		batch := data[:j]
+		batch := make([]sample, 0, j)
+		for _, s := range data[:j] {
+			if value.IsStaleNaN(s.v) || math.Float64bits(s.v) == value.NormalNaN || math.IsNaN(s.v) {
+				continue
+			}
+			batch = append(batch, s)
+		}
 		data = data[j:]
+		if len(batch) == 0 {
+			continue
+		}
 
 		ab := newAggrChunkBuilder()