perf: parallelize S3 .index + .kfs downloads in processors

[klaudworks]

Problem

Both processors download .index and .kfs files sequentially. Most of the time is just spent idling waiting for s3 downloads.

Proposed Fix

Parallelize it 😅

[novatechflow]

Took a pass on this since it's stale a few month - the best fix is a bit different from just parallelizing the two downloads.

In both processor decoders, we're downloading .index and .kfs sequentially, but the decoder path is not actually using the parsed .index entries for decoding. It;s validating the index and then throwing that result away. So parallelizing those two GETs would reduce wall time a bit, but it would still keep an unnecessary S3 read in the hot path.

Better approach is to remove the .index download from the processor decode path entirely and keeps segment/index pairing validation in discovery, where it already belongs. That gives us a cleaner fix than parallel fetches for this specific path.

While tracing this, we also found a related storage-side issue: for PITR/restore, .index does matter, just not in the processor decoder. That bug will handled separately in the same branch by making the restore path truncate the final segment at the cutoff and regenerate a matching .index, instead of only restoring whole segments.

So for this issue, the outcome in my local tests are:

• processor hot path: stop downloading .index when decode does not use it
• discovery: still requires valid .kfs + .index pairing
• PITR/restore: keep .index as part of the exact recovery path where it is actually needed

I’ll link the PR once it’s up.