Some questions about the experimental part of the article

[bingyang-lei]

Hi. First, thank you for your interesting work in the domain of long context!
I have some questions about the experimental part of the article. Specifically, is the experiments in Table 2 and Table 3.
Why in these two experiments, you only show the performance of LIFT and ICT method? I think a set of effects that use other previously proposed techniques for fine-tuning should be added, such as LoRA.

[LuckyGlass]

Thanks for your attention to our work!

We have noticed the lack of baselines in the current released version. We are continuing to improve LIFT and have added more baselines. To answer your question, we have conducted an ablation study comparing LoRA, PiSSA (a method improving LoRA with a better initialization), and our Gated Memory.

Task	LoRA	PiSSA	Gated Memory
ShortQA	43.31	42.03	47.51
LongQA	29.15	29.06	29.97

In addition, we have extended our main experiments to include more baselines, such as RAG-based approaches and memory-augmented LLMs. Below are the updated results:

LooGLE (ShortQA and LongQA)

Methods	ShortQA	LongQA	Comprehension & Reasoning	Multiple info retrieval	Computation	Timeline reorder
MemoryLLM	33.06	20.44	29.31	15.53	8.00	18.14
LlamaIndex(Llama-3)	41.93	21.07	33.00	17.11	12.00	9.77
LlamaIndex(Gemma-2)	42.95	22.98	31.03	18.16	11.00	21.86
ICL(Llama-3)	44.49	15.44	25.37	15.26	5.00	1.86
LIFT(Llama-3)	47.51	29.97	39.90	27.89	17.00	17.21
ICL(Gemma-2)	37.37	29.79	36.95	21.58	10.0	40.00
LIFT(Gemma-2)	50.33	31.24	40.39	27.11	12.00	30.23

LongBench

Methods	Musique	NarrativeQA	Qmsum	GovReport	PassageRetrievalEN
MemoryLLM	13.47	20.64	20.98	20.89	9.03
LlamaIndex(Llama-3)	18.23	17.21	21.95	28.03	59.38
ICL(Llama-3)	26.89	19.45	21.64	30.18	58.33
LIFT(Llama-3)	21.19	23.33	23.07	33.62	62.50

For details on the setup of MemoryLLM and LlamaIndex baselines, please refer to the updated version of our paper on OpenReview.