C'est le seul et unique objectif de la campagne de destruction méthodique (à la fois effective et symbolique) de la retraite par répartition menée par la droite: faire en sorte de développer la retraite par capitalisation (qui est par ailleurs très mal nommée, ça sera toujours les actifs du futur qui paieront pour votre retraite, mais ils le feront sous forme de dividendes au lieu de cotisations).

La retraite est toujours une retraite par répartition quoi qu'il arrive, parce que ce sont les travailleurs qui créent les richesse que les retraités consomment, simplement on peut avoir une répartition socialisée ou une répartition sur la base du capital (avec bien plus de frais, mais plus inégalitaire ce qui est un vu comme un avantage pour les classes aisées).

L'antiparlementarisme primaire des rentiers du libéralisme …

via localllama

Abstract:

Advancements in distributed training and efficient attention mechanisms have significantly expanded the context window sizes of large language models (LLMs). However, recent work reveals that the effective context lengths of open-source LLMs often fall short, typically not exceeding half of their training lengths. In this work, we attribute this limitation to the left-skewed frequency distribution of relative positions formed in LLMs pretraining and post-training stages, which impedes their ability to effectively gather distant information. To address this challenge, we introduce ShifTed Rotray position embeddING (StRing). StRing shifts well-trained positions to overwrite the original ineffective positions during inference, enhancing performance within their existing training lengths. Experimental results show that without additional training, StRing dramatically improves the performance of the latest large-scale models, such as Llama3.1 70B and Qwen2 72B, by over 10 points on popular long-context benchmarks RULER and InfiniteBench, establishing new state-of-the-art results for open-source LLMs. Compared to commercial models, Llama 3.1 70B with StRing even achieves better performance than GPT-4-128K and clearly surpasses Claude 2 and Kimi-chat.

Conclusion:

This work uncovers the limitations of current open-source large language models in effectively utilizing their extended training context windows. We show that using positions at the tail of the left-skewed position frequency distributions strongly hinders models’ long-range dependency modeling ability. We introduce StRing, a novel approach that shifts well-trained positions to replace ineffective ones during inference, thereby enhancing the model’s ability to capture distant contextual information without requiring additional training. Our experiments demonstrate that StRing significantly boosts the performance of strong baselines like Llama 3.1 70B and Qwen-2 72B on prominent long-context benchmarks, setting new state-of-the-art results for open-source LLMs.

via localllama