Par Michael Pichat, William Pogrund, Paloma Pichat, Armanouche Gasparian, Samuel Demarchi, Martin Corbet, Alois Georgeon et Michael Veillet-Guillem.
Article publié au sein de la revue en ligne Arxiv (Cornell University) le 17 mars 2025.
Abstract
This study investigates the ability of perceptron-type neurons in language models to perform intra-neuronal attention ; that is, to identify different homogeneous categorical segments within the synthetic thought category they encode, based on a segmentation of specific activation zones for the tokens to which they are particularly responsive. The objective of this work is therefore to determine to what extent formal neurons can establish a homomorphic relationship between activation-based and categorical segmentations. The results suggest the existence of such a relationship, albeit tenuous, only at the level of tokens with very high activation levels. This intra-neuronal attention subsequently enables categorical restructuring processes at the level of neurons in the following layer, thereby contributing to the progressive formation of high-level categorical abstractions.
DOI : https://doi.org/10.48550/arXiv.2503.12992
Michael Pichat est MCF en psychologie, fondateur du Cabinet Chrysippe-R&D et de Neocognition, co-directeur Diplômes « Management & Coaching » Université Paris Dauphine & Cabinet Chrysippe-R&D et membre titulaire de l’ER IPC.
