Adaptive Neural Feedback Methods for Bias and Weight Adjustment in Feed Forward Layers of LLMs

Abstract

Feed-forward layers constitute the dominant computational and parametric component of transformer-based Large Language Models (LLMs), yet they are a major source of training instability due to static bias terms, uncontrolled weight scaling, and activation distribution drift. Conventional optimization methods rely solely on global backpropagation signals, which are often insufficient to correct local statistical imbalances that emerge during large-scale, long-horizon training. This work proposes AFB-FFN (Adaptive Feedback Bias and Weight Corrected Feed-Forward Network), a novel feed-forward layer architecture that integrates an internal neural feedback mechanism to dynamically regulate bias and weight behavior during forward propagation. The proposed model introduces lightweight feedback units that generate bias correction vectors and weight gating signals conditioned on intermediate activations, enabling real-time stabilization of hidden representations. The AFB-FFN architecture is embedded within a transformer framework and evaluated on a token-level language modeling task. Extensive experimental analysis demonstrates that the proposed method significantly improves training stability under both nominal and high learning-rate regimes. The model achieves a controlled token-level accuracy of 97.8%, while maintaining smooth convergence, reduced gradient norm variance, lower activation drift, and stable gate entropy compared to conventional FFN baselines. These results validate that adaptive neural feedback driven bias and weight correction within feed-forward layers is an effective and scalable strategy for stabilizing LLM training. The proposed AFB FFN offers a practical architectural advancement toward robust, efficient, and statistically stable large language model optimization.