Efficient representation learning is essential for optimal information storage and classification. However, it is frequently overlooked in artificial neural networks (ANNs). This neglect results in networks that can become overparameterized by factors of up to 13, increasing redundancy and energy consumption. As the demand for large language models (LLMs) and their scale increase, these issues are further highlighted, raising significant ethical and environmental concerns. We analyze our previously developed biologically inspired learning rule using information-theoretic concepts, evaluating its efficiency on the MNIST classification task. The proposed rule, which emulates the brain's structural plasticity, naturally prevents overparameterization by optimizing synaptic usage and retaining only the essential number of synapses. Furthermore, it outperforms backpropagation (BP) in terms of efficiency and storage capacity. It also eliminates the need for pre-optimization of network architecture, enhances adaptability, and reflects the brain's ability to reserve 'space' for new memories. This approach advances scalable and energy-efficient AI and provides a promising framework for developing brain-inspired models that optimize resource allocation and adaptability.
While deep neural networks (DNNs) have achieved remarkable performance in tasks such as image recognition, they often struggle with generalization, learning from few examples, and continuous adaptation - abilities inherent in biological neural systems. These challenges arise due to DNNs' failure to emulate the efficient, adaptive learning mechanisms of biological networks. To address these issues, we explore the integration of neurobiologically inspired assumptions in neural network learning. This study introduces a biologically inspired learning rule that naturally integrates neurobiological principles, including sparsity, lognormal weight distributions, and adherence to Dale's law, without requiring explicit enforcement. By aligning with these core neurobiological principles, our model enhances robustness against adversarial attacks and demonstrates superior generalization, particularly in few-shot learning scenarios. Notably, integrating these constraints leads to the emergence of biologically plausible neural representations, underscoring the efficacy of incorporating neurobiological assumptions into neural network design. Preliminary results suggest that this approach could extend from feature-specific to task-specific encoding, potentially offering insights into neural resource allocation for complex tasks.
Nanocellulose, a biodegradable and renewable nanomaterial derived from biomass, has emerged as a promising sustainable building block for flexible functional devices due to its renewability, low density, excellent mechanical strength, tunable surface chemistry, and outstanding film-forming capability. This paper provides a critical review of the evaluations and synthesis of recent progress in the manufacturing, functionalization, and incorporation of nanocellulose and its composite materials for electronic devices and electrical systems applications. The paper also highlights the contributions of nanocellulose to performance, durability, and environmental sustainability, along with its potential uses in flexible electrical equipment, energy storage devices, sensors, and conductive components. Furthermore, the review examines the combined effects of nanocellulose with metallic nanoparticles, carbon-based materials, and polymers in developing superior electrically conductive composites. In addition, the article highlights research gaps and suggests future directions for advancing sustainable, high-performance conductive materials. Finally, the paper critically analyzes key challenges such as reliability, interface compatibility, and long-term stability, and proposes strategies to address these limitations.
Continual learning (CL) aims to train models sequentially over multiple domains without forgetting previously learned knowledge. However, existing CL methods optimize for in-domain performance and are therefore prone to learning spurious, domain-specific cues (``shortcut learning''), which limits generalization to unseen domains after deployment. In this paper, we address this limitation through . We introduce a broad class of CL methods that sequentially learn representations capturing invariant structures across domains. Our methods are motivated by the observation that such invariant structures often preserve the underlying causal mechanisms, which can reduce the risk of overfitting to domain-specific cues and thus offer better out-of-domain generalization. Our proposed CL methods combine replay-based training with a tailored sequential invariance alignment to learn---and preserve---invariant structures over time. We evaluate our methods under a deployment-oriented protocol that measures performance on unseen target domains. Across six benchmark and real-world datasets spanning vision, medicine, manufacturing, and ecology, our methods consistently outperform existing CL baselines in terms of generalization to unseen target domains. As an ablation, we further show that na\"ive extensions of sequential training with existing domain-invariant representation learning (DIRL) methods provide only limited benefits. To the best of our knowledge, this is the first work to develop domain-invariant representation methods for CL.
Modern machine learning applications employ deep neural networks training with the error backpropagation algorithm. Although this algorithm is very effective, it lacks biological realism. For example, backpropagation requires symmetric connectivity, and a separate neural processing channel for error signals. Prior works have therefore proposed a number of more realistic alternatives for error backpropagation. However, most of them still suffer from demanding preassumptions that may be not fulfilled in the real brain, for example, they often still require either symmetric connectivity or two separate processing channels, and often require also special mathematical operations like subtractions or function inversions. Here I propose supervised counterstream learning in deep associative networks as a simpler approach that requires only recognition of errors during training, and then backpropagates correcting target activity through the same activity channel as used for forward propagation. For this, two activity waves are initiated at the same time in input and output layers and then traveling in opposite directions to meet in one of the hidden layers. By employing simple local Hebbian-type learning rules, the corresponding activity pattern sequences get linked bidirectionally, thereby decreasing error rates over time. Despite its simplicity and an incomplete hyperparameter optimzation, a high high test accuracy is achieved on the (binarized) MNIST data set that is comparable to more demanding architectures.
