Alexander Kyuroson

NeurIPS 2026

TIDE: Asymmetric Neural Circuits for Stabilized Temporal Inhibitory-Excitatory Dynamics

TIDE

One internal step of TIDE: E-I synapse model, excitatory and inhibitory neuron-level models, rate update, lateral inhibition, three-stream synchronization, attention, output head and memory. One internal step of TIDE: E-I synapse model, excitatory and inhibitory neuron-level models, rate update, lateral inhibition, three-stream synchronization, attention, output head and memory.

CTM

One internal step of the Continuous Thought Machine: a U-Net synapse model with weights of either sign, one shared neuron-level model, post-activations, synchronization over random neuron pairs, attention and output head. Lateral inhibition and memory are marked as absent. One internal step of the Continuous Thought Machine: a U-Net synapse model with weights of either sign, one shared neuron-level model, post-activations, synchronization over random neuron pairs, attention and output head. Lateral inhibition and memory are marked as absent.

Scroll the figure sideways, or tap it to open it full size.

One internal step of TIDE and of the Continuous Thought Machine (CTM), drawn on the same grid; switch between them to see what changes. Excitatory units are blueyellow and inhibitory units are red. Round markers 1 to 10 follow the numbering of CTM's own figure; square markers 11 to 13 are the blocks specific to TIDE: the HRF backbone, lateral inhibition and the memory. Dashed boxes in the CTM view mark the two blocks it does not have.

Abstract

Recent Continuous Thought Machine (CTM) architecture decouples internal computation from external inputs via neural dynamics, but relies on unconstrained multi-layer perceptrons for its recurrent computations. We propose to model neural dynamics using asymmetric Excitatory-Inhibitory (E-I) networks, which can be stabilized via principles from network theory, while obeying Dale's principle and can be expressed as energy-based systems optimized through a game-theoretic loss. Building on this perspective, we introduce Temporal Inhibitory-Excitatory Dynamic Engine (TIDE), a neuro-inspired architecture that computes internal representations through neural dynamics stabilized by incorporating the Wilson-Cowan dynamics and lateral inhibition. TIDE balances biological realism by, for instance, using Hierarchical Receptive Fields and enforcing Dale's principle to ensure a realistic 80 : 20 E-I balance ratio with an end-to-end trainable architecture. The aim of this paper is to introduce a new architecture that brings neuro-inspired learning and architectural principles to the forefront. We present proofs of convergence, stability, and complexity bounds, along with empirical ablation studies. Overall, TIDE achieves higher top-1 accuracy than CTM on the majority of the investigated datasets while requiring a smaller training budget. Furthermore, across varying training budgets, TIDE improves top-1 accuracy on ImageNet-C by an average of +1.65% to +6.42% under different perturbation severity levels.

Demo

Each clip follows one ImageNet validation image through the 50 internal steps. The top row is TIDE and the bottom row is CTM. From left to right: the input, the mean attention, the attention overlaid on the image, and the five most likely classes with the certainty at that step. Press play to start a clip.

Common iguana. TIDE is correct at step 50, CTM is wrong.
African elephant. TIDE is correct, CTM is wrong.
Ram. CTM is correct, TIDE is wrong.
Stingray. CTM is correct, TIDE is wrong.
European gallinule. Both are correct.
Chiffonier. Both are wrong.

Six of the twelve clips in the supplementary material, chosen to cover every outcome: two where only TIDE is correct at the last step, two where only CTM is, one where both are and one where neither is.

Method

TIDE keeps the loop of the Continuous Thought Machine, a sequence of internal steps that is decoupled from the input, and changes what runs inside it. The neurons are split 80:20 into an excitatory and an inhibitory population. Every recurrent weight is non-negative, and whether a connection excites or inhibits is decided by the population it leaves, which is Dale's principle. The two populations follow a forward-Euler Wilson-Cowan update with their own time constants.

rE(t)=(1−αE) rE(t−1)+αE φ(hE(t))r_E^{(t)} = (1-\alpha_E)\,r_E^{(t-1)} + \alpha_E\,\varphi\big(h_E^{(t)}\big)
hE(t)=WEE rE(t−1)− WEI rI(t−1)+WEin a(t)h_E^{(t)} = W_{EE}\,r_E^{(t-1)} \textcolor{#B3261E}{-}\, W_{EI}\,r_I^{(t-1)} + W_E^{\mathrm{in}}\,a^{(t)}
rI(t)=(1−αI) rI(t−1)+αI φ(hI(t))r_I^{(t)} = (1-\alpha_I)\,r_I^{(t-1)} + \alpha_I\,\varphi\big(h_I^{(t)}\big)
hI(t)=WIE rE(t−1)− WII rI(t−1)+WIin a(t)h_I^{(t)} = W_{IE}\,r_E^{(t-1)} \textcolor{#B3261E}{-}\, W_{II}\,r_I^{(t-1)} + W_I^{\mathrm{in}}\,a^{(t)}

α∙=Δt/τ∙\alpha_\bullet = \Delta t/\tau_\bullet,   τE=20 ms\tau_E = 20\,\mathrm{ms},   τI=5 ms\tau_I = 5\,\mathrm{ms},   and WXY←max⁡(WXY, 0)W_{XY} \leftarrow \max(W_{XY},\,0) after every optimizer step.

Forward-Euler Wilson-Cowan update of the E-I rates. In TIDE, h is normalized by RMSNorm. The minus sign in front of the inhibitory term is explicit because the weights themselves are never negative.

From CTM to TIDE

The figure at the top of the page shows both models on the same grid. The table lists the same changes block by block.

BlockCTMTIDEChange
Feature extractorResNet backboneHierarchical receptive fields (HRF): learnable center-surround filters, trained end to endReplaced
Neuron interactionSynapse model: U-Net MLP with weights of either signE-I populations under Dale's law; all four weight blocks are non-negative and inhibition enters with an explicit signReplaced, split 80:20
Neuron-level modelsA single set of per-neuron models on the pre-activation historyTwo sets on the rate history, one per population: τE = 20 ms and τI = 5 msDuplicated
NormalizationShared LayerNormRMSNorm per populationModified
SynchronizationSingle streamThree streams: E-E (binding), E-I (gating), I-I (rhythm)Decomposed
Cross-attentionQuery from synchronizationAdds a residual stream, a ← a + z, on ImageNet-1KAdded, conditional
Lateral inhibitionAbsentWinner-take-all on the excitatory population, five iterationsAdded
MemoryAbsentSurprise-gated persistent buffer with momentum; an E-I retention gate weakens writes when the E/I activity ratio is highAdded
Output headReads the synchronization vectorReads the latent and the memory, [z; m]Extended

What changes block by block, after Figure 1 of the paper.

Training and stability

Training under Dale's principle
Projected gradient descent: after every optimizer step the recurrent weights are clamped to W ≥ 0.
Game-theoretic loss
The E-I circuit is read as a game between neurons, and the loss penalizes the fixed-point residual of the per-neuron energies (Betteti et al., 2025).
Stability
Lyapunov diagonal stability is monitored during training through a differentiable Perron sum-ratio surrogate that caps ρ(WEE) and ρ(WII).

Results

TIDE and CTM are trained with the same seeds and the same number of optimizer steps on five image-classification datasets. CTM is retrained for this comparison.

DatasetStepsdmodel TIDECTM
mean ± stdbest seedmean ± stdbest seed
MNIST50K25699.64 ± 0.0399.6799.54 ± 0.0399.56
MNIST200K25699.60 ± 0.0599.6499.59 ± 0.0699.64
Fashion-MNIST50K25694.21 ± 0.2194.4292.51 ± 0.1192.63
Fashion-MNIST200K25694.12 ± 0.1994.2392.53 ± 0.2492.80
CIFAR-10600K51290.62 ± 0.0690.6886.23 ± 0.3486.59
CIFAR-100600K71862.23 ± 0.1662.3865.19 ± 0.4865.70
CIFAR-100600K‡71865.68 ± 0.1865.79
ImageNet-1K100K409667.22 ± 1.5268.7452.67 ± 1.7954.46
ImageNet-1K200K409668.40 ± 1.9169.5562.78 ± 1.3164.06
ImageNet-1K500K4096n/a69.90n/a71.78

Best top-1 accuracy (%) during training, as mean ± standard deviation over seeds and for the best seed (Table 1 of the paper). Bold marks the highest value per dataset and step budget, separately for the mean and for the best seed. ‡ Effective batch size of 64. At 500K steps on ImageNet-1K only one run per model is reported, and CTM attains the higher accuracy.

Component ablation

Each row removes one component from the full model, or adds the spectral regularizer, and retrains with the same seeds. MNIST and Fashion-MNIST use 50K steps, CIFAR-10 and CIFAR-100 use 600K.

ConfigurationMNISTFashion-MNISTCIFAR-10CIFAR-100
Reference
CTM99.54 ± 0.0392.51 ± 0.1186.23 ± 0.3465.19 ± 0.48
TIDE, full (λspec = 0)99.64 ± 0.0294.21 ± 0.2190.62 ± 0.0665.68 ± 0.18
Component removed
Wilson-Cowan update99.61 ± 0.0293.70 ± 0.1290.18 ± 0.2664.52 ± 0.67
Dale's law constraint99.59 ± 0.0393.55 ± 0.1889.01 ± 1.7465.39 ± 0.39
Lateral inhibition (WTA)99.61 ± 0.0293.80 ± 0.1789.13 ± 1.2064.94 ± 0.30
Surprise-gated memory99.59 ± 0.0192.95 ± 0.1688.97 ± 1.4164.32 ± 0.30
HRF backbone (ResNet-18 instead)99.60 ± 0.0393.03 ± 0.1283.02 ± 2.0254.49 ± 0.76
Component added
Spectral regularizer (λspec = 0.1)99.64 ± 0.0394.31 ± 0.0689.02 ± 2.9365.38 ± 0.35

Leave-one-out ablation: best top-1 accuracy (%) during training, mean ± standard deviation over seeds (Table 3 of the paper). Bold marks the highest value per dataset. Removing the memory costs the most on MNIST and Fashion-MNIST; swapping the HRF backbone for ResNet-18 costs the most on CIFAR-10 and CIFAR-100.

Attention over internal steps

Two examples, a little blue heron with Gaussian noise and a coyote rotated by 90 degrees. For each, rows of attention maps at internal steps 1, 5, 10, 20, 30, 40 and 50 for TIDE and for CTM. TIDE predicts the correct class from step 5 onward in both examples; CTM is wrong at every step shown. Two examples, a little blue heron with Gaussian noise and a coyote rotated by 90 degrees. For each, rows of attention maps at internal steps 1, 5, 10, 20, 30, 40 and 50 for TIDE and for CTM. TIDE predicts the correct class from step 5 onward in both examples; CTM is wrong at every step shown.

Scroll the figure sideways, or tap it to open it full size.

Attention of TIDE and CTM over internal steps on perturbed ImageNet images. The frame and the label under each tile give the prediction at that step: green is correct, red is wrong.

Robustness

Both models are trained on clean ImageNet-1K only and then evaluated on ImageNet-C, which applies 19 corruption types at five severities. At 500K training steps TIDE is ahead of CTM by 2.68 points on average over all 95 cells. With smaller training budgets the gap is wider: 16.8 points at 100K steps and 6.4 points at 200K. CTM is the more robust of the two under fog, frost and snow.

Three heatmaps over 19 corruption types and 5 severities on ImageNet-C: TIDE accuracy, CTM accuracy and their difference. TIDE is higher for most blur, digital and noise corruptions, by up to 14.1 points for elastic transform; CTM is higher for fog at every severity, by up to 5.2 points. Three heatmaps over 19 corruption types and 5 severities on ImageNet-C: TIDE accuracy, CTM accuracy and their difference. TIDE is higher for most blur, digital and noise corruptions, by up to 14.1 points for elastic transform; CTM is higher for fog at every severity, by up to 5.2 points.

Scroll the figure sideways, or tap it to open it full size.

ImageNet-C top-1 accuracy (%) at 500K training steps. Left: TIDE. Center: CTM. Right: the difference TIDE − CTM, where blue means TIDE is higher (Figure 3 of the paper).

On CIFAR-10-C the results are mixed. TIDE leads under noise and under snow and frost, while CTM is clearly better at high severity under contrast, fog and defocus blur.

Three heatmaps over 15 corruption types and 5 severities on CIFAR-10-C: TIDE accuracy, CTM accuracy and their difference. TIDE is higher by about 7 to 10 points under Gaussian, shot and impulse noise; CTM is higher by up to 40.3 points under contrast at severity 5 and by 17.9 points under fog at severity 5. Three heatmaps over 15 corruption types and 5 severities on CIFAR-10-C: TIDE accuracy, CTM accuracy and their difference. TIDE is higher by about 7 to 10 points under Gaussian, shot and impulse noise; CTM is higher by up to 40.3 points under contrast at severity 5 and by 17.9 points under fog at severity 5.

Scroll the figure sideways, or tap it to open it full size.

CIFAR-10-C top-1 accuracy (%) of TIDE, of CTM and their difference TIDE − CTM, where blue means TIDE is higher. This panel is from the conference poster.

Limitations

The main limitation of both TIDE and CTM is their required sequential feedforward data processing, which prohibits shorter runtime and prevents more robust investigation. Therefore, further investigation is required to assess the feasibility of leveraging neuromorphic hardware and its dynamics, not only to enhance data preprocessing but also to implement a Look-up-Table-based approach to further reduce the training duration of the proposed framework. Given that we have already lowered the training boundary by around 50% to achieve results comparable to CTM, indicating the soundness of our approach, further studies are still needed to assess the reachability of an even sparser representation. Additionally, due to the lack of comparable implementations using similar approaches, detailed benchmarking within the same architectural family remains limited and challenging. Although CTM uses LSTM as its baseline, it is not a direct comparison; however, their results can be directly translated to TIDE given that we use CTM as our baseline.

The limitations paragraph of Appendix H of the paper, word for word.

Citation

BibTeX
@inproceedings{
anonymous2026tide,
title={{TIDE}: Asymmetric Neural Circuits for Stabilized Temporal Inhibitory-Excitatory Dynamics},
author={Anonymous},
booktitle={The Fortieth Annual Conference on Neural Information Processing Systems},
year={2026},
url={https://openreview.net/forum?id=dwcrK8NHpm}
}

All projects     About Alexander Kyuroson