Long-range Adaptive Memory Propagation
A differential-equation GNN for snapshot-based dynamic graphs. It combines impulsive neural ODEs with an antisymmetric parameterization to model conservative information flow, alongside data-driven dissipative dynamics that regulate information retention over space and time.
The problem
Modeling graphs demands a careful balance between long-range propagation of information across nodes and the controlled dissipation of noisy or redundant signals, so as to ensure stable learning and generalization. This challenge is exacerbated in dynamic graphs, where structural and temporal information interact, leading to uncontrolled information accumulation and amplifying noise. LAMP incorporates adaptive, learned dissipation within a principled dynamical systems framework, yielding stable yet expressive representations and enabling effective long-range dependency modeling while avoiding pathological information buildup.
§2.2 · Impulses
To capture the underlying evolving dynamics of discrete-time dynamic graphs, we combine the perspective of neural ordinary differential equations with that of impulsive ODEs. This formulation unifies the internal latent evolution of the graph features with the external arrival of discrete graph snapshots, treating the system as a hybrid dynamical system governed by both continuous flows and discrete events. Temporal inputs act as impulses driving the system dynamics: the evolution of the hidden state \(\mathbf{H}(t)\) is governed by a neural impulsive differential equation.
The continuous flow, the jump map at the impulse times, and the initial condition.
Between two snapshots the latent state \(\mathbf{H}(t)\) evolves continuously under the vector field \(F_\theta\). At each impulse time \(t_k\) the jump map \(G_\phi\) injects the newly observed node features \(\mathbf{X}(t_k)\) into the latent state, the discrete event that interrupts the continuous flow.
§2.2 · Continuous flow
In the interval between two snapshots, LAMP solves \(\dot{\mathbf{H}}(t) = F_\theta(\mathbf{H}(t);\, \mathcal{G}_{t_k})\), enabling temporal diffusion at a pace dictated by the sequence. To effectively propagate information jointly in space and time, we leverage the theory of non-dissipative neural ODEs, which links non-dissipative behavior to the stability of the system and to the sensitivity of its solutions to initial conditions. This sensitivity is governed by the Jacobian eigenvalues, with purely imaginary spectra ensuring stable dynamics that preserve information throughout the evolution.
Here \(\mathbf{D}(\mathbf{h}) = \operatorname{diag}(\sigma'(\mathbf{H}))\) collects the non-negative activation derivatives, and \(\mathbf{M}_{\mathrm{sk}}\) is the total mixing matrix. A key property of this formulation is the explicit control over the spectrum of \(\mathbf{J}_{F_\theta}\) induced by \(\boldsymbol{\Gamma}\): the stability of the flow is set directly by the diagonal damping.
§2.2 · Stability
The forward Euler method used to discretize the flow is stable when the spectrum of the Jacobian, scaled by the step size, resides within the unit circle centered at (−1, 0). The skew-symmetric contribution \(\mathbf{D}(\mathbf{h})\mathbf{M}_{\mathrm{sk}}\) alone places the spectrum on the imaginary axis, the boundary between contraction and expansion, where trajectories neither contract nor expand and information injected at any time is preserved. The diagonal term \(-\mathbf{D}(\mathbf{h})(\boldsymbol{\Gamma} \otimes \mathbf{I}_n)\) then shifts this spectrum into the left half-plane by a channel-wise amount proportional to the entries of \(\boldsymbol{\Gamma}\).
With \(\boldsymbol{\Gamma}=0\) the eigenvalues lie on the imaginary axis (non-dissipative). Increasing \(\boldsymbol{\Gamma}\) shifts the spectrum into the stable left half-plane, with \(\operatorname{Re}(\lambda)\) bounded between \(-1\) and \(-(\min_k \sigma'_k)\,\lambda_{\min}(\boldsymbol{\Gamma})\). This mirrors the trained spectra reported for LAMP, whose real parts strictly reside in the stable left-half plane.
§2 · Parameterization
Two ingredients produce this behavior. First, an antisymmetric parameterization: the spatial aggregation operators are defined so that the total mixing matrix \(\mathbf{M}_{\mathrm{sk}}\) is skew-symmetric (\(\mathbf{M}_{\mathrm{sk}}^{\top} = -\mathbf{M}_{\mathrm{sk}}\)), which ensures that the Jacobian of the conservative vector field has purely imaginary eigenvalues and the system possesses a non-dissipative dynamics. Second, an adaptive damping term \(\boldsymbol{\Gamma} = \operatorname{diag}(\operatorname{sigmoid}(\boldsymbol{\gamma}))\), subtracted from the channel mixing. This mechanism allows the model to continuously and dimension-wise balance dissipative and non-dissipative dynamics during latent-state evolution, effectively regulating how much information is preserved or attenuated as it propagates through time and across the graph.
\(\boldsymbol{\Gamma}\) is the per-channel dissipation. With \(\boldsymbol{\Gamma}\) near zero the signal is preserved over long temporal horizons (the long-range regime); as \(\boldsymbol{\Gamma}\) increases, only recent signal remains and noise is filtered, while strictly conservative dynamics would retain all history with equal magnitude.
§2.3 · Guarantees
We provide theoretical statements about information conservation, dissipation, and stability, showing that the model performs long-range propagation between nodes while controlling the amount of information to conserve in each channel.
The instantaneous rate of change \(\mu_2(\mathbf{J}_{F_\theta})\) is strictly bounded by the coupling expansion rate minus the minimum dissipation. Strict contraction is guaranteed whenever the dissipation is stronger than the expansion rate. \(\boldsymbol{\Gamma}\) thus plays a dual role: it bounds the Jacobian's spectrum in the negative half-plane, and it lets the model learn a task-driven timescale of information retention.
Under bounded inputs and the memory-normalization condition \(\xi > e^{\omega \varepsilon L}\), the latent state \(\mathbf{H}(t)\) remains globally bounded as \(t \to \infty\). Through Input-to-State Stability, the state magnitude is bounded by a function of the input magnitude plus a decaying transient, so the latent state does not diverge over infinitely long sequences.
Node-to-node sensitivity factorizes into a model term and a topology term \(\boldsymbol{\Psi}\), and is not limited by a rigid, pre-defined decay. The learned dissipation \(\boldsymbol{\Gamma}\) acts not as a structural bottleneck, but as an adaptive filter that selectively removes signals deemed noisy or redundant, while leaving the intrinsic long-range propagation paths intact. LAMP thus decouples stability from memory capacity.
Strict contraction holds when \(\mu_2 < 0\), that is, when the dissipation is stronger than the expansion rate.
A spatial view of the sensitivity bound. With low \(\boldsymbol{\Gamma}\) the signal reaches distant nodes, the long-range propagation regime in space; with strong \(\boldsymbol{\Gamma}\) the dissipation dominates and the signal dies at the horizon, recovering the over-squashing of standard GNNs over long spatial paths.
§4.1 · Benchmark
To rigorously evaluate long-range propagation jointly in time and space, we introduce a novel synthetic benchmark, ColoredLeafCounting. The benchmark considers a rooted tree where the root node and all leaf nodes are colored, while internal nodes act solely as intermediaries for message passing. Over a sequence of discrete timesteps, each leaf node may emit a signal, a blink, independently across time. At the end of the temporal sequence, the objective is to predict the total number of blinks produced over time by leaf nodes whose color matches that of the root. Solving this task requires the model to propagate information across long spatial paths, from leaves to root, while simultaneously aggregating signals over multiple timesteps.
The objective at the final timestep: the number of blinks emitted by leaves whose colour matches the root. The count is held hidden during the sequence, as the model must aggregate it from the deepest leaves over the full horizon.
§4.1 · Results
On the three-colour setting of ColoredLeafCounting, LAMP reduces test MSE by a substantial margin across all sequence lengths, for instance 0.057 versus 1.283 at T = 200. The conservative dynamics preserve the target signal over long horizons, while the adaptive dissipation selectively filters the noise from mismatched colours. Lower MSE is better.
| Model | T=200 | T=400 | T=600 |
|---|---|---|---|
| DCRNN | 1.283 | 0.811 | 0.675 |
| GCRN-LSTM | 1.248 | 1.012 | 0.837 |
| EvolveGCN-O | 1.307 | 0.991 | 0.868 |
| T-GCN | 1.307 | 1.004 | 0.838 |
| LAMP (ours) | 0.057 | 0.264 | 0.223 |
Test MSE on ColoredLeafCounting, three-colour setting, averaged over 5 runs. Full tables, ablations, and real-world forecasting and link-prediction results are reported in the paper.