RAFT: Redundancy-Aware Input Filtering Accelerates Temporal Graph Neural Network Training

Title

RAFT: Redundancy-Aware Input Filtering Accelerates Temporal Graph Neural Network Training

Meta Description

RAFT reduces training inputs by an average of 35.23% through redundancy-aware input filtering, cutting training time by up to 52.56% with negligible accuracy loss, offering a new solution for accelerating temporal graph neural network training.

Article Content

Temporal Graph Neural Networks (TGNNs) excel at dynamic graph learning tasks by continuously updating node memory from evolving interactions (i.e., events) and using these memories for downstream predictions. However, as graphs in modern applications grow rapidly, the number of events that must be processed during training surges. Moreover, each event may update node memories that subsequent events depend on—a dependency that limits parallel execution and diminishes the gains from additional compute resources. These costs prompt an upstream question: are all events and the dependencies they introduce truly necessary for learning?

Recently, a paper titled "RAFT: Redundancy-Aware Input Filtering for Temporal Graph Neural Network Training" investigated this question in depth. By analyzing two roles of temporal events—prediction targets and historical events sampled as inputs—the study found that prediction targets involving stable node memories tend to exhibit lower prediction loss, and that under various evaluation settings, filtering out historical events associated with stable memories preserves similar training loss trajectories. Together, these observations suggest that input redundancy can be identified through changes in node memory. Building on this, the researchers propose RAFT, a redundancy-aware input filtering framework for TGNN training.

The Bottleneck of Temporal Graph Neural Network Training

TGNN training involves processing a large number of events, each of which updates node memories that are then used by subsequent events, forming complex dependency chains. These dependencies not only increase computational burden but also limit parallelization, making it difficult to significantly speed up training by simply adding compute resources. Therefore, reducing unnecessary event processing becomes key to improving training efficiency.

Node memory is one of the core components of TGNNs, typically a vector indexed by node that summarizes the historical interaction information of that node up to the current moment. Whenever an event (e.g., an interaction on an edge) occurs, the TGNN reads the memories of the involved nodes as input, computes, and updates these memories to incorporate the new event's information. Thus, node memory acts as a "compressed summary" of historical interactions, enabling the model to make predictions without回溯ing the full history—that is, without revisiting all past events. Memory updates are serial: subsequent events depend on the memories updated by preceding events, which is the fundamental reason why parallelization is difficult in training. RAFT leverages the magnitude of memory changes to judge event importance—small memory changes mean the event has limited impact on subsequent predictions, and can thus serve as a basis for redundancy filtering.

The RAFT Framework: Memory-Change-Based Input Filtering

The core idea of RAFT is to use changes in node memory during training to guide input filtering. Specifically, it includes two filters:

• Root-event filter: Removes selected prediction targets before training. These targets correspond to relatively stable node memories and low prediction loss, thus contributing little to training.
• Support-event filter: Excludes selected historical context during sampling. These historical events are associated with stable node memories, and filtering them still preserves similar training loss trajectories.

By filtering before input preparation and processing, RAFT reduces the work triggered by these inputs, thereby accelerating training.

Root events and support events are the two roles of events in TGNN training. A root event is the event selected as the prediction target; the model must predict its occurrence (e.g., whether an edge exists, node category, etc.) based on node memories prior to it. Support events are historical events sampled retroactively to predict the root event; they provide context and participate in updating node memories. A training iteration typically consists of one root event and a batch of sampled support events, which together form the input. RAFT designs filters for these two roles respectively: the root-event filter eliminates prediction targets with stable memories and low loss before training; the support-event filter excludes historical context associated with stable memories during sampling. This role-specific filtering strategy enables redundancy identification to precisely target different stages of the training pipeline.

Training loss trajectory refers to the curve of loss values over iteration steps during training. In TGNN training, because events are processed in temporal order and have dependencies, the stability of the loss trajectory reflects the coherence of the model's learning process. When evaluating filtering effectiveness, RAFT does not merely look at final accuracy but compares the similarity of loss trajectories before and after filtering—if the loss trajectory remains similar after filtering out certain events, it indicates that these events contribute limitedly to the learning process and are redundant inputs. This metric captures the impact of filtering on training dynamics better than single-point accuracy and provides finer-grained evidence that "filtering does not harm learning."

Experimental Results: 35% Fewer Inputs, Training Time Halved

The researchers evaluated RAFT on three TGNN models and five datasets. Results show that RAFT reduces training inputs by an average of 35.23%, up to 97.11%; meanwhile, recorded training time is reduced by up to 52.56%, with negligible changes in model accuracy. This indicates that RAFT significantly improves training efficiency while barely affecting model performance.

Implications and Outlook

RAFT offers a new perspective for optimizing temporal graph neural network training: by identifying and filtering redundant inputs, the dependency bottleneck in training can be effectively alleviated. In the future, this method is expected to extend to more dynamic graph learning scenarios, further promoting the deployment of TGNNs in real-world applications.

One-line Summary

RAFT leverages node memory changes to identify and filter redundant events in temporal graph neural network training, speeding up training while preserving accuracy.

Paragraph Summary

Temporal Graph Neural Networks (TGNNs) require processing a large number of evolving events during training, and events form serial dependencies through node memory, limiting parallelization and acceleration. The paper proposes the RAFT framework, which analyzes events in their two roles—prediction targets (root events) and historical inputs (support events)—and finds that root events with stable memories have low prediction loss, and filtering support events with stable memories preserves similar training loss trajectories. Based on this, RAFT uses node memory changes to identify redundant inputs. RAFT includes a root-event filter and a support-event filter, removing redundancy before training and during sampling, respectively. On three TGNN models and five datasets, RAFT reduces training inputs by an average of 35.23% (up to 97.11%) and training time by up to 52.56%, with negligible accuracy changes.