Recent Releases of pytorch_geometric

pytorch_geometric - PyG 2.6.1: Bugfixes

PyG 2.6.1 includes a bugfix in the WebQSDataset.

Bug Fixes

Fixed a bug in the WebQSDataset dataset where empty edges were not treated gracefully (#9665)

Full Changelog: https://github.com/pyg-team/pytorch_geometric/compare/2.6.0...2.6.1

- Python
Published by rusty1s over 1 year ago

pytorch_geometric - PyG 2.6.0

We are excited to announce the release of PyG 2.6 🎉🎉🎉

Highlights
Breaking Changes
Features
Bugfixes
Changes
Full Changelog

PyG 2.6 is the culmination of work from 59 contributors who have worked on features and bug-fixes for a total of over 238 commits since torch-geometric==2.5.0.

Highlights

PyTorch 2.4 Support

PyG 2.6 is fully compatible with PyTorch 2.4, and supports the following combinations:

| PyTorch 2.2 | cpu | cu118 | cu121 | cu124 | |--------------|-------|---------|---------|---------| | Linux | ✅ | ✅ | ✅ | ✅ | | macOS | ✅ | | | | | Windows | ✅ | ✅ | ✅ | ✅ |

You can still install PyG 2.6 with an older PyTorch release up to PyTorch 1.13 in case you are not eager to update your PyTorch version.

GNNs+LLMs

In order to facilitate further research on combining GNNs with LLMs, PyG 2.6 introduces

a new sub-package torch_geometric.nn.nlp with fast access to SentenceTransformer models and LLMs
a new model GRetriever that is able to co-train LLAMA2 with GAT for answering questions based on knowledge graph information
a new example folder examples/llm that shows how to utilize these models in practice

`Index` Tensor Representation

Similar to the EdgeIndex class introduced in PyG 2.5, torch-geometric==2.6.0 introduces the Index class for efficient storage of 1D indices. While Index sub-classes a general torch.Tensor, it can hold additional (meta)data, i.e.:

dim_size: The size of the underlying sparse vector, i.e. the size of a dimension that can be indexed via Index. By default, it is inferred as dim_size=index.max() + 1
is_sorted: Whether indices are sorted in ascending order.

Additionally, Index caches data via indptr for fast CSR conversion in case its representation is sorted. Caches are filled based on demand (e.g., when calling Index.get_indptr() or when explicitly requested via Index.fill_cache_(), and are maintained and adjusted over its lifespan.

```python from torch_geometric import Index

index = Index([0, 1, 1, 2], dimsize=3, issorted=True) assert index.dimsize == 3 assert index.issorted

Flipping order:

index.flip(0) assert not index.is_sorted

Filtering:

mask = torch.tensor([True, True, True, False]) index[:, mask] assert index.is_sorted ```

EdgeIndex and Index will interact seamlessly together, e.g., edge_index[0] will now return a Index instance. This ensures optimal computation in GNN message passing schemes, while preserving the ease-of-use of regular COO-based PyG workflows. EdgeIndex and Index will fully deprecate the usage of SparseTensor from torch-sparse in later releases, leaving us with just a single source of truth for representing graph structure information in PyG.

Breaking Changes

Allow None outputs in FeatureStore.get_tensor() - KeyError should now be raised based on the implementation in FeatureStore._get_tensor() (#9102)
cugraph-based GNN layers such as CuGraphSAGEConv now expect EdgeIndex-based inputs (#8938)

Features

Examples

Added a multi-GPU example for training GNNs on the PCQM4M graph-level regression task (#9070)
Added a multi-GPU ogbn-mag240m example (#8249)
Added support for cugraph data loading capabilities in the papers100m examples (#8173)
Improved the hyper-parameters of the [single-node](ogbn-papers100m example](https://github.com/pyg-team/pytorch_geometric/blob/master/examples/ogbn_papers_100m.py) and [multi-node](https://github.com/pyg-team/pytorch_geometric/blob/master/examples/multi_gpu/papers100m_gcn_cugraph_multinode.py)ogbn-papers100m examples, and added evaluation on all ranks (#8823, #9386, #9445)

`EdgeIndex` and `Index`

Added torch_geometric.Index (#9276, #9277, #9278, #9279, #9280, #9281, #9284, #9285, #9286, #9287, #9288, #9289, #9296, #9297)
Added support for EdgeIndex in MessagePassing (#9007, #9026, #9131)
Added support for torch.compile in combination with EdgeIndex (#9007)
Added support for EdgeIndex.unbind() (#9298)
Added support for EdgeIndex.sparse_narrow() (#9291)
Added support for EdgeIndex.sparse_resize_() (#8983)

`torch_geometric.nn`

Added the GRetriever model (#9480)
Added the ClusterPooling layer (#9627)
Added the PatchTransformerAggregation layer (#9487)
Added a residual option in GATConv and GATv2Conv (#9515)
Added a nlp.LLM model wrapper (#9462)
Added a nlp.SentenceTransformer model wrapper (#9350)
Added the heterogeneous HeteroJumpingKnowledge module for applying jumping knowledge in heterogeneous graphs (#9380)
Added the VariancePreservingAggregation layer (#9075)
Added approximate faiss-based KNN-search capabilities via ApproxKNN (#8952, #9046)

`torch_geometric.metrics`

Added the LinkPredMRR metric (#9632)

`torch_geometric.transforms`

Added the RemoveSelfLoops transformation (#9562)

`torch_geometric.utils`

Added normalize_edge_index() for symmetric/asymmetric normalization of graph edges (#9554)
Added from_rdmol/to_rdmol functionality (#9452)
Added ONNX export for scatter with min/max reductions (#9587)

`torch_geometric.datasets`

Added the WebQSPDataset (#9481)
Added the OPFDataset (#9379)
Added CornellTemporalHyperGraphDataset hypergraph dataset (#9090)
Added option to pass customfrom_smiles functionality to PCQM4Mv2 and MoleculeNet (#9073)

`torch_geometric.loader`

Added support for negative sampling in LinkLoader acccording to source and destination node weights (#9316)

Bugfixes

Fixed VirtualNode transform for empty edge indices (#9605)
Fixed an issue where the import order in the multi-GPU cugraph example could cause an rmm error (#9577)
Fixed load_state_dict behavior with lazy parameters in HeteroDictLinear (#9493)
Sequential modules can now be properly pickled (#9369)
Fixed pickle.load for jittable MessagePassing modules (#9368)
Fixed batching of sparse tensors saved via data.edge_index (#9317)
Fixed arbitrary keyword ordering in MessagePassing.propagate() (#9245)
Fixed the node mapping in the RCDD dataset (#9234)
Fixed incorrect treatment of edge_label and edge_label_index in ToSparseTensor transform (#9199)
Fixed EgoData processing in SnapDataset in case filenames are unsorted (#9195)
Fixed empty graph and isolated node handling in to_dgl() function (#9188)
Fixed bug in to_scipy_sparse_matrix() when CUDA is set as default torch device (#9146)
Fixed the MetaPath2Vec model in case the last node is isolated (#9145)
Ensured backward compatibility in MessagePassing via torch.load() (#9105)
Prevented model compilation on custom MessagePassing.propagate() functions (#9079)
Ignore self.propagate appearances in comments when parsing MessagePassing implementation (#9044)
Fixed OSError on read-only file systems within MessagePassing (#9032)
Fixed metaclass conflict in Dataset (#8999)
Fixed import errors on MessagePassing modules with nested inheritance (#8973)
Fixed TorchScript compilation error for MessagePassing._check_input() on older torch versions (#9564)

Changes

Use torch.load(weights_only=True) by default (#9618)
Allow optional but untyped tensors in MessagePassing (#9494)
Added support for modifying filename of the stored partitioned file in ClusterLoader (#9448)
Support other than two-dimensional inputs in AttentionalAggregation (#9433)
Added the fmt argument to Dataset.print_summary() (#9408)
Skip zero atom molecules in MoleculeNet (#9318)
Ensure proper parallelism in OnDiskDataset for multi-threaded get calls (#9140)
Allow mini-batching of uncoalesced sparse matrices (#9099)
Default to scatter() operations in MessagePassing in case torch.use_deterministic_algorithms is not set (#9009)
Added XPU support to basic GNN examples (#9421, #9439)

New Contributors

@arthurdjn made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8918
@project-delphi made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8946
@arlesniak made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8978
@simon-forb made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9057
@rf523281 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9075
@BenediktAlkin made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9076
@luckynozomi made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9113
@MatthieuMelennec made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9116
@brenting made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9146
@drivanov made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9178
@Akkete made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9204
@1taroh made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9230
@nelsonaloysio made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9334
@chaojun-zhang made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9396
@zhouyu5 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9407
@mzgubic made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9379
@guanxingithub made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9386
@devanshamin made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9445
@ihkao made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9436
@oiao made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9452
@kano5266 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9473
@andyhuang-kumo made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9487
@bryceForrest made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9533
@ECMGit made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9543
@nilserranestle made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9499
@hnsgrvr made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9498
@Linnore made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9515
@alexbarghi-nv made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9541
@MFairley made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9564
@viktor-ktorvi made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9569
@lee-clement-oxb made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9587
@Danial-sb made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9554
@DonBoscoBlaiseA made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9611
@kaarthiksundar made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9621
@rhjohnstone made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/9632

Full Changelog: https://github.com/pyg-team/pytorch_geometric/compare/2.5.0...2.6.0

- Python
Published by akihironitta over 1 year ago

pytorch_geometric - PyG 2.5.3: Bugfixes

PyG 2.5.3 includes a variety of bug fixes related to the MessagePassing refactoring.

Bug Fixes

Ensure backward compatibility in MessagePassing via torch.load (#9105)
Prevent model compilation on custom propagate functions (#9079)
Flush template file before closing it (#9151)
Do not set propagate method twice in MessagePassing for decomposed_layers > 1 (#9198)

Full Changelog: https://github.com/pyg-team/pytorch_geometric/compare/2.5.2...2.5.3

- Python
Published by rusty1s almost 2 years ago

pytorch_geometric - PyG 2.5.2: Bugfixes

PyG 2.5.2 includes a bug fix for implementing MessagePassing layers in Google Colab.

Bug Fixes

Raise error in case inspect.get_source is not supported (#9068)

Full Changelog: https://github.com/pyg-team/pytorch_geometric/compare/2.5.1...2.5.2

- Python
Published by rusty1s almost 2 years ago

pytorch_geometric - PyG 2.5.1: Bugfixes

PyG 2.5.1 includes a variety of bugfixes.

Bug Fixes

Ignore self.propagate appearances in comments when parsing MessagePassing implementation (#9044)
Fixed OSError on read-only file systems within MessagePassing (#9032)
Made MessagePassing interface thread-safe (#9001)
Fixed metaclass conflict in Dataset (#8999)
Fixed import errors on MessagePassing modules with nested inheritance (#8973)
Fix OSError when downloading datasets with simplecache (#8932)

Full Changelog: https://github.com/pyg-team/pytorch_geometric/compare/2.5.0...2.5.1

- Python
Published by rusty1s almost 2 years ago

pytorch_geometric - PyG 2.5.0: Distributed training, graph tensor representation, RecSys support, native compilation

We are excited to announce the release of PyG 2.5 🎉🎉🎉

Highlights
Breaking Changes
Deprecations
Features
Bugfixes
Changes
Full Changelog

PyG 2.5 is the culmination of work from 38 contributors who have worked on features and bug-fixes for a total of over 360 commits since torch-geometric==2.4.0.

Highlights

`torch_geometric.distributed`

We are thrilled to announce the first in-house distributed training solution for PyG via the torch_geometric.distributed sub-package. Developers and researchers can now take full advantage of distributed training on large-scale datasets which cannot be fully loaded in memory of one machine at the same time. This implementation doesn't require any additional packages to be installed on top of the default PyG stack.

Key Advantages

Balanced graph partitioning via METIS ensures minimal communication overhead when sampling subgraphs across compute nodes.
Utilizing DDP for model training in conjunction with RPC for remote sampling and feature fetching routines (with TCP/IP protocol and gloo communication backend) allows for data parallelism with distinct data partitions at each node.
The implementation via custom GraphStore and FeatureStore APIs provides a flexible and tailored interface for distributing large graph structure information and feature storage.
Distributed neighbor sampling is capable of sampling in both local and remote partitions through RPC communication channels. All advanced functionality of single-node sampling are also applicable for distributed training, e.g., heterogeneous sampling, link-level sampling, temporal sampling, etc.
Distributed data loaders offer a high-level abstraction for managing sampler processes, ensuring simplicity and seamless integration with standard PyG data loaders.

See here for the accompanying tutorial. In addition, we provide two distributed examples in examples/distributed/pyg to get started: * Distributed node-level classification on ogbn-products * Distributed temporal link prediction on MovieLens

`EdgeIndex` Tensor Representation

torch-geometric==2.5.0 introduces the EdgeIndex class.

EdgeIndex is a torch.Tensor, that holds an edge_index representation of shape [2, num_edges]. Edges are given as pairwise source and destination node indices in sparse COO format. While EdgeIndex sub-classes a general torch.Tensor, it can hold additional (meta)data, i.e.: * sparse_size: The underlying sparse matrix size * sort_order: The sort order (if present), either by row or column * is_undirected: Whether edges are bidirectional.

Additionally, EdgeIndex caches data for fast CSR or CSC conversion in case its representation is sorted (i.e. its rowptr or colptr). Caches are filled based on demand (e.g., when calling EdgeIndex.sort_by()), or when explicitly requested via EdgeIndex.fill_cache_(), and are maintained and adjusted over its lifespan (e.g., when calling EdgeIndex.flip()).

```python from torch_geometric import EdgeIndex

edgeindex = EdgeIndex( [[0, 1, 1, 2], [1, 0, 2, 1]] sparsesize=(3, 3), sortorder='row', isundirected=True, device='cpu', )

EdgeIndex([[0, 1, 1, 2], ... [1, 0, 2, 1]]) assert edgeindex.issortedbyrow assert edgeindex.isundirected

Flipping order:

edgeindex = edgeindex.flip(0)

EdgeIndex([[1, 0, 2, 1], ... [0, 1, 1, 2]]) assert edgeindex.issortedbycol assert edgeindex.isundirected

Filtering:

mask = torch.tensor([True, True, True, False]) edgeindex = edgeindex[:, mask]

EdgeIndex([[1, 0, 2], ... [0, 1, 1]]) assert edgeindex.issortedbycol assert not edgeindex.isundirected

Sparse-Dense Matrix Multiplication:

out = edge_index.flip(0) @ torch.randn(3, 16) assert out.size() == (3, 16) ```

EdgeIndex is implemented through extending torch.Tensor via the __torch_function__ interface (see here for the highly recommended tutorial).

EdgeIndex ensures for optimal computation in GNN message passing schemes, while preserving the ease-of-use of regular COO-based PyG workflows. EdgeIndex will fully deprecate the usage of SparseTensor from torch-sparse in later releases, leaving us with just a single source of truth for representing graph structure information in PyG.

RecSys Support

Previously, all/most of our link prediction models were trained and evaluated using binary classification metrics. However, this usually requires that we have a set of candidates in advance, from which we can then infer the existence of links. This is not necessarily practical, since in most cases, we want to find the top-k most likely links from the full set of O(N^2) pairs.

torch-geometric==2.5.0 brings full support for using GNNs as a recommender system (#8452), including support for * Maximum Inner Product Search (MIPS) via MIPSKNNIndex * Retrieval metrics such as f1@k, map@k, precision@k, recall@k and ndcg@k, including mini-batch support

```python mips = MIPSKNNIndex(dst_emb)

for srcbatch in srcloader: srcemb = model(srcbatch.xdict, srcbatch.edgeindexdict) , predindexmat = mips.search(srcemb, k)

for metric in retrieval_metrics:
     metric.update(pred_index_mat, edge_label_index)

for metric in retrieval_metrics: metric.compute() ```

See here for the accompanying example.

PyTorch 2.2 Support

PyG 2.5 is fully compatible with PyTorch 2.2 (#8857), and supports the following combinations:

| PyTorch 2.2 | cpu | cu118 | cu121 | |--------------|-------|---------|---------| | Linux | ✅ | ✅ | ✅ | | macOS | ✅ | | | | Windows | ✅ | ✅ | ✅ |

You can still install PyG 2.5 with an older PyTorch release up to PyTorch 1.12 in case you are not eager to update your PyTorch version.

Native `torch.compile(...)` and TorchScript Support

torch-geometric==2.5.0 introduces a full re-implementation of the MessagePassing interface, which makes it natively applicable to both torch.compile and TorchScript. As such, torch_geometric.compile is now fully deprecated in favor of torch.compile

diff - model = torch_geometric.compile(model) + model = torch.compile(model)

and MessagePassing.jittable() is now a no-op:

diff - conv = torch.jit.script(conv.jittable()) + model = torch.jit.script(conv)

In addition, torch.compile usage has been fixed to not require disabling of extension packages such as torch-scatter or torch-sparse.

New Tutorials, Examples, Models and Improvements

Tutorials
- Multi-Node Training using SLURM (#8071)
- Point Cloud Processing (#8015)
Examples
- Distributed training via torch_geometric.distributed (examples/distributed/pyg/) (#8713)
- Edge-level temporal sampling on a heterogeneous graph (examples/hetero/temporal_link_pred.py) (#8383)
- Edge-level temporal sampling on a heterogeneous graph with distributed training (examples/distributed/pyg/temporal_link_movielens_cpu.py) (#8820)
- Distributed training on XPU device (examples/multi_gpu/distributed_sampling_xpu.py) (#8032)
- Multi-node multi-GPU training on ogbn-papers100M (examples/multi_gpu/papers100m_gcn_multinode.py) (#8070)
- Naive model parallelism on multiple GPUs (examples/multi_gpu/model_parallel.py) (#8309)
Models
- Added the equivariant ViSNet from "ViSNet: an equivariant geometry-enhanced graph neural network with vector-scalar interactive message passing for molecules" (#8287)
Improvements
- Enabled multi-GPU evaluation in distributed sampling example (examples/multi_gpu/distributed_sampling.py) (#8880)

Breaking Changes

GATConv now initializes modules differently depending on whether their input is bipartite or non-bipartite (#8397). This will lead to issues when loading model state for GATConv layers trained on earlier PyG versions.

Deprecations

Deprecated torch_geometric.compile in favor of torch.compile (#8780)
Deprecated torch_geometric.nn.DataParallel in favor of torch.nn.parallel.DistributedDataParallel (#8250)
Deprecated MessagePassing.jittable (#8781, #8731)
Deprecated torch_geometric.data.makedirs in favor of os.makedirs (#8421)

Features

Package-wide Improvements - Added support for type checking via mypy (#8254) - Added fsspec as file system backend (#8379, #8426, #8434, #8474) - Added fallback code path for segment-based reductions in case torch-scatter is not installed (#8852)

Temporal Graph Support - Added support for edge-level temporal sampling in NeighborLoader and LinkNeighborLoader (#8372, #8428) - Added Data.{sort_by_time,is_sorted_by_time,snapshot,up_to} for temporal graph use-cases (#8454) - Added support for graph partitioning for temporal data in torch_geometric.distributed (#8718, #8815)

torch_geometric.datasets - Added the Risk Commodity Detection Dataset (RCDD) from "Datasets and Interfaces for Benchmarking Heterogeneous Graph Neural Networks" (#8196) - Added the StochasticBlockModelDataset(num_graphs: int) argument (#8648) - Added support for floating-point average degree numbers in FakeDataset and FakeHeteroDataset (#8404) - Added InMemoryDataset.to(device) (#8402) - Added the force_reload: bool = False argument to Dataset and InMemoryDataset in order to enforce re-processing of datasets (#8352, #8357, #8436) - Added the TreeGraph and GridMotif generators (#8736)

torch_geometric.nn - Added KNNIndex exclusion logic (#8573) - Added support for MRR computation in KGEModel.test() (#8298) - Added support for nn.to_hetero_with_bases on static graphs (#8247) - Addressed graph breaks in ModuleDict, ParameterDict, MultiAggregation and HeteroConv for better support for torch.compile (#8363, #8345, #8344)

torch_geometric.metrics - Added support for f1@k, map@k, precision@k, recall@k and ndcg@k metrics for link-prediction retrieval tasks (#8499, #8326, #8566, #8647)

torch_geometric.explain - Enabled skipping explanations of certain message passing layers via conv.explain = False (#8216) - Added support for visualizing explanations with node labels via visualize_graph(node_labels: list[str] | None) argument (#8816)

torch_geometric.transforms - Added a faster dense computation code path in AddRandomWalkPE (#8431)

Other Improvements - Added support for returning multi graphs in utils.to_networkx (#8575) - Added noise scheduler utilities utils.noise_scheduler.{get_smld_sigma_schedule,get_diffusion_beta_schedule} for diffusion-based graph generative models (#8347) - Added a relabel node functionality to utils.dropout_node via relabel_nodes: bool argument (#8524) - Added support for weighted utils.cross_entropy.sparse_cross_entropy (#8340) - Added support for profiling on XPU device via profile.profileit("xpu") (#8532) - Added METIS partitioning with CSC/CSR format selection in ClusterData (#8438)

Bugfixes

Fixed dummy value creation of boolean tensors in HeteroData.to_homogeneous() (#8858)
Fixed Google Drive download issues (#8804)
Fixed InMemoryDataset to reconstruct the correct data class when a pre_transform has modified it (#8692)
Fixed a bug in which transforms were not applied for OnDiskDataset (#8663)
Fixed mini-batch computation in DMoNPooing loss function (#8285)
Fixed NaN handling in SQLDatabase (#8479)
Fixed CaptumExplainer in case no index is passed (#8440)
Fixed edge_index construction in the UPFD dataset (#8413)
Fixed TorchScript support in AttentionalAggregation and DeepSetsAggregation (#8406)
Fixed GraphMaskExplainer for GNNs with more than two layers (#8401)
Fixed input_id computation in NeighborLoader in case a mask is given (#8312)
Respect current device when deep-copying Linear layers (#8311)
Fixed Data.subgraph()/HeteroData.subgraph() in case edge_index is not defined (#8277)
Fixed empty edge handling in MetaPath2Vec (#8248)
Fixed AttentionExplainer usage within AttentiveFP (#8244)
Fixed load_from_state_dict in lazy Linear modules (#8242)
Fixed pre-trained DimeNet++ performance on QM9 (#8239)
Fixed GNNExplainer usage within AttentiveFP (#8216)
Fixed to_networkx(to_undirected=True) in case the input graph is not undirected (#8204)
Fixed sparse-sparse matrix multiplication support on Windows in TwoHop and AddRandomWalkPE transformations (#8197, #8225)
Fixed mini-batching of HeteroData objects converted via ToSparseTensor() when torch-sparse is not installed (#8356)

Changes

Disallow the usage of add_self_loops=True in GCNConv(normalize=False) (#8210)
Changed the default inference mode for use_segment_matmul based on benchmarking results (from a heuristic-based version) (#8615)
Sparse node features in NELL and AttributedGraphDataset are now represented as torch.sparse_csr_tensor instead of torch_sparse.SparseTensor (#8679)
Accelerated mini-batching of torch.sparse tensors (#8670)
ExplainerDataset will now contain node labels for any motif generator (#8519)
Made utils.softmax faster via the in-house pyg_lib.ops.softmax_csr kernel (#8399)
Made utils.mask.mask_select faster (#8369)
Added a warning when calling Dataset.num_classes on regression datasets (#8550)

New Contributors

@stadlmax made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8207
@joaquincabezas made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8215
@SZiesche made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8201
@666even666 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8268
@irustandi made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8274
@joao-alex-cunha made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8223
@chaous made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8309
@songsong0425 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8298
@rachitk made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8356
@pmpalang made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8372
@flxmr made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8353
@GuyAglionby made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8401
@plutonium-239 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8404
@asherbondy made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8379
@wwang-chcn made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8285
@SimonPop made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8326
@brovatten made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8566
@XJTUNR made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8287
@kativenOG made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8648
@ilsenatorov made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8663
@dependabot made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8706
@Sutongtong233 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8736
@m-atalla made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8755
@A-LOST-WAPITI made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8818
@AtomicVar made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8825
@vahanhov made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8816
@rraadd88 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8842
@mashaan14 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8884

Full Changelog: https://github.com/pyg-team/pytorch_geometric/compare/2.4.0...2.5.0

- Python
Published by akihironitta about 2 years ago

pytorch_geometric - PyG 2.4.0: Model compilation, on-disk datasets, hierarchical sampling

We are excited to announce the release of PyG 2.4 🎉🎉🎉

Highlights
Breaking Changes
Deprecations
Features
Bugfixes
Full Changelog

PyG 2.4 is the culmination of work from 62 contributors who have worked on features and bug-fixes for a total of over 500 commits since torch-geometric==2.3.1.

Highlights

PyTorch 2.1 and `torch.compile(dynamic=True)` support

The long wait has an end! With the release of PyTorch 2.1, PyG 2.4 now brings full support for torch.compile to graphs of varying size via the dynamic=True option, which is especially useful for use-cases that involve the usage of DataLoader or NeighborLoader. Examples and tutorials have been updated to reflect this support accordingly (#8134), and models and layers in torch_geometric.nn have been tested to produce zero graph breaks:

```python import torch_geometric

model = torch_geometric.compile(model, dynamic=True) ```

When enabling the dynamic=True option, PyTorch will up-front attempt to generate a kernel that is as dynamic as possible to avoid recompilations when sizes change across mini-batches changes. As such, you should only ever not specify dynamic=True when graph sizes are guaranteed to never change. Note that dynamic=True requires PyTorch >= 2.1.0 to be installed.

PyG 2.4 is fully compatible with PyTorch 2.1, and supports the following combinations:

| PyTorch 2.1 | cpu | cu118 | cu121 | |--------------|-------|---------|---------| | Linux | ✅ | ✅ | ✅ | | macOS | ✅ | | | | Windows | ✅ | ✅ | ✅ |

You can still install PyG 2.4 on older PyTorch releases up to PyTorch 1.11 in case you are not eager to update your PyTorch version.

`OnDiskDataset` Interface

We added the OnDiskDataset base class for creating large graph datasets (e.g., molecular databases with billions of graphs), which do not easily fit into CPU memory at once (#8028, #8044, #8046, #8051, #8052, #8054, #8057, #8058, #8066, #8088, #8092, #8106). OnDiskDataset leverages our newly introduced Database backend (sqlite3 by default) for on-disk storage and access of graphs, supports DataLoader out-of-the-box, and is optimized for maximum performance.

OnDiskDataset utilizes a user-specified schema to store data as efficient as possible (instead of Python pickling). The schema can take int, float str, object or a dictionary with dtype and size keys (for specifying tensor data) as input, and can be nested as a dictionary. For example,

python dataset = OnDiskDataset(root, schema={ 'x': dict(dtype=torch.float, size=(-1, 16)), 'edge_index': dict(dtype=torch.long, size=(2, -1)), 'y': float, })

creates a database with three columns, where x and edge_index are stored as binary data, and y is stored as a float.

Afterwards, you can append data to the OnDiskDataset and retrieve data from it via dataset.append()/dataset.extend(), and dataset.get()/dataset.multi_get(), respectively. We added a fully working example on how to set up your own OnDiskDataset here (#8102). You can also convert in-memory dataset instances to an OnDiskDataset instance by running InMemoryDataset.to_on_disk_dataset() (#8116).

Neighbor Sampling Improvements

Hierarchical Sampling

One drawback of NeighborLoader is that it computes a representations for all sampled nodes at all depths of the network. However, nodes sampled in later hops no longer contribute to the node representations of seed nodes in later GNN layers, thus performing useless computation. NeighborLoader will be marginally slower since we are computing node embeddings for nodes we no longer need. This is a trade-off we have made to obtain a clean, modular and experimental-friendly GNN design, which does not tie the definition of the model to its utilized data loader routine.

With PyG 2.4, we introduced the option to eliminate this overhead and speed-up training and inference in mini-batch GNNs further, which we call "Hierarchical Neighborhood Sampling" (see here for the full tutorial) (#6661, #7089, #7244, #7425, #7594, #7942). Its main idea is to progressively trim the adjacency matrix of the returned subgraph before inputting it to each GNN layer, and works seamlessly across several models, both in the homogeneous and heterogeneous graph setting. To support this trimming and implement it effectively, the NeighborLoader implementation in PyG and in pyg-lib additionally return the number of nodes and edges sampled in hop, which are then on a per-layer basis to trim the adjacency matrix and the various feature matrices to only maintain the required amount (see the trim_to_layer method):

```python class GNN(torch.nn.Module): def init(self, inchannels: int, outchannels: int, numlayers: int): super().init_()

    self.convs = ModuleList([SAGEConv(in_channels, 64)])
    for _ in range(num_layers - 1):
        self.convs.append(SAGEConv(hidden_channels, hidden_channels))
    self.lin = Linear(hidden_channels, out_channels)

def forward(
    self,
    x: Tensor,
    edge_index: Tensor,
    num_sampled_nodes_per_hop: List[int],
    num_sampled_edges_per_hop: List[int],
) -> Tensor:

    for i, conv in enumerate(self.convs):
        # Trim edge and node information to the current layer `i`.
        x, edge_index, _ = trim_to_layer(
            i, num_sampled_nodes_per_hop, num_sampled_edges_per_hop,
            x, edge_index)

        x = conv(x, edge_index).relu()

    return self.lin(x)

```

Corresponding examples can be found here and here.

Biased Sampling

Additionally, we added support for weighted/biased sampling in NeighborLoader/LinkNeighborLoader scenarios. For this, simply specify your edge_weight attribute during NeighborLoader initialization, and PyG will pick up these weights to perform weighted/biased sampling (#8038):

```python data = Data(numnodes=5, edgeindex=edgeindex, edgeweight=edge_weight)

loader = NeighborLoader( data, numneighbors=[10, 10], weightattr='edge_weight', )

batch = next(iter(loader)) ```

New models, datasets, examples & tutorials

As part of our algorithm and documentation sprints (#7892), we have added:

Model components:
- MixHopConv: “MixHop: Higher-Order Graph Convolutional Architecturesvia Sparsified Neighborhood Mixing” (examples/mixhop.py) (#8025)
- LCMAggregation: “Learnable Commutative Monoids for Graph Neural Networks” (examples/lcm_aggr_2nd_min.py) (#7976, #8020, #8023, #8026, #8075)
- DirGNNConv: “Edge Directionality Improves Learning on Heterophilic Graphs” (examples/dir_gnn.py) (#7458)
- Support for Performer in GPSConv: “Recipe for a General, Powerful, Scalable Graph Transformer” (examples/graph_gps.py) (#7465)
- PMLP: “Graph Neural Networks are Inherently Good Generalizers: Insights by Bridging GNNs and MLPs” (examples/pmlp.py) (#7470, #7543)
- RotateE: “RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space” (examples/kge_fb15k_237.py) (#7026)
- NeuralFingerprint: “Convolutional Networks on Graphs for Learning Molecular Fingerprints” (#7919)
Datasets: HM (#7515), BrcaTcga (#7994), MyketDataset (#7959), Wikidata5M (#7864), OSE_GVCS (#7811), MovieLens1M (#7479), AmazonBook (#7483), GDELTLite (#7442), IGMCDataset (#7441), MovieLens100K (#7398), EllipticBitcoinTemporalDataset (#7011), NeuroGraphDataset (#8112), PCQM4Mv2 (#8102)
Tutorials:
- Scaling GNNs via NeighborLoader (#7931)
- Shallow Node Embeddings via Node2Vec and MetaPath2Vec (#7938)
- Multi-GPU training with pure PyTorch (#7894)
- Hierarchical Neighborhood Sampling (#7594)
Examples:
- Heterogeneous link-level GNN explanations via CaptumExplainer (examples/captum_explainer_hetero_link.py) (#7096)
- Training LightGCN on AmazonBook for recommendation (examples/lightgcn.py) (#7603)
- Using the Kùzu remote backend as FeatureStore (examples/kuzu) (#7298)
- Multi-GPU training on ogbn-papers100M (examples/papers100m_multigpu.py) (#7921)
- The OGC model on Cora (examples/ogc.py) (#8168)
- Distributed training via graphlearn-for-pytorch (examples/distributed/graphlearn_for_pytorch) (#7402)

Join our Slack here if you're interested in joining community sprints in the future!

Breaking Changes

Data.keys() is now a method instead of a property (#7629):

<=2.3	2.4
`data = Data(x=x, edgeindex=edgeindex) print(data.keys) # ['x', 'edgeindex']`	`data = Data(x=x, edgeindex=edgeindex) print(data.keys()) # ['x', 'edgeindex']`

Dropped Python 3.7 support (#7939)
Removed FastHGTConv in favor of HGTConv (#7117)
Removed the layer_type argument from GraphMaskExplainer (#7445)
Renamed dest argument to dst in utils.geodesic_distance (#7708)

Deprecations

Deprecated contrib.explain.GraphMaskExplainer in favor of explain.algorithm.GraphMaskExplainer (#7779)

Features

Data and HeteroData improvements - Added a warning for isolated/non-existing node types in HeteroData.validate() (#7995) - Added HeteroData support in to_networkx (#7713) - Added Data.sort() and HeteroData.sort() (#7649) - Added padding capabilities to HeteroData.to_homogeneous() in case feature dimensionalities do not match (#7374) - Added torch.nested_tensor support in Data and Batch (#7643, #7647) - Added keep_inter_cluster_edges option to ClusterData to support inter-subgraph edge connections when doing graph partitioning (#7326)

Data-loading improvements - Added support for floating-point slicing in Dataset, e.g., dataset[:0.9] (#7915) - Added save and load methods to InMemoryDataset (#7250, #7413) - Beta: Added IBMBNodeLoader and IBMBBatchLoader data loaders (#6230) - Beta: Added HyperGraphData to support hypergraphs (#7611) - Added CachedLoader (#7896, #7897) - Allowed GPU tensors as input to NodeLoader and LinkLoader (#7572) - Added PrefetchLoader capabilities (#7376, #7378, #7383) - Added manual sampling interface to NodeLoader and LinkLoader (#7197)

Better support for sparse tensors - Added SparseTensor support to WLConvContinuous, GeneralConv, PDNConv and ARMAConv (#8013) - Change torch_sparse.SparseTensor logic to utilize torch.sparse_csr instead (#7041) - Added support for torch.sparse.Tensor in DataLoader (#7252) - Added support for torch.jit.script within MessagePassing layers without torch_sparse being installed (#7061, #7062) - Added unbatching logic for torch.sparse.Tensor (#7037) - Added support for Data.num_edges for native torch.sparse.Tensor adjacency matrices (#7104) - Accelerated sparse tensor conversion routines (#7042, #7043) - Added a sparse cross_entropy implementation (#7447, #7466)

Integration with 3rd-party libraries - Added FlopsCount support via fvcore (#7693) - Added to_dgl and from_dgl conversion functions (#7053)

torch_geometric.transforms - All transforms are now immutable, i.e. they perform a shallow-copy of the data and therefore do not longer modify data in-place (#7429) - Added the HalfHop graph upsampling augmentation (#7827) - Added interval argument to Cartesian, LocalCartesian and Distance transformations (#7533, #7614, #7700) - Added an optional add_pad_mask argument to the Pad transform (#7339) - Added NodePropertySplit transformation for creating node-level splits using structural node properties (#6894) - Added a AddRemainingSelfLoops transformation (#7192)

Bugfixes

Fixed HeteroConv for layers that have a non-default argument order, e.g., GCN2Conv (#8166)
Handle reserved keywords as keys in ModuleDict and ParameterDict (#8163)
Fixed DynamicBatchSampler.__len__ to raise an error in case num_steps is undefined (#8137)
Enabled pickling of DimeNet models (#8019)
Fixed a bug in which batch.e_id was not correctly computed on unsorted graph inputs (#7953)
Fixed from_networkx conversion from nx.stochastic_block_model graphs (#7941)
Fixed the usage of bias_initializer in HeteroLinear (#7923)
Fixed broken URLs in HGBDataset (#7907)
Fixed an issue where SetTransformerAggregation produced NaN values for isolates nodes (#7902)
Fixed summary on modules with uninitialized parameters (#7884)
Fixed tracing of add_self_loops for a dynamic number of nodes (#7330)
Fixed device issue in PNAConv.get_degree_histogram (#7830)
Fixed the shape of edge_label_time when using temporal sampling on homogeneous graphs (#7807)
Fixed edge_label_index computation in LinkNeighborLoader for the homogeneous+disjoint mode (#7791)
Fixed CaptumExplainer for binary classification tasks (#7787)
Raise error when collecting non-existing attributes in HeteroData (#7714)
Fixed get_mesh_laplacian for normalization="sym" (#7544)
Use dim_size to initialize output size of the EquilibriumAggregation layer (#7530)
Fixed empty edge indices handling in SparseTensor (#7519)
Move the scaler tensor in GeneralConv to the correct device (#7484)
Fixed HeteroLinear bug when used via mixed precision (#7473)
Fixed gradient computation of edge weights in utils.spmm (#7428)
Fixed an index-out-of-range bug in QuantileAggregation when dim_size is passed (#7407)
Fixed a bug in LightGCN.recommendation_loss() to only use the embeddings of the nodes involved in the current mini-batch (#7384)
Fixed a bug in which inputs where modified in-place in to_hetero_with_bases (#7363)
Do not load node_default and edge_default attributes in from_networkx (#7348)
Fixed HGTConv utility function _construct_src_node_feat (#7194)
Fixed subgraph on unordered inputs (#7187)
Allow missing node types in HeteroDictLinear (#7185)
Fix numpy incompatiblity when reading files for Planetoid datasets (#7141)
Fixed crash of heterogeneous data loaders if node or edge types are missing (#7060, #7087)
Allowed CaptumExplainer to be called multiple times in a row (#7391)

Changes

Enabled dense eigenvalue computation in AddLaplacianEigenvectorPE for small-scale graphs (#8143)
Accelerated and simplified top_k computation in TopKPooling (#7737)
Updated GIN implementation in benchmarks to apply sequential batch normalization (#7955)
Updated QM9 data pre-processing to include the SMILES string (#7867)
Warn user when using the training flag in to_hetero modules (#7772)
Changed add_random_edge to only add true negative edges (#7654)
Allowed the usage of BasicGNN models in DeepGraphInfomax (#7648)
Added a num_edges parameter to the forward method of HypergraphConv (#7560)
Added a max_num_elements parameter to the forward method of GraphMultisetTransformer, GRUAggregation, LSTMAggregation, SetTransformerAggregation and SortAggregation (#7529, #7367)
Re-factored ClusterLoader to integrate pyg-lib METIS routine (#7416)
The filter_per_worker option will not get automatically inferred by default based on the device of the underlying data (#7399)
Added the option to pass fill_value as a torch.tensor to utils.to_dense_batch (#7367)
Updated examples to use NeighborLoader instead of NeighborSampler (#7152)
Extend dataset summary to create stats for each node/edge type (#7203)
Added an optional batch_size argument to avg_pool_x and max_pool_x (#7216)
Optimized from_networkx memory footprint by reducing unnecessary copies (#7119)
Added an optional batch_size argument to LayerNorm, GraphNorm, InstanceNorm, GraphSizeNorm and PairNorm (#7135)
Accelerated attention-based MultiAggregation (#7077)
Edges in HeterophilousGraphDataset are now undirected by default (#7065)
Added an optional batch_size and max_num_nodes arguments to MemPooling layer (#7239)

Full Changelog

Full Changelog: https://github.com/pyg-team/pytorch_geometric/compare/2.3.0...2.4.0

New Contributors

@zoryzhang made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7027
@DomInvivo made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7037
@OlegPlatonov made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7065
@hbenedek made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7053
@rishiagarwal2000 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7011
@sisaman made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7104
@amorehead made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7110
@EulerPascal404 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7093
@Looong01 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7143
@kamil-andrzejewski made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7135
@andreazanetti made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7089
@akihironitta made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7195
@kjkozlowski made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7216
@vstenby made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7221
@piotrchmiel made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7239
@vedal made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7272
@gvbazhenov made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6894
@Saydemr made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7313
@HaoyuLu1022 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7325
@Vuenc made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7330
@mewim made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7298
@volltin made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7355
@kasper-piskorski made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7377
@happykygo made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7384
@ThomasKLY made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7398
@sky-2002 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7421
@denadai2 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7456
@chrisgo-gc made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7484
@furkanakkurt1335 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7507
@mzamini92 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7497
@n-patricia made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7543
@SalvishGoomanee made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7573
@emalgorithm made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7458
@marshka made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7595
@djm93dev made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7598
@NripeshN made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7770
@ATheCoder made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7774
@ebrahimpichka made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7775
@kaidic made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7814
@Wesxdz made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7811
@daviddavo made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7888
@frinkleko made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7907
@chendiqian made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7917
@rajveer43 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7885
@erfanloghmani made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7959
@xnuohz made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7937
@Favourj-bit made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7905
@apfelsinecode made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7996
@ArchieGertsman made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7976
@bkmi made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8019
@harshit5674 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7919
@erikhuck made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8024
@jay-bhambhani made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8028
@Barcavin made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8049
@royvelich made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8048
@CodeTal made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/7611
@filipekstrm made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8117
@Anwar-Said made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8122
@xYix made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/8168

- Python
Published by akihironitta over 2 years ago

pytorch_geometric - Pyg 2.3.1: Bugfixes

PyG 2.3.1 includes a variety of bugfixes.

Bug Fixes

Fixed cugraph GNN layer support for pylibcugraphops==23.04 (#7023)
Removed DeprecationWarning of TypedStorage usage in DataLoader (#7034)
Fixed a bug in FastHGTConv that computed values via parameters used to compute the keys (#7050)
Fixed numpy incompatiblity when reading files in Planetoid datasets (#7141)
Fixed utils.subgraph on unordered inputs (#7187)
Fixed support for Data.num_edges for native torch.sparse.Tensor adjacency matrices (#7104)

Full Changelog: https://github.com/pyg-team/pytorch_geometric/compare/2.3.0...2.3.1

- Python
Published by rusty1s almost 3 years ago

pytorch_geometric - Pyg 2.3.0: PyTorch 2.0 support, native sparse tensor support, explainability and accelerations

We are thrilled to announce the release of PyG 2.3 🎉

Highlights
Breaking Changes
Deprecations
Features
Bugfixes
Full Changelog

PyG 2.3 is the culmination of work from 59 contributors who have worked on features and bug-fixes for a total of over 470 commits since torch-geometric==2.2.0.

Highlights

PyTorch 2.0 Support

PyG 2.3 is fully compatible with the next generation release of PyTorch, bringing many new innovations and features such as torch.compile() and Python 3.11 support to PyG out-of-the-box. In particular, many PyG models and functions are speeded up significantly using torch.compile() in torch >= 2.0.0.

We have prepared a full tutorial and a set of examples to get you going with torch.compile() immediately:

```python import torchgeometric from torchgeometric.nn import GraphSAGE

model = GraphSAGE(inchannels, hiddenchannels, numlayers, outchannels) model = model.to(device)

model = torch_geometric.compile(model) ```

Overall, we observed runtime improvements of nearly up to 300%:

Model | Mode | Forward | Backward | Total | Speedup -- | -- | -- | -- | -- | -- GCN | Eager | 2.6396s | 2.1697s | 4.8093s | GCN | Compiled | 1.1082s | 0.5896s | 1.6978s | 2.83x GraphSAGE | Eager | 1.6023s | 1.6428s | 3.2451s | GraphSAGE | Compiled | 0.7033s | 0.7465s | 1.4498s | 2.24x GIN | Eager | 1.6701s | 1.6990s | 3.3690s | GIN | Compiled | 0.7320s | 0.7407s | 1.4727s | 2.29x

Please note that torch.compile() within PyG is in beta mode and under active development. For example, currently torch.compile(model, dynamic=True) does not yet work seamlessly, but fixes are on its way. We are very eager to improve its support across the whole PyG code base, so do not hesitate to reach out if you notice anything unexpected.

Infrastructure Changes

With the recent upstreams of torch-scatter and torch-sparse to native PyTorch, we are happy to announce that any installation of the extension packages torch-scatter, torch-sparse, torch-cluster and torch-spline-conv is now fully optional.

All it takes to install PyG is now encapsulated into a single command pip install torch-geometric and finally resolves a lot of previous installation issues.

Extension packages are still picked up for the following use-cases (if installed):

pyg-lib: Heterogeneous GNN operators and graph sampling routines like NeighborLoader
torch-scatter: Accelerated "min" and "max" reductions
torch-sparse: SparseTensor support
torch-cluster: Graph clustering routines like knn or radius
torch-spline-conv: SplineConv support

We recommend to start with a minimal installation, and only install additional dependencies once you actually get notified about them being missing during PyG usage.

Native PyTorch Sparse Tensor Support

With the recent additions of torch.sparse_csr_tensor and torch.sparse_csc_tensor classes and accelerated sparse matrix multiplication routines to PyTorch, we finally enable MessagePassing on pure PyTorch sparse tensors as well. In particular, you can now use torch.sparse_csr_tensor and torch.sparse_csc_tensor as a drop-in replacement for torch_sparse.SparseTensor:

```python from torchgeometric.nn import GCN import torchgeometric.transforms as T from torch_geometric.datasets import Planetoid

transform = T.ToSparseTensor(layout=torch.sparse_csr) dataset = Planetoid("Planetoid", name="Cora", transform=transform)

model = GCN(inchannels, hiddenchannels, numlayers=2) model = model(data.x, data.adjt) ```

Nearly all of the native PyG layers have been tested to work seamlessly with native PyTorch sparse tensors (#5906, #5944, #6003, #6033, #6514, #6532, #6748, #6847, #6868, #6874, #6897, #6930, #6932, #6936, #6937, #6939, #6947, #6950, #6951, #6957).

Explainability Framework

In PyG 2.2 we introduced the torch_geometric.explain package that provides a flexible interface to generate and visualize GNN explanations using various algorithms. We are happy to add the following key improvements on this front:

Support for explaining heterogeneous GNNs via HeteroExplanation
New visualization tools to visualize_feature_importance and to visualize_graph explanations
May new datasets and metrics to evaluate explanation algorithms
Several new explanation algorithms such as CaptumExplainer, PGExplainer, AttentionExplainer, PGMExplainer, and GraphMaskExplainer
Support for node-level, link-level and graph-level explanations

Using the new explainer interface is as simple as: ```python explainer = Explainer( model=model, algorithm=CaptumExplainer('IntegratedGradients'), explanationtype='model', modelconfig=dict( mode='multiclassclassification', tasklevel='node', returntype='logprobs', ), nodemasktype='attributes', edgemasktype='object', )

explanation = explainer(data.x, data.edge_index) ```

Read more about torch_geometric.explain in our newly added tutorial and example scripts. We also added a blog post that describes the new interface and functionality in depth.

Accelerations

Together with Intel and NVIDIA, we are excited about new PyG accelerations:

[Experimental] Support for native cugraph-based GNN layers for commonly used layers such as CuGraphSAGEConv, CuGraphGATConv, and CuGraphRGCNConv (#6278, #6388, #6412):

RGCN with neighbor sampling:

Dataset | CuGraphRGCNConv (ms) | FastRGCNConv (ms) | RGCNConv (ms) -- | -- | -- | -- AIFB | 7,2 | 13,4 | 70 BGS | 7,1 | 8,8 | 146,9 MUTAG | 8,3 | 21,8 | 47,6 AM | 17,5 | 51 | 330,1

Full-batch GAT:

Dataset | CuGraphGATConv (ms) | GATConv (ms) -- | -- | -- Cora | 7 | 8,7 Citeseer | 7 | 9 Pubmed | 8,2 | 11,4

GraphSAGE with neighbor sampling:

Dataset | CuGraphSAGEConv (ms) | SAGEConv (ms) -- | -- | -- ogbn-products | 2591,8 | 3040,3

A fast alternative to HGTConv via FastHGTConv that utilizes pyg-lib integration for improved runtimes. Overall, FastHGTConv achieves a speed-up of approximately 300% compared to the original implementation (#6178).
A fast implementation of HeteroDictLinear that utilizes pyg-lib integration for improved runtimes (#6178).
GNN inference and training optimizations on CPU within native PyTorch 2.0. Optimizations include:
- scatter_reduce: performance hotspot in message passing when edge_index is stored in Coordinate format (COO).
- gather: backward of scatter_reduce, specially tuned for the GNN compute when the index is an expanded tensor.
- torch.sparse.mm with reduce flag: performance hotspot in message passing when the edge_index is stored in Compressed Sparse Row (CSR). Supported reduce flags are "sum", "mean", "amax" and "amin". On OGB benchmarks, a 1.12x - 4.07x performance speedup is measured (PyTorch 1.13.1 vs PyTorch 2.0) for single node inference and training.
Introduction of index_sort via pyg-lib>=0.2.0, which implements a (way) faster alternative to sorting one-dimensional indices compared to torch.sort (#6554). Overall, this achieves speed-ups in dataset loading times by up to 600%.
Introduction of AffinityMixin to accelerate PyG workflows on CPU. CPU affinity can be enabled via the AffinityMixin.enable_cpu_affinity() method for num_workers > 0 data loading use-cases, and will guarantee that a separate core is assigned to each worker at initialization. Over all benchmarked model/dataset samples, the average training time is decreased by up to 1.85x. We added an in-depth tutorial on how to speed-up your PyG workflows on CPU.

Additional Highlights

Documentation Revamp

The documentation has undergone a revision of design and structure, making it faster to load and easier to navigate. Take a look at its new design here.

Community Sprint: Improving Code Coverage

We had our third community sprint in the last two weeks of January. The goal was to improve code coverage by writing more thorough tests. Thanks to the efforts of many contributors, the total code coverage went from ~85% to ~92% (#6528, #6523, #6538, #6555, #6558, #6568, #6573, #6578, #6597, #6600, #6618, #6619, #6621, #6623, #6637, #6638, #6640, #6645, #6648, #6647, #6653, #6657, #6662, #6664, #6667, #6668, #6669, #6670, #6671, #6673, #6675, #6676, #6677, #6678, #6681, #6683, #6703, #6720, #6735, #6736, #6763, #6781, #6797, #6799, #6824, #6858)

Breaking Changes

Temporal sampling in NeighborLoader will now also sample nodes with an equal timestamp to the seed time. Changed from sampling only nodes with a smaller timestamp (requires pyg-lib>=0.2.0) (#6517)
Changed the interface and implementation of GraphMultisetTransformer such that GNN execution is no longer performed inside its module (#634))
Unify Explanation.node_mask and Explanation.node_feat_mask into a single attribute in Explainer (#6267)
Moved ExplainerConfig arguments to the Explainer class (#6176)
Moved PyTorch Lightning data modules to torch_geometric.data.lightning (#6140)
Removed target_index argument in the Explainer interface (#6270)
Removed the Aggregation.set_validate_args option (#6175)
Removed the usage of __dunder__ names in MessagePassing (#6999)

Deprecations

The usage of datasets.BAShapes is now deprecated. Use the BAGraph graph generator to generate Barabasi-Albert graphs instead (#6072)

Features

Layers, Models and Examples

Added the DenseGATConv layer (https://github.com/pyg-team/pytorch_geometric/pull/6928)
Added the DistMult KGE model (https://github.com/pyg-team/pytorch_geometric/pull/6958)
Added the ComplEx KGE model (#6898)
Added the TransE KGE model (#6314)
Added HeteroLayerNorm and HeteroBatchNorm layers (#6838)
Added a TemporalEncoding module (#6785)
Added the SimpleConv to perform non-trainable propagation (#6718)
Added torch.jit examples for example/film.py and example/gcn.py (#6602)
Added the AntiSymmetricConv layer (#6577)
Added a PyGModelHubMixin for Huggingface model hub integration (#5930, #6591)
Added the PGMExplainer (#6149, #6588, #6589)
Added internal ToHeteroLinear and ToHeteroMessagePassing modules to accelerate to_hetero functionality (#5992, #6456)
Added the GraphMaskExplainer (#6284)
Added the GRBCDAttack and PRBCDAttack adversarial attack models (#5972)
Added the CaptumExplainer (#6383, #6387, #6433, #6487)
Added the GNNFF model (#5866)
Added MLPAggregation, SetTransformerAggregation, GRUAggregation, and DeepSetsAggregation as adaptive readout functions (#6301, #6336, #6338) (https://github.com/pyg-team/pytorch_geometric/pull/6331), #6332)
Added the GPSConv Graph Transformer layer (#6326, #6327)
Added the PGExplainer (#6204)
Added the AttentionExplainer (#6279)
Added the PointGNNConv layer (#6194)
Added the RandLA-Net architecture as classification and segmentation examples (#5117)

Datasets

Added the AirfRANS dataset (#6287)
Added the HeterophilousGraphDataset suite (#6846)
Added support for the revised version of the MD17 dataset (#6734)
Added the BAMultiShapesDataset (#6541)
Added the Taobao dataset and a corresponding example (#6144)
Added the Freebase FB15k_237 dataset (#3204)
Added the BA2MotifDataset explainer dataset (#6257)
Added the CycleMotif motif generator to generate n-node cycle shaped motifs (#6256)
Added the InfectionDataset to evaluate explanations (#6222)
Added a CustomMotif motif generator (#6179)
Added the ERGraph graph generator to generate Ergos-Renyi (ER) graphs (#6073)
Added a general ExplainerDataset to evaluate explanation methods (#6104)

Loaders

Enabled NeighborLoader to return the number of sampled nodes and edges per hop, and added corresponding trim_to_layer functionality for more efficient NeighborLoader use-cases (#6661, #6834)
Added a ZipLoader to execute multiple NodeLoader or LinkLoader instances (#6829)
Added a seed_time attribute to temporal NodeLoader outputs in case input_time is given (#6196)

Transformations

Added the Pad transformation (#5940, #6697, #6731, #6758)
Added a RemoveDuplicatedEdges transformation (#6709)

General Improvements

Added a memory-efficient utils.one_hot implementation (https://github.com/pyg-team/pytorch_geometric/pull/7005)
Optimized utils.softmax implementation (#6113, #6155, #6805)
Optimized topk implementation for graph pooling on large graphs (#6123)
Added common utils.select and utils.narrow functionality to support filtering of both tensors and lists (#6162)
Support normalization customization in get_mesh_laplacian (#6790)
Added CPU-optimized spmm functionality via CSR format (#6699, #6759)
Added TorchScript support to the RECT_L model (#6727)
Added TorchScript support to the Node2Vec model (#6726)
Added utils.to_edge_index to convert sparse tensors to edge indices and edge attributes (#6728)
Added TorchScript support to the LINKX model (#6712)
Added dropout option to GraphMultisetTransformer (#6484)
Added option to customize loader arguments for evaluation in LightningNodeData and LightningLinkData (#6450, #6456)
Added option to customize num_neighbors in NeighborSampler after instantiation (#6446)
Added support to define a custom HeteroData mini-batch class in remote backends (#6377)
Allow the usage of ChebConv within GNNExplainer (https://github.com/pyg-team/pytorch_geometric/pull/6778)
Added Dataset.to_datapipe functionality for converting PyG datasets into a PyTorch DataPipe(#6141)
Added to_nested_tensor and from_nested_tensor functionality (#6329, #6330, [#6331]
Added networkit conversion utilities (#6321)
Added Data.update() and HeteroData.update functionality (#6313)
Added HeteroData.set_value_dict functionality (https://github.com/pyg-team/pytorchgeometric/pull/6961, https://github.com/pyg-team/pytorchgeometric/pull/6974)
Added the (un)faithfulness explainability metric (#6090)
Added the fidelity explainability metric (#6116, #6510)
Added characterization_score and fidelity_curve_auc explainer metrics (#6188)
Added subgraph visualization of GNN explanations (#6235, #6271)
Added a weighted negative sampling option in LinkNeighborLoader (#6264)
Added a get_embeddings function (#6201)
Added Explanation.visualize_feature_importance to support node feature importance visualizations (#6094)
Added heterogeneous graph support to explainers via HeteroExplanation (#6091, #6218)
Added a summary method for PyG/PyTorch models (#5859, #6161)
Added an input_time option to LightningNodeData and transform_sampler_output to NodeLoader and LinkLoader (#6187)
Added Data.edge_subgraph and HeteroData.edge_subgraph functionalities (#6193)

Bug Fixes

Fixed a bug in Data.subgraph() and HeteroData.subgraph() for bipartite graphs (#6613, #6654)
Fixed a bug in PNAConv and DegreeScalerAggregation to correctly incorporate degree statistics of isolated nodes (#6609)
Fixed a bug in which Data.to_heterogeneous filtered attributes in the wrong dimension (#6522)
Fixed a bug in to_hetero when using an uninitialized submodule without implementing reset_parameters (#6863)
Fixed a bug in get_mesh_laplacian (#6790)
Fixed a bug in which masks were not properly masked in GNNExplainer on link prediction tasks (#6787)
Fixed the ImbalancedSampler when operating on a sliced InMemoryDataset (#6374)
Fixed the approximate PPR variant in transforms.GDC to not crash on graphs with isolated nodes (#6242)
Fixed the filtering of node features in transforms.RemoveIsolatedNodes (#6308)
Fixed a bug in DimeNet that causes an output dimension mismatch (#6305)
Fixed Data.to_heterogeneous when used with an empty edge_index (#6304)
Fixed a bug in the output order in HeteroLinear for un-sorted type vectors (#6198)

Full Changelog

Added

- Added a memory-efficient `utils.one_hot` implementation ([#7005](https://github.com/pyg-team/pytorch_geometric/pull/7005)) - Added `HeteroDictLinear` and an optimized `FastHGTConv` module ([#6178](https://github.com/pyg-team/pytorch_geometric/pull/6178), [#6998](https://github.com/pyg-team/pytorch_geometric/pull/6998)) - Added the `DenseGATConv` module ([#6928](https://github.com/pyg-team/pytorch_geometric/pull/6928)) - Added `trim_to_layer` utility function for more efficient `NeighborLoader` use-cases ([#6661](https://github.com/pyg-team/pytorch_geometric/pull/6661)) - Added the `DistMult` KGE model ([#6958](https://github.com/pyg-team/pytorch_geometric/pull/6958)) - Added `HeteroData.set_value_dict` functionality ([#6961](https://github.com/pyg-team/pytorch_geometric/pull/6961), [#6974](https://github.com/pyg-team/pytorch_geometric/pull/6974)) - Added PyTorch >= 2.0 support ([#6934](https://github.com/pyg-team/pytorch_geometric/pull/6934), [#7000](https://github.com/pyg-team/pytorch_geometric/pull/7000)) - Added PyTorch Lightning >= 2.0 support ([#6929](https://github.com/pyg-team/pytorch_geometric/pull/6929)) - Added the `ComplEx` KGE model ([#6898](https://github.com/pyg-team/pytorch_geometric/pull/6898)) - Added option to write benchmark results to csv ([#6888](https://github.com/pyg-team/pytorch_geometric/pull/6888)) - Added `HeteroLayerNorm` and `HeteroBatchNorm` layers ([#6838](https://github.com/pyg-team/pytorch_geometric/pull/6838)) - Added the `HeterophilousGraphDataset` suite ([#6846](https://github.com/pyg-team/pytorch_geometric/pull/6846)) - Added support for sparse tensor in full batch mode inference benchmark ([#6843](https://github.com/pyg-team/pytorch_geometric/pull/6843)) - Enabled `NeighborLoader` to return number of sampled nodes and edges per hop ([#6834](https://github.com/pyg-team/pytorch_geometric/pull/6834)) - Added `ZipLoader` to execute multiple `NodeLoader` or `LinkLoader` instances ([#6829](https://github.com/pyg-team/pytorch_geometric/issues/6829)) - Added common `utils.select` and `utils.narrow` functionality to support filtering of both tensors and lists ([#6162](https://github.com/pyg-team/pytorch_geometric/issues/6162)) - Support `normalization` customization in `get_mesh_laplacian` ([#6790](https://github.com/pyg-team/pytorch_geometric/issues/6790)) - Added the `TemporalEncoding` module ([#6785](https://github.com/pyg-team/pytorch_geometric/pull/6785)) - Added CPU-optimized `spmm_reduce` functionality via CSR format ([#6699](https://github.com/pyg-team/pytorch_geometric/pull/6699), [#6759](https://github.com/pyg-team/pytorch_geometric/pull/6759)) - Added support for the revised version of the `MD17` dataset ([#6734](https://github.com/pyg-team/pytorch_geometric/pull/6734)) - Added TorchScript support to the `RECT_L` model ([#6727](https://github.com/pyg-team/pytorch_geometric/pull/6727)) - Added TorchScript support to the `Node2Vec` model ([#6726](https://github.com/pyg-team/pytorch_geometric/pull/6726)) - Added `utils.to_edge_index` to convert sparse tensors to edge indices and edge attributes ([#6728](https://github.com/pyg-team/pytorch_geometric/issues/6728)) - Fixed expected data format in `PolBlogs` dataset ([#6714](https://github.com/pyg-team/pytorch_geometric/issues/6714)) - Added `SimpleConv` to perform non-trainable propagation ([#6718](https://github.com/pyg-team/pytorch_geometric/pull/6718)) - Added a `RemoveDuplicatedEdges` transform ([#6709](https://github.com/pyg-team/pytorch_geometric/pull/6709)) - Added TorchScript support to the `LINKX` model ([#6712](https://github.com/pyg-team/pytorch_geometric/pull/6712)) - Added `torch.jit` examples for `example/film.py` and `example/gcn.py`([#6602](https://github.com/pyg-team/pytorch_geometric/pull/6692)) - Added `Pad` transform ([#5940](https://github.com/pyg-team/pytorch_geometric/pull/5940), [#6697](https://github.com/pyg-team/pytorch_geometric/pull/6697), [#6731](https://github.com/pyg-team/pytorch_geometric/pull/6731), [#6758](https://github.com/pyg-team/pytorch_geometric/pull/6758)) - Added full batch mode to the inference benchmark ([#6631](https://github.com/pyg-team/pytorch_geometric/pull/6631)) - Added `cat` aggregation type to the `HeteroConv` class so that features can be concatenated during grouping ([#6634](https://github.com/pyg-team/pytorch_geometric/pull/6634)) - Added `torch.compile` support and benchmark study ([#6610](https://github.com/pyg-team/pytorch_geometric/pull/6610), [#6952](https://github.com/pyg-team/pytorch_geometric/pull/6952), [#6953](https://github.com/pyg-team/pytorch_geometric/pull/6953), [#6980](https://github.com/pyg-team/pytorch_geometric/pull/6980), [#6983](https://github.com/pyg-team/pytorch_geometric/pull/6983), [#6984](https://github.com/pyg-team/pytorch_geometric/pull/6984), [#6985](https://github.com/pyg-team/pytorch_geometric/pull/6985), [#6986](https://github.com/pyg-team/pytorch_geometric/pull/6986), [#6989](https://github.com/pyg-team/pytorch_geometric/pull/6989), [#7002](https://github.com/pyg-team/pytorch_geometric/pull/7002)) - Added the `AntiSymmetricConv` layer ([#6577](https://github.com/pyg-team/pytorch_geometric/pull/6577)) - Added a mixin for Huggingface model hub integration ([#5930](https://github.com/pyg-team/pytorch_geometric/pull/5930), [#6591](https://github.com/pyg-team/pytorch_geometric/pull/6591)) - Added support for accelerated GNN layers in `nn.conv.cugraph` via `cugraph-ops` ([#6278](https://github.com/pyg-team/pytorch_geometric/pull/6278), [#6388](https://github.com/pyg-team/pytorch_geometric/pull/6388), [#6412](https://github.com/pyg-team/pytorch_geometric/pull/6412)) - Added accelerated `index_sort` function from `pyg-lib` for faster sorting ([#6554](https://github.com/pyg-team/pytorch_geometric/pull/6554)) - Fix incorrect device in `EquilibriumAggregration` ([#6560](https://github.com/pyg-team/pytorch_geometric/pull/6560)) - Added bipartite graph support in `dense_to_sparse()` ([#6546](https://github.com/pyg-team/pytorch_geometric/pull/6546)) - Add CPU affinity support for more data loaders ([#6534](https://github.com/pyg-team/pytorch_geometric/pull/6534), [#6922](https://github.com/pyg-team/pytorch_geometric/pull/6922)) - Added the `BAMultiShapesDataset` ([#6541](https://github.com/pyg-team/pytorch_geometric/pull/6541)) - Added the interfaces of a graph pooling framework ([#6540](https://github.com/pyg-team/pytorch_geometric/pull/6540)) - Added automatic `n_id` and `e_id` attributes to mini-batches produced by `NodeLoader` and `LinkLoader` ([#6524](https://github.com/pyg-team/pytorch_geometric/pull/6524)) - Added `PGMExplainer` to `torch_geometric.contrib` ([#6149](https://github.com/pyg-team/pytorch_geometric/pull/6149), [#6588](https://github.com/pyg-team/pytorch_geometric/pull/6588), [#6589](https://github.com/pyg-team/pytorch_geometric/pull/6589)) - Added a `NumNeighbors` helper class for specifying the number of neighbors when sampling ([#6501](https://github.com/pyg-team/pytorch_geometric/pull/6501), [#6505](https://github.com/pyg-team/pytorch_geometric/pull/6505), [#6690](https://github.com/pyg-team/pytorch_geometric/pull/6690)) - Added caching to `is_node_attr()` and `is_edge_attr()` calls ([#6492](https://github.com/pyg-team/pytorch_geometric/pull/6492)) - Added `ToHeteroLinear` and `ToHeteroMessagePassing` modules to accelerate `to_hetero` functionality ([#5992](https://github.com/pyg-team/pytorch_geometric/pull/5992), [#6456](https://github.com/pyg-team/pytorch_geometric/pull/6456)) - Added `GraphMaskExplainer` ([#6284](https://github.com/pyg-team/pytorch_geometric/pull/6284)) - Added the `GRBCD` and `PRBCD` adversarial attack models ([#5972](https://github.com/pyg-team/pytorch_geometric/pull/5972)) - Added `dropout` option to `SetTransformer` and `GraphMultisetTransformer` ([#6484](https://github.com/pyg-team/pytorch_geometric/pull/6484)) - Added option to customize loader arguments for evaluation in `LightningNodeData` and `LightningLinkData` ([#6450](https://github.com/pyg-team/pytorch_geometric/pull/6450), [#6456](https://github.com/pyg-team/pytorch_geometric/pull/6456)) - Added option to customize `num_neighbors` in `NeighborSampler` after instantiation ([#6446](https://github.com/pyg-team/pytorch_geometric/pull/6446)) - Added the `Taobao` dataset and a corresponding example for it ([#6144](https://github.com/pyg-team/pytorch_geometric/pull/6144)) - Added `pyproject.toml` ([#6431](https://github.com/pyg-team/pytorch_geometric/pull/6431)) - Added the `torch_geometric.contrib` sub-package ([#6422](https://github.com/pyg-team/pytorch_geometric/pull/6422)) - Warn on using latest documentation ([#6418](https://github.com/pyg-team/pytorch_geometric/pull/6418)) - Added basic `pyright` type checker support ([#6415](https://github.com/pyg-team/pytorch_geometric/pull/6415)) - Added a new external resource for link prediction ([#6396](https://github.com/pyg-team/pytorch_geometric/pull/6396)) - Added `CaptumExplainer` ([#6383](https://github.com/pyg-team/pytorch_geometric/pull/6383), [#6387](https://github.com/pyg-team/pytorch_geometric/pull/6387), [#6433](https://github.com/pyg-team/pytorch_geometric/pull/6433), [#6487](https://github.com/pyg-team/pytorch_geometric/pull/6487), [#6966](https://github.com/pyg-team/pytorch_geometric/pull/6966)) - Added support for custom `HeteroData` mini-batch class in remote backends ([#6377](https://github.com/pyg-team/pytorch_geometric/pull/6377)) - Added the `GNNFF` model ([#5866](https://github.com/pyg-team/pytorch_geometric/pull/5866)) - Added `MLPAggregation`, `SetTransformerAggregation`, `GRUAggregation`, and `DeepSetsAggregation` as adaptive readout functions ([#6301](https://github.com/pyg-team/pytorch_geometric/pull/6301), [#6336](https://github.com/pyg-team/pytorch_geometric/pull/6336), [#6338](https://github.com/pyg-team/pytorch_geometric/pull/6338)) - Added `Dataset.to_datapipe` for converting PyG datasets into a torchdata `DataPipe`([#6141](https://github.com/pyg-team/pytorch_geometric/pull/6141)) - Added `to_nested_tensor` and `from_nested_tensor` functionality ([#6329](https://github.com/pyg-team/pytorch_geometric/pull/6329), [#6330](https://github.com/pyg-team/pytorch_geometric/pull/6330), [#6331](https://github.com/pyg-team/pytorch_geometric/pull/6331), [#6332](https://github.com/pyg-team/pytorch_geometric/pull/6332)) - Added the `GPSConv` Graph Transformer layer and example ([#6326](https://github.com/pyg-team/pytorch_geometric/pull/6326), [#6327](https://github.com/pyg-team/pytorch_geometric/pull/6327)) - Added `networkit` conversion utilities ([#6321](https://github.com/pyg-team/pytorch_geometric/pull/6321)) - Added global dataset attribute access via `dataset.{attr_name}` ([#6319](https://github.com/pyg-team/pytorch_geometric/pull/6319)) - Added the `TransE` KGE model and example ([#6314](https://github.com/pyg-team/pytorch_geometric/pull/6314)) - Added the Freebase `FB15k_237` dataset ([#3204](https://github.com/pyg-team/pytorch_geometric/pull/3204)) - Added `Data.update()` and `HeteroData.update()` functionality ([#6313](https://github.com/pyg-team/pytorch_geometric/pull/6313)) - Added `PGExplainer` ([#6204](https://github.com/pyg-team/pytorch_geometric/pull/6204)) - Added the `AirfRANS` dataset ([#6287](https://github.com/pyg-team/pytorch_geometric/pull/6287)) - Added `AttentionExplainer` ([#6279](https://github.com/pyg-team/pytorch_geometric/pull/6279)) - Added (un)faithfulness explainability metric ([#6090](https://github.com/pyg-team/pytorch_geometric/pull/6090)) - Added fidelity explainability metric ([#6116](https://github.com/pyg-team/pytorch_geometric/pull/6116), [#6510](https://github.com/pyg-team/pytorch_geometric/pull/6510)) - Added subgraph visualization of GNN explanations ([#6235](https://github.com/pyg-team/pytorch_geometric/pull/6235), [#6271](https://github.com/pyg-team/pytorch_geometric/pull/6271)) - Added weighted negative sampling option in `LinkNeighborLoader` ([#6264](https://github.com/pyg-team/pytorch_geometric/pull/6264)) - Added the `BA2MotifDataset` explainer dataset ([#6257](https://github.com/pyg-team/pytorch_geometric/pull/6257)) - Added `CycleMotif` motif generator to generate `n`-node cycle shaped motifs ([#6256](https://github.com/pyg-team/pytorch_geometric/pull/6256)) - Added the `InfectionDataset` to evaluate explanations ([#6222](https://github.com/pyg-team/pytorch_geometric/pull/6222)) - Added `characterization_score` and `fidelity_curve_auc` explainer metrics ([#6188](https://github.com/pyg-team/pytorch_geometric/pull/6188)) - Added `get_message_passing_embeddings` ([#6201](https://github.com/pyg-team/pytorch_geometric/pull/6201)) - Added the `PointGNNConv` layer ([#6194](https://github.com/pyg-team/pytorch_geometric/pull/6194)) - Added `GridGraph` graph generator to generate grid graphs ([#6220](https://github.com/pyg-team/pytorch_geometric/pull/6220) - Added explainability metrics for when ground truth is available ([#6137](https://github.com/pyg-team/pytorch_geometric/pull/6137)) - Added `visualize_feature_importance` to support node feature visualizations ([#6094](https://github.com/pyg-team/pytorch_geometric/pull/6094)) - Added heterogeneous graph support to `Explanation` framework ([#6091](https://github.com/pyg-team/pytorch_geometric/pull/6091), [#6218](https://github.com/pyg-team/pytorch_geometric/pull/6218)) - Added a `CustomMotif` motif generator ([#6179](https://github.com/pyg-team/pytorch_geometric/pull/6179)) - Added `ERGraph` graph generator to generate Ergos-Renyi (ER) graphs ([#6073](https://github.com/pyg-team/pytorch_geometric/pull/6073)) - Added `BAGraph` graph generator to generate Barabasi-Albert graphs - the usage of `datasets.BAShapes` is now deprecated ([#6072](https://github.com/pyg-team/pytorch_geometric/pull/6072) - Added explainability benchmark dataset framework ([#6104](https://github.com/pyg-team/pytorch_geometric/pull/6104)) - Added `seed_time` attribute to temporal `NodeLoader` outputs in case `input_time` is given ([#6196](https://github.com/pyg-team/pytorch_geometric/pull/6196)) - Added `Data.edge_subgraph` and `HeteroData.edge_subgraph` functionalities ([#6193](https://github.com/pyg-team/pytorch_geometric/pull/6193)) - Added `input_time` option to `LightningNodeData` and `transform_sampler_output` to `NodeLoader` and `LinkLoader` ([#6187](https://github.com/pyg-team/pytorch_geometric/pull/6187)) - Added `summary` for PyG/PyTorch models ([#5859](https://github.com/pyg-team/pytorch_geometric/pull/5859), [#6161](https://github.com/pyg-team/pytorch_geometric/pull/6161)) - Started adding `torch.sparse` support to PyG ([#5906](https://github.com/pyg-team/pytorch_geometric/pull/5906), [#5944](https://github.com/pyg-team/pytorch_geometric/pull/5944), [#6003](https://github.com/pyg-team/pytorch_geometric/pull/6003), [#6033](https://github.com/pyg-team/pytorch_geometric/pull/6033), [#6514](https://github.com/pyg-team/pytorch_geometric/pull/6514), [#6532](https://github.com/pyg-team/pytorch_geometric/pull/6532), [#6748](https://github.com/pyg-team/pytorch_geometric/pull/6748), [#6847](https://github.com/pyg-team/pytorch_geometric/pull/6847), [#6868](https://github.com/pyg-team/pytorch_geometric/pull/6868), [#6874](https://github.com/pyg-team/pytorch_geometric/pull/6874), [#6897](https://github.com/pyg-team/pytorch_geometric/pull/6897), [#6930](https://github.com/pyg-team/pytorch_geometric/pull/6930), [#6932](https://github.com/pyg-team/pytorch_geometric/pull/6932), [#6936](https://github.com/pyg-team/pytorch_geometric/pull/6936), [#6937](https://github.com/pyg-team/pytorch_geometric/pull/6937), [#6939](https://github.com/pyg-team/pytorch_geometric/pull/6939), [#6947](https://github.com/pyg-team/pytorch_geometric/pull/6947), [#6950](https://github.com/pyg-team/pytorch_geometric/pull/6950), [#6951](https://github.com/pyg-team/pytorch_geometric/pull/6951), [#6957](https://github.com/pyg-team/pytorch_geometric/pull/6957)) - Add `inputs_channels` back in training benchmark ([#6154](https://github.com/pyg-team/pytorch_geometric/pull/6154)) - Added support for dropping nodes in `utils.to_dense_batch` in case `max_num_nodes` is smaller than the number of nodes ([#6124](https://github.com/pyg-team/pytorch_geometric/pull/6124)) - Added the RandLA-Net architecture as an example ([#5117](https://github.com/pyg-team/pytorch_geometric/pull/5117))

Changed

- Drop internal usage of `__dunder__` names ([#6999](https://github.com/pyg-team/pytorch_geometric/issues/6999)) - Changed the interface of `sort_edge_index`, `coalesce` and `to_undirected` to only return single `edge_index` information in case the `edge_attr` argument is not specified ([#6875](https://github.com/pyg-team/pytorch_geometric/issues/6875), [#6879](https://github.com/pyg-team/pytorch_geometric/issues/6879), [#6893](https://github.com/pyg-team/pytorch_geometric/issues/6893)) - Fixed a bug in `to_hetero` when using an uninitialized submodule without implementing `reset_parameters` ([#6863](https://github.com/pyg-team/pytorch_geometric/issues/6790)) - Fixed a bug in `get_mesh_laplacian` ([#6790](https://github.com/pyg-team/pytorch_geometric/issues/6790)) - Fixed a bug in which masks were not properly masked in `GNNExplainer` on link prediction tasks ([#6787](https://github.com/pyg-team/pytorch_geometric/pull/6787)) - Allow the usage of `ChebConv` within `GNNExplainer` ([#6778](https://github.com/pyg-team/pytorch_geometric/pull/6778)) - Allow setting the `EdgeStorage.num_edges` property ([#6710](https://github.com/pyg-team/pytorch_geometric/pull/6710)) - Fixed a bug in `utils.bipartite_subgraph()` and updated docs of `HeteroData.subgraph()` ([#6654](https://github.com/pyg-team/pytorch_geometric/pull/6654)) - Properly reset the `data_list` cache of an `InMemoryDataset` when accessing `dataset.data` ([#6685](https://github.com/pyg-team/pytorch_geometric/pull/6685)) - Fixed a bug in `Data.subgraph()` and `HeteroData.subgraph()` ([#6613](https://github.com/pyg-team/pytorch_geometric/pull/6613)) - Fixed a bug in `PNAConv` and `DegreeScalerAggregation` to correctly incorporate degree statistics of isolated nodes ([#6609](https://github.com/pyg-team/pytorch_geometric/pull/6609)) - Improved code coverage ([#6523](https://github.com/pyg-team/pytorch_geometric/pull/6523), [#6538](https://github.com/pyg-team/pytorch_geometric/pull/6538), [#6555](https://github.com/pyg-team/pytorch_geometric/pull/6555), [#6558](https://github.com/pyg-team/pytorch_geometric/pull/6558), [#6568](https://github.com/pyg-team/pytorch_geometric/pull/6568), [#6573](https://github.com/pyg-team/pytorch_geometric/pull/6573), [#6578](https://github.com/pyg-team/pytorch_geometric/pull/6578), [#6597](https://github.com/pyg-team/pytorch_geometric/pull/6597), [#6600](https://github.com/pyg-team/pytorch_geometric/pull/6600), [#6618](https://github.com/pyg-team/pytorch_geometric/pull/6618), [#6619](https://github.com/pyg-team/pytorch_geometric/pull/6619), [#6621](https://github.com/pyg-team/pytorch_geometric/pull/6621), [#6623](https://github.com/pyg-team/pytorch_geometric/pull/6623), [#6637](https://github.com/pyg-team/pytorch_geometric/pull/6637), [#6638](https://github.com/pyg-team/pytorch_geometric/pull/6638), [#6640](https://github.com/pyg-team/pytorch_geometric/pull/6640), [#6645](https://github.com/pyg-team/pytorch_geometric/pull/6645), [#6648](https://github.com/pyg-team/pytorch_geometric/pull/6648), [#6647](https://github.com/pyg-team/pytorch_geometric/pull/6647), [#6653](https://github.com/pyg-team/pytorch_geometric/pull/6653), [#6657](https://github.com/pyg-team/pytorch_geometric/pull/6657), [#6662](https://github.com/pyg-team/pytorch_geometric/pull/6662), [#6664](https://github.com/pyg-team/pytorch_geometric/pull/6664), [#6667](https://github.com/pyg-team/pytorch_geometric/pull/6667), [#6668](https://github.com/pyg-team/pytorch_geometric/pull/6668), [#6669](https://github.com/pyg-team/pytorch_geometric/pull/6669), [#6670](https://github.com/pyg-team/pytorch_geometric/pull/6670), [#6671](https://github.com/pyg-team/pytorch_geometric/pull/6671), [#6673](https://github.com/pyg-team/pytorch_geometric/pull/6673), [#6675](https://github.com/pyg-team/pytorch_geometric/pull/6675), [#6676](https://github.com/pyg-team/pytorch_geometric/pull/6676), [#6677](https://github.com/pyg-team/pytorch_geometric/pull/6677), [#6678](https://github.com/pyg-team/pytorch_geometric/pull/6678), [#6681](https://github.com/pyg-team/pytorch_geometric/pull/6681), [#6683](https://github.com/pyg-team/pytorch_geometric/pull/6683), [#6703](https://github.com/pyg-team/pytorch_geometric/pull/6703), [#6720](https://github.com/pyg-team/pytorch_geometric/pull/6720), [#6735](https://github.com/pyg-team/pytorch_geometric/pull/6735), [#6736](https://github.com/pyg-team/pytorch_geometric/pull/6736), [#6763](https://github.com/pyg-team/pytorch_geometric/pull/6763), [#6781](https://github.com/pyg-team/pytorch_geometric/pull/6781), [#6797](https://github.com/pyg-team/pytorch_geometric/pull/6797), [#6799](https://github.com/pyg-team/pytorch_geometric/pull/6799), [#6824](https://github.com/pyg-team/pytorch_geometric/pull/6824), [#6858](https://github.com/pyg-team/pytorch_geometric/pull/6858)) - Fixed a bug in which `data.to_heterogeneous()` filtered attributs in the wrong dimension ([#6522](https://github.com/pyg-team/pytorch_geometric/pull/6522)) - Breaking Change: Temporal sampling will now also sample nodes with an equal timestamp to the seed time (requires `pyg-lib>0.1.0`) ([#6517](https://github.com/pyg-team/pytorch_geometric/pull/6517)) - Changed `DataLoader` workers with affinity to start at `cpu0` ([#6512](https://github.com/pyg-team/pytorch_geometric/pull/6512)) - Allow 1D input to `global_*_pool` functions ([#6504](https://github.com/pyg-team/pytorch_geometric/pull/6504)) - Add information about dynamic shapes in `RGCNConv` ([#6482](https://github.com/pyg-team/pytorch_geometric/pull/6482)) - Fixed the use of types removed in `numpy 1.24.0` ([#6495](https://github.com/pyg-team/pytorch_geometric/pull/6495)) - Fixed keyword parameters in `examples/mnist_voxel_grid.py` ([#6478](https://github.com/pyg-team/pytorch_geometric/pull/6478)) - Unified `LightningNodeData` and `LightningLinkData` code paths ([#6473](https://github.com/pyg-team/pytorch_geometric/pull/6473)) - Allow indices with any integer type in `RGCNConv` ([#6463](https://github.com/pyg-team/pytorch_geometric/pull/6463)) - Re-structured the documentation ([#6420](https://github.com/pyg-team/pytorch_geometric/pull/6420), [#6423](https://github.com/pyg-team/pytorch_geometric/pull/6423), [#6429](https://github.com/pyg-team/pytorch_geometric/pull/6429), [#6440](https://github.com/pyg-team/pytorch_geometric/pull/6440), [#6443](https://github.com/pyg-team/pytorch_geometric/pull/6443), [#6445](https://github.com/pyg-team/pytorch_geometric/pull/6445), [#6452](https://github.com/pyg-team/pytorch_geometric/pull/6452), [#6453](https://github.com/pyg-team/pytorch_geometric/pull/6453), [#6458](https://github.com/pyg-team/pytorch_geometric/pull/6458), [#6459](https://github.com/pyg-team/pytorch_geometric/pull/6459), [#6460](https://github.com/pyg-team/pytorch_geometric/pull/6460), [#6490](https://github.com/pyg-team/pytorch_geometric/pull/6490), [#6491](https://github.com/pyg-team/pytorch_geometric/pull/6491), [#6693](https://github.com/pyg-team/pytorch_geometric/pull/6693), [#6744](https://github.com/pyg-team/pytorch_geometric/pull/6744)) - Fix the default arguments of `DataParallel` class ([#6376](https://github.com/pyg-team/pytorch_geometric/pull/6376)) - Fix `ImbalancedSampler` on sliced `InMemoryDataset` ([#6374](https://github.com/pyg-team/pytorch_geometric/pull/6374)) - Breaking Change: Changed the interface and implementation of `GraphMultisetTransformer` ([#6343](https://github.com/pyg-team/pytorch_geometric/pull/6343)) - Fixed the approximate PPR variant in `transforms.GDC` to not crash on graphs with isolated nodes ([#6242](https://github.com/pyg-team/pytorch_geometric/pull/6242)) - Added a warning when accesing `InMemoryDataset.data` ([#6318](https://github.com/pyg-team/pytorch_geometric/pull/6318)) - Drop `SparseTensor` dependency in `GraphStore` ([#5517](https://github.com/pyg-team/pytorch_geometric/pull/5517)) - Replace `NeighborSampler` with `NeighborLoader` in the distributed sampling example ([#6204](https://github.com/pyg-team/pytorch_geometric/pull/6307)) - Fixed the filtering of node features in `transforms.RemoveIsolatedNodes` ([#6308](https://github.com/pyg-team/pytorch_geometric/pull/6308)) - Fixed a bug in `DimeNet` that causes a output dimension mismatch ([#6305](https://github.com/pyg-team/pytorch_geometric/pull/6305)) - Fixed `Data.to_heterogeneous()` with empty `edge_index` ([#6304](https://github.com/pyg-team/pytorch_geometric/pull/6304)) - Unify `Explanation.node_mask` and `Explanation.node_feat_mask` ([#6267](https://github.com/pyg-team/pytorch_geometric/pull/6267)) - Moved thresholding config of the `Explainer` to `Explanation` ([#6215](https://github.com/pyg-team/pytorch_geometric/pull/6215)) - Fixed a bug in the output order in `HeteroLinear` for un-sorted type vectors ([#6198](https://github.com/pyg-team/pytorch_geometric/pull/6198)) - Breaking Change: Move `ExplainerConfig` arguments to the `Explainer` class ([#6176](https://github.com/pyg-team/pytorch_geometric/pull/6176)) - Refactored `NeighborSampler` to be input-type agnostic ([#6173](https://github.com/pyg-team/pytorch_geometric/pull/6173)) - Infer correct CUDA device ID in `profileit` decorator ([#6164](https://github.com/pyg-team/pytorch_geometric/pull/6164)) - Correctly use edge weights in `GDC` example ([#6159](https://github.com/pyg-team/pytorch_geometric/pull/6159)) - Breaking Change: Moved PyTorch Lightning data modules to `torch_geometric.data.lightning` ([#6140](https://github.com/pyg-team/pytorch_geometric/pull/6140)) - Make `torch_sparse` an optional dependency ([#6132](https://github.com/pyg-team/pytorch_geometric/pull/6132), [#6134](https://github.com/pyg-team/pytorch_geometric/pull/6134), [#6138](https://github.com/pyg-team/pytorch_geometric/pull/6138), [#6139](https://github.com/pyg-team/pytorch_geometric/pull/6139)) - Optimized `utils.softmax` implementation ([#6113](https://github.com/pyg-team/pytorch_geometric/pull/6113), [#6155](https://github.com/pyg-team/pytorch_geometric/pull/6155), [#6805](https://github.com/pyg-team/pytorch_geometric/pull/6805)) - Optimized `topk` implementation for large enough graphs ([#6123](https://github.com/pyg-team/pytorch_geometric/pull/6123))

Removed

- `torch-sparse` is now an optional dependency ([#6625](https://github.com/pyg-team/pytorch_geometric/pull/6625), [#6626](https://github.com/pyg-team/pytorch_geometric/pull/6626), [#6627](https://github.com/pyg-team/pytorch_geometric/pull/6627), [#6628](https://github.com/pyg-team/pytorch_geometric/pull/6628), [#6629](https://github.com/pyg-team/pytorch_geometric/pull/6629), [#6630](https://github.com/pyg-team/pytorch_geometric/pull/6630)) - Removed most of the `torch-scatter` dependencies ([#6394](https://github.com/pyg-team/pytorch_geometric/pull/6394), [#6395](https://github.com/pyg-team/pytorch_geometric/pull/6395), [#6399](https://github.com/pyg-team/pytorch_geometric/pull/6399), [#6400](https://github.com/pyg-team/pytorch_geometric/pull/6400), [#6615](https://github.com/pyg-team/pytorch_geometric/pull/6615), [#6617](https://github.com/pyg-team/pytorch_geometric/pull/6617)) - Removed the deprecated classes `GNNExplainer` and `Explainer` from `nn.models` ([#6382](https://github.com/pyg-team/pytorch_geometric/pull/6382)) - Removed `target_index` argument in the `Explainer` interface ([#6270](https://github.com/pyg-team/pytorch_geometric/pull/6270)) - Removed `Aggregation.set_validate_args` option ([#6175](https://github.com/pyg-team/pytorch_geometric/pull/6175))

Full Changelog: https://github.com/pyg-team/pytorch_geometric/compare/2.2.0...2.3.0

New Contributors

@CharlesGaydon made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/5117
@mova made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6123
@Humbertzhang made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6143
@dongyukang1 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6159
@edwag made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6183
@toenshoff made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6198
@shenoynikhil made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6073
@andreasbinder made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6194
@ZeynepP made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6090
@karimsr4 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6279
@FlorentExtrality made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6287
@anton-bushuiev made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6306
@marekdedic made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6242
@davidbuterez made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6301
@ken2403 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/5866
@bwroblew made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6376
@manangoel99 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6396
@binlee52 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6439
@bartekxk made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6463
@forest1040 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6478
@sigeisler made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/5972
@steveazzolin made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6541
@JinL0 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6557
@HelgeS made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6560
@jaypmorgan made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6562
@jreniecki made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6570
@tingyu66 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6388
@gravins made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6577
@karuna-bhaila made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6597
@DylanSand made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6634
@soumik12345 made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6641
@sivonxay made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6648
@kimfalk made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6651
@dsciebu made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6710
@bmarenco made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6715
@RobDHess made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6734
@cemunds made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6810
@berkekisin made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6838
@varun-tandon made their first contribution in https://github.com/pyg-team/pytorch_geometric/pull/6904

- Python
Published by wsad1 almost 3 years ago

pytorch_geometric - PyG 2.2.0: Accelerations and Scalability

We are excited to announce the release of PyG 2.2 🎉🎉🎉

Highlights
Breaking Changes
Deprecations
Features
Bugfixes
Full Changelog

PyG 2.2 is the culmination of work from 78 contributors who have worked on features and bug-fixes for a total of over 320 commits since torch-geometric==2.1.0.

Highlights

`pyg-lib` Integration

We are proud to release and integrate pyg-lib==0.1.0 into PyG, the first stable version of our new low-level Graph Neural Network library to drive all CPU and GPU acceleration needs of PyG (#5330, #5347, #5384, #5388).

You can install pyg-lib as described in our README.md:

pip install pyg-lib -f https://data.pyg.org/whl/torch-${TORCH}+${CUDA}.html

python import pyg_lib

Once pyg-lib is installed, it will get automatically picked up by PyG, e.g., to accelerate neighborhood sampling routines or to accelerate heterogeneous GNN execution:

pyg-lib provides fast and optimized CPU routines to iteratively sample neighbors in homogeneous and heterogeneous graphs, and heavily improves upon the previously used neighborhood sampling techniques utilized in PyG.

Screenshot 2022-11-30 at 08 44 08

pyg-lib provides efficient GPU-based routines to parallelize workloads in heterogeneous graphs across different node types and edge types. We achieve this by leveraging type-dependent transformations via NVIDIA CUTLASS integration, which is flexible to implement most heterogeneous GNNs with, and efficient, even for sparse edge types or a large number of different node types.

Screenshot 2022-11-30 at 08 44 38

`GraphStore` and `FeatureStore` Abstractions

PyG 2.2 includes numerous primitives to easily integrate with simple paradigms for scalable graph machine learning, enabling users to train GNNs on graphs far larger than the size of their machine's available memory. It does so by introducing simple, easy-to-use, and extensible abstractions of a FeatureStore and a GraphStore that plug directly into existing familiar PyG interfaces (see here for the accompanying tutorial).

```python featurestore = CustomFeatureStore() featurestore['paper', 'x', None] = ... # Add paper features feature_store['author', 'x', None] = ... # Add author features

graphstore = CustomGraphStore() graphstore['edge', 'coo'] = ... # Add edges in "COO" format

`CustomGraphSampler` knows how to sample on `CustomGraphStore`:

graphsampler = CustomGraphSampler( graphstore=graphstore, numneighbors=[10, 20], ... )

from torchgeometric.loader import NodeLoader loader = NodeLoader( data=(featurestore, graphstore), nodesampler=graphsampler, batchsize=20, input_nodes='paper', )

for batch in loader: pass ``Data loading and sampling routines are refactored and decomposed into [torch_geometric.loader](https://pytorch-geometric.readthedocs.io/en/latest/modules/loader.html) and [torch_geometric.sampler`](https://pytorch-geometric.readthedocs.io/en/latest/modules/sampler.html) modules, respectively (#5563, #5820, #5456, #5457, #5312, #5365, #5402, #5404, #5418).

Optimized and Fused Aggregations

PyG 2.2 further accelerates scatter aggregations based on CPU/GPU and with/without backward computation paths (requires torch>=1.12.0 and torch-scatter>=2.1.0) (#5232, #5241, #5353, #5386, #5399, #6051, #6052).

We also optimized the usage of nn.aggr.MultiAggregation by fusing the computation of multiple aggregations together (see here for more details) (#6036, #6040).

Here are some benchmarking results on PyTorch 1.12 (summed over 1000 runs):

| Aggregators | Vanilla | Fusion | |-------------------------|---------|---------| | [sum, mean] | 0.3325s | 0.1996s | | [sum, mean, min, max] | 0.7139s | 0.5037s | | [sum, mean, var] | 0.6849s | 0.3871s | | [sum, mean, var, std] | 1.0955s | 0.3973s |

Lastly, we have incorporated "fused" GNN operators via the dgNN package, starting with a FusedGATConv implementation (#5140).

Community Sprint: Type Hints and TorchScript Support

We are running regular community sprints to get our community more involved in building PyG. Whether you are just beginning to use graph learning or have been leveraging GNNs in research or production, the community sprints welcome members of all levels with different types of projects.

We had our first community sprint on 10/12 to fully-incorporate type hints and TorchScript support over the entire code base. The goal was to improve usability and cleanliness of our codebase. We had 20 contributors participating, contributing to 120 type hints within 2 weeks, adding around 2400 lines of code (#5842, #5603, #5659, #5664, #5665, #5666, #5667, #5668, #5669, #5673, #5675, #5673, #5678, #5682, #5683, #5684, #5685, #5687, #5688, #5695, #5699, #5701, #5702, #5703, #5706, #5707, #5710, #5714, #5715, #5716, #5722, #5724, #5725, #5726, #5729, #5730, #5731, #5732, #5733, #5743, #5734, #5735, #5736, #5737, #5738, #5747, #5752, #5753, #5754, #5756, #5757, #5758, #5760, #5766, #5767, #5768, #5781, #5778, #5797, #5798, #5799, #5800, #5806, #5810, #5811, #5828, #5847, #5851, #5852).

Explainability

Our second community sprint began on 11/15 with the goal to improve the explainability capabilities of PyG. With this, we introduce the torch_geometric.explain module to provide a unified set of tools to explain the predictions of a PyG model or to explain the underlying phenomenon of a dataset.

Some of the features developed in the sprint are incorporated into this release:

Added the torch_geometric.explain module (#5804, #6054, #6089)
Moved and adapted the GNNExplainer module to torch_geometric.explain (#5967, #6065). See here and here for the accompanying examples.
Extended GNNExplainer to support edge level explanations (#6056)
Added explainability support for heterogeneous GNNs via to_captum_model and to_captum_input (#5886, #5934)

```python data = HeteroData(...) model = HeteroGNN(...)

Explain predictions on heterogenenous graphs for output node 10:

captummodel = tocaptummodel(model, masktype, outputidx, metadata) inputs, additionalforwardargs = tocaptuminput(data.xdict, data.edgeindexdict, mask_type)

ig = IntegratedGradients(captummodel) igattr = ig.attribute( inputs=inputs, target=int(y[outputidx]), additionalforwardargs=additionalforwardargs, internalbatch_size=1, ) ```

Breaking Changes

Renamed drop_unconnected_nodes to drop_unconnected_node_types and drop_orig_edges to drop_orig_edge_types in AddMetapaths (#5490)

Deprecations

The usage of nn.models.GNNExplainer is now deprecated in favor of explain.GNNExplainer
The usage of utils.dropout_adj is now deprecated in favor of utils.dropout_edge
The usage of loader.RandomNodeSampler is now deprecated in favor of loader.RandomNodeLoader
The usage of to_captum is now deprecated in favor of to_captum_model.

Features

Layers, Models and Examples

Added a "Link Prediction on MovieLens" Colab notebook (#5823)
Added a bipartite link-prediction example (#5834)
Added the SSGConv layer (#5599)
Added the WLConvContinuous layer for performing WL-refinement with continuous attributes (#5316)
Added the PositionalEncoding module (#5381)
Added a node classification example instrumented with Weights and Biases (#5192)

Data Loaders

Added support for triplet sampling in LinkNeighborLoader (#6004)
Added temporal_strategy = uniform/last option to NeighborLoader and LinkNeighborLoader (#5576)
Added a disjoint option to NeighborLoader and LinkNeighborLoader (#5717, #5775)
Added HeteroData support in RandomNodeLoader (#6007
Added int32-based edge_index support in NeighborLoader (#5948)
Added support for input_time in NeighborLoader (#5763)
Added np.memmap support in NeighborLoader (#5696)
Added CPU affinitization support to NeighborLoader (#6005)

Transformations

Added a FeaturePropagation transform (#5387)
Added IndexToMask and MaskToIndex transforms (#5375, #5455)
Added shuffle_node, mask_feature and add_random_edge augmentations (#5548)
Added dropout_node, dropout_edge and dropout_path augmentations (#5481, #5495, #5531)
Added a AddRandomMetaPaths transform that adds edges based on random walks along a metapath (#5397)
Added a utils.to_smiles function (#6038)
Added HeteroData support for transforms.Constant (#5700)

Datasets

Added the LRGBDataset to include 5 datasets from the Long Range Graph Benchmark (#5935)
Added the HydroNet water cluster dataset (#5537, #5902, #5903)
Added the DGraphFin dynamic graph dataset (#5504)
Added the official splits to the MalNetTiny dataset (#5078)
Added a print_summary method to torch_geometric.data.Dataset (#5438)

General Improvements

Added training and inference benchmark scripts (#5774, #5830, #5878, #5293, #5341, #5242, #5258, #5881, #5254)
Added the utils.assortativity function to compute the degree assortativity coefficient (#5587)
Add support for filling labels with dummy values in HeteroData.to_homogeneous() (#5540)
Added torch.onnx.export support (see here for an example) (#5877, #5997)
Added option to make normalization coefficients trainable in PNAConv (#6039)
Added a semi_grad option in VarAggregation and StdAggregation (#6042)
Added a warning for invalid node and edge type names in HeteroData (#5990)
Added lr_scheduler_solver and customized lr_scheduler classes (#5942)
Added to_fixed_size graph transformer (#5939)
Added support for symbolic tracing in the SchNet model (#5938)
Added support for customizing the interaction graph in the SchNet model (#5919)
Added SparseTensor support to SuperGATConv (#5888)
Added TorchScript support for AttentiveFP (#5868)
Added a return_semantic_attention_weights argument HANConv (#5787)
Added temperature value customization in dense_mincut_pool (#5908)
Added support for a tuple of in_channels in GENConv for bipartite message passing (#5627, #5641)
Added Aggregation.set_validate_args option to skip validation of dim_size (#5290)
Added BaseStorage.get() functionality (#5240)
Added support for batches of size one in BatchNorm (#5530, #5614)
The AttentionalAggregation module can now be applied to compute attention on a per-feature level (#5449)
Added TorchScript support to ASAPooling (#5395)
Updated the unsupervised GraphSAGE example to leverage LinkNeighborLoader (#5317)
Added better out-of-bounds error message in MessagePassing (#5339)
Added support to customize the activation function in PNAConv (#5262)

Bugfixes

Fixed a bug in TUDataset, in which node features were wrongly constructed whenever node_attributes only hold a single feature (e.g., in PROTEINS) (#5441)
Fixed a bug in the VirtualNode transform, in which node features were mistakenly treated as edge features (#5819)
Fixed a bug when applying several scalers with PNAConv (#5514)
Fixed setter and getter handling in BaseStorage (#5815)
Fixed the auto_select_device routine in GraphGym for pytorch_lightning>=1.7 (#5677)
Fixed RandomLinkSplit in case there aren't enough negative edges to sample (#5642)
Fixed the in-place modification to mode_kwargs in MultiAggregation (#5601)
Fixed the utils.to_dense_adj routine in case edge_index is empty (#5476)
Fixed the PointTransformerConv to now correctly use sum aggregation (#5332)
Fixed the output of InMemoryDataset.num_classes in case a transform modifies data.y (#5274)
Fail gracefully on GLIBC errors within torch-spline-conv (#5276)

Full Changelog

Added

- Extended `GNNExplainer` to support edge level explanations ([#6056](https://github.com/pyg-team/pytorch_geometric/pull/6056)) - Added CPU affinitization for `NodeLoader` ([#6005](https://github.com/pyg-team/pytorch_geometric/pull/6005)) - Added triplet sampling in `LinkNeighborLoader` ([#6004](https://github.com/pyg-team/pytorch_geometric/pull/6004)) - Added `FusedAggregation` of simple scatter reductions ([#6036](https://github.com/pyg-team/pytorch_geometric/pull/6036)) - Added a `to_smiles` function ([#6038](https://github.com/pyg-team/pytorch_geometric/pull/6038)) - Added option to make normalization coefficients trainable in `PNAConv` ([#6039](https://github.com/pyg-team/pytorch_geometric/pull/6039)) - Added `semi_grad` option in `VarAggregation` and `StdAggregation` ([#6042](https://github.com/pyg-team/pytorch_geometric/pull/6042)) - Allow for fused aggregations in `MultiAggregation` ([#6036](https://github.com/pyg-team/pytorch_geometric/pull/6036), [#6040](https://github.com/pyg-team/pytorch_geometric/pull/6040)) - Added `HeteroData` support for `to_captum_model` and added `to_captum_input` ([#5934](https://github.com/pyg-team/pytorch_geometric/pull/5934)) - Added `HeteroData` support in `RandomNodeLoader` ([#6007](https://github.com/pyg-team/pytorch_geometric/pull/6007)) - Added bipartite `GraphSAGE` example ([#5834](https://github.com/pyg-team/pytorch_geometric/pull/5834)) - Added `LRGBDataset` to include 5 datasets from the [Long Range Graph Benchmark](https://openreview.net/pdf?id=in7XC5RcjEn) ([#5935](https://github.com/pyg-team/pytorch_geometric/pull/5935)) - Added a warning for invalid node and edge type names in `HeteroData` ([#5990](https://github.com/pyg-team/pytorch_geometric/pull/5990)) - Added PyTorch 1.13 support ([#5975](https://github.com/pyg-team/pytorch_geometric/pull/5975)) - Added `int32` support in `NeighborLoader` ([#5948](https://github.com/pyg-team/pytorch_geometric/pull/5948)) - Add `dgNN` support and `FusedGATConv` implementation ([#5140](https://github.com/pyg-team/pytorch_geometric/pull/5140)) - Added `lr_scheduler_solver` and customized `lr_scheduler` classes ([#5942](https://github.com/pyg-team/pytorch_geometric/pull/5942)) - Add `to_fixed_size` graph transformer ([#5939](https://github.com/pyg-team/pytorch_geometric/pull/5939)) - Add support for symbolic tracing of `SchNet` model ([#5938](https://github.com/pyg-team/pytorch_geometric/pull/5938)) - Add support for customizable interaction graph in `SchNet` model ([#5919](https://github.com/pyg-team/pytorch_geometric/pull/5919)) - Started adding `torch.sparse` support to PyG ([#5906](https://github.com/pyg-team/pytorch_geometric/pull/5906), [#5944](https://github.com/pyg-team/pytorch_geometric/pull/5944), [#6003](https://github.com/pyg-team/pytorch_geometric/pull/6003)) - Added `HydroNet` water cluster dataset ([#5537](https://github.com/pyg-team/pytorch_geometric/pull/5537), [#5902](https://github.com/pyg-team/pytorch_geometric/pull/5902), [#5903](https://github.com/pyg-team/pytorch_geometric/pull/5903)) - Added explainability support for heterogeneous GNNs ([#5886](https://github.com/pyg-team/pytorch_geometric/pull/5886)) - Added `SparseTensor` support to `SuperGATConv` ([#5888](https://github.com/pyg-team/pytorch_geometric/pull/5888)) - Added TorchScript support for `AttentiveFP `([#5868](https://github.com/pyg-team/pytorch_geometric/pull/5868)) - Added `num_steps` argument to training and inference benchmarks ([#5898](https://github.com/pyg-team/pytorch_geometric/pull/5898)) - Added `torch.onnx.export` support ([#5877](https://github.com/pyg-team/pytorch_geometric/pull/5877), [#5997](https://github.com/pyg-team/pytorch_geometric/pull/5997)) - Enable VTune ITT in inference and training benchmarks ([#5830](https://github.com/pyg-team/pytorch_geometric/pull/5830), [#5878](https://github.com/pyg-team/pytorch_geometric/pull/5878)) - Add training benchmark ([#5774](https://github.com/pyg-team/pytorch_geometric/pull/5774)) - Added a "Link Prediction on MovieLens" Colab notebook ([#5823](https://github.com/pyg-team/pytorch_geometric/pull/5823)) - Added custom `sampler` support in `LightningDataModule` ([#5820](https://github.com/pyg-team/pytorch_geometric/pull/5820)) - Added a `return_semantic_attention_weights` argument `HANConv` ([#5787](https://github.com/pyg-team/pytorch_geometric/pull/5787)) - Added `disjoint` argument to `NeighborLoader` and `LinkNeighborLoader` ([#5775](https://github.com/pyg-team/pytorch_geometric/pull/5775)) - Added support for `input_time` in `NeighborLoader` ([#5763](https://github.com/pyg-team/pytorch_geometric/pull/5763)) - Added `disjoint` mode for temporal `LinkNeighborLoader` ([#5717](https://github.com/pyg-team/pytorch_geometric/pull/5717)) - Added `HeteroData` support for `transforms.Constant` ([#5700](https://github.com/pyg-team/pytorch_geometric/pull/5700)) - Added `np.memmap` support in `NeighborLoader` ([#5696](https://github.com/pyg-team/pytorch_geometric/pull/5696)) - Added `assortativity` that computes degree assortativity coefficient ([#5587](https://github.com/pyg-team/pytorch_geometric/pull/5587)) - Added `SSGConv` layer ([#5599](https://github.com/pyg-team/pytorch_geometric/pull/5599)) - Added `shuffle_node`, `mask_feature` and `add_random_edge` augmentation methdos ([#5548](https://github.com/pyg-team/pytorch_geometric/pull/5548)) - Added `dropout_path` augmentation that drops edges from a graph based on random walks ([#5531](https://github.com/pyg-team/pytorch_geometric/pull/5531)) - Add support for filling labels with dummy values in `HeteroData.to_homogeneous()` ([#5540](https://github.com/pyg-team/pytorch_geometric/pull/5540)) - Added `temporal_strategy` option to `neighbor_sample` ([#5576](https://github.com/pyg-team/pyg-lib/pull/5576)) - Added `torch_geometric.sampler` package to docs ([#5563](https://github.com/pyg-team/pytorch_geometric/pull/5563)) - Added the `DGraphFin` dynamic graph dataset ([#5504](https://github.com/pyg-team/pytorch_geometric/pull/5504)) - Added `dropout_edge` augmentation that randomly drops edges from a graph - the usage of `dropout_adj` is now deprecated ([#5495](https://github.com/pyg-team/pytorch_geometric/pull/5495)) - Added `dropout_node` augmentation that randomly drops nodes from a graph ([#5481](https://github.com/pyg-team/pytorch_geometric/pull/5481)) - Added `AddRandomMetaPaths` that adds edges based on random walks along a metapath ([#5397](https://github.com/pyg-team/pytorch_geometric/pull/5397)) - Added `WLConvContinuous` for performing WL refinement with continuous attributes ([#5316](https://github.com/pyg-team/pytorch_geometric/pull/5316)) - Added `print_summary` method for the `torch_geometric.data.Dataset` interface ([#5438](https://github.com/pyg-team/pytorch_geometric/pull/5438)) - Added `sampler` support to `LightningDataModule` ([#5456](https://github.com/pyg-team/pytorch_geometric/pull/5456), [#5457](https://github.com/pyg-team/pytorch_geometric/pull/5457)) - Added official splits to `MalNetTiny` dataset ([#5078](https://github.com/pyg-team/pytorch_geometric/pull/5078)) - Added `IndexToMask` and `MaskToIndex` transforms ([#5375](https://github.com/pyg-team/pytorch_geometric/pull/5375), [#5455](https://github.com/pyg-team/pytorch_geometric/pull/5455)) - Added `FeaturePropagation` transform ([#5387](https://github.com/pyg-team/pytorch_geometric/pull/5387)) - Added `PositionalEncoding` ([#5381](https://github.com/pyg-team/pytorch_geometric/pull/5381)) - Consolidated sampler routines behind `torch_geometric.sampler`, enabling ease of extensibility in the future ([#5312](https://github.com/pyg-team/pytorch_geometric/pull/5312), [#5365](https://github.com/pyg-team/pytorch_geometric/pull/5365), [#5402](https://github.com/pyg-team/pytorch_geometric/pull/5402), [#5404](https://github.com/pyg-team/pytorch_geometric/pull/5404)), [#5418](https://github.com/pyg-team/pytorch_geometric/pull/5418)) - Added `pyg-lib` neighbor sampling ([#5384](https://github.com/pyg-team/pytorch_geometric/pull/5384), [#5388](https://github.com/pyg-team/pytorch_geometric/pull/5388)) - Added `pyg_lib.segment_matmul` integration within `HeteroLinear` ([#5330](https://github.com/pyg-team/pytorch_geometric/pull/5330), [#5347](https://github.com/pyg-team/pytorch_geometric/pull/5347))) - Enabled `bf16` support in benchmark scripts ([#5293](https://github.com/pyg-team/pytorch_geometric/pull/5293), [#5341](https://github.com/pyg-team/pytorch_geometric/pull/5341)) - Added `Aggregation.set_validate_args` option to skip validation of `dim_size` ([#5290](https://github.com/pyg-team/pytorch_geometric/pull/5290)) - Added `SparseTensor` support to inference and training benchmark suite ([#5242](https://github.com/pyg-team/pytorch_geometric/pull/5242), [#5258](https://github.com/pyg-team/pytorch_geometric/pull/5258), [#5881](https://github.com/pyg-team/pytorch_geometric/pull/5881)) - Added experimental mode in inference benchmarks ([#5254](https://github.com/pyg-team/pytorch_geometric/pull/5254)) - Added node classification example instrumented with [Weights and Biases (W&B) logging](https://wandb.com) and [W&B Sweeps](https://wandb.com/sweeps) ([#5192](https://github.com/pyg-team/pytorch_geometric/pull/5192)) - Added experimental mode for `utils.scatter` ([#5232](https://github.com/pyg-team/pytorch_geometric/pull/5232), [#5241](https://github.com/pyg-team/pytorch_geometric/pull/5241), [#5386](https://github.com/pyg-team/pytorch_geometric/pull/5386)) - Added missing test labels in `HGBDataset` ([#5233](https://github.com/pyg-team/pytorch_geometric/pull/5233)) - Added `BaseStorage.get()` functionality ([#5240](https://github.com/pyg-team/pytorch_geometric/pull/5240)) - Added a test to confirm that `to_hetero` works with `SparseTensor` ([#5222](https://github.com/pyg-team/pytorch_geometric/pull/5222)) - Added `torch_geometric.explain` module with base functionality for explainability methods ([#5804](https://github.com/pyg-team/pytorch_geometric/pull/5804), [#6054](https://github.com/pyg-team/pytorch_geometric/pull/6054), [#6089](https://github.com/pyg-team/pytorch_geometric/pull/6089))

Changed

- Moved and adapted `GNNExplainer` from `torch_geometric.nn` to `torch_geometric.explain.algorithm` ([#5967](https://github.com/pyg-team/pytorch_geometric/pull/5967), [#6065](https://github.com/pyg-team/pytorch_geometric/pull/6065)) - Optimized scatter implementations for CPU/GPU, both with and without backward computation ([#6051](https://github.com/pyg-team/pytorch_geometric/pull/6051), [#6052](https://github.com/pyg-team/pytorch_geometric/pull/6052)) - Support temperature value in `dense_mincut_pool` ([#5908](https://github.com/pyg-team/pytorch_geometric/pull/5908)) - Fixed a bug in which `VirtualNode` mistakenly treated node features as edge features ([#5819](https://github.com/pyg-team/pytorch_geometric/pull/5819)) - Fixed `setter` and `getter` handling in `BaseStorage` ([#5815](https://github.com/pyg-team/pytorch_geometric/pull/5815)) - Fixed `path` in `hetero_conv_dblp.py` example ([#5686](https://github.com/pyg-team/pytorch_geometric/pull/5686)) - Fix `auto_select_device` routine in GraphGym for PyTorch Lightning>=1.7 ([#5677](https://github.com/pyg-team/pytorch_geometric/pull/5677)) - Support `in_channels` with `tuple` in `GENConv` for bipartite message passing ([#5627](https://github.com/pyg-team/pytorch_geometric/pull/5627), [#5641](https://github.com/pyg-team/pytorch_geometric/pull/5641)) - Handle cases of not having enough possible negative edges in `RandomLinkSplit` ([#5642](https://github.com/pyg-team/pytorch_geometric/pull/5642)) - Fix `RGCN+pyg-lib` for `LongTensor` input ([#5610](https://github.com/pyg-team/pytorch_geometric/pull/5610)) - Improved type hint support ([#5842](https://github.com/pyg-team/pytorch_geometric/pull/5842), [#5603](https://github.com/pyg-team/pytorch_geometric/pull/5603), [#5659](https://github.com/pyg-team/pytorch_geometric/pull/5659), [#5664](https://github.com/pyg-team/pytorch_geometric/pull/5664), [#5665](https://github.com/pyg-team/pytorch_geometric/pull/5665), [#5666](https://github.com/pyg-team/pytorch_geometric/pull/5666), [#5667](https://github.com/pyg-team/pytorch_geometric/pull/5667), [#5668](https://github.com/pyg-team/pytorch_geometric/pull/5668), [#5669](https://github.com/pyg-team/pytorch_geometric/pull/5669), [#5673](https://github.com/pyg-team/pytorch_geometric/pull/5673), [#5675](https://github.com/pyg-team/pytorch_geometric/pull/5675), [#5673](https://github.com/pyg-team/pytorch_geometric/pull/5676), [#5678](https://github.com/pyg-team/pytorch_geometric/pull/5678), [#5682](https://github.com/pyg-team/pytorch_geometric/pull/5682), [#5683](https://github.com/pyg-team/pytorch_geometric/pull/5683), [#5684](https://github.com/pyg-team/pytorch_geometric/pull/5684), [#5685](https://github.com/pyg-team/pytorch_geometric/pull/5685), [#5687](https://github.com/pyg-team/pytorch_geometric/pull/5687), [#5688](https://github.com/pyg-team/pytorch_geometric/pull/5688), [#5695](https://github.com/pyg-team/pytorch_geometric/pull/5695), [#5699](https://github.com/pyg-team/pytorch_geometric/pull/5699), [#5701](https://github.com/pyg-team/pytorch_geometric/pull/5701), [#5702](https://github.com/pyg-team/pytorch_geometric/pull/5702), [#5703](https://github.com/pyg-team/pytorch_geometric/pull/5703), [#5706](https://github.com/pyg-team/pytorch_geometric/pull/5706), [#5707](https://github.com/pyg-team/pytorch_geometric/pull/5707), [#5710](https://github.com/pyg-team/pytorch_geometric/pull/5710), [#5714](https://github.com/pyg-team/pytorch_geometric/pull/5714), [#5715](https://github.com/pyg-team/pytorch_geometric/pull/5715), [#5716](https://github.com/pyg-team/pytorch_geometric/pull/5716), [#5722](https://github.com/pyg-team/pytorch_geometric/pull/5722), [#5724](https://github.com/pyg-team/pytorch_geometric/pull/5724), [#5725](https://github.com/pyg-team/pytorch_geometric/pull/5725), [#5726](https://github.com/pyg-team/pytorch_geometric/pull/5726), [#5729](https://github.com/pyg-team/pytorch_geometric/pull/5729), [#5730](https://github.com/pyg-team/pytorch_geometric/pull/5730), [#5731](https://github.com/pyg-team/pytorch_geometric/pull/5731), [#5732](https://github.com/pyg-team/pytorch_geometric/pull/5732), [#5733](https://github.com/pyg-team/pytorch_geometric/pull/5733), [#5743](https://github.com/pyg-team/pytorch_geometric/pull/5743), [#5734](https://github.com/pyg-team/pytorch_geometric/pull/5734), [#5735](https://github.com/pyg-team/pytorch_geometric/pull/5735), [#5736](https://github.com/pyg-team/pytorch_geometric/pull/5736), [#5737](https://github.com/pyg-team/pytorch_geometric/pull/5737), [#5738](https://github.com/pyg-team/pytorch_geometric/pull/5738), [#5747](https://github.com/pyg-team/pytorch_geometric/pull/5747), [#5752](https://github.com/pyg-team/pytorch_geometric/pull/5752), [#5753](https://github.com/pyg-team/pytorch_geometric/pull/5753), [#5754](https://github.com/pyg-team/pytorch_geometric/pull/5754), [#5756](https://github.com/pyg-team/pytorch_geometric/pull/5756), [#5757](https://github.com/pyg-team/pytorch_geometric/pull/5757), [#5758](https://github.com/pyg-team/pytorch_geometric/pull/5758), [#5760](https://github.com/pyg-team/pytorch_geometric/pull/5760), [#5766](https://github.com/pyg-team/pytorch_geometric/pull/5766), [#5767](https://github.com/pyg-team/pytorch_geometric/pull/5767), [#5768](https://github.com/pyg-team/pytorch_geometric/pull/5768)), [#5781](https://github.com/pyg-team/pytorch_geometric/pull/5781), [#5778](https://github.com/pyg-team/pytorch_geometric/pull/5778), [#5797](https://github.com/pyg-team/pytorch_geometric/pull/5797), [#5798](https://github.com/pyg-team/pytorch_geometric/pull/5798), [#5799](https://github.com/pyg-team/pytorch_geometric/pull/5799), [#5800](https://github.com/pyg-team/pytorch_geometric/pull/5800), [#5806](https://github.com/pyg-team/pytorch_geometric/pull/5806), [#5810](https://github.com/pyg-team/pytorch_geometric/pull/5810), [#5811](https://github.com/pyg-team/pytorch_geometric/pull/5811), [#5828](https://github.com/pyg-team/pytorch_geometric/pull/5828), [#5847](https://github.com/pyg-team/pytorch_geometric/pull/5847), [#5851](https://github.com/pyg-team/pytorch_geometric/pull/5851), [#5852](https://github.com/pyg-team/pytorch_geometric/pull/5852)) - Avoid modifying `mode_kwargs` in `MultiAggregation` ([#5601](https://github.com/pyg-team/pytorch_geometric/pull/5601)) - Changed `BatchNorm` to allow for batches of size one during training ([#5530](https://github.com/pyg-team/pytorch_geometric/pull/5530), [#5614](https://github.com/pyg-team/pytorch_geometric/pull/5614)) - Integrated better temporal sampling support by requiring that local neighborhoods are sorted according to time ([#5516](https://github.com/pyg-team/pytorch_geometric/issues/5516), [#5602](https://github.com/pyg-team/pytorch_geometric/issues/5602)) - Fixed a bug when applying several scalers with `PNAConv` ([#5514](https://github.com/pyg-team/pytorch_geometric/issues/5514)) - Allow `.` in `ParameterDict` key names ([#5494](https://github.com/pyg-team/pytorch_geometric/pull/5494)) - Renamed `drop_unconnected_nodes` to `drop_unconnected_node_types` and `drop_orig_edges` to `drop_orig_edge_types` in `AddMetapaths` ([#5490](https://github.com/pyg-team/pytorch_geometric/pull/5490)) - Improved `utils.scatter` performance by explicitly choosing better implementation for `add` and `mean` reduction ([#5399](https://github.com/pyg-team/pytorch_geometric/pull/5399)) - Fix `to_dense_adj` with empty `edge_index` ([#5476](https://github.com/pyg-team/pytorch_geometric/pull/5476)) - The `AttentionalAggregation` module can now be applied to compute attentin on a per-feature level ([#5449](https://github.com/pyg-team/pytorch_geometric/pull/5449)) - Ensure equal lenghts of `num_neighbors` across edge types in `NeighborLoader` ([#5444](https://github.com/pyg-team/pytorch_geometric/pull/5444)) - Fixed a bug in `TUDataset` in which node features were wrongly constructed whenever `node_attributes` only hold a single feature (*e.g.*, in `PROTEINS`) ([#5441](https://github.com/pyg-team/pytorch_geometric/pull/5441)) - Breaking change: removed `num_neighbors` as an attribute of loader ([#5404](https://github.com/pyg-team/pytorch_geometric/pull/5404)) - `ASAPooling` is now jittable ([#5395](https://github.com/pyg-team/pytorch_geometric/pull/5395)) - Updated unsupervised `GraphSAGE` example to leverage `LinkNeighborLoader` ([#5317](https://github.com/pyg-team/pytorch_geometric/pull/5317)) - Replace in-place operations with out-of-place ones to align with `torch.scatter_reduce` API ([#5353](https://github.com/pyg-team/pytorch_geometric/pull/5353)) - Breaking bugfix: `PointTransformerConv` now correctly uses `sum` aggregation ([#5332](https://github.com/pyg-team/pytorch_geometric/pull/5332)) - Improve out-of-bounds error message in `MessagePassing` ([#5339](https://github.com/pyg-team/pytorch_geometric/pull/5339)) - Allow file names of a `Dataset` to be specified as either property and method ([#5338](https://github.com/pyg-team/pytorch_geometric/pull/5338)) - Fixed separating a list of `SparseTensor` within `InMemoryDataset` ([#5299](https://github.com/pyg-team/pytorch_geometric/pull/5299)) - Improved name resolving of normalization layers ([#5277](https://github.com/pyg-team/pytorch_geometric/pull/5277)) - Fail gracefully on `GLIBC` errors within `torch-spline-conv` ([#5276](https://github.com/pyg-team/pytorch_geometric/pull/5276)) - Fixed `Dataset.num_classes` in case a `transform` modifies `data.y` ([#5274](https://github.com/pyg-team/pytorch_geometric/pull/5274)) - Allow customization of the activation function within `PNAConv` ([#5262](https://github.com/pyg-team/pytorch_geometric/pull/5262)) - Do not fill `InMemoryDataset` cache on `dataset.num_features` ([#5264](https://github.com/pyg-team/pytorch_geometric/pull/5264)) - Changed tests relying on `dblp` datasets to instead use synthetic data ([#5250](https://github.com/pyg-team/pytorch_geometric/pull/5250)) - Fixed a bug for the initialization of activation function examples in `custom_graphgym` ([#5243](https://github.com/pyg-team/pytorch_geometric/pull/5243)) - Allow any integer tensors when checking edge_index input to message passing ([5281](https://github.com/pyg-team/pytorch_geometric/pull/5281))

Removed

- Removed `scatter_reduce` option from experimental mode ([#5399](https://github.com/pyg-team/pytorch_geometric/pull/5399))

Full commit list: https://github.com/pyg-team/pytorch_geometric/compare/2.1.0...2.2.0

- Python
Published by rusty1s about 3 years ago

pytorch_geometric - PyG 2.1.0: Principled aggregations, link-level and temporal samplers, data pipe support, ...

We are excited to announce the release of PyG 2.1.0 🎉🎉🎉

Highlights
Breaking Changes
Deprecations
Features
Bugfixes
Full Changelog

PyG 2.1.0 is the culmination of work from over 60 contributors who have worked on features and bug-fixes for a total of over 320 commits since torch-geometric==2.0.4.

Highlights

Principled Aggregations

See here for the accompanying tutorial.

Aggregation functions play an important role in the message passing framework and the readout functions of Graph Neural Networks. Specifically, many works in the literature (Hamilton et al. (2017), Xu et al. (2018), Corso et al. (2020), Li et al. (2020), Tailor et al. (2021), Bartunov et al. (2022)) demonstrate that the choice of aggregation functions contributes significantly to the representational power and performance of the model.

To facilitate further experimentation and unify the concepts of aggregation within GNNs across both MessagePassing and global readouts, we have made the concept of Aggregation a first-class principle in PyG (#4379, #4522, #4687, #4721, #4731, #4762, #4749, #4779, #4863, #4864, #4865, #4866, #4872, #4927, #4934, #4935, #4957, #4973, #4973, #4986, #4995, #5000, #5021, #5034, #5036, #5039, #4522, #5033, #5085, #5097, #5099, #5104, #5113, #5130, #5098, #5191). As of now, PyG provides support for various aggregations — from simple ones (e.g., mean, max, sum), to advanced ones (e.g., median, var, std), learnable ones (e.g., SoftmaxAggregation, PowerMeanAggregation), and exotic ones (e.g., LSTMAggregation, SortAggregation, EquilibriumAggregation). Furthermore, multiple aggregations can be combined and stacked together:

```python from torch_geometric.nn import MessagePassing, SoftmaxAggregation

class MyConv(MessagePassing): def init(self, ...): # Combines a set of aggregations and concatenates their results. # The interface also supports automatic resolution. super().init(aggr=['mean', 'std', SoftmaxAggregation(learn=True)]) ```

Link-level Neighbor Loader

We added a new LinkNeighborLoader class for training scalable GNNs that perform edge-level predictions on giant graphs (#4396, #4439, #4441, #4446, #4508, #4509, #4868). LinkNeighborLoader comes with automatic support for both homogeneous and heterogenous data, and supports link prediction via automatic negative sampling as well as edge-level classification and regression models:

```python from torch_geometric.loader import LinkNeighborLoader

loader = LinkNeighborLoader( data, numneighbors=[30] * 2, # Sample 30 neighbors for each node for 2 iterations batchsize=128, # Use a batch size of 128 for sampling training links edgelabelindex=data.edgeindex, # Use the entire graph for supervision negativesampling_ratio=1.0, # Sample negative edges )

sampleddata = next(iter(loader)) print(sampleddata)

Data(x=[1368, 1433], edgeindex=[2, 3103], edgelabelindex=[2, 256], edgelabel=[256]) ```

Neighborhood Sampling based on Temporal Constraints

Both NeighborLoader and LinkNeighborLoader now support temporal sampling via the time_attr argument (#4025, #4877, #4908, #5137, #5173). If set, temporal sampling will be used such that neighbors are guaranteed to fulfill temporal constraints, i.e. neighbors have an earlier timestamp than the center node:

```python from torch_geometric.loader import NeighborLoader

data['paper'].time = torch.arange(data['paper'].num_nodes)

loader = NeighborLoader( data, inputnodes='paper', timeattr='time', # Only sample papers that appeared before the seed paper numneighbors=[30] * 2, batchsize=128, ) ```

Note that this feature requires torch-sparse>=0.6.14.

Functional `DataPipes`

See here for the accompanying example.

PyG now fully supports data loading using the newly introduced concept of DataPipes in PyTorch for easily constructing flexible and performant data pipelines (#4302, #4345, #4349). PyG provides DataPipe support for batching multiple PyG data objects together and for applying any PyG transform:

python datapipe = FileOpener(['SMILES_HIV.csv']) datapipe = datapipe.parse_csv_as_dict() datapipe = datapipe.parse_smiles(target_key='HIV_active') datapipe = datapipe.in_memory_cache() # Cache graph instances in-memory. datapipe = datapipe.shuffle() datapipe = datapipe.batch_graphs(batch_size=32)

python datapipe = FileLister([root_dir], masks='*.off', recursive=True) datapipe = datapipe.read_mesh() datapipe = datapipe.in_memory_cache() # Cache graph instances in-memory. datapipe = datapipe.sample_points(1024) # Use PyG transforms from here. datapipe = datapipe.knn_graph(k=8) datapipe = datapipe.shuffle() datapipe = datapipe.batch_graphs(batch_size=32)

Breaking Changes

The torch_geometric.utils.metric package has been removed. We now recommend to use the torchmetrics package instead.
len(batch) of the data.Batch class will now return the number of graphs inside the batch, not the number of attributes (#4931)

Deprecations

The usage of the torch_geometric.nn.glob package is now deprecated in favor of torch_geometric.nn.aggr
The usage of RandomTranslate is now deprecated in favor of RandomJitter (#4828)

Features

Layers, Models and Examples

Added the GroupAddRev module with support for reducing training GPU memory (#4671, #4701, #4715, #4730) [Example]
Added the MaskLabel module for performing masked label propagation (#4197) **[Example]
Added the DimeNetPlusPlus module (#4432, #4699, #4700, #4800)
Added the MeanSubtractionNorm module (#5068)
Added the DynamicBatchSampler for filling a mini-batch with a variable number of samples up to a maximum size (#4972)
Added PyTorch Lightning support in GraphGym (#4511, #4516 #4531, #4689, #4843)
Added an example of using PyG with PyTorch Ignite (#4487)
Added an example for unsupervised heterogeneous graph learning (#3189)
Added an example for unsupervised GraphSAGE on the PPI dataset (#4416)
Added the EdgeCNN model (#4991)
Added an example to load a trained PyG model in C++ (#4307)

Transformations

Added the AddPositionalEncoding transforms with two implementations: AddLaplacianEigenvectorPE and AddRandomWalkPE (#4521)
Added the Rooted transform with two implementations: RootedEgoNets and RootedRWSubgraph (#3926)
Added support for computing weighted metapaths in AddMetapaths (#5049)

Datasets

Added the Genius and Wiki datasets to the LINKXDataset (#4570, #4600)
Added the AQSOL dataset (#4626)
Added Geom-GCN splits to the Planetoid datasets (#4442)

General Improvements

Added support for GATv2Conv in the GAT model (#4357)
Added support for projecting features before propagation in SAGEConv (#4437)
Added a MessagePassing.explain_message() method to customize making explanations on messages (#4278, #4448))
Added the MLP.plain_last = False option (4652)
Added support graph-level attributes in networkx conversion (#4343)
Added Data.validate() and HeteroData.validate() functionality to validate the correctness of the data (#4885)
Added TorchScript support to JumpingKnowledge module (#4805)
Added predict() support to the LightningNodeData module (#4884)
Added support for renaming node types via HeteroData.rename() (#4329)
Added HeteroData.num_features functionality (#4504)
Added HeteroData.subgraph, HeteroData.node_type_subgraph and HeteroData.edge_type_subgraph functionality (#4243)
Added HeteroData support to the RemoveIsolatedNodes transform (#4479)
Added support for graph-level outputs in to_hetero (#4582)
Added HeteroData.is_undirected() support (#4604)
Added HeteroData.node_items() and HeteroData.edge_items() functionality (#4644)
Added HeteroData.subgraph() support (#4635)
Added node-wise normalization mode in LayerNorm (#4944)
Added utils.unbatch and utils.unbatch_edge_index functionality for splitting an edge_index tensor according to a batch vector (#4628, #4903)
Added scalable inference mode in BasicGNN with layer-wise neighbor loading (#4977)
Added fine grained options for setting bias and dropout per layer in the MLP model (#4981)
Added support for BasicGNN models within to_hetero (#5091)
Let ImbalancedSampler accept torch.Tensor as input (#5138)
Allow edge_type == rev_edge_type argument in RandomLinkSplit (#4757)

Bugfixes

Fixed a bug in RGATConv that produced device mismatches for "f-scaled" mode (#5187]
Fixed a bug in GINEConv bug for non-Sequential neural network layers (#5154]
Fixed a bug in HGTLoader which produced outputs with missing edge types, will require torch-sparse>=0.6.15 (#5067)
Fixed a bug in load_state_dict for Linear with strict=False mode (5094)
Fixed data.num_node_features computation for sparse matrices (5089)
Fixed a bug in which GraphGym did not create new non-linearity functions but re-used existing ones (4978)
Fixed BasicGNN for num_layers=1, which now respects a desired number of out_channels (#4943)
Fixed a bug in data.subgraph for 0-dim tensors (#4932)
Fixed a bug in InMemoryDataset inferring wrong length for lists of tensors (#4837)
Fixed a bug in TUDataset where pre_filter was not applied whenever pre_transform was present (#4842)
Fixed access of edge types in HeteroData via two node types when there exists multiple relations between them (#4782)
Fixed a bug in HANConv in which destination node features rather than source node features were propagated (#4753)
Fixed a ranking protocol bug in the RGCN link prediction example (#4688)
Fixed the interplay between TUDataset and pre_transform transformations that modify node features (#4669)
The bias argument in TAGConv is now correctly applied (#4597)
Fixed filtering of attributes in samplers in case __cat_dim__ != 0 (#4629)
Fixed SparseTensor support in NeighborLoader (#4320)
Fixed average degree handling in PNAConv (#4312)
Fixed a bug in from_networkx in case some attributes are PyTorch tensors (#4486)
Fixed a missing clamp in the DimeNet model (#4506, #4562)
Fixed the download link in DBP15K (#4428)
Fixed an autograd bug in DimeNet when resetting parameters (#4424)
Fixed bipartite message passing in case flow="target_to_source" (#4418)
Fixed a bug in which num_nodes was not properly updated in the FixedPoints transform (#4394)
Fixed a bug in which GATConv was not jittable (#4347)
Fixed a bug in which nn.models.GAT did not produce out_channels many output channels (#4299)
Fixed a bug in mini-batching with empty lists as attributes (#4293)
Fixed a bug in which GCNConv could not be combined with to_hetero on heterogeneous graphs with one node type (#4279)

Full Changelog

Added

- Added `edge_label_time` argument to `LinkNeighborLoader` ([#5137](https://github.com/pyg-team/pytorch_geometric/pull/5137), [#5173](https://github.com/pyg-team/pytorch_geometric/pull/5173)) - Let `ImbalancedSampler` accept `torch.Tensor` as input ([#5138](https://github.com/pyg-team/pytorch_geometric/pull/5138)) - Added `flow` argument to `gcn_norm` to correctly normalize the adjacency matrix in `GCNConv` ([#5149](https://github.com/pyg-team/pytorch_geometric/pull/5149)) - `NeighborSampler` supports graphs without edges ([#5072](https://github.com/pyg-team/pytorch_geometric/pull/5072)) - Added the `MeanSubtractionNorm` layer ([#5068](https://github.com/pyg-team/pytorch_geometric/pull/5068)) - Added `pyg_lib.segment_matmul` integration within `RGCNConv` ([#5052](https://github.com/pyg-team/pytorch_geometric/pull/5052), [#5096](https://github.com/pyg-team/pytorch_geometric/pull/5096)) - Support `SparseTensor` as edge label in `LightGCN` (#[5046](https://github.com/pyg-team/pytorch_geometric/issues/5046)) - Added support for `BasicGNN` models within `to_hetero` ([#5091](https://github.com/pyg-team/pytorch_geometric/pull/5091)) - Added support for computing weighted metapaths in `AddMetapaths` ([#5049](https://github.com/pyg-team/pytorch_geometric/pull/5049)) - Added inference benchmark suite ([#4915](https://github.com/pyg-team/pytorch_geometric/pull/4915)) - Added a dynamically sized batch sampler for filling a mini-batch with a variable number of samples up to a maximum size ([#4972](https://github.com/pyg-team/pytorch_geometric/pull/4972)) - Added fine grained options for setting `bias` and `dropout` per layer in the `MLP` model ([#4981](https://github.com/pyg-team/pytorch_geometric/pull/4981)) - Added `EdgeCNN` model ([#4991](https://github.com/pyg-team/pytorch_geometric/pull/4991)) - Added scalable `inference` mode in `BasicGNN` with layer-wise neighbor loading ([#4977](https://github.com/pyg-team/pytorch_geometric/pull/4977)) - Added inference benchmarks ([#4892](https://github.com/pyg-team/pytorch_geometric/pull/4892), [#5107](https://github.com/pyg-team/pytorch_geometric/pull/5107)) - Added PyTorch 1.12 support ([#4975](https://github.com/pyg-team/pytorch_geometric/pull/4975)) - Added `unbatch_edge_index` functionality for splitting an `edge_index` tensor according to a `batch` vector ([#4903](https://github.com/pyg-team/pytorch_geometric/pull/4903)) - Added node-wise normalization mode in `LayerNorm` ([#4944](https://github.com/pyg-team/pytorch_geometric/pull/4944)) - Added support for `normalization_resolver` ([#4926](https://github.com/pyg-team/pytorch_geometric/pull/4926), [#4951](https://github.com/pyg-team/pytorch_geometric/pull/4951), [#4958](https://github.com/pyg-team/pytorch_geometric/pull/4958), [#4959](https://github.com/pyg-team/pytorch_geometric/pull/4959)) - Added notebook tutorial for `torch_geometric.nn.aggr` package to documentation ([#4927](https://github.com/pyg-team/pytorch_geometric/pull/4927)) - Added support for `follow_batch` for lists or dictionaries of tensors ([#4837](https://github.com/pyg-team/pytorch_geometric/pull/4837)) - Added `Data.validate()` and `HeteroData.validate()` functionality ([#4885](https://github.com/pyg-team/pytorch_geometric/pull/4885)) - Added `LinkNeighborLoader` support to `LightningDataModule` ([#4868](https://github.com/pyg-team/pytorch_geometric/pull/4868)) - Added `predict()` support to the `LightningNodeData` module ([#4884](https://github.com/pyg-team/pytorch_geometric/pull/4884)) - Added `time_attr` argument to `LinkNeighborLoader` ([#4877](https://github.com/pyg-team/pytorch_geometric/pull/4877), [#4908](https://github.com/pyg-team/pytorch_geometric/pull/4908)) - Added a `filter_per_worker` argument to data loaders to allow filtering of data within sub-processes ([#4873](https://github.com/pyg-team/pytorch_geometric/pull/4873)) - Added a `NeighborLoader` benchmark script ([#4815](https://github.com/pyg-team/pytorch_geometric/pull/4815), [#4862](https://github.com/pyg-team/pytorch_geometric/pull/4862/files)) - Added support for `FeatureStore` and `GraphStore` in `NeighborLoader` ([#4817](https://github.com/pyg-team/pytorch_geometric/pull/4817), [#4851](https://github.com/pyg-team/pytorch_geometric/pull/4851), [#4854](https://github.com/pyg-team/pytorch_geometric/pull/4854), [#4856](https://github.com/pyg-team/pytorch_geometric/pull/4856), [#4857](https://github.com/pyg-team/pytorch_geometric/pull/4857), [#4882](https://github.com/pyg-team/pytorch_geometric/pull/4882), [#4883](https://github.com/pyg-team/pytorch_geometric/pull/4883), [#4929](https://github.com/pyg-team/pytorch_geometric/pull/4929), [#4992](https://github.com/pyg-team/pytorch_geometric/pull/4922), [#4962](https://github.com/pyg-team/pytorch_geometric/pull/4962), [#4968](https://github.com/pyg-team/pytorch_geometric/pull/4968), [#5037](https://github.com/pyg-team/pytorch_geometric/pull/5037), [#5088](https://github.com/pyg-team/pytorch_geometric/pull/5088)) - Added a `normalize` parameter to `dense_diff_pool` ([#4847](https://github.com/pyg-team/pytorch_geometric/pull/4847)) - Added `size=None` explanation to jittable `MessagePassing` modules in the documentation ([#4850](https://github.com/pyg-team/pytorch_geometric/pull/4850)) - Added documentation to the `DataLoaderIterator` class ([#4838](https://github.com/pyg-team/pytorch_geometric/pull/4838)) - Added `GraphStore` support to `Data` and `HeteroData` ([#4816](https://github.com/pyg-team/pytorch_geometric/pull/4816)) - Added `FeatureStore` support to `Data` and `HeteroData` ([#4807](https://github.com/pyg-team/pytorch_geometric/pull/4807), [#4853](https://github.com/pyg-team/pytorch_geometric/pull/4853)) - Added `FeatureStore` and `GraphStore` abstractions ([#4534](https://github.com/pyg-team/pytorch_geometric/pull/4534), [#4568](https://github.com/pyg-team/pytorch_geometric/pull/4568)) - Added support for dense aggregations in `global_*_pool` ([#4827](https://github.com/pyg-team/pytorch_geometric/pull/4827)) - Added Python version requirement ([#4825](https://github.com/pyg-team/pytorch_geometric/pull/4825)) - Added TorchScript support to `JumpingKnowledge` module ([#4805](https://github.com/pyg-team/pytorch_geometric/pull/4805)) - Added a `max_sample` argument to `AddMetaPaths` in order to tackle very dense metapath edges ([#4750](https://github.com/pyg-team/pytorch_geometric/pull/4750)) - Test `HANConv` with empty tensors ([#4756](https://github.com/pyg-team/pytorch_geometric/pull/4756), [#4841](https://github.com/pyg-team/pytorch_geometric/pull/4841)) - Added the `bias` vector to the `GCN` model definition in the "Create Message Passing Networks" tutorial ([#4755](https://github.com/pyg-team/pytorch_geometric/pull/4755)) - Added `transforms.RootedSubgraph` interface with two implementations: `RootedEgoNets` and `RootedRWSubgraph` ([#3926](https://github.com/pyg-team/pytorch_geometric/pull/3926)) - Added `ptr` vectors for `follow_batch` attributes within `Batch.from_data_list` ([#4723](https://github.com/pyg-team/pytorch_geometric/pull/4723)) - Added `torch_geometric.nn.aggr` package ([#4687](https://github.com/pyg-team/pytorch_geometric/pull/4687), [#4721](https://github.com/pyg-team/pytorch_geometric/pull/4721), [#4731](https://github.com/pyg-team/pytorch_geometric/pull/4731), [#4762](https://github.com/pyg-team/pytorch_geometric/pull/4762), [#4749](https://github.com/pyg-team/pytorch_geometric/pull/4749), [#4779](https://github.com/pyg-team/pytorch_geometric/pull/4779), [#4863](https://github.com/pyg-team/pytorch_geometric/pull/4863), [#4864](https://github.com/pyg-team/pytorch_geometric/pull/4864), [#4865](https://github.com/pyg-team/pytorch_geometric/pull/4865), [#4866](https://github.com/pyg-team/pytorch_geometric/pull/4866), [#4872](https://github.com/pyg-team/pytorch_geometric/pull/4872), [#4934](https://github.com/pyg-team/pytorch_geometric/pull/4934), [#4935](https://github.com/pyg-team/pytorch_geometric/pull/4935), [#4957](https://github.com/pyg-team/pytorch_geometric/pull/4957), [#4973](https://github.com/pyg-team/pytorch_geometric/pull/4973), [#4973](https://github.com/pyg-team/pytorch_geometric/pull/4973), [#4986](https://github.com/pyg-team/pytorch_geometric/pull/4986), [#4995](https://github.com/pyg-team/pytorch_geometric/pull/4995), [#5000](https://github.com/pyg-team/pytorch_geometric/pull/5000), [#5034](https://github.com/pyg-team/pytorch_geometric/pull/5034), [#5036](https://github.com/pyg-team/pytorch_geometric/pull/5036), [#5039](https://github.com/pyg-team/pytorch_geometric/issues/5039), [#4522](https://github.com/pyg-team/pytorch_geometric/pull/4522), [#5033](https://github.com/pyg-team/pytorch_geometric/pull/5033]), [#5085](https://github.com/pyg-team/pytorch_geometric/pull/5085), [#5097](https://github.com/pyg-team/pytorch_geometric/pull/5097), [#5099](https://github.com/pyg-team/pytorch_geometric/pull/5099), [#5104](https://github.com/pyg-team/pytorch_geometric/pull/5104), [#5113](https://github.com/pyg-team/pytorch_geometric/pull/5113), [#5130](https://github.com/pyg-team/pytorch_geometric/pull/5130), [#5098](https://github.com/pyg-team/pytorch_geometric/pull/5098), [#5191](https://github.com/pyg-team/pytorch_geometric/pull/5191)) - Added the `DimeNet++` model ([#4432](https://github.com/pyg-team/pytorch_geometric/pull/4432), [#4699](https://github.com/pyg-team/pytorch_geometric/pull/4699), [#4700](https://github.com/pyg-team/pytorch_geometric/pull/4700), [#4800](https://github.com/pyg-team/pytorch_geometric/pull/4800)) - Added an example of using PyG with PyTorch Ignite ([#4487](https://github.com/pyg-team/pytorch_geometric/pull/4487)) - Added `GroupAddRev` module with support for reducing training GPU memory ([#4671](https://github.com/pyg-team/pytorch_geometric/pull/4671), [#4701](https://github.com/pyg-team/pytorch_geometric/pull/4701), [#4715](https://github.com/pyg-team/pytorch_geometric/pull/4715), [#4730](https://github.com/pyg-team/pytorch_geometric/pull/4730)) - Added benchmarks via [`wandb`](https://wandb.ai/site) ([#4656](https://github.com/pyg-team/pytorch_geometric/pull/4656), [#4672](https://github.com/pyg-team/pytorch_geometric/pull/4672), [#4676](https://github.com/pyg-team/pytorch_geometric/pull/4676)) - Added `unbatch` functionality ([#4628](https://github.com/pyg-team/pytorch_geometric/pull/4628)) - Confirm that `to_hetero()` works with custom functions, *e.g.*, `dropout_adj` ([4653](https://github.com/pyg-team/pytorch_geometric/pull/4653)) - Added the `MLP.plain_last=False` option ([4652](https://github.com/pyg-team/pytorch_geometric/pull/4652)) - Added a check in `HeteroConv` and `to_hetero()` to ensure that `MessagePassing.add_self_loops` is disabled ([4647](https://github.com/pyg-team/pytorch_geometric/pull/4647)) - Added `HeteroData.subgraph()` support ([#4635](https://github.com/pyg-team/pytorch_geometric/pull/4635)) - Added the `AQSOL` dataset ([#4626](https://github.com/pyg-team/pytorch_geometric/pull/4626)) - Added `HeteroData.node_items()` and `HeteroData.edge_items()` functionality ([#4644](https://github.com/pyg-team/pytorch_geometric/pull/4644)) - Added PyTorch Lightning support in GraphGym ([#4511](https://github.com/pyg-team/pytorch_geometric/pull/4511), [#4516](https://github.com/pyg-team/pytorch_geometric/pull/4516) [#4531](https://github.com/pyg-team/pytorch_geometric/pull/4531), [#4689](https://github.com/pyg-team/pytorch_geometric/pull/4689), [#4843](https://github.com/pyg-team/pytorch_geometric/pull/4843)) - Added support for returning embeddings in `MLP` models ([#4625](https://github.com/pyg-team/pytorch_geometric/pull/4625)) - Added faster initialization of `NeighborLoader` in case edge indices are already sorted (via `is_sorted=True`) ([#4620](https://github.com/pyg-team/pytorch_geometric/pull/4620), [#4702](https://github.com/pyg-team/pytorch_geometric/pull/4702)) - Added `AddPositionalEncoding` transform ([#4521](https://github.com/pyg-team/pytorch_geometric/pull/4521)) - Added `HeteroData.is_undirected()` support ([#4604](https://github.com/pyg-team/pytorch_geometric/pull/4604)) - Added the `Genius` and `Wiki` datasets to `nn.datasets.LINKXDataset` ([#4570](https://github.com/pyg-team/pytorch_geometric/pull/4570), [#4600](https://github.com/pyg-team/pytorch_geometric/pull/4600)) - Added `nn.aggr.EquilibrumAggregation` implicit global layer ([#4522](https://github.com/pyg-team/pytorch_geometric/pull/4522)) - Added support for graph-level outputs in `to_hetero` ([#4582](https://github.com/pyg-team/pytorch_geometric/pull/4582)) - Added `CHANGELOG.md` ([#4581](https://github.com/pyg-team/pytorch_geometric/pull/4581)) - Added `HeteroData` support to the `RemoveIsolatedNodes` transform ([#4479](https://github.com/pyg-team/pytorch_geometric/pull/4479)) - Added `HeteroData.num_features` functionality ([#4504](https://github.com/pyg-team/pytorch_geometric/pull/4504)) - Added support for projecting features before propagation in `SAGEConv` ([#4437](https://github.com/pyg-team/pytorch_geometric/pull/4437)) - Added `Geom-GCN` splits to the `Planetoid` datasets ([#4442](https://github.com/pyg-team/pytorch_geometric/pull/4442)) - Added a `LinkNeighborLoader` for training scalable link predictions models [#4396](https://github.com/pyg-team/pytorch_geometric/pull/4396), [#4439](https://github.com/pyg-team/pytorch_geometric/pull/4439), [#4441](https://github.com/pyg-team/pytorch_geometric/pull/4441), [#4446](https://github.com/pyg-team/pytorch_geometric/pull/4446), [#4508](https://github.com/pyg-team/pytorch_geometric/pull/4508), [#4509](https://github.com/pyg-team/pytorch_geometric/pull/4509)) - Added an unsupervised `GraphSAGE` example on `PPI` ([#4416](https://github.com/pyg-team/pytorch_geometric/pull/4416)) - Added support for `LSTM` aggregation in `SAGEConv` ([#4379](https://github.com/pyg-team/pytorch_geometric/pull/4379)) - Added support for floating-point labels in `RandomLinkSplit` ([#4311](https://github.com/pyg-team/pytorch_geometric/pull/4311), [#4383](https://github.com/pyg-team/pytorch_geometric/pull/4383)) - Added support for `torch.data` `DataPipes` ([#4302](https://github.com/pyg-team/pytorch_geometric/pull/4302), [#4345](https://github.com/pyg-team/pytorch_geometric/pull/4345), [#4349](https://github.com/pyg-team/pytorch_geometric/pull/4349)) - Added support for the `cosine` argument in the `KNNGraph`/`RadiusGraph` transforms ([#4344](https://github.com/pyg-team/pytorch_geometric/pull/4344)) - Added support graph-level attributes in `networkx` conversion ([#4343](https://github.com/pyg-team/pytorch_geometric/pull/4343)) - Added support for renaming node types via `HeteroData.rename` ([#4329](https://github.com/pyg-team/pytorch_geometric/pull/4329)) - Added an example to load a trained PyG model in C++ ([#4307](https://github.com/pyg-team/pytorch_geometric/pull/4307)) - Added a `MessagePassing.explain_message` method to customize making explanations on messages ([#4278](https://github.com/pyg-team/pytorch_geometric/pull/4278), [#4448](https://github.com/pyg-team/pytorch_geometric/pull/4448))) - Added support for `GATv2Conv` in the `nn.models.GAT` model ([#4357](https://github.com/pyg-team/pytorch_geometric/pull/4357)) - Added `HeteroData.subgraph` functionality ([#4243](https://github.com/pyg-team/pytorch_geometric/pull/4243)) - Added the `MaskLabel` module and a corresponding masked label propagation example ([#4197](https://github.com/pyg-team/pytorch_geometric/pull/4197)) - Added temporal sampling support to `NeighborLoader` ([#4025](https://github.com/pyg-team/pytorch_geometric/pull/4025)) - Added an example for unsupervised heterogeneous graph learning based on "Deep Multiplex Graph Infomax" ([#3189](https://github.com/pyg-team/pytorch_geometric/pull/3189))

Changed

- Changed docstring for `RandomLinkSplit` ([#5190](https://github.com/pyg-team/pytorch_geometric/issues/5190)) - Switched to PyTorch `scatter_reduce` implementation - experimental feature ([#5120](https://github.com/pyg-team/pytorch_geometric/pull/5120)) - Fixed `RGATConv` device mismatches for `f-scaled` mode ([#5187](https://github.com/pyg-team/pytorch_geometric/pull/5187)] - Allow for multi-dimensional `edge_labels` in `LinkNeighborLoader` ([#5186](https://github.com/pyg-team/pytorch_geometric/pull/5186)] - Fixed `GINEConv` bug with non-sequential input ([#5154](https://github.com/pyg-team/pytorch_geometric/pull/5154)] - Improved error message ([#5095](https://github.com/pyg-team/pytorch_geometric/pull/5095)) - Fixed `HGTLoader` bug which produced outputs with missing edge types ([#5067](https://github.com/pyg-team/pytorch_geometric/pull/5067)) - Fixed dynamic inheritance issue in data batching ([#5051](https://github.com/pyg-team/pytorch_geometric/pull/5051)) - Fixed `load_state_dict` in `Linear` with `strict=False` mode ([5094](https://github.com/pyg-team/pytorch_geometric/pull/5094)) - Fixed typo in `MaskLabel.ratio_mask` ([5093](https://github.com/pyg-team/pytorch_geometric/pull/5093)) - Fixed `data.num_node_features` computation for sparse matrices ([5089](https://github.com/pyg-team/pytorch_geometric/pull/5089)) - Fixed `torch.fx` bug with `torch.nn.aggr` package ([#5021](https://github.com/pyg-team/pytorch_geometric/pull/5021))) - Fixed `GenConv` test ([4993](https://github.com/pyg-team/pytorch_geometric/pull/4993)) - Fixed packaging tests for Python 3.10 ([4982](https://github.com/pyg-team/pytorch_geometric/pull/4982)) - Changed `act_dict` (part of `graphgym`) to create individual instances instead of reusing the same ones everywhere ([4978](https://github.com/pyg-team/pytorch_geometric/pull/4978)) - Fixed issue where one-hot tensors were passed to `F.one_hot` ([4970](https://github.com/pyg-team/pytorch_geometric/pull/4970)) - Fixed `bool` arugments in `argparse` in `benchmark/` ([#4967](https://github.com/pyg-team/pytorch_geometric/pull/4967)) - Fixed `BasicGNN` for `num_layers=1`, which now respects a desired number of `out_channels` ([#4943](https://github.com/pyg-team/pytorch_geometric/pull/4943)) - `len(batch)` will now return the number of graphs inside the batch, not the number of attributes ([#4931](https://github.com/pyg-team/pytorch_geometric/pull/4931)) - Fixed `data.subgraph` generation for 0-dim tensors ([#4932](https://github.com/pyg-team/pytorch_geometric/pull/4932)) - Removed unnecssary inclusion of self-loops when sampling negative edges ([#4880](https://github.com/pyg-team/pytorch_geometric/pull/4880)) - Fixed `InMemoryDataset` inferring wrong `len` for lists of tensors ([#4837](https://github.com/pyg-team/pytorch_geometric/pull/4837)) - Fixed `Batch.separate` when using it for lists of tensors ([#4837](https://github.com/pyg-team/pytorch_geometric/pull/4837)) - Correct docstring for SAGEConv ([#4852](https://github.com/pyg-team/pytorch_geometric/pull/4852)) - Fixed a bug in `TUDataset` where `pre_filter` was not applied whenever `pre_transform` was present ([#4842](https://github.com/pyg-team/pytorch_geometric/pull/4842)) - Renamed `RandomTranslate` to `RandomJitter` - the usage of `RandomTranslate` is now deprecated ([#4828](https://github.com/pyg-team/pytorch_geometric/pull/4828)) - Do not allow accessing edge types in `HeteroData` with two node types when there exists multiple relations between these types ([#4782](https://github.com/pyg-team/pytorch_geometric/pull/4782)) - Allow `edge_type == rev_edge_type` argument in `RandomLinkSplit` ([#4757](https://github.com/pyg-team/pytorch_geometric/pull/4757)) - Fixed a numerical instability in the `GeneralConv` and `neighbor_sample` tests ([#4754](https://github.com/pyg-team/pytorch_geometric/pull/4754)) - Fixed a bug in `HANConv` in which destination node features rather than source node features were propagated ([#4753](https://github.com/pyg-team/pytorch_geometric/pull/4753)) - Fixed versions of `checkout` and `setup-python` in CI ([#4751](https://github.com/pyg-team/pytorch_geometric/pull/4751)) - Fixed `protobuf` version ([#4719](https://github.com/pyg-team/pytorch_geometric/pull/4719)) - Fixed the ranking protocol bug in the RGCN link prediction example ([#4688](https://github.com/pyg-team/pytorch_geometric/pull/4688)) - Math support in Markdown ([#4683](https://github.com/pyg-team/pytorch_geometric/pull/4683)) - Allow for `setter` properties in `Data` ([#4682](https://github.com/pyg-team/pytorch_geometric/pull/4682), [#4686](https://github.com/pyg-team/pytorch_geometric/pull/4686)) - Allow for optional `edge_weight` in `GCN2Conv` ([#4670](https://github.com/pyg-team/pytorch_geometric/pull/4670)) - Fixed the interplay between `TUDataset` and `pre_transform` that modify node features ([#4669](https://github.com/pyg-team/pytorch_geometric/pull/4669)) - Make use of the `pyg_sphinx_theme` documentation template ([#4664](https://github.com/pyg-team/pyg-lib/pull/4664), [#4667](https://github.com/pyg-team/pyg-lib/pull/4667)) - Refactored reading molecular positions from sdf file for qm9 datasets ([4654](https://github.com/pyg-team/pytorch_geometric/pull/4654)) - Fixed `MLP.jittable()` bug in case `return_emb=True` ([#4645](https://github.com/pyg-team/pytorch_geometric/pull/4645), [#4648](https://github.com/pyg-team/pytorch_geometric/pull/4648)) - The generated node features of `StochasticBlockModelDataset` are now ordered with respect to their labels ([#4617](https://github.com/pyg-team/pytorch_geometric/pull/4617)) - Fixed typos in the documentation ([#4616](https://github.com/pyg-team/pytorch_geometric/pull/4616), [#4824](https://github.com/pyg-team/pytorch_geometric/pull/4824), [#4895](https://github.com/pyg-team/pytorch_geometric/pull/4895), [#5161](https://github.com/pyg-team/pytorch_geometric/pull/5161)) - The `bias` argument in `TAGConv` is now actually applied ([#4597](https://github.com/pyg-team/pytorch_geometric/pull/4597)) - Fixed subclass behaviour of `process` and `download` in `Datsaet` ([#4586](https://github.com/pyg-team/pytorch_geometric/pull/4586)) - Fixed filtering of attributes for loaders in case `__cat_dim__ != 0` ([#4629](https://github.com/pyg-team/pytorch_geometric/pull/4629)) - Fixed `SparseTensor` support in `NeighborLoader` ([#4320](https://github.com/pyg-team/pytorch_geometric/pull/4320)) - Fixed average degree handling in `PNAConv` ([#4312](https://github.com/pyg-team/pytorch_geometric/pull/4312)) - Fixed a bug in `from_networkx` in case some attributes are PyTorch tensors ([#4486](https://github.com/pyg-team/pytorch_geometric/pull/4486)) - Added a missing clamp in `DimeNet` ([#4506](https://github.com/pyg-team/pytorch_geometric/pull/4506), [#4562](https://github.com/pyg-team/pytorch_geometric/pull/4562)) - Fixed the download link in `DBP15K` ([#4428](https://github.com/pyg-team/pytorch_geometric/pull/4428)) - Fixed an autograd bug in `DimeNet` when resetting parameters ([#4424](https://github.com/pyg-team/pytorch_geometric/pull/4424)) - Fixed bipartite message passing in case `flow="target_to_source"` ([#4418](https://github.com/pyg-team/pytorch_geometric/pull/4418)) - Fixed a bug in which `num_nodes` was not properly updated in the `FixedPoints` transform ([#4394](https://github.com/pyg-team/pytorch_geometric/pull/4394)) - PyTorch Lightning >= 1.6 support ([#4377](https://github.com/pyg-team/pytorch_geometric/pull/4377)) - Fixed a bug in which `GATConv` was not jittable ([#4347](https://github.com/pyg-team/pytorch_geometric/pull/4347)) - Fixed a bug in which the GraphGym config was not stored in each specific experiment directory ([#4338](https://github.com/pyg-team/pytorch_geometric/pull/4338)) - Fixed a bug in which `nn.models.GAT` did not produce `out_channels`-many output channels ([#4299](https://github.com/pyg-team/pytorch_geometric/pull/4299)) - Fixed mini-batching with empty lists as attributes ([#4293](https://github.com/pyg-team/pytorch_geometric/pull/4293)) - Fixed a bug in which `GCNConv` could not be combined with `to_hetero` on heterogeneous graphs with one node type ([#4279](https://github.com/pyg-team/pytorch_geometric/pull/4279))

Removed

- Remove internal metrics in favor of `torchmetrics` ([#4287](https://github.com/pyg-team/pytorch_geometric/pull/4287))

Full commit list: https://github.com/pyg-team/pytorch_geometric/compare/2.0.4...2.1.0

- Python
Published by rusty1s over 3 years ago

pytorch_geometric - 2.0.4

PyG 2.0.4 🎉

A new minor PyG version release, bringing PyTorch 1.11 support to PyG. It further includes a variety of new features and bugfixes:

Features

Added Quiver examples for multi-GU training using GraphSAGE (#4103), thanks to @eedalong and @luomai
nn.model.to_captum: Full integration of explainability methods provided by the Captum library (#3990, #4076), thanks to @RBendias
nn.conv.RGATConv: The relational graph attentional operator (#4031, #4110), thanks to @fork123aniket
nn.pool.DMoNPooling: The spectral modularity pooling operator (#4166, #4242), thanks to @fork123aniket
nn.*: Support for shape information in the documentation (#3739, #3889, #3893, #3946, #3981, #4009, #4120, #4158), thanks to @saiden89 and @arunppsg and @konstantinosKokos
loader.TemporalDataLoader: A dataloader to load a TemporalData object in mini-batches (#3985, #3988), thanks to @otaviocx
loader.ImbalancedSampler: A weighted random sampler that randomly samples elements according to class distribution (#4198)
transforms.VirtualNode: A transform that adds a virtual node to a graph (#4163)
transforms.LargestConnectedComponents: Selects the subgraph that corresponds to the largest connected components in the graph (#3949), thanks to @abojchevski
utils.homophily: Support for class-insensitive edge homophily (#3977, #4152), thanks to @hash-ir and @jinjh0123
utils.get_mesh_laplacian: Mesh Laplacian computation (#4187), thanks to @daniel-unyi-42

Datasets

Added a dataset cheatsheet to the documentation that collects import graph statistics across a variety of datasets supported in PyG (#3807, #3817) (please consider helping us filling its remaining content)
datasets.EllipticBitcoinDataset: A dataset of Bitcoin transactions (#3815), thanks to @shravankumar147

Minor Changes

nn.models.MLP: MLPs can now either be initialized via a list of channels or by specifying hidden_channels and num_layers (#3957)
nn.models.BasicGNN: Final Linear transformations are now always applied (except for jk=None) (#4042)
nn.conv.MessagePassing: Message passing modules that make use of edge_updater are now jittable (#3765), thanks to @Padarn
nn.conv.MessagePassing: (Official) support for min and mul aggregations (#4219)
nn.LightGCN: Initialize embeddings via xavier_uniform for better model performance (#4083), thanks to @nishithshowri006
nn.conv.ChebConv: Automatic eigenvalue approximation (#4106), thanks to @daniel-unyi-42
nn.conv.APPNP: Added support for optional edge_weight, (690a01d), thanks to @YueeXiang
nn.conv.GravNetConv: Support for torch.jit.script (#3885), thanks to @RobMcH
nn.pool.global_*_pool: The batch vector is now optional (#4161)
nn.to_hetero: Added a warning in case to_hetero is used on HeteroData metadata with unused destination node types (#3775)
nn.to_hetero: Support for nested modules (ea135bf)
nn.Sequential: Support for indexing (#3790)
nn.Sequential: Support for OrderedDict as input (#4075)
datasets.ZINC: Added an in-depth description of the task (#3832), thanks to @gasteigerjo
datasets.FakeDataset: Support for different feature distributions across different labels (#4065), thanks to @arunppsg
datasets.FakeDataset: Support for custom global attributes (#4074), thanks to @arunppsg
transforms.NormalizeFeatures: Features will no longer be transformed in-place (ada5b9a)
transforms.NormalizeFeatures: Support for negative feature values (6008e30)
utils.is_undirected: Improved efficiency (#3789)
utils.dropout_adj: Improved efficiency (#4059)
utils.contains_isolated_nodes: Improved efficiency (970de13)
utils.to_networkx: Support for to_undirected options (upper triangle vs. lower triangle) (#3901, #3948), thanks to @RemyLau
graphgym: Support for custom metrics and loggers (#3494), thanks to @RemyLau
graphgym.register: Register operations can now be used as class decorators (#3779, #3782)
Documentation: Added a few exercises at the end of documentation tutorials (#3780), thanks to @PabloAMC
Documentation: Added better installation instructions to CONTRIBUTUNG.md (#3803, #3991, #3995), thanks to @Cho-Geonwoo and @RBendias and @RodrigoVillatoro
Refactor: Clean-up dependencies (#3908, #4133, #4172), thanks to @adelizer
CI: Improved test runtimes (#4241)
CI: Additional linting check via yamllint (#3886)
CI: Additional linting check via isort (66b1780), thanks to @mananshah99
torch.package: Model packaging via torch.package (#3997)

Bugfixes

data.HeteroData: Fixed a bug in data.{attr_name}_dict in case data.{attr_name} does not exist (#3897)
data.Data: Fixed data.is_edge_attr in case data.num_edges == 1 (#3880)
data.Batch: Fixed a device mismatch bug in case a batch object was indexed that was created from GPU tensors (e6aa4c9, c549b3b)
data.InMemoryDataset: Fixed a bug in which copy did not respect the underlying slice (d478dcb, #4223)
nn.conv.MessagePassing: Fixed message passing with zero nodes/edges (#4222)
nn.conv.MessagePassing: Fixed bipartite message passing with flow="target_to_source" (#3907)
nn.conv.GeneralConv: Fixed an issue in case skip_linear=False and in_channels=out_channels (#3751), thanks to @danielegrattarola
nn.to_hetero: Fixed model transformation in case node type names or edge type names contain whitespaces or dashes (#3882, b63a660)
nn.dense.Linear: Fixed a bug in lazy initialization for PyTorch < 1.8.0 (973d17d, #4086)
nn.norm.LayerNorm: Fixed a bug in the shape of weights and biases (#4030), thanks to @marshka
nn.pool: Fixed torch.jit.script support for torch-cluster functions (#4047)
datasets.TOSCA: Fixed a bug in which indices of faces started at 1 rather than 0 (8c282a0), thanks to @JRowbottomGit
datasets.WikiCS: Fixed WikiCS to be undirected by default (#3796), thanks to @pmernyei
Resolved inconsistency between utils.contains_isolated_nodes and data.has_isolated_nodes (#4138)
graphgym: Fixed the loss function regarding multi-label classification (#4206), thanks to @RemyLau
Documentation: Fixed typos, grammar and bugs (#3840, #3874, #3875, #4149), thanks to @itamblyn and @chrisyeh96 and @finquick

- Python
Published by rusty1s almost 4 years ago

pytorch_geometric - 2.0.3

PyG 2.0.3 🎉

A new minor PyG version release, including a variety of new features and bugfixes:

Features

GLNN: Graph-less Neural Networks Example
LINKX: Large Scale Learning on Non-Homophilous Graphs Example
Added an example for heterogeneous link classification (#3350) - thanks to @anniekmyatt
HANConv: The Heterogenous Graph Attention operator Example - thanks to @rishubhkhurana and @wsad1
LGConv and LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation (#3685) - thanks to @LukasHaas and @KathyFeiyang
PyTorch Lightning DataModule wrappers for PyG+PL multi-GPU training/inference without replicating datasets across processes :
- torch_geometric.data.LightningDataset for multi-GPU training via PL on graph-level tasks Example
- torch_geometric.data.LightningNodeData for multi-GPU training via PL on node-level tasks Example
NeighborLoader: Added CUDA support leading to major runtime improvements Example
MessagePassing: Added the edge_updater/edge_update interface for updating edge features (#3450) - thanks to @Padarn
GNNExplainer: Added an example that reproduces the official BA-Shapes experiment (#3386) - thanks to @RBendias
torch_geometric.graphgym: Support for heterogeneous graphs and lazy initialization (#3460) - thanks to @JiaxuanYou
MLP: Added a basic MLP implementation (#3553)
PointTransformer: Classification and segmentation examples (#3344) - thanks to @QuanticDisaster and @wsad1
ShaDowKHopSampler: Added an example (#3411) - thanks to @SubhajitDuttaChowdhury
Data.subgraph(...) implementation (#3521)

Datasets

HGBDataset benchmark suite (#3454)
MalNetTiny dataset (#3472) - thanks to @rampasek
OMDB: Organic Materials Database (#3506)
BAShapes: The BA-Shapes dataset (#3386) - thanks to @RBendias
PolBlogs and EmailEUCore datasets (#3534) - thanks to @AlexDuvalinho
StochasticBlockModel and RandomPartition graph datasets (#3586) - thanks to @dongkwan-kim
LINKXDataset: A subset of the non-homophilous benchmark datasets from LINKX
FakeDataset and FakeHeteroDataset for testing purposes (#3741) - thanks to @levulinh

Minor Changes

torch_geometric.nn.norm: Improved the runtimes of normalization layers - thanks to @johnpeterflynn
DataLoader and NeighborLoader: Output tensors are now written to shared memory to avoid an extra copy in case num_workers > 0 (#3401 and #3734) - thanks to @johnpeterflynn
GATv2Conv: Support for edge features (#3421) - thanks to @Kenneth-Schroeder
Batch.from_data_list: Runtime improvements
TransformerConv: Runtime and memory consumption improvements (#3392) - thanks to @wsad1
mean_iou: Added IoU computation via omitting NaNs (#3464) - thanks to @GericoVi
DataLoader: follow_batch and exclude_keys are now optional arguments
Improvements to the package metadata (#3445) - thanks to @cthoyt
Updated the quick start widget to support PyTorch 1.10 (#3474) - thanks to @kathyfan
NeighborLoader and HGTLoader: Removed the persistent_workers=True default
voxel_grid: The batch argument is now optional (#3533) - thanks to @QuanticDisaster
TransformerConv: JIT support (#3538) - thanks to @RobMcH
Lazy modules can now correctly be saved and loaded via state_dict() and load_state_dict() (#3651) - thanks to @shubham-gupta-iitr
from_networkx: Support for nx.MultiDiGraph (#3646) - thanks to @max-zipfl-fzi
GATv2Conv: Support for lazy initialization (#3678) - thanks to @richcmwang
torch_geometric.graphgym: register_* functions can now be used as decorators (#3684)
AddSelfLoops: Now supports the full argument set of torch_geometric.utils.add_self_loops (#3702) - thanks to @dongkwan-kim
Documentation: Added shape information to a variety of GNN operators, e.g., GATConv or ChebConv (#3697) - thanks to @saiden89
GATv2Conv and HEATConv: Removed unnecessary size argument in forward (#3744) - thanks to @saiden89

Bugfixes

GNNExplainer: Fixed a bug in the GCN example normalization coefficients were wrongly calculated (#3508) - thanks to @RBendias
HGTConv: Fixed a bug in the residual connection formulation - thanks to @zzhnobug
torch_geometric.grapghym: Fixed a bug in the creation of MLP (#3431) - thanks to @JiaxuanYou
torch_geometric.graphgym: Fixed a bug in the dimensionality of GeneralMultiLayer (#3456) - thanks to @JiaxuanYou
RandomLinkSplit: Fixed a bug in negative edge sampling for undirected graphs (#3440) - thanks to @panisson
add_self_loops: Fixed a bug in adding self-loops with scalar-valued weights
SchNet: Fixed a bug in which a bias vector was not correctly initialized as zero - thanks to @nec4
Batch.from_data_list: Replaced the torch.repeat_interleave call due to errors in forked processes (#3566) - thanks to @Enolerobotti
NeighborLoader: Fixed a bug in conjunction with PyTorch Lightning (#3602) - thanks to @pbielak
NeighborLoader and ToSparseTensor: Fixed a bug in case num_nodes == num_edges (#3683) - thanks to @WuliangHuang
ToUndirected: Fixed a bug in case num_nodes == 2 (#3627) - thanks to @aur3l14no
FiLMConv: Fixed a bug in the backward pass due to the usage of in-place operations - thanks to @Jokeren
GDC: Fixed a bug in case K > num_nodes - thanks to @Misterion777
LabelPropagation: Fixed a bug in the order of transformations (#3639) - thanks to @Riyer01
negative_sampling: Fixed execution for GPU input tensors - thanks to @Sticksword and @lmy86263
HeteroData: Fixed a bug in which node types were interpreted as edge types in case they were described by two characters (#3692)
FastRGCNConv: Fixed a bug in which weights were indexed on destination node index rather than source node index (#3690) - thanks to @Jokeren
WikipediaNetwork: Fixed a bug in downloading due to a change in URLs - thanks to @csbobby and @Kousaka-Honoka

- Python
Published by rusty1s about 4 years ago

pytorch_geometric - 2.0.2

A new minor version release, including further bugfixes, official PyTorch 1.10 support, as well as additional features and operators:

Features

Added video tutorials and Colabs from the PyTorch Geometric Tutorial project (thanks to @AntonioLonga)
Added the GraphMultisetTransformer operator (thanks to @JinheonBaek)
Added the PointTransformerConv operator (thanks to @QuanticDisaster)
Added the HEATConv operator (thanks to @Xiaoyu006)
Added the PNA GNN model (thanks to @RBendias)
Added the AddMetaPaths transform, which will add additional edge types to a HeteroData object based on a list of metapaths (thanks to @wsad1)
Added the Data.to_heterogeneous method to allow for the conversion from Data to HeteroData objects
Added the AttributedGraphDataset, containing a variety of attributes graphs
Added the Airports datasets
Added the structured_negative_sampling_feasible method, which checks if structured_negative_sampling is feasible (thanks to @WuliangHuang)
GATConv can now make use of multi-dimensional edge features to compute attention scores (thanks to @dongkwan-kim)
RandomNodeSplit and RandomLinkSplit now support HeteroData as input
MessagePassing inference can now be sped up via the decomposed_layers argument (thanks to @ZhouAo-ZA)
negative_sampling and batched_negative_sampling now support negative sampling in bipartite graphs
HeteroConv now supports the inclusion of arbitrary node-level or edge-level information for the underlying MessagePassing operators
GNNExplainer now supports multiple node-level masks and explaining regression problems (thanks to @gregorkrz)

Minor Changes

Data.to_homogeneous will now add node_type information to the homogeneous Data object
GINEConv now allows to transform edge features automatically in case their dimensionalities do not match (thanks to @CaypoH)
OGB_MAG will now add node_year information to paper nodes
Entities datasets do now allow the processing of HeteroData objects via the hetero=True option
Batch objects can now be batched together to form super batches
Added heterogeneous graph support for Center, Constant and LinearTransformation transformations
HeteroConv now allows to return "stacked" embeddings
The batch vector of a Batch object will now be initialized on the GPU in case other attributes are held in GPU memory

Bugfixes

Fixed the num_neighbors argument of NeighborLoader in order to specify an edge-type specific number of neighbors
Fixed the collate policy of lists of integers/strings to return nested lists
Fixed the Delaunay transformation in case the face attribute is not present in the data
Fixed the TGNMemory module to only read from the latest update (thanks to @cwh104504)
Fixed the pickle.PicklingError when Batch objects are used in a torch.multiprocessing.manager.Queue() (thanks to @RasmusOrsoe)
Fixed an issue with _parent state changing after pickling of Data objects (thanks to @zepx)
Fixed the ToUndirected transformation in case the number of edges and nodes are equal (thanks to @lmkmkrcc)
Fixed the from_networkx routine in case node-level and edge-level features share the same names
Removed the num_nodes warning when creating PairData objects
Fixed the initialization of the GeneralMultiLayer module in GraphGym (thanks to @fjulian)
Fixed custom model registration in GraphGym
Fixed a clash in the run_dir naming of GraphGym (thanks to @fjulian)
Includes a fix to prevent a GraphGym crash in case ROC-score is undefined (thanks to @fjulian)
Fixed the Batch.from_data_list routine on dataset slices (thanks to @dtortorella)
Fixed the MetaPath2Vec model in case there exists isolated nodes
Fixed torch_geometric.utils.coalesce with CUDA tensors

- Python
Published by rusty1s over 4 years ago

pytorch_geometric - 2.0.1

PyG 2.0.1

This is a minor release, bringing some emergency fixes to PyG 2.0.

Bugfixes

Fixed a bug in loader.DataLoader that raised a PicklingError for num_workers > 0 (thanks to @r-echeveste, @arglog and @RishabhPandit-00)
Fixed a bug in the creation of data.Batch objects in case customized data.Data objects expect non-default arguments (thanks to @Emiyalzn)
Fixed a bug in which SparseTensor attributes could not be batched along single dimensions (thanks to @rubenwiersma)

- Python
Published by rusty1s over 4 years ago

pytorch_geometric - 2.0.0

PyG 2.0 :tada: :tada: :tada:

PyG (PyTorch Geometric) has been moved from my own personal account rusty1s to its own organization account pyg-team to emphasize the ongoing collaboration between TU Dortmund University, Stanford University and many great external contributors. With this, we are releasing PyG 2.0, a new major release that brings sophisticated heterogeneous graph support, GraphGym integration and many other exciting features to PyG.

If you encounter any bugs in this new release, please do not hesitate to create an issue.

Heterogeneous Graph Support

We finally provide full heterogeneous graph support in PyG 2.0. See here for the accompanying tutorial.

Highlights

Heterogeneous Graph Storage: Heterogeneous graphs can now be stored in their own dedicated data.HeteroData class (thanks to @yaoyaowd): ```python from torch_geometric.data import HeteroData

data = HeteroData()

# Create two node types "paper" and "author" holding a single feature matrix: data['paper'].x = torch.randn(numpapers, numpaperfeatures) data['author'].x = torch.randn(numauthors, numauthorsfeatures)

# Create an edge type ("paper", "writtenby", "author") holding its graph connectivity: data['paper', 'writtenby', 'author'].edgeindex = ... # [2, numedges] ```

data.HeteroData behaves similar to a regular homgeneous data.Data object:

python print(data['paper'].num_nodes) print(data['paper', 'written_by', 'author'].num_edges) data = data.to('cuda')

Heterogeneous Mini-Batch Loading: Heterogeneous graphs can be converted to mini-batches for many small and single giant graphs via the loader.DataLoader and loader.NeighborLoader loaders, respectively. These loaders can now handle both homogeneous and heterogeneous graphs: ```python from torch_geometric.loader import DataLoader

loader = DataLoader(heterogeneousgraphdataset, batch_size=32, shuffle=True)

from torch_geometric.loader import NeighborLoader

loader = NeighborLoader(heterogeneousgraph, numneighbors=[30, 30], batchsize=128, inputnodes=('paper', data['paper'].train_mask), shuffle=True) ```

Heterogeneous Graph Neural Networks: Heterogeneous GNNs can now easily be created from homogeneous ones via nn.to_hetero and nn.to_hetero_with_bases. These processes take an existing GNN model and duplicate their message functions to account for different node and edge types: ```python from torchgeometric.nn import SAGEConv, tohetero

class GNN(torch.nn.Module): def init(hiddenchannels, outchannels): super().init() self.conv1 = SAGEConv((-1, -1), hiddenchannels) self.conv2 = SAGEConv((-1, -1), outchannels)

  def forward(self, x, edge_index):
      x = self.conv1(x, edge_index).relu()
      x = self.conv2(x, edge_index)
      return x

model = GNN(hiddenchannels=64, outchannels=dataset.numclasses) model = tohetero(model, data.metadata(), aggr='sum') ```

Additional Features

A heterogeneous graph tutorial describing all newly released features (thanks to @mrjel)
A variety of heterogeneous GNN examples
Support for lazy initialization of GNN operators by passing -1 to the in_channels argument (implemented via nn.dense.Linear). This allows to avoid calculating and keeping track of input tensor sizes, simplyfing the creation of heterogeneous graph models with varying feature dimensionalities across different node and edge types. Lazy initialization is supported for all existing PyG operators (thanks to @yaoyaowd): ```python from torch_geometric.nn import GATConv

conv = GATConv(-1, 64)

# We can initialize the model’s parameters by calling it once: conv(x, edge_index) ``* [nn.conv.HeteroConv](https://pytorch-geometric.readthedocs.io/en/latest/modules/nn.html#torch_geometric.nn.conv.HeteroConv): A generic wrapper for computing graph convolution on heterogeneous graphs (thanks to @RexYing) * [nn.conv.HGTConv](https://pytorch-geometric.readthedocs.io/en/latest/modules/nn.html#torch_geometric.nn.conv.HGTConv): The heterogeneous graph transformer operator from the ["Heterogeneous Graph Transformer"](https://arxiv.org/abs/2003.01332) paper * [loader.HGTLoader](https://pytorch-geometric.readthedocs.io/en/latest/modules/loader.html#torch_geometric.loader.HGTLoader): The heterogeneous graph sampler from the ["Heterogeneous Graph Transformer"](https://arxiv.org/abs/2003.01332) paper for learning on large-scale heterogeneous graphs (thanks to @chantat) * Support for heterogeneous graph transformations in [transforms.AddSelfLoops](https://pytorch-geometric.readthedocs.io/en/latest/modules/transforms.html#torch_geometric.transforms.AddSelfLoops), [transforms.ToSparseTensor](https://pytorch-geometric.readthedocs.io/en/latest/modules/transforms.html#torch_geometric.transforms.ToSparseTensor), [transforms.NormalizeFeatures](https://pytorch-geometric.readthedocs.io/en/latest/modules/transforms.html#torch_geometric.transforms.NormalizeFeatures) and [transforms.ToUndirected](https://pytorch-geometric.readthedocs.io/en/latest/modules/transforms.html#torch_geometric.transforms.ToUndirected) * New heterogeneous graph datasets: [datasets.OGBMAG`](https://pytorch-geometric.readthedocs.io/en/latest/modules/datasets.html#torchgeometric.datasets.OGB_MAG), datasets.IMDB, datasets.DBLP and datasets.LastFM * Support for converting heterogeneous graphs to "typed" homogeneous ones via data.HeteroData.to_homogeneous (thanks to @yzhao062) * A tutorial on creating a data.HeteroData object from raw *.csv files (thanks to @yaoyaowd and @mrjel) * An example to scale heterogeneous graph models via PyTorch Lightning

Managing Experiments with GraphGym

GraphGym is now officially supported in PyG 2.0 via torch_geometric.graphgym. See here for the accompanying tutorial. Overall, GraphGym is a platform for designing and evaluating Graph Neural Networks from configuration files via a highly modularized pipeline (thanks to @JiaxuanYou):

GraphGym is the perfect place to start learning about standardized GNN implementation and evaluation
GraphGym provides a simple interface to try out thousands of GNN architectures in parallel to find the best design for your specific task
GraphGym lets you easily do hyper-parameter search and visualize what design choices are better

Breaking Changes

The datasets.AMiner dataset now returns a data.HeteroData object. See here for our updated MetaPath2Vec example on AMiner.
transforms.AddTrainValTestMask has been replaced in favour of transforms.RandomNodeSplit
Since the storage layout of data.Data significantly changed in order to support heterogenous graphs, already processed datasets need to be re-processed by deleting the root/processed folder.
data.Data.__cat_dim__ and data.Data.__inc__ now expect additional input arguments: ```python def cat_dim(self, key, value, args, *kwargs): pass

def inc(self, key, value, args, *kwargs): pass ``In case you modified [cat_dim](https://pytorch-geometric.readthedocs.io/en/latest/modules/data.html#torch_geometric.data.Data.__cat_dim__) or [inc](https://pytorch-geometric.readthedocs.io/en/latest/modules/data.html#torch_geometric.data.Data.__inc__) functionality in a customized [data.Data`](https://pytorch-geometric.readthedocs.io/en/latest/modules/data.html#torch_geometric.data.Data) object, please ensure to apply the above changes.

Deprecations

nn.conv.PointConv is deprecated in favour of nn.conv.PointNetConv (thanks to @lelouedec and @QuanticDisaster)
utils.train_test_split_edges is deprecated in favour of the new transforms.RandomLinkSplit transform
All data loaders were moved from torch_geometric.data to torch_geometric.loader, e.g.: python from torch_geometric.loader import DataLoader
loader.NeighborSampler is deprecated in favour of loader.NeighborLoader in order to simplify the application of neighbor sampling and to support both neighbor sampling in homogeneous and heterogeneous graphs
Data.contains_isolated_nodes and Data.contains_self_loops are deprecated in favour of Data.has_isolated_nodes and Data.has_self_loops, respectively

Additional Features

torch-scatter and torch-sparse now support half-precision computation via torch.half, bringing half-precision support to PyG
Added a GNN cheatsheet to the documentation, which lets you more easily choose a GNN operator for your specific need
Added the transforms.RandomLinkSplit transform to easily perform a random edge-level split (thanks to @RexYing)
Added the torch_geometric.profile package which provides a variety of utility functions for benchmarking runtimes and memory consumptions of GNN models (thanks to @yzhao062)
nn.conv.MessagePassing now supports hooks for propagate, message, aggregate and update functions, e.g. via nn.conv.MessagePassing.register_propagate_forward_hook
Added the nn.conv.GeneralConv operator that can handle most GNN use-cases (e.g., w/ or w/o edge features, ...) and has enough design options to be tuned (e.g., attention, skip-connections, ...) (thanks to @JiaxuanYou)
Added the nn.models.RECT_L model for learning with completely-imbalanced labels (thanks to @Fizyhsp)
Added the Pathfinder Discovery Network Convolutional operator nn.conv.PDNConv (thanks to @benedekrozemberczki)
Added basic GNN model support as part of the nn.models package, e.g., nn.model.GCN, nn.models.GraphSAGE, nn.models.GAT and nn.models.GIN. Pre-defined models support customizing hidden feature dimensionality, number of layers, activation, normalization and jumping knowledge (thanks to @PabloAMC)
Added the datasets.MD17 datasets (thanks to @M-R-Schaefer)
Added a link-prediction example of nn.conv.RGCNConv (thanks to @moritzblum)
Added an example of nn.pool.MemPooling (thanks to @wsad1)
Added a return_attention_weights argument for nn.conv.TransformerConv (thanks to @wsad1)
Batch support for utils.homophily (thanks to @wsad1)
Added a batch_size argument to utils.to_dense_batch (thanks to @jimmiebtlr)

Minor Changes

Heavily improved loading times of import torch_geometric
nn.Sequential is now fully jittable
nn.conv.LEConv is now fully jittable (thanks to @lucagrementieri)
nn.conv.GENConv can now make use of "add", "mean" or "max" aggregations (thanks to @riskiem)
Attributes of type torch.nn.utils.rnn.PackedSequence are now correctly handled by data.Data and data.HeteroData (thanks to @WuliangHuang)
Added support for data.record_stream() in order to allow for data prefetching (thanks to @FarzanT)
Added a max_num_neighbors attribute to nn.models.SchNet and nn.models.DimeNet (thanks to @nec4)
nn.conv.MessagePassing is now jittable in case message, aggregate and update return multiple arguments (thanks to @PhilippThoelke)
utils.from_networkx now supports grouping of node-level and edge-level features (thanks to @PabloAMC)
Transforms now inherit from transforms.BaseTransform to ease type checking (thanks to @CCInc)
Added support for the deletion of data attributes via del data[key] (thanks to @Linux-cpp-lisp)

Bugfixes

The transforms.LinearTransformation transform now correctly transposes the input matrix before applying the transformation (thanks to @beneisner)
Fixed a bug in benchmark/kernel that prevented the application of DiffPool on the IMDB-BINARY dataset (thanks to @dongZheX)
Feature dimensionalities of datasets.WikipediaNetwork do now match which the official reported ones in case geom_gcn_preprocess=True (thanks to @ZhuYun97 and @GitEventhandler)
Fixed a bug in the datasets.DynamicFAUST dataset in which data.num_nodes was undefined (thanks to @koustav123)
Fixed a bug in which nn.models.GNNExplainer could not handle GNN operators that add self-loops to the graph in case self-loops were already present (thanks to @tw200464tw and @NithyaBhasker)
nn.norm.LayerNorm may no longer produce NaN gradients (thanks to @fbragman)
Fixed a bug in which it was not possible to customize networkx drawing arguments in nn.models.GNNExplainer.visualize_subgraph() (thanks to @jvansan)
transforms.RemoveIsolatedNodes now correctly removes isolated nodes in case data.num_nodes is explicitely set (thanks to @blakechi)

- Python
Published by rusty1s over 4 years ago

pytorch_geometric - 1.7.2

Datasets

The GitHub Web and ML developer dataset (thanks to @benedekrozemberczki)
The FacebookPagePage dataset (thanks to @benedekrozemberczki)
The Twitch gamer datasets (thanks to @benedekrozemberczki)
The DeezerEurope dataset (thanks to @benedekrozemberczki)
The GemsecDeezer dataset (thanks to @benedekrozemberczki)
The LastFMAsia dataset (thanks to @benedekrozemberczki)
The WikipediaNetwork datasets does now allow usage of the raw dataset as introduced in Multi-scale Attributed Node Embedding (thanks to @benedekrozemberczki)

Bugfixes

Fixed an error in DeepGCNLayer in case no normalization layer is provided (thanks to @lukasfolle)
Fixed a bug in GNNExplainer which mixed the loss computation for graph-level and node-level predictions (thanks to @panisson and @wsad1)

- Python
Published by rusty1s over 4 years ago

pytorch_geometric - 1.7.1

A minor release that brings PyTorch 1.9.0 and Python 3.9 support to PyTorch Geometric. In case you are in the process of updating to PyTorch 1.9.0, please re-install the external dependencies for PyTorch 1.9.0 as well (torch-scatter and torch-sparse).

Features

EGConv (thanks to @shyam196)
GATv2Conv (thanks to @shakedbr)
GraphNorm normalization layer
GNNExplainer now supports explaining graph-level predictions (thanks to @wsad1)
bro and gini regularization (thanks to @rhsimplex)
train_test_split_edges() and to_undirected() can now edge features (thanks to @saiden89 and @SherylHYX)
Datasets can now be accessed with np.ndarray as well (thanks to @josephenguehard)
dense_to_sparse can now handle batched adjacency matrices
numba is now an optional dependency

Datasets

The tree-structured fake news propagation UPFD dataset (thanks to @YingtongDou)
The large-scale AmazonProducts graph from the GraphSAINT paper
Added support for two more datasets in the SNAPDataset benchmark suite (thanks to @SherylHYX)

Issues

Fixed an issue in which SuperGATConv used all positive edges for computing the auxiliary loss (thanks to @anniekmyatt)
Fixed a bug in which MemPooling produced NaN gradients (thanks to @wsad1)
Fixed an issue in which the schnetpack package was required for training SchNet (thanks to @mshuaibii)
Modfied XConv to sample without replacement in case dilation > 1 (thanks to @mayur-ag)
GraphSAINTSampler can now be used in combination with PyTorch Lightning
Fixed a bug in HypergraphConv in case num_nodes > num_edges (thanks to @THinnerichs)

- Python
Published by rusty1s over 4 years ago

pytorch_geometric - 1.7.0

Major Features

Temporal Graph Network and an example utilizing graph attention, (thanks to @emalgorithm)
CorrectAndSmooth and an example on ogbn-products
PyTorch Lightning support, see here for the accompanying examples (thanks to @tchaton)
Sequential API, see here for the accompanying example
FiLMConv and an example on PPI (thanks to @ldv1)
SuperGAT and an example on Cora (thanks to @dongkwan-kim)
MemPooling (thanks to @wsad1)
PANConv (thanks to @YuGuangWang)
DiffGroupNorm (thanks to @wsad1)
ResGatedGraphConv (thanks to @ldv1)
FAConv (thanks to @wsad1)
AttentiveFP model for molecular graph learning and an example on ESOL (thanks to @thegodone)
Shadow k-hop Sampler (currently requires torch-sparse from master)

Additional Features

Inductive Deep Graph Infomax example (thanks to @harrygcoppock)
WLConv and an example of the Weisfeiler-Lehman subtree kernel (thanks to @chrsmrrs)
LabelPropagation
AddTrainValTestMask transform for creating various splitting strategies (thanks to @dongkwan-kim)
homophily measurement (thanks to @ldv1)
to_cugraph conversion

Minor Changes

More memory-efficient implementation of GCN2Conv
Improved TransformerConv with the beta argument being input and message dependent (thanks to @ldv1)
NeighborSampler now works with SparseTensor and supports an additional transform argument
Batch.from_data_list now supports batching along a new dimension via returning None in Data.__cat_dim__, see here for the accompanying tutorial (thanks to @Linux-cpp-lisp)
MetaLayer is now "jittable"
Lazy loading of torch_geometric.nn and torch_geometric.datasets, leading to faster imports (thanks to @Linux-cpp-lisp)
GNNExplainer now supports various output formats of the underlying GNN model (thanks to @wsad1)

Datasets

JODIE datasets for temporal graph learning
WordNet18RR (thanks to @minhtriet)
Reddit2
MixHopSyntheticDataset (thanks to @ldv1)
NELL

Bugfixes

Fixed SparseAdam usage in examples/metapath2vec.py (thanks to @declanmillar)
Fixed from_networkx to support empty edge lists (thanks to @shakedbr)
Fixed a numerical issue in softmax
Fixed an issue in DenseGraphConv with aggr="max" (thanks to @quqixun)
Fixed the norm computation in GraphSAINTSampler (thanks to @austintwang)
Cartesian and LocalCartesian now compute Cartesian coordinates from target to source nodes (thanks to @ldv1)

- Python
Published by rusty1s almost 5 years ago

pytorch_geometric - 1.6.3

Fixed a crucial bug in which InMemoryDatasets with the usage of pre_transform led to an error
New datasets: WikipediaNetwork and Actor
Added homophily ratio utility function: torch_geometric.utils.homophily_ratio

- Python
Published by rusty1s about 5 years ago

pytorch_geometric - 1.6.2

Features

GCN2Conv [Cora example, PPI example]
TransformerConv
New Dataset: WebKB
New Google Colab: Explaining GNN Model Predictions using Captum (thanks to @m30m)
Distributed training examples for node classification and graph classification (thanks to @maqy1995)
Node2Vec can now handle different p and q values other than 1 (torch-cluster update required)
GraphSAGE unsupervised training example (thanks to @yuanx749)
Linear GAE example (thanks to @GuillaumeSalha)

Minor improvements

The SIGN example now operates on mini-batches of nodes
Improved data loading runtime of InMemoryDatasets
NeighborSampler does now work with SparseTensor as input
ToUndirected transform in order to convert directed graphs to undirected ones
GNNExplainer does now allow for customizable edge and node feature loss reduction
aggr can now passed to any GNN based on the MessagePassing interface (thanks to @m30m)
Runtime improvements in SEAL (thanks to @muhanzhang)
Runtime improvements in torch_geometric.utils.softmax (thanks to @Book1996)
GAE.recon_loss now supports custom negative edge indices (thanks to @reshinthadithyan)
Faster spmm computation and random_walk sampling on CPU (torch-sparse and torch-cluster updates required)
DataParallel does now support the follow_batch argument
Parallel approximate PPR computation in the GDC transform (thanks to @klicperajo)
Improved documentation by providing an autosummary of all subpackages (thanks to @m30m)
Improved documentation on how edge weights are handled in various GNNs (thanks to @m30m)

Bugfixes

Fixed a bug in GATConv when computing attention coefficients in bipartite graphs
Fixed a bug in GraphSAINTSampler that led to wrong edge feature sampling
Fixed the DimeNet pretraining link
Fixed a bug in processing ego-twitter and ego-gplus of the SNAPDataset collection
Fixed a number of broken dataset URLs (ICEWS18, QM9, QM7b, MoleculeNet, Entities, PPI, Reddit, MNISTSuperpixels, ShapeNet)
Fixed a bug in which MessagePassing.jittable() tried to write to a file without permission (thanks to @twoertwein)
GCNConv does not require edge_weight in case normalize=False
Batch.num_graphs will now report the correct amount of graphs in case of zero-sized graphs

- Python
Published by rusty1s over 5 years ago

pytorch_geometric - 1.6.1

This is a minor release, mostly focusing on PyTorch 1.6.0 support. All external wheels are now also available for PyTorch 1.6.0.

New Features

WikiCS dataset
DeepGCN via GENConv and DeepGCNLayer (thanks to @lightaime)
PairNorm (thanks to @gupta-abhay)
LayerNorm (thanks to @aluo-x)

Bugfixes

Fixed a bug which prevented GNNExplainer to work with GATConv
Fixed the MessagePassing.jittable call when installing PyG via pip
Fixed a bug in torch-sparse where reduce functions with dim=0 did not yield the correct result
Fixed a bug in torch-sparse which suppressed all warnings

- Python
Published by rusty1s over 5 years ago

pytorch_geometric - 1.6.0

A new major release, introducing TorchScript support, memory-efficient aggregations, bipartite GNN modules, static graphs and much more!

Major Features

TorchScript support, see here for the accompanying tutorial (thanks to @lgray and @liaopeiyuan)
Memory-efficient aggregations via torch_sparse.SparseTensor, see here for the accompanying tutorial
Most GNN modules can now operate on bipartite graphs (and some of them can also operate on different feature dimensionalities for source and target nodes), useful for neighbor sampling or heterogeneous graphs: python conv = SAGEConv(in_channels=(32, 64), out_channels=64) out = conv((x_src, x_dst), edge_index)
Static graph support: ```python conv = GCNConv(inchannels=32, outchannels=64)

x = torch.randn(batchsize, numnodes, inchannels) out = conv(x, edgeindex) print(out.size())

torch.Size([batchsize, numnodes, out_channels]) ```

Additional Features

PNAConv (thanks to @lukecavabarrett and @gcorso)
Pre-Trained DimeNet on QM9
SEAL link prediction example (thanks to @muhanzhang)
ClusterGCNConv
Cluster-GCN PPI example (thanks to @CFF-Dream)
WeightedEdgeSampler for GraphSAINT (thanks to @KiddoZhu)
Better num_workers support for GraphSAINT
The automatic addition of self-loops can now be disabled via the add_self_loops argument, e.g., for GCNConv

Breaking Changes

Memory-efficient RGCNConv: The old RGCNConv implementation has been moved to FastRGCNConv

Complementary Frameworks

DeepSNAP: A PyTorch library that bridges between graph libraries such as NetworkX and PyTorch Geometric
PyTorch Geometric Temporal: A temporal GNN library built upon PyTorch Geometric

Datasets

GNNBenchmarkDataset suite from the Benchmarking Graph Neural Networks paper
WordNet18

Bugfixes

Fixed a bug in the VGAE KL-loss computation (thanks to @GuillaumeSalha)

- Python
Published by rusty1s over 5 years ago

pytorch_geometric - 1.5.0

This release is a big one thanks to many wonderful contributors. You guys are awesome!

Breaking Changes and Highlights

NeighborSampler got completely revamped: it's now much faster, allows for parallel sampling, and allows to easily apply skip-connections or self-loops. See examples/reddit.py or the newly introduced OGB examples (examples/ogbn_products_sage.py and examples/ogbn_products_gat.py). The latter also sets a new SOTA on the OGB leaderboards (reaching 0.7945 ± 0.0059 test accuracy)
SAGEConv now uses concat=True by default, and there is no option to disable it anymore
Node2Vec got enhanced by a parallel sampling mechanism, and as a result, its API slightly changed
MetaPath2Vec: The first model in PyG that is able to operate on heteregenous graphs
GNNExplainer: Generating explanations for graph neural networks
GraphSAINT: A graph sampling based inductive learning method
SchNet model for learning on molecular graphs, comes with pre-trained weights for each target of the QM9 dataset (thanks to @Nyuten)

Additional Features

ASAPooling: Adaptive structure aware pooling for learning hierarchical graph representations (thanks to @ekagra-ranjan)
ARGVA node clustering example, see examples/argva_node_clustering.py (thanks to @gsoosk)
MFConv: Molecular fingerprint graph convolution operator (thanks to @rhsimplex)
GIN-E-Conv that extends the GINConv to also account for edge features
DimeNet: Directional message passing for molecular graphs
SIGN: Scalable inception graph neural networks
GravNetConv (thanks to @jkiesele)

Datasets

Yelp
Flickr
AMiner (first real heterogeneous graph)

Minor changes

GATConv can now return attention weights via the return_attention_weights argument (thanks to @douglasrizzo)
InMemoryDataset now has a copy method that converts sliced datasets back into a contiguous memory layout
Planetoid got enhanced by the ability to let users choose between different splitting methods (thanks to @dongkwan-kim)
k_hop_subgraph: Computes the k-hop subgraph around a subset of nodes
geodesic_distance: Geodesic distances can now be computed in parallel (thanks to @jannessm)
tree_decomposition: The tree decompostion algorithm for generating junction trees from molecules
SortPool benchmark script now uses 1-D convolutions after pooling, leading to better performance (thanks to @muhanzhang)

Bugfixes

Fixed a bug in write_off
Fixed a bug in the processing of the GEDDataset dataset
to_networkx conversion can now also properly handle non-tensor attributes
Fixed a bug in read_obj (thanks to @mwussow)

- Python
Published by rusty1s almost 6 years ago

pytorch_geometric - 1.4.3

Features

Cluster-GCN via ClusterData and ClusterLoader for operating on large-scale graphs, see examples/cluster_gcn.py for an example on how to use
Added a tutorial about advanced mini-batching scenarios
Added a tensorboard logging example

Datasets

CitationFull: The full citation network dataset suite
SNAPDataset: A subset of graph datasets from the SNAP dataset collection
SuiteSparseMatrixCollection
TrackMLParticleTrackingDataset

Minor Changes

Added the concat argument to SAGEConv
Outsourced the train_test_split_edges method of the graph autoencoder GAE class to torch_geometric.utils

Bugfixes

Fixed SplineConv compatibility with latest torch-spline-conv package
trimesh conversion utilities do not longer result in a permutation of the input data

- Python
Published by rusty1s almost 6 years ago

pytorch_geometric - 1.4.2

Minor Changes

There are now Python wheels available for torch-scatter and torch-sparse which should make the installation procedure much more user-friendly. Simply run pip install torch-scatter==latest+${CUDA} torch-sparse==latest+${CUDA} -f https://pytorch-geometric.com/whl/torch-1.4.0.html pip install torch-geometric where ${CUDA} should be replaced by either cpu, cu92, cu100 or cu101 depending on your PyTorch installation.
torch-cluster is now an optional dependency. All methods that rely on torch-cluster will result in an error requesting you to install torch-cluster.
torch_geometric.data.Dataset can now also be indexed and shuffled: python dataset.shuffle()[:50]

Bugfixes
Fixed a bug that prevented the user from saving MessagePassing modules.
Fixed a bug in RGCNConv when using root_weight=False.

- Python
Published by rusty1s about 6 years ago

pytorch_geometric - 1.4.1

This release mainly focuses on torch-scatter=2.0 support. As a result, PyTorch Geometric now requires PyTorch 1.4. If you are in the process of updating to PyTorch 1.4, please ensure that you also re-install all related external packages.

Features

Graph Diffusion Convolution
MinCUT Pooling
CGCNNConv
TUDataset cleaned versions, containing only non-isomorphic graphs
GridSampling transform
ShapeNet dataset now comes with normals and better split options
TriMesh conversion utilities
ToSLIC transform for superpixel generation from images
Re-writing of MessagePassing interface with custom aggregate methods (no API changes)

Bugfixes

Fixed some failure modes of from_networkx.

- Python
Published by rusty1s about 6 years ago

pytorch_geometric - 1.3.2

This release focuses on Pytorch 1.2 support and removes all torch.bool deprecation warnings. As a result, this release now requires PyTorch 1.2. If you are in the process of updating to PyTorch 1.2, please ensure that you also re-install all related external packages.

Overall, this release brings the following new features/bugfixes:

Features

Prints out a warning in case the pre_transform and pre_filter arguments differ from an already processed version

Bugfixes

Removed all torch.bool deprecation warnings
Fixed ARGA initialization bug
Fixed a pre-processing bug in QM9

- Python
Published by rusty1s over 6 years ago

pytorch_geometric - 1.3.1

This is a minor release which is mostly distributed for official PyTorch 1.2 support. In addition, it provides minor bugfixes and the following new features:

Modules

Non-normalized ChebConv in combination with a largest eigenvalue transform
TAGCN
Graph U-Net
Node2Vec
EdgePooling
Alternative GMMConv formulation with separate kernels
Alternative Top-K pooling formulation based on thresholds with examples on synthetic COLORS and TRIANGLES datasets

Datasets

Pascal VOC 2011 with Berkeley keypoint annotations (PascalVOCKeypoints)
DBP15K dataset
WILLOWObjectClass dataset

Please also update related external packages via, e.g.: $ pip install --upgrade torch-cluster

- Python
Published by rusty1s over 6 years ago

pytorch_geometric - 1.3.0

Support for giant graph handling using NeighborSampler and bipartite message passing operators
Debugging support using the new debug API
Fixed TUDataset download errors
Added FeasStConv module
Improved networkx conversion functionality
Improved Data and DataLoader handling with customizable number_of_nodes (e.g. for holding two graphs in a single Data object)
Added GeniePath example
Added SAGPool module
Added geodesic distance computation using gdist (optional)
Improved PointNet and DGCNN classification and segmentation examples
Added subgraph functionality
Fixed GMMConv
Added a bunch of new datasets
Added fast implementations for random graph generation
Improved loop API
Minor bugfixes

Thanks to all contributors!

- Python
Published by rusty1s over 6 years ago