GraphScope GraphScope

GraphScope Python GRAPEMaxGraphGraph-Learn Vineyard Gremlin GraphScope

GraphScope SIGMOD2017 VLDB2017 VLDB2020 SAILGraphScope NSDI 2021 GraphScope TODSSIGMODVLDBKDD

graphscope.io

JupyterLab Playground GraphScope

GraphScope Kubernetes (k8s)

pip GraphScope

bash pip3 install graphscope

graphscope Python >= 3.7 pip >= 19.0.

GraphScope Linux (Ubuntu 20.04+ / Centos 7+) macOS 12+ (Intel/Apple silicon) Windows WSL2 Ubuntu GraphScope

GraphScope

:

ogbn-mag Microsoft Academic Graph4

ogbn-mag 128 word2vec

GraphScope property graphlabelproperty ogbn-mag

GraphScope

```python import graphscope from graphscope.dataset import loadogbnmag

g = loadogbnmag() ```

ogb snap . here. .

GraphScope Gremlin

ID 2 4307

```python

get the endpoint for submitting Gremlin queries on graph g.

interactive = graphscope.gremlin(g)

count the number of papers two authors (with id 2 and 4307) have co-authored

papers = interactive.execute("g.V().has('author', 'id', 2).out('writes').where(__.in('writes').has('id', 4307)).count()").one() ```

GraphScope

Gremlin k-core

```python

extract a subgraph of publication within a time range

sub_graph = interactive.subgraph("g.V().has('year', gte(2014).and(lte(2020))).outE('cites')")

project the projected graph to simple graph.

simpleg = subgraph.project(vertices={"paper": []}, edges={"cites": []})

ret1 = graphscope.kcore(simpleg, k=5) ret2 = graphscope.triangles(simple_g)

add the results as new columns to the citation graph

subgraph = subgraph.addcolumn(ret1, {"kcore": "r"}) subgraph = subgraph.addcolumn(ret2, {"tc": "r"}) ```

GraphScope GraphScope Pregel PIE

(GNNs)

GNN

GCN 349

```python

define the features for learning

paperfeatures = [f"feat{i}" for i in range(128)]

paperfeatures.append("kcore") paperfeatures.append("tc")

launch a learning engine.

lg = graphscope.graphlearn(subgraph, nodes=[("paper", paperfeatures)], edges=[("paper", "cites", "paper")], gen_labels=[ ("train", "paper", 100, (0, 75)), ("val", "paper", 100, (75, 85)), ("test", "paper", 100, (85, 100)) ]) ```

```python

Note: Here we use tensorflow as NN backend to train GNN model. so please

install tensorflow.

try: # https://www.tensorflow.org/guide/migrate import tensorflow.compat.v1 as tf tf.disablev2behavior() except ImportError: import tensorflow as tf

import graphscope.learning from graphscope.learning.examples import EgoGraphSAGE from graphscope.learning.examples import EgoSAGESupervisedDataLoader from graphscope.learning.examples.tf.trainer import LocalTrainer

supervised GCN.

def traingcn(graph, nodetype, edgetype, classnum, featuresnum, hopsnum=2, nbrsnum=[25, 10], epochs=2, hiddendim=256, indroprate=0.5, learningrate=0.01, ): graphscope.learning.resetdefaulttfgraph()

dimensions = [features_num] + [hidden_dim] * (hops_num - 1) + [class_num]
model = EgoGraphSAGE(dimensions, act_func=tf.nn.relu, dropout=in_drop_rate)

# prepare train dataset
train_data = EgoSAGESupervisedDataLoader(
    graph, graphscope.learning.Mask.TRAIN,
    node_type=node_type, edge_type=edge_type, nbrs_num=nbrs_num, hops_num=hops_num,
)
train_embedding = model.forward(train_data.src_ego)
train_labels = train_data.src_ego.src.labels
loss = tf.reduce_mean(
    tf.nn.sparse_softmax_cross_entropy_with_logits(
        labels=train_labels, logits=train_embedding,
    )
)
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)

# prepare test dataset
test_data = EgoSAGESupervisedDataLoader(
    graph, graphscope.learning.Mask.TEST,
    node_type=node_type, edge_type=edge_type, nbrs_num=nbrs_num, hops_num=hops_num,
)
test_embedding = model.forward(test_data.src_ego)
test_labels = test_data.src_ego.src.labels
test_indices = tf.math.argmax(test_embedding, 1, output_type=tf.int32)
test_acc = tf.div(
    tf.reduce_sum(tf.cast(tf.math.equal(test_indices, test_labels), tf.float32)),
    tf.cast(tf.shape(test_labels)[0], tf.float32),
)

# train and test
trainer = LocalTrainer()
trainer.train(train_data.iterator, loss, optimizer, epochs=epochs)
trainer.test(test_data.iterator, test_acc)

traingcn(lg, nodetype="paper", edgetype="cites", classnum=349, # output dimension features_num=130, # input dimension, 128 + kcore + triangle count ) ```

nodeclassificationon_citation.ipynb

Kubernetes

GraphScope Vineyard GraphScope Kubernetes

k8s k8s API ~/.kube/config

Kind k8s

```bash

for usage, type -h

./scripts/install_deps.sh --k8s ```

graphscope

bash pip3 install graphscope-client

Python GraphScope

• * 1*. session GraphScope
• * 2 - 5*. ( graphscope.gremlin graphscope.graphlearn sess.gremlin sess.graphlearnsess Session )
• * 6*.

GraphScope Python session

with_dataset Pod /dataset

```python import graphscope

sess = graphscope.session(with_dataset=True) ```

macOS LoadBalancer NodePort

python sess = graphscope.session(with_dataset=True, k8s_service_type="LoadBalancer")

coordinator coordinator k8s pods coordinator pod vineyard

```python from graphscope.dataset import loadogbnmag

/dataset

ogbnmagsmall

container /dataset

g = loadogbnmag(sess, "/dataset/ogbnmagsmall") ```

g vineyard k8s pods

Gremlin

nodeclassificationon_citation.ipynb

Coordinator

python sess.close()

graphscope Python

```bash python3 gsctl.py install-deps dev

--cn

python3 gsctl.py install-deps dev --cn ```

make GraphScope

```bash

Python

sudo make install

make interactive

make analytical

make learning

```

Docker

GraphScope Dockerfile Docker GraphScope k8s/internal

```bash

tag graphscope/graphscope:SHORTSHA

cd k8s

make graphscope ```

Python

GraphScope Python GraphScope docker Python protobuf GraphScope docker Python

bash make client

C/C++ Python Python Wheels.

bash make test

GraphScope Sphinx

```bash

build the docs

make graphscope-docs

to open preview on local

open docs/_build/html/index.html ``` https://graphscope.io/docs

GraphScope Apache License 2.0

• Wenfei Fan, Tao He, Longbin Lai, Xue Li, Yong Li, Zhao Li, Zhengping Qian, Chao Tian, Lei Wang, Jingbo Xu, Youyang Yao, Qiang Yin, Wenyuan Yu, Jingren Zhou, Diwen Zhu, Rong Zhu. GraphScope: A Unified Engine For Big Graph Processing. The 47th International Conference on Very Large Data Bases (VLDB), industry, 2021.
• Jingbo Xu, Zhanning Bai, Wenfei Fan, Longbin Lai, Xue Li, Zhao Li, Zhengping Qian, Lei Wang, Yanyan Wang, Wenyuan Yu, Jingren Zhou. GraphScope: A One-Stop Large Graph Processing System. The 47th International Conference on Very Large Data Bases (VLDB), demo, 2021

• Slack channel
• GitHub issue bug
• Pull requests