AAAI 2022-Knowledge Graph Alignment Network with Gated Multi-hop Neighborhood Aggregation.pdf

波风水门

193

8页

5次

2023-05-31

免费下载

The Thirty-Fourth AAAI Conference on Artiﬁcial Intelligence (AAAI-20)

Knowledge Graph Alignment Network

with Gated Multi-Hop Neighborhood Aggregation

Zequn Sun,

1∗

Chengming Wang,

1∗

Wei Hu,

1†

Muhao Chen,

Jian Dai,

Wei Zhang,

Yuzhong Qu

State Key Laboratory for Novel Software Technology, Nanjing University, China

Department of Computer Science, University of California, Los Angeles, USA

Alibaba Group, China

{zqsun.nju, cmwang.nju}@gmail.com, {whu, yzqu}@nju.edu.cn, muhaochen@ucla.edu, {yiding.dj, lantu.zw}@alibaba-inc.com

Abstract

Graph neural networks (GNNs) have emerged as a powerful

paradigm for embedding-based entity alignment due to their

capability of identifying isomorphic subgraphs. However, in

real knowledge graphs (KGs), the counterpart entities usually

have non-isomorphic neighborhood structures, which easily

causes GNNs to yield different representations for them. To

tackle this problem, we propose a new KG alignment network,

namely AliNet, aiming at mitigating the non-isomorphism of

neighborhood structures in an end-to-end manner. As the direct

neighbors of counterpart entities are usually dissimilar due to

the schema heterogeneity, AliNet introduces distant neighbors

to expand the overlap between their neighborhood structures.

It employs an attention mechanism to highlight helpful distant

neighbors and reduce noises. Then, it controls the aggregation

of both direct and distant neighborhood information using a

gating mechanism. We further propose a relation loss to reﬁne

entity representations. We perform thorough experiments with

detailed ablation studies and analyses on ﬁve entity alignment

datasets, demonstrating the effectiveness of AliNet.

1 Introduction

Entity alignment is the task of ﬁnding entities from different

knowledge graphs (KGs) that refer to the same real-world

identity. Recently, increasing attention has been paid to the

utilization of KG representation learning rather than symbolic

formalism for tackling this task. Representation learning mod-

els encode KGs into vector spaces, where relation semantics

of entities can be assessed by the learned embedding opera-

tions, such as the relation-speciﬁc translation (Bordes et al

2013) or rotation (Sun et al

2019). For embedding-based

entity alignment, the similarity of entities is measured by

the distance of entity embeddings. It has shown great poten-

tials in dealing with the symbolic heterogeneity problem and

beneﬁts the entity alignment task in both monolingual and

cross-lingual scenarios (Zhu et al. 2017; Chen et al. 2017).

Most recently, graph neural networks (GNNs) (Kipf and

Welling 2017; Velickovic et al

2018; Abu-El-Haija et al

∗

This work was done while Zequn Sun was a research intern at

Alibaba Group. The ﬁrst two authors contributed equally.

†

Corresponding author



2020, Association for the Advancement of Artiﬁcial

birthplace

draft team

high school

career position

Kobe Bryant

Shooting guard

Lower Merion High School

Charlotte Hornets

country

Philadelphia

United States of America

place of birth

member of sports team

country of citizenship

father

drafted by

educated at

Kobe Bryant

Lower Merion

High School

Charlotte Hornets

Joe Bryant

United States

of America

Los Angeles

Lakers

Philadelphia

Figure 1: Non-isomorphic relational neighborhood of Kobe

Bryant in DBpedia (left) and Wikidata (right), respectively.

2019) have emerged as a powerful model to learn vector

representations for graph-structured data. In GNNs, the rep-

resentation of a node is learned by recursively aggregating

the representations of its neighboring nodes. A recent work

(Morris et al

2019) has proved that GNNs have the same

expressiveness as the Weisfeiler-Leman (WL) test (Weisfeiler

and Lehman 1968) in terms of identifying isomorphic sub-

graphs. It provides the theory basis of using GNNs for entity

alignment between different KGs as similar entities usually

have similar neighborhood. Recently, several studies (Wang

et al

2018; Xu et al

2019b; Cao et al

2019; Wu et al

2019;

Ye et al

2019) have exploited GNNs for embedding-based

entity alignment, and have achieved promising results.

However, existing GNN-based entity alignment models

still face a critical problem. As different KGs usually have

heterogeneous schemas and data incompleteness (Pujara et

2013), the counterpart entities usually have dissimilar

neighborhood structures. Figure 1 gives an example. The

neighborhood of the two entities referring to Kobe Bryant

is inconsistent to each other, especially containing different

sets of neighboring entities. The statistics on DBpedia-based

benchmark datasets for entity alignment (Sun, Hu, and Li

2017) also show that the majority of aligned entity pairs

have different neighboring entities. Particularly, the percent-

ages of such entity pairs reach

89.97%

between Chinese-

English,

86.19%

between Japanese-English and

90.71%

be-

tween French-English, respectively. Different neighborhood

structures would easily cause a GNN to yield different repre-

sentations for counterpart entities.

The challenge of resolving this issue lies in the difﬁculty

222

of fully mitigating the non-isomorphism in the neighborhood

structures of counterpart entities from different KGs. Even

though we assume that the two KGs are complete (the goal of

MuGNN (Cao et al

2019)), due to the schema heterogeneity,

the counterpart entities still inevitably have dissimilar neigh-

borhood structures. For example, in Figure 1, United States

of America is among the one-hop (direct) neighbors of Kobe

Bryant in Wikidata. However in DBpedia, it is a two-hop

neighbor. Motivated by the fact that the semantically-related

information can appear in both direct and distant neighbors of

counterpart entities, we propose the KG alignment network

AliNet which aggregates both direct and distant neighbor-

hood information. Speciﬁcally, each AliNet layer has mul-

tiple functions to aggregate the neighborhood information

within multiple hops. To reduce noise information, we further

employ an attention mechanism for the distant neighborhood

aggregation to ﬁnd out important neighbors in an end-to-end

manner. Finally, we use the gating mechanism to combine the

output representations of the multiple aggregation functions,

obtaining the hidden representations in the current layer. We

also design a relation loss to reﬁne entity representations and

enable AliNet to capture some special structures such as the

triangular relational structure. We perform thorough exper-

iments with detailed ablation studies and analyses on ﬁve

entity alignment datasets, demonstrating the effectiveness of

AliNet and each of its technical contributions.

2 Preliminaries

2.1 GNNs

In GNNs, the representation of a node is learned by recur-

sively aggregating the feature vectors of its neighbors. Differ-

ent aggregation strategies lead to different variants of GNNs.

GCN

A very popular variant of GNNs is the vanilla GCN

(Kipf and Welling 2017). The hidden representation of node

i at the l-th layer (l ≥ 1), denoted as h

(l)

, is computed by:

(l)

= σ





j∈N

(i)∪{i}

(l)

(l−1)



, (1)

where

(·)

represents the set of one-hop neighbors of the

given entity,

(l)

is the weight matrix of the

-th layer and

is the normalization constant.

σ(·)

is an activation function.

The vanilla GCN encodes a node as the mean pooling of the

representations of its neighbors and itself from the last layer.

The input vector fed to the ﬁrst layer is denoted as h

(0)

R-GCN

Conventional GNNs only consider the node-wise

connectivity in a graph and ignore edge labels such as the

relations in KGs. R-GCN (Schlichtkrull et al

2018) addresses

this issue by distinguishing different neighbors with relation-

speciﬁc weight matrices. It computes h

(l)

as follows:

(l)

= σ



(l)

(l−1)



r∈R



j∈N

(i)

i,r

(l)

(l−1)



, (2)

where

(l)

is the weight matrix for the node itself and

(l)

is used speciﬁcally for the neighbors having relation

, i.e.,

(i). R is the relation set and c

i,r

is for normalization.

2.2 Entity Alignment of KGs

We formally represent a KG as

G =(E, R, T )

, where

the set of entities, R is the set of relations, and T = E×R

×E

is the set of triples. Without loss of generality, we con-

sider the entity alignment task between two KGs, i.e.,

, R

, T

)

and

=(E

, R

, T

)

. Given partial pre-

aligned entity pairs

= {(i, j) ∈E

×E

|i ≡ j}

where

≡

means the alignment relationship, the goal of the task is to

ﬁnd alignment of remaining entities via entity embeddings.

2.3 GNNs for Entity Alignment

Recent GNN-based entity alignment models include GCN-

Align (Wang et al

2018), GMNN (Xu et al

2019b), MuGNN

(Cao et al

2019), RDGCN (Wu et al

2019) and AVR-GCN

(Yeetal

2019). GCN-Align and GMNN are built based on

the vanilla GCN. RDGCN introduces dual relation graphs to

enhance the vanilla GCN. AVR-GCN extends R-GCN using

a TransE-like relation-speciﬁc translation operation (Bordes

et al

2013). Before aggregation, each entity representation is

translated from its tail entity representations using relation

vectors. We argue that such relation-speciﬁc translation and

R-GCN introduce a high complexity with the overhead of

trainable parameters. More importantly, the aforementioned

models do not take the non-isomorphism in KG structures

into consideration. While MuGNN (Cao et al

2019) notices

the structure incompleteness of KGs and proposes a two-

step method of rule-based KG completion and multi-channel

GNNs for entity alignment. However, the learned rules rely

on relation alignment to resolve schema heterogeneity.

Isomorphic structures are beneﬁcial

GNNs would learn

the same representation for the entities that have isomorphic

neighborhood structures with identical feature vectors repre-

senting corresponding neighbors (Xu et al

2019a). We show

that, in some cases, if two entities have isomorphic neigh-

borhood structures and only partially pre-aligned neighbor

representations, GNNs can also capture the similarity of other

neighbors to be aligned. Figure 2 (i) gives an example. For

simplicity, here we consider a single-layer GCN. We can let

pre-aligned entities have the same representation by minimiz-

ing their Euclidean distance, i.e.,

(0)

= h

(0)



(0)

= h

(0)



and

(0)

= h

(0)



as well as

(1)

= h

(1)



(1)

= h

(1)



and

(1)

= h

(1)



in the ideal condition. By the mean-pooling

based aggregation, we have

(1)

= σ(W

(1)

(0)

+ h

(0)

)/3)

and

(1)



= σ(W

(1)

(0)



+ h

(0)



+ h

(0)



)/3)

, yield-

ing

(0)

= h

(0)



. Finally, the counterpart entities would have

the same representation. This indicates that the alignment

information between entities can be propagated across the

different GNN layers and different isomorphic graphs given

partially pre-aligned neighborhood. However, for entity align-

ment between different KGs, it is impossible to require the

two KGs to have isomorphic structures due to the schema het-

erogeneity. Figure 2 (ii) gives an example of non-isomorphic

graph structures, where

and



would have different repre-

sentations due to their different neighborhood structures.

Only structures are not enough

Conventional GNNs fall

short of characterizing some special subgraph structures such

223

The Thirty-Fourth AAAI Conference on Artiﬁcial Intelligence (AAAI-20)

Knowledge Graph Alignment Network

with Gated Multi-Hop Neighborhood Aggregation

Zequn Sun,

1∗

Chengming Wang,

1∗

Wei Hu,

1†

Muhao Chen,

Jian Dai,

Wei Zhang,

Yuzhong Qu

State Key Laboratory for Novel Software Technology, Nanjing University, China

Department of Computer Science, University of California, Los Angeles, USA

Alibaba Group, China

{zqsun.nju, cmwang.nju}@gmail.com, {whu, yzqu}@nju.edu.cn, muhaochen@ucla.edu, {yiding.dj, lantu.zw}@alibaba-inc.com

Abstract

Graph neural networks (GNNs) have emerged as a powerful

paradigm for embedding-based entity alignment due to their

capability of identifying isomorphic subgraphs. However, in

real knowledge graphs (KGs), the counterpart entities usually

have non-isomorphic neighborhood structures, which easily

causes GNNs to yield different representations for them. To

tackle this problem, we propose a new KG alignment network,

namely AliNet, aiming at mitigating the non-isomorphism of

neighborhood structures in an end-to-end manner. As the direct

neighbors of counterpart entities are usually dissimilar due to

the schema heterogeneity, AliNet introduces distant neighbors

to expand the overlap between their neighborhood structures.

It employs an attention mechanism to highlight helpful distant

neighbors and reduce noises. Then, it controls the aggregation

of both direct and distant neighborhood information using a

gating mechanism. We further propose a relation loss to reﬁne

entity representations. We perform thorough experiments with

detailed ablation studies and analyses on ﬁve entity alignment

datasets, demonstrating the effectiveness of AliNet.

1 Introduction

Entity alignment is the task of ﬁnding entities from different

knowledge graphs (KGs) that refer to the same real-world

identity. Recently, increasing attention has been paid to the

utilization of KG representation learning rather than symbolic

formalism for tackling this task. Representation learning mod-

els encode KGs into vector spaces, where relation semantics

of entities can be assessed by the learned embedding opera-

tions, such as the relation-speciﬁc translation (Bordes et al

2013) or rotation (Sun et al

2019). For embedding-based

entity alignment, the similarity of entities is measured by

the distance of entity embeddings. It has shown great poten-

tials in dealing with the symbolic heterogeneity problem and

beneﬁts the entity alignment task in both monolingual and

cross-lingual scenarios (Zhu et al. 2017; Chen et al. 2017).

Most recently, graph neural networks (GNNs) (Kipf and

Welling 2017; Velickovic et al

2018; Abu-El-Haija et al

∗

This work was done while Zequn Sun was a research intern at

Alibaba Group. The ﬁrst two authors contributed equally.

†

Corresponding author



2020, Association for the Advancement of Artiﬁcial

birthplace

draft team

high school

career position

Kobe Bryant

Shooting guard

Lower Merion High School

Charlotte Hornets

country

Philadelphia

United States of America

place of birth

member of sports team

country of citizenship

father

drafted by

educated at

Kobe Bryant

Lower Merion

High School

Charlotte Hornets

Joe Bryant

United States

of America

Los Angeles

Lakers

Philadelphia

Figure 1: Non-isomorphic relational neighborhood of Kobe

Bryant in DBpedia (left) and Wikidata (right), respectively.

2019) have emerged as a powerful model to learn vector

representations for graph-structured data. In GNNs, the rep-

resentation of a node is learned by recursively aggregating

the representations of its neighboring nodes. A recent work

(Morris et al

2019) has proved that GNNs have the same

expressiveness as the Weisfeiler-Leman (WL) test (Weisfeiler

and Lehman 1968) in terms of identifying isomorphic sub-

graphs. It provides the theory basis of using GNNs for entity

alignment between different KGs as similar entities usually

have similar neighborhood. Recently, several studies (Wang

et al

2018; Xu et al

2019b; Cao et al

2019; Wu et al

2019;

Ye et al

2019) have exploited GNNs for embedding-based

entity alignment, and have achieved promising results.

However, existing GNN-based entity alignment models

still face a critical problem. As different KGs usually have

heterogeneous schemas and data incompleteness (Pujara et

2013), the counterpart entities usually have dissimilar

neighborhood structures. Figure 1 gives an example. The

neighborhood of the two entities referring to Kobe Bryant

is inconsistent to each other, especially containing different

sets of neighboring entities. The statistics on DBpedia-based

benchmark datasets for entity alignment (Sun, Hu, and Li

2017) also show that the majority of aligned entity pairs

have different neighboring entities. Particularly, the percent-

ages of such entity pairs reach

89.97%

between Chinese-

English,

86.19%

between Japanese-English and

90.71%

be-

tween French-English, respectively. Different neighborhood

structures would easily cause a GNN to yield different repre-

sentations for counterpart entities.

The challenge of resolving this issue lies in the difﬁculty

222

of fully mitigating the non-isomorphism in the neighborhood

structures of counterpart entities from different KGs. Even

though we assume that the two KGs are complete (the goal of

MuGNN (Cao et al

2019)), due to the schema heterogeneity,

the counterpart entities still inevitably have dissimilar neigh-

borhood structures. For example, in Figure 1, United States

of America is among the one-hop (direct) neighbors of Kobe

Bryant in Wikidata. However in DBpedia, it is a two-hop

neighbor. Motivated by the fact that the semantically-related

information can appear in both direct and distant neighbors of

counterpart entities, we propose the KG alignment network

AliNet which aggregates both direct and distant neighbor-

hood information. Speciﬁcally, each AliNet layer has mul-

tiple functions to aggregate the neighborhood information

within multiple hops. To reduce noise information, we further

employ an attention mechanism for the distant neighborhood

aggregation to ﬁnd out important neighbors in an end-to-end

manner. Finally, we use the gating mechanism to combine the

output representations of the multiple aggregation functions,

obtaining the hidden representations in the current layer. We

also design a relation loss to reﬁne entity representations and

enable AliNet to capture some special structures such as the

triangular relational structure. We perform thorough exper-

iments with detailed ablation studies and analyses on ﬁve

entity alignment datasets, demonstrating the effectiveness of

AliNet and each of its technical contributions.

2 Preliminaries

2.1 GNNs

In GNNs, the representation of a node is learned by recur-

sively aggregating the feature vectors of its neighbors. Differ-

ent aggregation strategies lead to different variants of GNNs.

GCN

A very popular variant of GNNs is the vanilla GCN

(Kipf and Welling 2017). The hidden representation of node

i at the l-th layer (l ≥ 1), denoted as h

(l)

, is computed by:

(l)

= σ





j∈N

(i)∪{i}

(l)

(l−1)



, (1)

where

(·)

represents the set of one-hop neighbors of the

given entity,

(l)

is the weight matrix of the

-th layer and

is the normalization constant.

σ(·)

is an activation function.

The vanilla GCN encodes a node as the mean pooling of the

representations of its neighbors and itself from the last layer.

The input vector fed to the ﬁrst layer is denoted as h

(0)

R-GCN

Conventional GNNs only consider the node-wise

connectivity in a graph and ignore edge labels such as the

relations in KGs. R-GCN (Schlichtkrull et al

2018) addresses

this issue by distinguishing different neighbors with relation-

speciﬁc weight matrices. It computes h

(l)

as follows:

(l)

= σ



(l)

(l−1)



r∈R



j∈N

(i)

i,r

(l)

(l−1)



, (2)

where

(l)

is the weight matrix for the node itself and

(l)

is used speciﬁcally for the neighbors having relation

, i.e.,

(i). R is the relation set and c

i,r

is for normalization.

2.2 Entity Alignment of KGs

We formally represent a KG as

G =(E, R, T )

, where

the set of entities, R is the set of relations, and T = E×R

×E

is the set of triples. Without loss of generality, we con-

sider the entity alignment task between two KGs, i.e.,

, R

, T

)

and

=(E

, R

, T

)

. Given partial pre-

aligned entity pairs

= {(i, j) ∈E

×E

|i ≡ j}

where

≡

means the alignment relationship, the goal of the task is to

ﬁnd alignment of remaining entities via entity embeddings.

2.3 GNNs for Entity Alignment

Recent GNN-based entity alignment models include GCN-

Align (Wang et al

2018), GMNN (Xu et al

2019b), MuGNN

(Cao et al

2019), RDGCN (Wu et al

2019) and AVR-GCN

(Yeetal

2019). GCN-Align and GMNN are built based on

the vanilla GCN. RDGCN introduces dual relation graphs to

enhance the vanilla GCN. AVR-GCN extends R-GCN using

a TransE-like relation-speciﬁc translation operation (Bordes

et al

2013). Before aggregation, each entity representation is

translated from its tail entity representations using relation

vectors. We argue that such relation-speciﬁc translation and

R-GCN introduce a high complexity with the overhead of

trainable parameters. More importantly, the aforementioned

models do not take the non-isomorphism in KG structures

into consideration. While MuGNN (Cao et al

2019) notices

the structure incompleteness of KGs and proposes a two-

step method of rule-based KG completion and multi-channel

GNNs for entity alignment. However, the learned rules rely

on relation alignment to resolve schema heterogeneity.

Isomorphic structures are beneﬁcial

GNNs would learn

the same representation for the entities that have isomorphic

neighborhood structures with identical feature vectors repre-

senting corresponding neighbors (Xu et al

2019a). We show

that, in some cases, if two entities have isomorphic neigh-

borhood structures and only partially pre-aligned neighbor

representations, GNNs can also capture the similarity of other

neighbors to be aligned. Figure 2 (i) gives an example. For

simplicity, here we consider a single-layer GCN. We can let

pre-aligned entities have the same representation by minimiz-

ing their Euclidean distance, i.e.,

(0)

= h

(0)



(0)

= h

(0)



and

(0)

= h

(0)



as well as

(1)

= h

(1)



(1)

= h

(1)



and

(1)

= h

(1)



in the ideal condition. By the mean-pooling

based aggregation, we have

(1)

= σ(W

(1)

(0)

+ h

(0)

)/3)

and

(1)



= σ(W

(1)

(0)



+ h

(0)



+ h

(0)



)/3)

, yield-

ing

(0)

= h

(0)



. Finally, the counterpart entities would have

the same representation. This indicates that the alignment

information between entities can be propagated across the

different GNN layers and different isomorphic graphs given

partially pre-aligned neighborhood. However, for entity align-

ment between different KGs, it is impossible to require the

two KGs to have isomorphic structures due to the schema het-

erogeneity. Figure 2 (ii) gives an example of non-isomorphic

graph structures, where

and



would have different repre-

sentations due to their different neighborhood structures.

Only structures are not enough

Conventional GNNs fall

short of characterizing some special subgraph structures such

223

of 8

免费下载

关注

评论