IMPROVE-QA An Interactive Mechanism for RDF-2018年-张欣勃-SIGMOD 2018.pdf
免费下载
IMPROVE-QA: An Interactive Mechanism for RDF
estion/Answering Systems
Xinbo Zhang
Peking University
Beijing, China
zhangxinbo@pku.edu.cn
Lei Zou
Peking University
Beijing, China
zoulei@pku.edu.cn
ABSTRACT
RDF Question/Answering(Q/A) systems can interpret user’s ques-
tion
N
as SPARQL query
Q
and return answer set
Q(D)
over RDF
repository
D
to the user. However, due to the complexity of linking
natural phrases with specic RDF items (e.g., entities and predi-
cates), it remains dicult to understand users’ questions precisely,
hence
Q(D)
may not meet users’ expectation, oering wrong an-
swers and dismissing some correct answers. In this demo, we design
an
I
nteractive
M
echanism aiming for
PRO
motion
V
ia f
e
edback to
Q
/
A
systems
(IMPROVE-QA)
, a whole platform to make existing
Q/A systems return more precise answers (denoted as
Q
′
(D)
) to
users. Based on user’s feedback over
Q(D)
, IMPROVE-QA auto-
matically renes the original query
Q
into a new query graph
Q
′
with minimum modications, where
Q
′
(D)
provides more precise
answers. We will also demonstrate how IMPROVE-QA can apply
the “lesson” learned from the user in each query to improve the
precision of Q/A systems on subsequent natural language questions.
ACM Reference Format:
Xinbo Zhang and Lei Zou. 2018. IMPROVE-QA: An Interactive Mechanism
for RDF Question/Answering Systems. In SIGMOD’18: 2018 International
Conference on Management of Data, June 10–15, 2018, Houston, TX, USA.
ACM, New York, NY, USA, 4 pages. https://doi.org/10.1145/3183713.3193555
1 INTRODUCTION
As a de facto standard of a knowledge graph, RDF (Resource De-
scription Framework) repository is a collection of triples, denoted
as
⟨
subject,predicate,object
⟩
. SPARQL is a standard way to access
RDF data, but it remains dicult for common users due to the com-
plexity of SPARQL syntax. As more and more knowledge graphs
become available, such as YAGO, DBpedia, Freebase, it becomes
crucial to enable users to access knowledge graphs easily without
mastering SPARQL. Natural language question/answering systems
over RDF (
RDF Q/A
for short) provide such an accessible way and
have attracted wide attentions in both academia and industry.
Given a natural language question
N
, an RDF Q/A system trans-
lates
N
into a SPARQL query
Q
, evaluates query
Q
over the knowl-
edge graph
D
and returns answers
Q(D)
to users. However, due to
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specic permission and/or a
fee. Request permissions from permissions@acm.org.
SIGMOD’18, June 10–15, 2018, Houston, TX, USA
© 2018 Association for Computing Machinery.
ACM ISBN 978-1-4503-4703-7/18/06.. . $15.00
https://doi.org/10.1145/3183713.3193555
the complexity and ambiguity of natural language questions, a fun-
damental challenge is how to understand users’ questions precisely,
i.e., interpreting users’ question
N
as a proper SPARQL query
Q
.
In existing RDF Q/A systems, when the returned answers
Q(D)
do
not meet users’ expectation, nothing could be done for users but
leaving the system with confusion. We call it passive acceptance.
In this demo, we design an
I
nteractive
M
echanism aiming for
PRO
motion
V
ia f
e
edback to
Q
/
A
systems
(IMPROVE-QA)
, a
whole platform to make existing Q/A systems return more pre-
cise answers (denoted as
Q
′
(D)
) to the user. Specically, when
the user does not satisfy the returned answers
Q(D)
, he can give
feedback about
Q(D)
, including crossing out some wrong answers
(
Q
−
(D)
), adding missing correct ones (
Q
∆
(D)
) and marking some
existing correct answers (
Q
+
(D)
). Note that our system does
not
require users to provide the full list of errors/omissions or mark
all correct answers. IMPROVE-QA learns from users’ feedback and
generates a more precise SPARQL query
Q
′
as a new translation
for N and nally returns more precise answers Q
′
(D) to users.
Although IMPROVE-QA is similar to QBE (query by example)
[
4
] for database queries, the major dierence is that sample answers
are only inputs for QBE problems, while IMPROVE-QA considers
not only the seed answers (including positive and negative ones)
given by users but also the original translated SPARQL query
Q
.
This is because that existing Q/A systems have made great progress
to generate
Q
. It’s hard to promote the quality of
Q
directly, while
many translation errors from question
N
to SPARQL
Q
are reected
on some parts of query graph
Q
, such as entity/class linking (node
error), relation paraphrasing (edge error) and sentence structure
error (template error). Thus, based on users’ feedback, our goal is
to nd “minimum edit” from original query
Q
to a new SPARQL
Q
′
, where
Q
′
(D)
provides more precise answers. We provide an
interactive mechanism aiming for promotion to RDF Q/A systems,
allowing for users’ feedback to obtain better answers. Furthermore,
the “lesson” learned from users in each query can be used to improve
RDF Q/A systems on subsequent natural language questions.
Example 1. Consider a question
N “Which actresses were born
in European countries?”
issued by a big fan of Elizabeth Taylor.
Figure 1 shows the answer set
Q(D)
which includes six answers re-
turned by an RDF Q/A system over a knowledge graph
D
. “Elizabeth
Taylor” is unexpectedly excluded from the answers. The big fan def-
initely knows Elizabeth Taylor was born in London in 1932. To her
more surprise and bewilderment, “Marilyn Monroe” is shown in the
answer list. As we know, Marilyn Monroe is an American-born actress
(born in Los Angeles in 1926). What happens to the RDF Q/A system?
The big fan gives her feedback over answer set
Q(D)
. As shown
in Figure 1, she adds one missing correct answer
Q
∆
(D)
(Elizabeth
Taylor), crosses out one wrong answer
Q
−
(D)
(Marilyn Monroe) and
Demonstrations
SIGMOD’18, June 10-15, 2018, Houston, TX, USA
1753
Evaluate
Ask: Which actress was born in countries in Europe?
Which actress
was born in
countries in Europe?
N
<Mariene_Dietrich>
<Audrey_Hepburn>
<Elizabeth_Taylor>
R1
R2
R3
( )
Q D
( )
Q D
R
<Marilyn_Monroe>
( )
Q D
His feedback
Entity/class-Linking Dictionary
countries in Europe<Country>
<EuropeanCountry>
Relation-Paraphrasing Dictionary
be born (in) <deathPlace>
<birthPlace>
Sentence-Structure Dictionary
Who verb place: <who> <verb> <place>
ADD:
<who> <verb> <?place>;
<?place> <verb> <place>
<Mariene_Dietrich>
<Audrey_Hepburn>
<Elizabeth_Taylor>
<Eva_Green>
<Vivien Leigh>
<Greta Garbo>
R1
R2
R3
R4
R5
R6
( )
Q D
Collect
r2 <Mariene_Dietrich>
r1 <Audrey_Hepburn>
r3 <Greta Garbo>
r5 <Judy_Garland>
r4 <Marilyn_Monroe>
r6 <Lana_Turner>
Where is
“
<Elizabeth_Taylor>
”
?
( )
Q D
Return
<occupation>
<Actress>?actress
<type>
<Country>
?country
Ordinary Query Q
<deathPlace>
Q
1
v
2
v
3
v
4
v
<occupation>
<Actress>
?actress
<country>
?city
?country
<type>
<EuropeanCountry>
<occupation>
<type>
<EuropeanCountry>
<Actress>
?actress
<birthPlace>
<birthPlace>
?country
Refined Query
1
Q
2
Q
1 2
Q Q Q
Q
1
u
2
u
3
u
4
u
5
u
4
u
3
u
2
u
1
u
Figure 1: Rening Queries based on Users’ Feedback.
marks two existing correct answers
Q
+
(D)
(Audrey Hepburn and
Mariene Dietrich). Let
R = Q
∆
(D)∪Q
+
(D)
denote all positive answers.
Based on the user’s feedback, IMPROVE-QA system nds a new query
Q
′
which contains two BGP
1
queries
Q
′
1
and
Q
′
2
, i.e.,
Q
′
= Q
′
1
UNION
Q
′
2
, each of which is denoted as a query graph. Note that in such cases
it is impossible to generate a single BGP query that covers all positive
answers
R
and meanwhile excludes any negative answer in
Q
−
(D)
.
Thus, we allow for multiple BGP queries in the revised query
Q
′
. On
the other hand, to avoid overtting, we require that the number of
BGP queries and the total
edits
from
Q
to
Q
′
should be minimized.
We will formally dene the problem in Section 2.2.
In the above running example, the revised answer set
Q
′
(D) =
Q
′
1
(D) ∪ Q
′
2
(D)
satises the big fan’s requirement. Thus,
Q
′
(D)
is
better than the original one for the user. If the user would like to
make further revision about
Q
′
(D)
, IMPROVE-QA can iterate the
above process until the returned answers are completely satisfac-
tory. Note that the edit from
Q
to
Q
′
species a graph alignment
between
Q
and
Q
′
. Based on the alignment, IMPROVE-QA can learn
“lessons” from users, which can benet answering other questions.
Example 2. Continuing with the previous example. The vertex
label
label(v
4
)
=“
⟨
Country
⟩
” is relabeled as
label(u
4
)
=“
⟨
European-
Country
⟩
” in both
Q
′
1
and
Q
′
2
. Actually, this is an
entity/class link-
ing error
in
Q
. In the original query, “European countries” in the nat-
ural language question
N
is linked to
⟨
Country
⟩
but the correct one
should be
⟨
EuropeanCountry
⟩
. We also record “countries in Europe”
→
⟨
EuropeanCountry
⟩
into the
entity/class-linking dictionary
for
the purpose of further querying. The edge label
label(
−−−→
v
1
v
3
)
=“
⟨
death-
place
⟩
” in
Q
is relabeled as
label(
−−−→
u
1
u
3
)
=“
⟨
birthplace
⟩
” in
Q
′
1
and
Q
′
2
. This is a
relation paraphrasing error
which maps phrase “(be)
born in” to “
⟨
deathplace
⟩
”. Based on this edit operation, we can correct
the error in the
relation-paraphrasing dictionary
[
8
]. The edge
−−−→
v
3
v
4
in
Q
is extended to a path
(
−−−→
u
3
u
5
,
−−−→
u
5
u
4
)
of length-2 in
Q
′
1
. Based
on this edit, we can add a template pattern “in European countries”
→
“(?city
⟨
country
⟩
?country. ?country
⟨
type
⟩
EuropeanCountry)” into
sentence-structure dictionary
. These templates can also benet
the translation in RDF Q/A systems [5].
Figure 1 shows the amendment of three dictionaries mentioned
above. Obviously, these amendments can be used to avoid similar
1
basic graph pattern
errors in answering the following questions, such as “Give me all
European countries
2
”, “Which NBA players were born in
3
USA?”.
In Section 2, we will illustrate the whole framework of our
IMPROVE-QA and present core and basic algorithm in IMPROVE-
QA. And we will demonstrate each component of IMPROVE-QA to
show how it improves RDF Q/A systems (Section 3).
2 SYSTEM OVERVIEW
2.1 Architecture
We set up an
I
nteractive
M
echanism aiming for
PRO
motion
V
ia
f
e
edback to
Q
/
A
systems
(IMPROVE-QA)
on top of our previ-
ous RDF Q/A system gAnswer[
8
]. Technically, gAnswer translates
users’ natural language question
N
into query graph
Q
and nds
the subgraph matches of Q over RDF graph D to nd the answers.
The translation is based on two dictionaries, one is entity-linking
dictionary and the other one is relation-paraphrasing dictionary.
The former is to identify the entity/class phrases in question
N
and
link them to the entities/classes in RDF graph
D
, while the latter
is to extract the relation phrases in question
N
and map them to
predicates in D. More details of gAnswer are given in [8].
IMPROVE-QA has three components: feedback collection, query
renement and dictionary amendment. We will briey review each
component. Figure 2 illustrates the architecture of IMPROVE-QA.
Feedback Collection
: When an RDF Q/A system (gAnswer) re-
turns answer set
Q(D)
to users, this component shows an interface
to allow for users’ feedback (as shown in Figure 2). Users can cross
out some wrong answers (
Q
−
(D)
), add missing correct ones (
Q
∆
(D)
)
and mark some existing correct answers (
Q
+
(D)
). Note that, users
are not required to judge all returned answers in
Q(D)
since
Q(D)
may be very large, such as “Show me all American movies”. Based
on users’ feedback, this component collects some positive answers
R = Q
∆
(D) ∪ Q
+
(D) and negative answers Q
−
(D).
Query Renement
: This is the core component of IMPROVE-
QA. According to the original query graph
Q
(given by RDF Q/A
system) and users’ feedback (
R
and
Q
−
(D)
, collected in the previous
component), this component aims to nd a rened query
Q
′
, where
Q
′
(D)
covers all positive answers in
R
and excludes any negative
one in
Q
−
(D)
. Furthermore, we hope that the edit from
Q
to
Q
′
2
should be mapped to ⟨EuropeanCountry⟩ rather than ⟨Country⟩.
3
should be mapped to ⟨birthplace⟩ rather than ⟨deathplace⟩.
Demonstrations
SIGMOD’18, June 10-15, 2018, Houston, TX, USA
1754
of 4
免费下载
【版权声明】本文为墨天轮用户原创内容，转载时必须标注文档的来源（墨天轮），文档链接，文档作者等基本信息，否则作者和墨天轮有权追究责任。如果您发现墨天轮中有涉嫌抄袭或者侵权的内容，欢迎发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。
下载排行榜
评论