Bases de Données / Databases

Site Web de l'équipe BD du LIP6 / LIP6 DB Web Site

User Tools

Site Tools


en:site:recherche:logiciels:sparqlwithspark:chain6

Chain6 query plans

The plans produced by each method are:

  • SPARQL RDD:
val d1 = triples.filter{case(s,(p,o))=> p==P1}.map{case(x1,(p, x2))=> (x2, x1)}
val d2 = triples.filter{case(s,(p,o))=> p==P2}.mapValues{case(p,x3) => x3}
val d3 = triples.filter{case(s,(p,o))=> p==P3}.mapValues{case(p,x4) => x4}
val d4 = triples.filter{case(s,(p,o))=> p==P4}.mapValues{case(p,x5) => x5}
val d5 = triples.filter{case(s,(p,o))=> p==P5}.mapValues{case(p,x6) => x6}
val d6 = triples.filter{case(s,(p,o))=> p==P6}.mapValues{case(p,x7) => x7}
val j1 = d1.join(d2).map{case(x2,(x1, x3))=> (x3, (x1,x2))}
val j2 = j1.join(d3).map{case(x3,((x1,x2), x4))=> (x4, (x1,x2,x3))}
val j3 = j2.join(d4).map{case(x4,((x1,x2,x3), x5))=> (x5, (x1,x2,x3,x4))}
val j4 = j3.join(d5).map{case(x5,((x1,x2,x3,x4), x6))=> (x6, (x1,x2,x3,x4,x5))}
val j5 = j4.join(d6).map{case(x6,((x1,x2,x3,x4,x5), x7))=> (x7, (x1,x2,x3,x4,x5,x6))}
j5.count
  • SPARQL DF:
val t1 = df.where(s"p =$P1").select("s","o").withColumnRenamed("s", "x1").withColumnRenamed("o", "x2")
val t2 = df.where(s"p=$P2").select("s","o").withColumnRenamed("s", "x2").withColumnRenamed("o", "x3")
val t3 = df.where(s"p=$P3").select("s","o").withColumnRenamed("s", "x3").withColumnRenamed("o", "x4")
val t4 = df.where(s"p=$P4").select("s","o").withColumnRenamed("s", "x4").withColumnRenamed("o", "x5")
val t5 = df.where(s"p=$P5").select("s","o").withColumnRenamed("s", "x5").withColumnRenamed("o", "x6")
val t6 = df.where(s"p=$P6").select("s","o").withColumnRenamed("s", "x6").withColumnRenamed("o", "x7")
val res = t1.join(t2,Seq("x2")).join(t3,Seq("x3")).join(t4,Seq("x4")).join(t5,Seq("x5")).join(t6,Seq("x6"))
res.count
  • SPARQL Hybrid DF:
val subg = df.where(s"p in ($P1, $P2, $P3, $P4, $P5, $P6)")
subg.persist
subg.count
val st1 = subg.where(s"p= $P1").select("s","o").withColumnRenamed("s", "x1").withColumnRenamed("o", "x2")
val st2 = subg.where(s"p= $P2").select("s","o").withColumnRenamed("s", "x2").withColumnRenamed("o", "x3")
val st3 = subg.where(s"p= $P3").select("s","o").withColumnRenamed("s", "x3").withColumnRenamed("o", "x4")
val st4 = subg.where(s"p= $P4").select("s","o").withColumnRenamed("s", "x4").withColumnRenamed("o", "x5")
val st5 = subg.where(s"p= $P5").select("s","o").withColumnRenamed("s", "x5").withColumnRenamed("o", "x6")
val st6 = subg.where(s"p= $P6").select("s","o").withColumnRenamed("s", "x6").withColumnRenamed("o", "x7")
val res = st1.join(st2,Seq("x2")).join(st3,Seq("x3")).join(st4,Seq("x4")).join(st5,Seq("x5")).join(st6,Seq("x6"))
res.count

Go to SPARQL query processing with Apache Spark

en/site/recherche/logiciels/sparqlwithspark/chain6.txt · Last modified: 16/09/2016 23:11 by hubert