'''Load:'''
      
||  Endpoint     || Cell Cycle ||Allie || PDBJ  || UniProt || DDBJ ||
|| Virtuoso(min) ||4 || 46   ||  103  || * 43hours17mins     || 78hours8mins     ||
|| OwlimSE(min)  || 3 ||28   ||  127  ||  testing       || testing     ||
|| Bigdata(min)  ||3  ||62   ||  537  ||        ||   X   ||
|| 4Store (min)  ||2 || 12   ||  6408 ||    #   ||   #   || 
|| Mulgara(min)  || 10 ||89   ||   X   ||       ||    X   ||

 
 ** It needs another 17 hours to decompress and split the data set.

 # We do not test 4Store on larger data set because its scalability is not ideal. From about 100M Allie data set to 500M PDBJ data set the time cost increases 500 times. 

 X Some Problem occurred when uploading the data. Please refer to its uploading procedure for details.
 
[[Image(triple.bmp)]] [[Image(loadtime.bmp)]]

Space used:

||  Endpoint     || Cell Cycle||Allie || PDBJ  || UniProt || DDBJ ||
|| Virtuoso ||     ||    ||       ||    308G  ||538G||
|| OwlimSE ||  5.7G   ||  18G  ||     57G    ||    481G  ||1079G ||
|| Bigdata  ||  780M   ||  6.9G  ||   34G     ||      || ||
|| 4Store   ||  2.2G   ||  14.7G ||    *    ||   *   || *||
|| Mulgara  ||   2.4G ||  15.8G   ||    X     ||     || X ||

'''Query for Cell Cycle:'''
      
|| Endpoint             || case1 ||case2|| case3|| case4|| case5|| case6 ||case7|| case8|| case9|| case10|| case11 ||case12|| case13|| case14|| case15|| case16 ||case17|| case18|| case19||
||Virtuoso (ms)         || '''24'''|| '''2'''|| 23280||3||42500||13073||5||7562||41||2||120||19||5||1||56058||'''46'''||'''15'''||'''16'''||16721||
||OwlimSE(ms)           ||110|| 6||2472||'''2'''||149||2071||'''2'''||'''33'''||'''22'''||'''0'''||'''6'''||'''6'''||'''2'''||'''0'''||46129||X||X||X||'''14''' ||
||Bigdata(ms)           || 331 ||42|| 3135|| 16 ||414 ||1191|| 21|| 97 ||43|| 14|| 23|| 43|| 13|| 13|| '''19093'''||X||X||X|| 37||
||4Store (ms)           ||  56|| 18|| '''1236'''|| 13|| '''33'''|| '''64'''|| 22|| 67|| 2035|| 7|| '''6'''|| 1563|| 8 ||7|| * ||X||X||X|| 15 ||
|| Mulgara(ms)          ||  1294     ||  20     ||   2207  ||   9  ||    343  ||2325|| 32|| 58||33||4||14||X||9||6||X||X||X||X||38||

 X or * shows that the endpoint does not support "count()" function or some unsupported function causes a wrong result.

[[Image(cellcycle_bar.bmp)]] [[Image(cellcycle_pie.bmp)]] The pie chart shows that how many percent an end point accounts for the fastest performers.

 

The data shows that in the cell cycle queries on the 10 million or so triples:

(1)Virtuoso supports more query. In some cases Virtuoso response fast but some others cost far more than others, such as case5 and case19;

(2)Except that OwlimSE cannot support count() function, it totally perform better and has no worst case;

(3)Bigdata and Mulgara perform averagely well;

(4) 4Store do not support count() and give no response in case15. However it performs distinctively better in some cases such as case5 and case6.
 

'''Query for Allie:'''
      
|| Endpoint             || case1 ||case2|| case3|| case4|| case5||                                               
||Virtuoso (ms)         || '''23'''   || 1413|| '''152'''  ||  95  ||'''27299'''||
||OwlimSE(ms)           || 145   || 1980|| 1476 || '''38'''   ||68369||
||Bigdata(ms)           || 427   ||  4206|| 2549   || 212   || 195021   ||
||4Store (ms)           ||  X    || '''217''' || X    || X    ||65128 ||
|| Mulgara(ms)          ||   373   ||   121    || X    ||   X  ||   X   ||

4Store: Donot support "lang()" function. 

Mulgara: Unable to support arbitrarily complex ORDER BY clause.

[[Image(allie_pie.bmp)]]

'''Query for PDBJ:'''

      
|| Endpoint             || case1 ||case2|| case3|| case4||
||Virtuoso (ms)         ||   147  ||2 ||''' 2'''  || 138   || 
||OwlimSE(ms)           ||''' 53'''   || '''1''' ||191 || '''4'''   ||
||Bigdata(ms)           ||  213 ||17  ||55    ||56     ||
||4Store (ms)           || 1025    ||  1274     || 131   || 1524 || 
|| Mulgara(ms)          ||    X   ||  X     || X    || X    ||   


In these two groups of queries on about 100 million triples:

(1) Virtuoso and OwlimSE works better than others. Although in Allie Virtuoso performs a little better and OwlimSE is better in PDBJ, there looks no overwhelming advantages over each other.

(2) In Allie 4Store is still limited but performs better when it executes the query such as in case2. However as increasing the number of triples in PDBJ, it performs worst.

(3) Bigdata still  keeps it situation: neither the best one nor the worst one.

  

'''Query for Uniprot:'''
      
|| Endpoint             || case1 ||case2|| case3|| case4|| case5|| case6 ||case7|| case8|| case9|| case10|| case11 ||case12|| case13|| case14|| case15|| case16 ||case17|| case18||
||Virtuoso (ms)         ||51||95||114||2||7||2206||34916||413||605||652||53||4||289||269||10631||9052||2||76||
||OwlimSE (ms)         ||429||383||519||139||21||628381||433913||6924||1923||58794||3266||117||22||42||35112||322058||5875||8900||


'''Query for DDBJ:'''

|| Endpoint             || case1 ||case2|| case3|| case4|| case5|| case6 ||case7|| case8|| case9|| case10|| 
||OwlimSE (ms)         ||18648||3276||3710||82||85||117||5534||10468||2010||1||
||Virtuoso (ms)         ||226||218||418||56||7||98||5||4||7||1||

In these two group of queries with about 4 billion and 8 billion triples, we found out that Virtuoso performs obviously  better.


'''Conclusion'''

Our evaluation shows that the importing cost of the data depends on the multiple factors: Server configuration(CPU,memory,harddisk and so on), the system property(vm.swappiness, JVM), the application configuration(cachememory,etc...), the data format, the size of data set and even data contents, e.g. DDBJ is nearly 2 times the triple size of Uniprot, but its importing cost is 2 times less than Uniprot(2 times longer expected if simply considering the proportional scaling).  

Without considering inference, when the number of triple size is