バージョン 5 から バージョン 6 における更新: bigdata
- 更新日時:
- 2012/03/08 18:40:00 (12 年 前)
凡例:
- 変更なし
- 追加
- 削除
- 変更
-
bigdata
v5 v6 21 21 The RW store (Read-Write) supports recycling of allocation slots on the backing file. It may be used as a time-bounded version of an immortal database where history is aged off of the database over time. This is a good choice for standalone workloads where updates are continuously arriving and older database states may be released. The RW store is also less sensitive to data skew because it can reuse B+Tree node and leaf revisions within a commit group on large data set loads. Scaling should be better than the WORM for standalone and could reach to 10B+ triples. The default property file is attachment:RWStore.properties. 22 22 23 In the test we modified the following two important parameters:24 {{{25 com.bigdata.btree.writeRetentionQueue.capacity=50000026 com.bigdata.rdf.sail.BigdataSail.bufferCapacity=100000027 }}}28 29 23 30 24 === Load Performance === #load … … 36 30 Approach 2: 37 31 32 Upload with com.bigdata.rdf.store.DataLoader tools and RW store default parameter. 33 34 And test the situation when adding GC in JVM. 38 35 36 {{{ 37 -Xmx55G -Xms30G -XX:+UseG1GC -XX:+TieredCompilation? -XX:+HeapDumpOnOutOfMemoryError 38 }}} 39 39 40 Approach 3: 41 42 We modified the following two important parameters(In the rest test we use this configure in default): 43 {{{ 44 com.bigdata.btree.writeRetentionQueue.capacity=500000 45 com.bigdata.rdf.sail.BigdataSail.bufferCapacity=1000000 46 }}} 47 48 Approach 4: Split the file into 12 small files. 40 49 41 50 === Allie upload === #allieload … … 44 53 45 54 Approach 2: 5.89hours 46 when Setting JVM GC : 6.75hours 55 when Setting JVM GC : 6.75hours 47 56 48 57 Approach 3: 2.61 hours 49 58 59 Approach 4: 1.03 hours 50 60 51 61 === PDBJ upload === #pdbjload 52 62 53 63 54 '''Result:''' 64 '''Result:''' 8.95 hours 55 65 56 66 === Uniprot upload === #uniprotload
