Skip to content

Commit f975693

Browse files
studyzystudyzy
committed
refactor: SharedingCountPerformance test
--story=1
1 parent ef1d5df commit f975693

File tree

2 files changed

+41
-9
lines changed

2 files changed

+41
-9
lines changed

‎README.md‎

Lines changed: 26 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ sdb, err := NewShardingDb(Sha256Sharding, db1, db2)
7777
7878
generate data by command:
7979
```bash
80-
go test -run "TestCompareDbPerformance"
80+
go test -timeout 60m -run "TestCompareDbPerformance"
8181
```
8282
Test case: total 1000000 key-value pairs, 100 go routines, 100 key-value pairs per batch.
8383
@@ -89,7 +89,7 @@ Test case: total 1000000 key-value pairs, 100 go routines, 100 key-value pairs p
8989
| 200B | 4.45s | 1.07s | 0.683s | 1.9s |
9090
| 500B | 15.3s | 3.36s | 1.49s | 6.4s |
9191
| 1KB | 48.9s | 9.42s | 3.74s | 17.69s |
92-
| 10KB | 1117s | 351s | 123s | 0.0016s |
92+
| 10KB | 1117s | 351s | 123s | 308s |
9393
9494
### 2. GetData
9595
@@ -99,7 +99,7 @@ Test case: total 1000000 key-value pairs, 100 go routines, 100 key-value pairs p
9999
| 200B | 3.09s | 1.42s | 1.27s | 2.24s |
100100
| 500B | 4.17s | 1.91s | 1.62s | 3.73s |
101101
| 1KB | 7.97s | 2.37s | 2.26s | 4.53s |
102-
| 10KB | 12.75s | 9.54s | 11.03s | 0.0016s |
102+
| 10KB | 12.75s | 9.54s | 11.03s | 13.85s |
103103
104104
### 3. GetData not found
105105
@@ -109,7 +109,7 @@ Test case: total 1000000 key-value pairs, 100 go routines, 100 key-value pairs p
109109
| 200B | 2.07s | 1.47s | 0.9s | 1.6s |
110110
| 500B | 2.05s | 1.51s | 0.93s | 1.81s |
111111
| 1KB | 2.35s | 1.64s | 0.891s | 2.28s |
112-
| 10KB | 8.68s | 5.56s | 2.48s | 0.0016s |
112+
| 10KB | 8.68s | 5.56s | 2.48s | 7.75s |
113113
114114
115115
### 4. DeleteData
@@ -120,7 +120,7 @@ Test case: total 1000000 key-value pairs, 100 go routines, 100 key-value pairs p
120120
| 200B | 3.81s | 1.71s | 1.02s | 1.74s |
121121
| 500B | 3.85s | 1.76s | 1.05s | 1.69s |
122122
| 1KB | 3.84s | 1.72s | 1.04s | 1.74s |
123-
| 10KB | 3.844s | 1.78s | 1.06s | 0.0016s |
123+
| 10KB | 3.844s | 1.78s | 1.06s | 1.76s |
124124
125125
### 5. Iterator
126126
@@ -130,8 +130,27 @@ Test case: total 1000000 key-value pairs, 100 go routines, 100 key-value pairs p
130130
| 200B | 0.151s | 0.246s | 0.246s | 0.191s |
131131
| 500B | 0.282s | 0.351s | 0.41s | 0.344s |
132132
| 1KB | 0.514s | 0.419s | 0.472s | 0.541s |
133-
| 10KB | 2.46s | 2.39s | 1.96s | 0.0016s |
134-
133+
| 10KB | 2.46s | 2.39s | 1.96s | 2.3s |
134+
135+
136+
### 6. Sharding count compare
137+
138+
#### 6.1 PutData
139+
| Data Size | ShardingDB(3 folders) | ShardingDB(6 folders) | ShardingDB(9 folders) | ShardingDB(30 folders) | ShardingDB(60 folders) |
140+
|:---------:|:---------------------:|:---------------------:|:---------------------:|:----------------------:|:-----------------------:|
141+
| 100B | 0.659s | 0.581s | | | |
142+
| 200B | 1.07s | 0.683s | | | |
143+
| 500B | 3.36s | 1.49s | | | |
144+
| 1KB | 9.42s | 3.74s | | | |
145+
| 10KB | 351s | 123s | | | |
146+
#### 6.2 GetData
147+
| Data Size | ShardingDB(3 folders) | ShardingDB(6 folders) | ShardingDB(9 folders) | ShardingDB(30 folders) | ShardingDB(60 folders) |
148+
|:---------:|:---------------------:|:---------------------:|:---------------------:|:----------------------:|:----------------------:|
149+
| 100B | 1.25s | 1.02s | | | |
150+
| 200B | 1.42s | 1.27s | | | |
151+
| 500B | 1.91s | 1.62s | | | |
152+
| 1KB | 2.37s | 2.26s | | | |
153+
| 10KB | 9.54s | 11.03s | | | |
135154
136155
137156
Most interfaces are the same as [goleveldb](https://github.com/syndtr/goleveldb). For my interface definition, please refer to [DbHandle](https://github.com/studyzy/shardingdb/blob/main/interfaces.go).

‎performance_test.go‎

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -99,7 +99,7 @@ func TestCompareDbPerformance(t *testing.T) {
9999
}
100100
func TestCompareShardingCountPerformance(t *testing.T) {
101101
sizes := []int{100, 200, 500, 1024, 10240}
102-
shardingCount := []int{9, 30, 60}
102+
shardingCount := []int{3, 6, 9, 30, 60}
103103
for _, size := range sizes {
104104
valueLength = size
105105
for _, count := range shardingCount {
@@ -144,7 +144,20 @@ func TestShardingNDbPerformance(t *testing.T) {
144144
}
145145
fmt.Printf("ShardingDb path[%v]", pathList)
146146
//Test shardingdb performance
147-
db, _ := OpenFile(pathList, nil)
147+
dbs := make([]LevelDbHandle, len(pathList))
148+
var err error
149+
for i := 0; i < len(pathList); i++ {
150+
dbs[i], err = leveldb.OpenFile(pathList[i], nil)
151+
if err != nil {
152+
//close all opened db
153+
for j := 0; j < i; j++ {
154+
dbs[j].Close()
155+
}
156+
panic(err)
157+
}
158+
}
159+
160+
db, _ := NewShardingDb(WithDbHandles(dbs...), WithShardingFunc(MurmurSharding))
148161
testDbPerformance(t, db, fmt.Sprintf("shardingdb%d", diskCount))
149162
db.Close()
150163

0 commit comments

Comments
 (0)