Skip to content

Conversation

@gumb0
Copy link
Collaborator

@gumb0 gumb0 commented Feb 12, 2021

No description provided.

@gumb0 gumb0 force-pushed the changelog-0.7 branch 4 times, most recently from cfad13f to 3a9f619 Compare February 12, 2021 17:20
@codecov
Copy link

codecov bot commented Feb 12, 2021

Codecov Report

Merging #734 (263558e) into master (fe665d8) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #734   +/-   ##
=======================================
  Coverage   99.26%   99.26%           
=======================================
  Files          74       74           
  Lines       11442    11442           
=======================================
  Hits        11358    11358           
  Misses         84       84           
Flag Coverage Δ
rust 99.86% <ø> (ø)
spectests 90.68% <ø> (ø)
unittests 99.22% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

@gumb0
Copy link
Collaborator Author

gumb0 commented Feb 24, 2021

Added recently merged #736 and #737

- 1009 of 1009 validation tests,
- 499 skipped due to testing text format parser.


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want benchmarks here?

@chfast
Copy link
Collaborator

chfast commented Feb 26, 2021

Raw benchmarks look like this, GCC10 LTO

fizzy/parse/blake2b_mean                                           +0.0390         +0.0390            26            27            26            27
fizzy/instantiate/blake2b_mean                                     +0.0330         +0.0330            32            33            32            33
fizzy/execute/blake2b/512_bytes_rounds_1_mean                      -0.0621         -0.0621            76            72            76            72
fizzy/execute/blake2b/512_bytes_rounds_16_mean                     -0.0308         -0.0308          1108          1074          1108          1074
fizzy/parse/ecpairing_mean                                         +0.0310         +0.0310          1535          1583          1535          1583
fizzy/instantiate/ecpairing_mean                                   +0.0297         +0.0297          1589          1636          1589          1636
fizzy/execute/ecpairing/onepoint_mean                              +0.0557         +0.0557        333335        351916        333339        351918
fizzy/parse/keccak256_mean                                         +0.0362         +0.0362            48            50            48            50
fizzy/instantiate/keccak256_mean                                   +0.0317         +0.0317            52            54            52            54
fizzy/execute/keccak256/512_bytes_rounds_1_mean                    -0.2310         -0.2310            95            73            95            73
fizzy/execute/keccak256/512_bytes_rounds_16_mean                   -0.2198         -0.2198          1370          1069          1370          1069
fizzy/parse/memset_mean                                            +0.0163         +0.0163             6             7             6             7
fizzy/instantiate/memset_mean                                      +0.0094         +0.0094            10            10            10            10
fizzy/execute/memset/256_bytes_mean                                +0.0646         +0.0646             6             6             6             6
fizzy/execute/memset/60000_bytes_mean                              +0.0688         +0.0688          1318          1408          1318          1408
fizzy/parse/mul256_opt0_mean                                       -0.0029         -0.0029             9             9             9             9
fizzy/instantiate/mul256_opt0_mean                                 +0.0045         +0.0045            12            13            12            13
fizzy/execute/mul256_opt0/input1_mean                              +0.1173         +0.1173            24            26            24            26
fizzy/parse/ramanujan_pi_mean                                      +0.0570         +0.0570            27            28            27            28
fizzy/instantiate/ramanujan_pi_mean                                +0.0469         +0.0468            31            32            31            32
fizzy/execute/ramanujan_pi/33_runs_mean                            +0.0020         +0.0020           101           101           101           101
fizzy/parse/sha1_mean                                              +0.0430         +0.0430            43            45            43            45
fizzy/instantiate/sha1_mean                                        +0.0387         +0.0387            48            49            48            49
fizzy/execute/sha1/512_bytes_rounds_1_mean                         +0.0242         +0.0242            77            79            77            79
fizzy/execute/sha1/512_bytes_rounds_16_mean                        +0.0209         +0.0209          1085          1107          1085          1107
fizzy/parse/sha256_mean                                            +0.0480         +0.0480            71            75            71            75
fizzy/instantiate/sha256_mean                                      +0.0182         +0.0182            78            79            78            79
fizzy/execute/sha256/512_bytes_rounds_1_mean                       +0.0541         +0.0541            75            80            75            80
fizzy/execute/sha256/512_bytes_rounds_16_mean                      +0.0462         +0.0462          1045          1094          1045          1094
fizzy/parse/taylor_pi_mean                                         +0.0166         +0.0166             3             3             3             3
fizzy/instantiate/taylor_pi_mean                                   -0.0011         -0.0011             6             6             6             6
fizzy/execute/taylor_pi/pi_1000000_runs_mean                       +0.1275         +0.1275         36645         41318         36645         41318
fizzy/parse/micro/eli_interpreter_mean                             +0.0027         +0.0027             4             4             4             4
fizzy/instantiate/micro/eli_interpreter_mean                       +0.0066         +0.0066             8             8             8             8
fizzy/execute/micro/eli_interpreter/exec105_mean                   +0.0965         +0.0965             4             5             4             5
fizzy/parse/micro/factorial_mean                                   +0.0265         +0.0265             1             1             1             1
fizzy/instantiate/micro/factorial_mean                             +0.0375         +0.0375             1             1             1             1
fizzy/execute/micro/factorial/20_mean                              +0.0043         +0.0043             1             1             1             1
fizzy/parse/micro/fibonacci_mean                                   -0.0000         -0.0000             1             1             1             1
fizzy/instantiate/micro/fibonacci_mean                             +0.0288         +0.0288             1             1             1             1
fizzy/execute/micro/fibonacci/24_mean                              -0.0048         -0.0048          4726          4704          4726          4704
fizzy/parse/micro/host_adler32_mean                                -0.0172         -0.0172             2             2             2             2
fizzy/instantiate/micro/host_adler32_mean                          +0.0066         +0.0066             4             4             4             4
fizzy/execute/micro/host_adler32/1_mean                            +0.0540         +0.0540             0             0             0             0
fizzy/execute/micro/host_adler32/1000_mean                         +0.0190         +0.0190            28            29            28            29
fizzy/parse/micro/icall_hash_mean                                  +0.0017         +0.0017             3             3             3             3
fizzy/instantiate/micro/icall_hash_mean                            -0.0140         -0.0140             7             7             7             7
fizzy/execute/micro/icall_hash/1000_steps_mean                     +0.0486         +0.0486            62            64            62            64
fizzy/parse/micro/spinner_mean                                     +0.0135         +0.0135             1             1             1             1
fizzy/instantiate/micro/spinner_mean                               +0.0146         +0.0146             1             1             1             1
fizzy/execute/micro/spinner/1_mean                                 +0.1548         +0.1548             0             0             0             0
fizzy/execute/micro/spinner/1000_mean                              +0.2054         +0.2054             7             9             7             9
fizzy/parse/stress/guido-fuzzer-find-1_mean                        +0.0404         +0.0404           137           142           137           142
fizzy/instantiate/stress/guido-fuzzer-find-1_mean                  +0.0324         +0.0324           166           171           166           171

@gumb0
Copy link
Collaborator Author

gumb0 commented Feb 26, 2021

Added recently merged #741 #742 #743

@chfast
Copy link
Collaborator

chfast commented Feb 26, 2021

Clang11 looks more balanced.

fizzy/parse/blake2b_mean                                           +0.0054         +0.0054            23            24            23            24
fizzy/instantiate/blake2b_mean                                     +0.0052         +0.0052            29            30            29            30
fizzy/execute/blake2b/512_bytes_rounds_1_mean                      +0.0285         +0.0285            80            83            80            83
fizzy/execute/blake2b/512_bytes_rounds_16_mean                     +0.0223         +0.0223          1210          1237          1210          1237
fizzy/parse/ecpairing_mean                                         -0.0139         -0.0139          1405          1386          1405          1386
fizzy/instantiate/ecpairing_mean                                   -0.0120         -0.0120          1463          1446          1463          1446
fizzy/execute/ecpairing/onepoint_mean                              +0.0529         +0.0529        396209        417152        396213        417156
fizzy/parse/keccak256_mean                                         +0.0028         +0.0028            43            44            44            44
fizzy/instantiate/keccak256_mean                                   -0.0117         -0.0117            48            48            48            48
fizzy/execute/keccak256/512_bytes_rounds_1_mean                    +0.0452         +0.0452            99           103            99           103
fizzy/execute/keccak256/512_bytes_rounds_16_mean                   +0.0374         +0.0374          1449          1504          1449          1504
fizzy/parse/memset_mean                                            +0.0270         +0.0270             6             6             6             6
fizzy/instantiate/memset_mean                                      +0.0069         +0.0069             9             9             9             9
fizzy/execute/memset/256_bytes_mean                                +0.0113         +0.0113             7             7             7             7
fizzy/execute/memset/60000_bytes_mean                              +0.0123         +0.0123          1539          1558          1539          1558
fizzy/parse/mul256_opt0_mean                                       +0.0071         +0.0071             8             8             8             8
fizzy/instantiate/mul256_opt0_mean                                 -0.0009         -0.0009            11            11            11            11
fizzy/execute/mul256_opt0/input1_mean                              +0.0710         +0.0710            28            30            28            30
fizzy/parse/ramanujan_pi_mean                                      +0.0037         +0.0037            25            25            25            25
fizzy/instantiate/ramanujan_pi_mean                                -0.0063         -0.0063            29            29            29            29
fizzy/execute/ramanujan_pi/33_runs_mean                            +0.0193         +0.0193           111           113           111           113
fizzy/parse/sha1_mean                                              -0.0010         -0.0010            39            39            39            39
fizzy/instantiate/sha1_mean                                        -0.0002         -0.0002            44            44            44            44
fizzy/execute/sha1/512_bytes_rounds_1_mean                         +0.0122         +0.0122            90            91            90            91
fizzy/execute/sha1/512_bytes_rounds_16_mean                        +0.0137         +0.0137          1251          1268          1251          1268
fizzy/parse/sha256_mean                                            -0.0080         -0.0080            66            65            66            65
fizzy/instantiate/sha256_mean                                      +0.0059         +0.0059            70            70            70            70
fizzy/execute/sha256/512_bytes_rounds_1_mean                       +0.0126         +0.0126            81            82            81            82
fizzy/execute/sha256/512_bytes_rounds_16_mean                      +0.0161         +0.0161          1111          1129          1111          1129
fizzy/parse/taylor_pi_mean                                         +0.0319         +0.0319             2             2             2             2
fizzy/instantiate/taylor_pi_mean                                   +0.0109         +0.0109             6             6             6             6
fizzy/execute/taylor_pi/pi_1000000_runs_mean                       -0.0004         -0.0004         40774         40757         40774         40757
fizzy/parse/micro/eli_interpreter_mean                             +0.0215         +0.0215             4             4             4             4
fizzy/instantiate/micro/eli_interpreter_mean                       -0.0002         -0.0002             8             8             8             8
fizzy/execute/micro/eli_interpreter/exec105_mean                   +0.0384         +0.0384             5             5             5             5
fizzy/parse/micro/factorial_mean                                   +0.0215         +0.0215             1             1             1             1
fizzy/instantiate/micro/factorial_mean                             +0.0190         +0.0190             1             1             1             1
fizzy/execute/micro/factorial/20_mean                              -0.0065         -0.0065             1             1             1             1
fizzy/parse/micro/fibonacci_mean                                   +0.0160         +0.0160             1             1             1             1
fizzy/instantiate/micro/fibonacci_mean                             +0.0101         +0.0101             1             1             1             1
fizzy/execute/micro/fibonacci/24_mean                              -0.0322         -0.0322          5229          5061          5229          5061
fizzy/parse/micro/host_adler32_mean                                +0.0414         +0.0414             1             2             1             2
fizzy/instantiate/micro/host_adler32_mean                          +0.0128         +0.0128             3             4             3             4
fizzy/execute/micro/host_adler32/1_mean                            +0.0322         +0.0322             0             0             0             0
fizzy/execute/micro/host_adler32/1000_mean                         +0.0114         +0.0114            31            31            31            31
fizzy/parse/micro/icall_hash_mean                                  +0.0389         +0.0389             3             3             3             3
fizzy/instantiate/micro/icall_hash_mean                            -0.0030         -0.0030             7             6             7             6
fizzy/execute/micro/icall_hash/1000_steps_mean                     +0.0021         +0.0021            69            69            69            69
fizzy/parse/micro/spinner_mean                                     +0.0143         +0.0143             1             1             1             1
fizzy/instantiate/micro/spinner_mean                               +0.0370         +0.0370             1             1             1             1
fizzy/execute/micro/spinner/1_mean                                 +0.0346         +0.0346             0             0             0             0
fizzy/execute/micro/spinner/1000_mean                              +0.0306         +0.0306             9             9             9             9
fizzy/parse/stress/guido-fuzzer-find-1_mean                        -0.0165         -0.0165           140           137           140           137
fizzy/instantiate/stress/guido-fuzzer-find-1_mean                  -0.0131         -0.0131           168           166           168           166

@chfast
Copy link
Collaborator

chfast commented Feb 26, 2021

GCC10, no LTO

fizzy/parse/blake2b_mean                                           +0.0124         +0.0124            25            26            25            26
fizzy/instantiate/blake2b_mean                                     -0.0287         -0.0287            31            30            31            30
fizzy/execute/blake2b/512_bytes_rounds_1_mean                      -0.1380         -0.1380            84            73            84            73
fizzy/execute/blake2b/512_bytes_rounds_16_mean                     -0.0483         -0.0483          1185          1128          1186          1128
fizzy/parse/ecpairing_mean                                         +0.0007         +0.0007          1481          1482          1481          1482
fizzy/instantiate/ecpairing_mean                                   +0.0029         +0.0029          1539          1543          1539          1543
fizzy/execute/ecpairing/onepoint_mean                              -0.0117         -0.0117        361605        357370        361609        357374
fizzy/parse/keccak256_mean                                         +0.0101         +0.0101            47            47            47            47
fizzy/instantiate/keccak256_mean                                   +0.0051         +0.0051            51            52            51            52
fizzy/execute/keccak256/512_bytes_rounds_1_mean                    -0.3119         -0.3119           108            74           108            74
fizzy/execute/keccak256/512_bytes_rounds_16_mean                   -0.3223         -0.3223          1582          1072          1582          1072
fizzy/parse/memset_mean                                            -0.0383         -0.0383             6             6             6             6
fizzy/instantiate/memset_mean                                      +0.0070         +0.0070            10            10            10            10
fizzy/execute/memset/256_bytes_mean                                +0.0194         +0.0194             6             6             6             6
fizzy/execute/memset/60000_bytes_mean                              +0.0244         +0.0244          1381          1414          1381          1414
fizzy/parse/mul256_opt0_mean                                       +0.0014         +0.0014             8             8             8             8
fizzy/instantiate/mul256_opt0_mean                                 +0.0041         +0.0041            12            12            12            12
fizzy/execute/mul256_opt0/input1_mean                              +0.0739         +0.0739            25            27            25            27
fizzy/parse/ramanujan_pi_mean                                      +0.0071         +0.0071            27            27            27            27
fizzy/instantiate/ramanujan_pi_mean                                +0.0162         +0.0162            31            31            31            31
fizzy/execute/ramanujan_pi/33_runs_mean                            +0.0343         +0.0343           102           106           102           106
fizzy/parse/sha1_mean                                              +0.0051         +0.0051            43            43            43            43
fizzy/instantiate/sha1_mean                                        +0.0211         +0.0211            47            48            47            48
fizzy/execute/sha1/512_bytes_rounds_1_mean                         +0.0206         +0.0206            80            81            80            81
fizzy/execute/sha1/512_bytes_rounds_16_mean                        +0.0164         +0.0164          1112          1130          1112          1130
fizzy/parse/sha256_mean                                            +0.0124         +0.0124            70            70            70            70
fizzy/instantiate/sha256_mean                                      +0.0031         +0.0031            74            75            74            75
fizzy/execute/sha256/512_bytes_rounds_1_mean                       +0.0137         +0.0137            77            78            77            78
fizzy/execute/sha256/512_bytes_rounds_16_mean                      +0.0162         +0.0162          1062          1079          1062          1079
fizzy/parse/taylor_pi_mean                                         -0.0090         -0.0090             3             3             3             3
fizzy/instantiate/taylor_pi_mean                                   -0.0078         -0.0078             6             6             6             6
fizzy/execute/taylor_pi/pi_1000000_runs_mean                       +0.1250         +0.1250         36860         41467         36860         41467
fizzy/parse/micro/eli_interpreter_mean                             +0.0060         +0.0060             4             4             4             4
fizzy/instantiate/micro/eli_interpreter_mean                       +0.0029         +0.0029             8             8             8             8
fizzy/execute/micro/eli_interpreter/exec105_mean                   +0.0082         +0.0082             4             5             4             5
fizzy/parse/micro/factorial_mean                                   -0.0060         -0.0060             1             1             1             1
fizzy/instantiate/micro/factorial_mean                             -0.0243         -0.0243             1             1             1             1
fizzy/execute/micro/factorial/20_mean                              -0.0082         -0.0082             1             1             1             1
fizzy/parse/micro/fibonacci_mean                                   -0.0019         -0.0019             1             1             1             1
fizzy/instantiate/micro/fibonacci_mean                             -0.0242         -0.0242             1             1             1             1
fizzy/execute/micro/fibonacci/24_mean                              -0.0374         -0.0374          4878          4695          4878          4695
fizzy/parse/micro/host_adler32_mean                                +0.0203         +0.0203             2             2             2             2
fizzy/instantiate/micro/host_adler32_mean                          +0.0297         +0.0297             4             4             4             4
fizzy/execute/micro/host_adler32/1_mean                            +0.0503         +0.0503             0             0             0             0
fizzy/execute/micro/host_adler32/1000_mean                         +0.0233         +0.0233            28            29            28            29
fizzy/parse/micro/icall_hash_mean                                  +0.0221         +0.0221             3             3             3             3
fizzy/instantiate/micro/icall_hash_mean                            +0.0045         +0.0045             7             7             7             7
fizzy/execute/micro/icall_hash/1000_steps_mean                     -0.0280         -0.0280            67            65            67            65
fizzy/parse/micro/spinner_mean                                     +0.0209         +0.0209             1             1             1             1
fizzy/instantiate/micro/spinner_mean                               +0.0219         +0.0219             1             1             1             1
fizzy/execute/micro/spinner/1_mean                                 +0.0710         +0.0710             0             0             0             0
fizzy/execute/micro/spinner/1000_mean                              +0.0436         +0.0436             9             9             9             9
fizzy/parse/stress/guido-fuzzer-find-1_mean                        -0.0334         -0.0334           139           134           139           134
fizzy/instantiate/stress/guido-fuzzer-find-1_mean                  +0.0147         +0.0147           164           166           164           166

CHANGELOG.md Outdated
- Test utils improvements. [#579](https://github.com/wasmx/fizzy/pull/579)
[#658](https://github.com/wasmx/fizzy/pull/658) [#661](https://github.com/wasmx/fizzy/pull/661)
[#658](https://github.com/wasmx/fizzy/pull/658) [#661](https://github.com/wasmx/fizzy/pull/661)
[#741](https://github.com/wasmx/fizzy/pull/741)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somehow 741/742/743 got added to older versions and not 0.7.0.

Co-authored-by: Alex Beregszaszi <[email protected]>
Co-authored-by: Paweł Bylica <[email protected]>
@axic axic merged commit a0c08f8 into master Mar 1, 2021
@axic axic deleted the changelog-0.7 branch March 1, 2021 10:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants