@@ -54,11 +54,10 @@ The Wasmtime maintainers appreciate bug reports with the following:
5454 well as your rationale for * why* that behavior is expected. For example, just
5555 because another Wasm engine or an alternative Wasmtime execution strategy
5656 produces a different result from default Wasmtime, that is not necessarily a
57- bug. Wasm has a variety of sources of
58- [ non-determinism] ( ./examples-deterministic-wasm-execution.md ) (e.g. ` NaN ` bit
59- patterns, maximum call stack depth, and whether growing memory succeeds) and
60- the two different results could both be valid and spec-conforming. Make sure
61- to account for this in your rationale and analysis of the bug.
57+ bug. See the [ documentation
58+ below] ( #divergent-webassembly-behavior-across-runtimes ) for examples of known divergent
59+ behavior of one module in two runtimes. If applicable, make sure to account
60+ for this in your rationale and analysis of the bug.
6261
6362* ** Actual behavior:** A description of the actual, buggy behavior. This should
6463 include things various things like incorrect computation results, assertion
@@ -72,6 +71,57 @@ Including the above information is extra important for bugs discovered
7271mechanically, whether by fuzzing or other means, since the associated test cases
7372will often be pseudo-random or otherwise unintuitive to debug.
7473
74+ ### Divergent WebAssembly behavior across runtimes
75+
76+ WebAssembly has a variety of sources of [ non-determinism] which means that the
77+ exact same module is allowed to behave differently under the same inputs
78+ across multiple runtimes. These specifics don't often arise in "real world"
79+ modules but can quickly arise during fuzzing. Some example behaviors are:
80+
81+ * ** NaN bit patterns** - floating-point operations which produce NaN as a result
82+ are allowed to produce any one of a set of patterns of NaN. This means that
83+ the exact bit-representation of the result of a floating-point operation may
84+ diverge across engines. When fuzzing you can update your source-generation to
85+ automatically canonicalize NaN values after all floating point operations.
86+ Wasmtime has built-in options to curb this [ non-determinism] as well.
87+
88+ * ** Relaxed SIMD** - the ` relaxed-simd ` proposal to WebAssembly explicitly has
89+ multiple allowed results for instructions given particular inputs. These
90+ instructions are inherently non-deterministic across implementations. When
91+ fuzzing you can avoid these instructions entirely, canonicalize the results,
92+ or use Wasmtime's built-in options to curb the [ non-determinism] .
93+
94+ * ** Call stack exhaustion** - the WebAssembly specification requires that all
95+ function calls consume a nonzero-amount of some resource which can eventually
96+ be exhausted. This means that infinite recursion is not allowed in any
97+ WebAssembly engine. Bounded, but very large, recursion is allowed in
98+ WebAssembly but is not guaranteed to work across WebAssembly engines. One
99+ engine may have different stack settings than another engine and/or runtime
100+ parameters may tune how much stack space is taken (e.g. optimizations on/off).
101+ If one engine stack overflows and another doesn't then that's not necessarily
102+ a bug in either engine. Short of banning recursion there's no known great way
103+ to handle this apart from throwing out fuzz test cases that stack overflow.
104+
105+ * ** Memory exhaustion** - the ` memory.grow ` and ` table.grow ` instructions in
106+ WebAssembly are not always guaranteed to either fail or succeed. This means
107+ that growth may succeed in one engine but fail in another depending on various
108+ settings. To handle this in fuzzing it's recommended to generate memories with
109+ a maximum size and ensure that each engine being fuzzed can grow memory all
110+ the way to the maximum size.
111+
112+ * ** WASIp1 API behavior** - the initial specification of WASI, WASIp1 or
113+ ` wasi_snapshot_preview1 ` , effectively is not suitable for differential fuzzing
114+ across engines. The APIs are not thoroughly specified enough nor is there a
115+ rigorous enough test suite to codify what exactly should happen in all
116+ situations on all platforms. This means that exactly what kind of error arises
117+ or various other edge cases may behave differently across engines. The lack of
118+ specificity of WASIp1 means that there is no great oracle as to whether an
119+ engine is right or wrong. Development of WASIp1 has ceased and the Component
120+ Model is being worked on instead (e.g. WASIp2 and beyond) which is more
121+ suitable for differential fuzzing.
122+
123+ [ non-determinism ] : ./examples-deterministic-wasm-execution.md
124+
75125### Do Not Report the Same Bug Multiple Times
76126
77127Fuzzers will often trigger the same bug multiple times in multiple different
0 commit comments