Skip to content

Commit 302b545

Browse files
annevksefeng211
annevk
authored and
sefeng211
committed
Define opaque-response blocking
This is good enough for early review, but there are a number of issues that still need resolving: https://github.com/annevk/orb/labels/mvp. There are also some inline TODO comments. A PR against HTML is needed to ensure it passes the appropriate metadata for media element and classic script requests. We might also want to depend on HTML for parsing JavaScript.
1 parent 8d0d207 commit 302b545

File tree

1 file changed

+305
-8
lines changed

1 file changed

+305
-8
lines changed

‎fetch.bs‎

Lines changed: 305 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,9 @@ urlPrefix:https://w3c.github.io/hr-time/#;spec:hr-time
5252
urlPrefix:https://tc39.es/ecma262/#;type:dfn;spec:ecma-262
5353
url:realm;text:realm
5454
url:sec-list-and-record-specification-type;text:Record
55-
url:current-realm;text:current realm
55+
url:sec-parsetext;text:ParseText
56+
url:prod-Script;text:Script
57+
url:script-record;text:Script Record
5658
</pre>
5759

5860
<pre class=biblio>
@@ -2162,6 +2164,17 @@ Unless stated otherwise, it is false.
21622164

21632165
<p class=note>This flag is for exclusive use by HTML's render-blocking mechanism. [[!HTML]]
21642166

2167+
<p class=XXX>A <a for=/>request</a> has an associated
2168+
<dfn export for=request>no-cors media request state</dfn> ...
2169+
2170+
<p class=note>This is for exclusive use by the <a>opaque-response-safelist check</a>.
2171+
2172+
<p>A <a for=/>request</a> has an associated
2173+
<dfn for=request>no-cors JavaScript fallback encoding</dfn> (an <a for=/>encoding</a>). Unless
2174+
stated otherwise, it is <a for=/>UTF-8</a>.
2175+
2176+
<p class=note>This is for exclusive use by the <a>opaque-response-safelist check</a>.
2177+
21652178
<hr>
21662179

21672180
<p>A <a for=/>request</a> has an associated
@@ -3270,6 +3283,285 @@ through TLS using ALPN. The protocol cannot be spoofed through HTTP requests in
32703283
</div>
32713284

32723285

3286+
<h3 id=orb>Opaque-response blocking</h3>
3287+
3288+
<div class=note>
3289+
<p>Opaque-response blocking, also known as <abbr>ORB</abbr>, is a network filter that blocks access
3290+
to <a>opaque filtered responses</a>. These responses would likely would not have been useful to the
3291+
fetching party. Blocking them reduces information leakage to potential attackers.
3292+
3293+
<p>Essentially, CSS, JavaScript, images, and media (audio and video) can be requested across
3294+
origins without the <a>CORS protocol</a>. And unfortunately except for CSS there is no MIME type
3295+
enforcement. This algorithm aims to block as many responses as possible that are not one of these
3296+
types (or are newer variants of those types) to avoid leaking their contents through side channels.
3297+
3298+
<p>The network filter combines pro-active blocking based on response headers, sniffing a limited
3299+
set of bytes, and ultimately falls back to a full parse due to unfortunate (lack of) design
3300+
decisions in the early days of the web platform. As a result there are still quite a few responses
3301+
whose secrets can end up being revealed to attackers. Web developers are strongly encouraged to use
3302+
the `<code http-header>Cross-Origin-Resource-Policy</code>` response header to defend them.
3303+
</div>
3304+
3305+
3306+
<h4 id=orb-algorithm>The opaque-response-safelist check</h4>
3307+
3308+
<p>The <dfn>opaque-response-safelist check</dfn>, given a <a for=/>request</a> <var>request</var>
3309+
and a <a for=/>response</a> <var>response</var>, is to run these steps:
3310+
3311+
<ol>
3312+
<li><p>Let <var>mimeType</var> be the result of <a>extracting a MIME type</a> from
3313+
<var>response</var>'s <a for=response>header list</a>.
3314+
3315+
<li><p>Let <var>nosniff</var> be the result of <a>determining nosniff</a> given
3316+
<var>response</var>'s <a for=response>header list</a>.
3317+
3318+
<li>
3319+
<p>If <var>mimeType</var> is not failure, then:
3320+
3321+
<ol>
3322+
<li><p>If <var>mimeType</var> is an <a>opaque-response-safelisted MIME type</a>, then return
3323+
true.
3324+
3325+
<li><p>If <var>mimeType</var> is an <a>opaque-response-blocklisted-never-sniffed MIME type</a>,
3326+
then return false.
3327+
3328+
<li><p>If <var>response</var>'s <a for=response>status</a> is 206 and <var>mimeType</var> is an
3329+
<a>opaque-response-blocklisted MIME type</a>, then return false.
3330+
3331+
<li><p>If <var>nosniff</var> is true and <var>mimeType</var> is an
3332+
<a>opaque-response-blocklisted MIME type</a> or its <a for="MIME type">essence</a> is
3333+
"<code>text/plain</code>", then return false.
3334+
</ol>
3335+
3336+
<li><p>If <var>request</var>'s <a for=request>no-cors media request state</a> is
3337+
"<code>subsequent</code>", then return true.
3338+
3339+
<li><p>If <var>response</var>'s <a for=response>status</a> is 206 and
3340+
<a>validate a partial response</a> given 0 and <var>response</var> returns invalid, then return
3341+
false.
3342+
<!-- TODO Integrate https://wicg.github.io/background-fetch/#validate-a-partial-response into Fetch -->
3343+
3344+
<li><p>Let <var>bytes</var> be the result of running
3345+
<a>obtain a copy of the first 1024 bytes of response</a> given <var>response</var>.
3346+
3347+
<li><p>If <var>bytes</var> is failure, then return false.
3348+
3349+
<li>
3350+
<p>If the <a>audio or video type pattern matching algorithm</a> given <var>bytes</var> does not
3351+
return undefined, then:
3352+
3353+
<ol>
3354+
<li><p>If <var>requests</var>'s <a for=request>no-cors media request state</a> is not
3355+
"<code>initial</code>", then return false.
3356+
3357+
<li><p>If <var>response</var>'s <a for=response>status</a> is not 200 or 206, then return false.
3358+
3359+
<li><p>Return true.
3360+
</ol>
3361+
3362+
<li><p>If <var>requests</var>'s <a for=request>no-cors media request state</a> is not
3363+
"<code>N/A</code>", then return false.
3364+
3365+
<li><p>If the <a>image type pattern matching algorithm</a> given <var>bytes</var> does not return
3366+
undefined, then return true.
3367+
3368+
<li>
3369+
<p>If <var>nosniff</var> is true, then return false.
3370+
3371+
<p class=note>This check is made late as unfortunately images and media are always sniffed.
3372+
3373+
<li><p>If <var>response</var>'s <a for=response>status</a> is not an <a>ok status</a>, then return
3374+
false.
3375+
3376+
<li>
3377+
<p>If <var>mimeType</var> is failure, then return true.
3378+
3379+
<p class=note>This could be improved at somewhat significant cost. See
3380+
<a href=https://github.com/annevk/orb/issues/28>annevk/orb #28</a>.
3381+
3382+
<li><p>If <var>mimeType</var>'s <a for="MIME type">essence</a> <a for=string>starts with</a>
3383+
"<code>audio/</code>", "<code>image/</code>", or "<code>video/</code>", then return false.
3384+
3385+
<li><p>Return <a>determine if response is JavaScript and not JSON</a> given <var>response</var>.
3386+
</ol>
3387+
3388+
<hr>
3389+
3390+
<p>To <dfn>obtain a copy of the first 1024 bytes of response</dfn>, given a <a for=/>response</a>
3391+
<var>response</var>, run these steps:
3392+
3393+
<ol>
3394+
<li><p>Let <var>first1024Bytes</var> be null.
3395+
3396+
<li>
3397+
<p><a for=/>In parallel</a>:
3398+
3399+
<ol>
3400+
<li><p>Let <var>bytes</var> be the empty <a for=/>byte sequence</a>.
3401+
3402+
<li><p>Let <var>transformStream</var> be a new {{TransformStream}}.
3403+
3404+
<li>
3405+
<p>Let <var>transformAlgorithm</var> given a <var>chunk</var> be these steps:
3406+
3407+
<ol>
3408+
<li><p><a for=ReadableStream>Enqueue</a> <var>chunk</var> in <var>transformStream</var>.
3409+
3410+
<li>
3411+
<p>If <var>first1024Bytes</var> is null, then:
3412+
3413+
<ol>
3414+
<li><p>Let <var>chunkBytes</var> be
3415+
<a lt="get a copy of the bytes held by the buffer source">a copy of the bytes held by</a>
3416+
<var>chunk</var>.
3417+
3418+
<li><p>Append <var>chunkBytes</var> to <var>bytes</var>.
3419+
3420+
<li>
3421+
<p>If <var>bytes</var>'s <a for="byte sequencue">length</a> is greater than 1024, then:
3422+
3423+
<ol>
3424+
<li><p>Truncate <var>bytes</var> from the end so that it only contains 1024 bytes.
3425+
3426+
<li><p>Set <var>first1024Bytes</var> to <var>bytes</var>.
3427+
</ol>
3428+
</ol>
3429+
</ol>
3430+
3431+
<li><p>Let <var>flushAlgorithm</var> be this step: if <var>first1024Bytes</var> is null, then set
3432+
<var>first1024Bytes</var> to <var>bytes</var>.
3433+
3434+
<li><p><a for=TransformStream>Set up</a> <var>transformStream</var> with
3435+
<a for="TransformStream/set up"><i>transformAlgorithm</i></a> set to
3436+
<var>transformAlgorithm</var> and <a for="TransformStream/set up"><i>flushAlgorithm</i></a> set
3437+
to <var>flushAlgorithm</var>.
3438+
3439+
<li><p>Set <var>response</var>'s <a for=response>body</a>'s <a for=body>stream</a> to the result
3440+
of <var>response</var>'s <a for=response>body</a>'s <a for=body>stream</a>
3441+
<a for=TransformStream>piped through</a> <var>transformStream</var>.
3442+
</ol>
3443+
3444+
<li><p>Wait until <var>first1024Bytes</var> is non-null or <var>response</var>'s
3445+
<a for=response>body</a>'s <a for=body>stream</a> is <a for=ReadableStream>errored</a>.
3446+
3447+
<li><p>If <var>first1024Bytes</var> is null, then return failure.
3448+
3449+
<li>Return <var>first1024Bytes</var>.
3450+
</ol>
3451+
3452+
<hr>
3453+
3454+
<p>To <dfn>determine if response is JavaScript and not JSON</dfn> given a <a for=/>response</a>
3455+
<var>response</var>, run these steps:</p>
3456+
3457+
<ol>
3458+
<li><p>Let <var>responseBodyBytes</var> be null.
3459+
3460+
<li>
3461+
<p>Let <var>processBody</var> given a <a for=/>byte sequence</a> <var>bytes</var> be these steps:
3462+
3463+
<ol>
3464+
<li><p>Set <var>responseBodyBytes</var> to <var>bytes</var>.
3465+
3466+
<li><p>Set <var>response</var>'s <a for=response>body</a> to the <a for="body with type">body</a>
3467+
of the result of <a for=BodyInit>safely extracting</a> <var>bytes</var>.
3468+
</ol>
3469+
3470+
<li><p>Let <var>processBodyError</var> be this step: set <var>responseBodyBytes</var> to failure.
3471+
3472+
<li><p><a>Fully read</a> <var>response</var>'s <a for=response>body</a> given <a>processBody</a>
3473+
and <var>processBodyError</var>.
3474+
3475+
<li><p>Wait for <var>responseBodyBytes</var> to be non-null.
3476+
3477+
<li><p>If <var>responseBodyBytes</var> is failure, then return false.
3478+
3479+
<li><p><a for=/>Assert</a>: <var>responseBodyBytes</var> is a <a for=/>byte sequence</a>.
3480+
3481+
<li>
3482+
<p>If <a>parse JSON bytes to a JavaScript value</a> given <var>responseBodyBytes</var> does not
3483+
throw, then return false. If it throws, catch the exception and ignore it.
3484+
3485+
<p class=note>If there is an exception, <var>response</var> is not JSON. If there is not, it is.
3486+
3487+
<li><p>Let <var>potentialMIMETypeForEncoding</var> be the result of <a>extracting a MIME type</a>
3488+
given <var>response</var>'s <a for=response>header list</a>.
3489+
3490+
<li>
3491+
<p>Let <var>encoding</var> be the result of <a>legacy extracting an encoding</a> given
3492+
<var>potentialMIMETypeForEncoding</var> and <var>request</var>'s
3493+
<a for=request>no-cors JavaScript fallback encoding</a>.
3494+
3495+
<p class=note>Equivalently to <a>fetch a classic script</a>, this ignores the
3496+
<a for="MIME type" lt=essence>MIME type essence</a>.
3497+
3498+
<li><p>Let <var>sourceText</var> be the result of <a for=/>decoding</a>
3499+
<var>responseBodyBytes</var> given <var>encoding</var>.
3500+
3501+
<li><p>If <a>ParseText</a>(<var>sourceText</var>, <a>Script</a>) returns a <a>Script Record</a>,
3502+
then return true.
3503+
<!-- Ideally HTML owns this so ECMAScript changes don't end up impacting Fetch. We could
3504+
potentially make this use "create a classic script" instead with some mock data. Maybe that is
3505+
better? -->
3506+
3507+
<li><p>Return false.
3508+
</ol>
3509+
3510+
3511+
<h4 id=orb-mime-type-sets>New MIME type sets</h4>
3512+
3513+
<p class=note>The definitions in this section are solely for the purpose of abstracting parts of the
3514+
<a>opaque-response-safelist check</a>. They are not suited for usage elsewhere.
3515+
3516+
<p>An <dfn>opaque-response-safelisted MIME type</dfn> is a <a>JavaScript MIME type</a> or a
3517+
<a for=/>MIME type</a> whose <a for="MIME type">essence</a> is "<code>text/css</code>" or
3518+
"<code>image/svg+xml</code>".
3519+
3520+
<p>An <dfn>opaque-response-blocklisted MIME type</dfn> is an <a>HTML MIME type</a>,
3521+
<a>JSON MIME type</a>, or <a>XML MIME type</a>.
3522+
3523+
<p>An <dfn>opaque-response-blocklisted-never-sniffed MIME type</dfn> is a <a for=/>MIME type</a>
3524+
whose <a for="MIME type">essence</a> is one of:
3525+
3526+
<ul class=brief>
3527+
<li>"<code>application/gzip</code>"
3528+
<li>"<code>application/msexcel</code>"
3529+
<li>"<code>application/mspowerpoint</code>"
3530+
<li>"<code>application/msword</code>"
3531+
<li>"<code>application/msword-template</code>"
3532+
<li>"<code>application/pdf</code>"
3533+
<li>"<code>application/vnd.ces-quickpoint</code>"
3534+
<li>"<code>application/vnd.ces-quicksheet</code>"
3535+
<li>"<code>application/vnd.ces-quickword</code>"
3536+
<li>"<code>application/vnd.ms-excel</code>"
3537+
<li>"<code>application/vnd.ms-excel.sheet.macroenabled.12</code>"
3538+
<li>"<code>application/vnd.ms-powerpoint</code>"
3539+
<li>"<code>application/vnd.ms-powerpoint.presentation.macroenabled.12</code>"
3540+
<li>"<code>application/vnd.ms-word</code>"
3541+
<li>"<code>application/vnd.ms-word.document.12</code>"
3542+
<li>"<code>application/vnd.ms-word.document.macroenabled.12</code>"
3543+
<li>"<code>application/vnd.msword</code>"
3544+
<li>"<code>application/vnd.openxmlformats-officedocument.presentationml.presentation</code>"
3545+
<li>"<code>application/vnd.openxmlformats-officedocument.presentationml.template</code>"
3546+
<li>"<code>application/vnd.openxmlformats-officedocument.spreadsheetml.sheet</code>"
3547+
<li>"<code>application/vnd.openxmlformats-officedocument.spreadsheetml.template</code>"
3548+
<li>"<code>application/vnd.openxmlformats-officedocument.wordprocessingml.document</code>"
3549+
<li>"<code>application/vnd.openxmlformats-officedocument.wordprocessingml.template</code>"
3550+
<li>"<code>application/vnd.presentation-openxml</code>"
3551+
<li>"<code>application/vnd.presentation-openxmlm</code>"
3552+
<li>"<code>application/vnd.spreadsheet-openxml</code>"
3553+
<li>"<code>application/vnd.wordprocessing-openxml</code>"
3554+
<li>"<code>application/x-gzip</code>"
3555+
<li>"<code>application/x-protobuf</code>"
3556+
<li>"<code>application/x-protobuffer</code>"
3557+
<li>"<code>application/zip</code>"
3558+
<li>"<code>multipart/byteranges</code>"
3559+
<li>"<code>multipart/signed</code>"
3560+
<li>"<code>text/event-stream</code>"
3561+
<li>"<code>text/csv</code>"
3562+
</ul>
3563+
3564+
32733565

32743566
<h2 id=http-extensions>HTTP extensions</h2>
32753567

@@ -5232,19 +5524,23 @@ these steps:
52325524
<li><p>Set <var>response</var> and <var>internalResponse</var> to the result of running
52335525
<a>HTTP-network-or-cache fetch</a> given <var>fetchParams</var>.
52345526

5235-
<li>
5236-
<p>If <var>request</var>'s <a for=request>response tainting</a> is "<code>cors</code>" and a
5237-
<a>CORS check</a> for <var>request</var> and <var>response</var> returns failure, then return a
5238-
<a>network error</a>.
5527+
<li><p>If <var>request</var>'s <a for=request>response tainting</a> is "<code>opaque</code>",
5528+
<var>response</var>'s <a for=response>status</a> is not a <a>redirect status</a>, and the
5529+
<a>opaque-response-safelist check</a> given <var>request</var> and <var>response</var> returns
5530+
false, then return a <a>network error</a>.
52395531

5240-
<p class=note>As the <a>CORS check</a> is not to be applied to <a for=/>responses</a> whose
5241-
<a for=response>status</a> is 304 or 407, or <a for=/>responses</a> from a service worker for
5242-
that matter, it is applied here.
5532+
<li><p>If <var>request</var>'s <a for=request>response tainting</a> is "<code>cors</code>" and
5533+
the <a>CORS check</a> for <var>request</var> and <var>response</var> returns failure, then return
5534+
a <a>network error</a>.
52435535

52445536
<li><p>If the <a>TAO check</a> for <var>request</var> and <var>response</var> returns failure,
52455537
then set <var>request</var>'s <a for=request>timing allow failed flag</a>.
52465538
</ol>
52475539

5540+
<p class=note>As the <a>opaque-response-safelist check</a>, <a>CORS check</a>, and
5541+
<a>TAO check</a> are not to be applied to <a for=/>responses</a> whose <a for=response>status</a>
5542+
is 304 or 407, or to <a for=/>responses</a> from a service worker, they are applied here.
5543+
52485544
<li>
52495545
<p>If either <var>request</var>'s <a for=request>response tainting</a> or <var>response</var>'s
52505546
<a for=response>type</a> is "<code>opaque</code>", and the
@@ -9132,6 +9428,7 @@ Mohamed Zergaoui,
91329428
Mohammed Zubair Ahmed<!-- M-ZubairAhmed; GitHub -->,
91339429
Moritz Kneilmann,
91349430
Ms2ger,
9431+
Nathan Froyd,
91359432
Nico Schlömer,
91369433
Nicolás Peña Moreno,
91379434
Nidhi Jaju,

0 commit comments

Comments
 (0)