Return a `content-encoding` header for resource timing and more #1796

guohuideng2024 · 2024-12-13T22:25:26Z

The major change is to add content-encoding to response header list. This PR also adds description on how content-encoding is determined. (content negotiation)

The purpose is to pass such value to resource timing. Further details are available at
w3c/resource-timing#381.

Note: Per discussion at 12/05/2024 webPerWG call (https://docs.google.com/document/d/1mpFDrAWuV6IgvJ1KiL9sgIlcboC5uArtF8r_oqS1Sco/edit?tab=t.0#heading=h.af6v74wysf4m), we decided to allow arbitrary "content-encoding" value at "fetch". We only filter such value at client side, before passing the value to resource timing.

Related PR to modify resource timing specification:
w3c/resource-timing#411

At least two implementers are interested (and none opposed):
Likely, Discussed and agreed at Feb 29, 2024 W3C WebPerf call, Chromium already receiving content-encoding through fetch. But not sure about other browsers.
Tests are written and can be reviewed and commented upon at:
[to be updated with a new link] https://chromium-review.googlesource.com/c/chromium/src/+/5958411
Implementation bugs are filed:
Chromium: https://issues.chromium.org/issues/327941462
Gecko: https://bugzilla.mozilla.org/show_bug.cgi?id=1886107
WebKit: https://bugs.webkit.org/show_bug.cgi?id=271632
Deno (not for CORS changes): …
MDN issue is filed:
New PerformanceResourceTiming.contentEncoding field mdn/content#32823
The top of this comment includes a clear commit message to use.

(See WHATWG Working Mode: Changes for more details.)

Bug: w3c/resource-timing#381

Preview | Diff

And some corrections.

fetch.bs

annevk · 2025-01-07T14:33:55Z

Thanks for taking the time to pick this up. However, it doesn't seem like this addresses all the issues with #1742? I recommend studying the feedback on that PR.

guohuideng2024 · 2025-01-14T21:55:15Z

Thanks for taking the time to pick this up. However, it doesn't seem like this addresses all the issues with #1742? I recommend studying the feedback on that PR.

Hi Anne! I think I should have put up some background information here.

You mentioned in Pass in Content-Encoding to resource-timing #1742 that the spec must define how the value is determined. This PR is trying to do that. The value is a result of the "content negotiation" (determine what encoding should be used) so I tried to add that into the existing text.
Note that Pass in Content-Encoding to resource-timing #1742 is a change similar to one for a previously added field contentType. But contentEncoding is very different, it's not an extracted MIME type, but a result of "content negotiation". So, this PR should be very different from Pass in Content-Encoding to resource-timing #1742
We originally thought that the filtering should happen at the "fetch" stage. But in the last web perf meeting Patrick brought up that the returned contentEncoding can be a proprietary value and that value is needed by service worker. So, the unfiltered value must be kept by the browser and the filtering should happen right before reported to resourceTiming.

https://docs.google.com/document/d/1mpFDrAWuV6IgvJ1KiL9sgIlcboC5uArtF8r_oqS1Sco/edit?tab=t.0#heading=h.af6v74wysf4m

Therefore, in this fetch doc I didn't mention filtering. I mentioned "filtering" in the resourceTiming spec:
w3c/resource-timing#411
And I am going to add more details about the filtering there.

Does this sound right to you? I am new to fetch and I may have missed a lot of things here. Thanks for your patience and guidance.
Guohui

This CL introduce a contentEncoding field to Performance resource timing object. This field is behind a feature flag. PR to resource timing specification: w3c/resource-timing#411 PR to fetch specification: whatwg/fetch#1796 Bug: 327941462 Change-Id: I70cad190fe658fb3dbf8b401ff8393bc1d0782f0

This CL introduce a contentEncoding field to Performance resource timing object. This field is behind a feature flag. PR to resource timing specification: w3c/resource-timing#411 PR to fetch specification: whatwg/fetch#1796 Bug: 327941462 Change-Id: I70cad190fe658fb3dbf8b401ff8393bc1d0782f0 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6098321 Commit-Queue: Guohui Deng <[email protected]> Reviewed-by: Noam Rosenthal <[email protected]> Reviewed-by: Matthew Denton <[email protected]> Reviewed-by: Yoav Weiss (@Shopify) <[email protected]> Cr-Commit-Position: refs/heads/main@{#1407331}

fetch.bs

…ourceTiming, a=testonly Automatic update from web-platform-tests Expose contentEncoding in PerformanceResourceTiming This CL introduce a contentEncoding field to Performance resource timing object. This field is behind a feature flag. PR to resource timing specification: w3c/resource-timing#411 PR to fetch specification: whatwg/fetch#1796 Bug: 327941462 Change-Id: I70cad190fe658fb3dbf8b401ff8393bc1d0782f0 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6098321 Commit-Queue: Guohui Deng <[email protected]> Reviewed-by: Noam Rosenthal <[email protected]> Reviewed-by: Matthew Denton <[email protected]> Reviewed-by: Yoav Weiss (@Shopify) <[email protected]> Cr-Commit-Position: refs/heads/main@{#1407331} -- wpt-commits: 1df2c3e47bcb6379ecf3a07735bd967101d02a5b wpt-pr: 50115

1) formatting; 2) "gzip, GZIP" is ok for they case-insensitive match. 3) there is a mistake saying that the "contentEncoding" consists of digits; 4) no longer returns "contentEncoding" for data url.

fetch.bs

That's on the client side getting the reponse header.

fetch.bs

Just add the content encoding to body info.

guohuideng2024 · 2025-01-22T22:22:18Z

Updated the patch, I just added the content encoding to the body info struct, and add the clause that updates it.

fetch.bs

restore a new line.

guohuideng2024 · 2025-01-23T21:31:12Z

very sorry for so many mistakes folks. Thanks for you guys' patence.

noamr · 2025-01-23T21:33:30Z

very sorry for so many mistakes folks. Thanks for you guys' patence.

No worries, we've all been there! (Or at least I have...)

…ourceTiming, a=testonly Automatic update from web-platform-tests Expose contentEncoding in PerformanceResourceTiming This CL introduce a contentEncoding field to Performance resource timing object. This field is behind a feature flag. PR to resource timing specification: w3c/resource-timing#411 PR to fetch specification: whatwg/fetch#1796 Bug: 327941462 Change-Id: I70cad190fe658fb3dbf8b401ff8393bc1d0782f0 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6098321 Commit-Queue: Guohui Deng <[email protected]> Reviewed-by: Noam Rosenthal <[email protected]> Reviewed-by: Matthew Denton <[email protected]> Reviewed-by: Yoav Weiss (@Shopify) <[email protected]> Cr-Commit-Position: refs/heads/main@{#1407331} -- wpt-commits: 1df2c3e47bcb6379ecf3a07735bd967101d02a5b wpt-pr: 50115

annevk · 2025-02-27T13:29:21Z

I think it should be specified here as that matches how we do MIME types and that reduces the chances of someone inadvertently exposing the information. In other words: the guarantee should come from Fetch, not from the caller.

guohuideng2024 · 2025-03-06T15:49:20Z

I think it should be specified here as that matches how we do MIME types and that reduces the chances of someone inadvertently exposing the information. In other words: the guarantee should come from Fetch, not from the caller.

The "raw" contentEncoding value can be arbitrary proprietary compression the app uses, and it's leaked as a response header.
So it's indeed a new communication channel that's created :(.

Meanwhile I think moving the filtering here guarantees that the only place where the raw contentEncoding is leaked is the fetch response header. I would say something here that contentEncoding is filtered when accessed anywhere else.

If there is any concern pls let me know. Thanks.

annevk · 2025-03-06T16:35:14Z

To be clear, the header is not exposed to the website passively embedding the resource, but this getter is. I don't think I understand your suggestion, could you rephrase?

noamr · 2025-03-07T08:47:54Z

I think it should be specified here as that matches how we do MIME types and that reduces the chances of someone inadvertently exposing the information. In other words: the guarantee should come from Fetch, not from the caller.

The "raw" contentEncoding value can be arbitrary proprietary compression the app uses, and it's leaked as a response header.
So it's indeed a new communication channel that's created :(.

Meanwhile I think moving the filtering here guarantees that the only place where the raw contentEncoding is leaked is the fetch response header. I would say something here that contentEncoding is filtered when accessed anywhere else.

If there is any concern pls let me know. Thanks.

Specifically, it needs to be explicitly filtered when assigned to the response body into struct.

It should be before copying "content-encoding" value to response bodyinfo.

guohuideng2024 · 2025-03-08T23:15:51Z

I think it should be specified here as that matches how we do MIME types and that reduces the chances of someone inadvertently exposing the information. In other words: the guarantee should come from Fetch, not from the caller.

The "raw" contentEncoding value can be arbitrary proprietary compression the app uses, and it's leaked as a response header.
So it's indeed a new communication channel that's created :(.
Meanwhile I think moving the filtering here guarantees that the only place where the raw contentEncoding is leaked is the fetch response header. I would say something here that contentEncoding is filtered when accessed anywhere else.
If there is any concern pls let me know. Thanks.

Specifically, it needs to be explicitly filtered when assigned to the response body into struct.

Got it, Thanks! I updated the PR accordingly.

guohuideng2024 · 2025-03-08T23:29:46Z

To be clear, the header is not exposed to the website passively embedding the resource, but this getter is. I don't think I understand your suggestion, could you rephrase?

I think the website can get the arbitrary value like this: (I am new to this area so please correct me if I am wrong)

let myCoding = myHeaders.get("Content-Encoding");  //  |myCoding| can be a proprietary compression, i.e., an arbitrary value.

And the reason for that is some use cases involving service workers. See
w3c/resource-timing#381

but the contentEncoding field in resourceTiming is filtered, where only a few pre-determined values are permitted.

noamr · 2025-03-11T14:31:19Z

To be clear, the header is not exposed to the website passively embedding the resource, but this getter is. I don't think I understand your suggestion, could you rephrase?

I think the website can get the arbitrary value like this: (I am new to this area so please correct me if I am wrong)
let myCoding = myHeaders.get("Content-Encoding");  //  |myCoding| can be a proprietary compression, i.e., an arbitrary value.

You would only get access to myHeaders if this is an actual fetch or via a service worker; Those channels are not always available.

And the reason for that is some use cases involving service workers. See w3c/resource-timing#381

but the contentEncoding field in resourceTiming is filtered, where only a few pre-determined values are permitted.

Yea, so filtering them when assigning to the struct wouldn't change anything observable, but any future user of that struct would get the filtered value.

guohuideng2024 · 2025-03-19T17:34:20Z

Thank you Noam!

@annevk : Would you take one more look? (I also left a response at WebKit/standards-positions#467 )

annevk

I noticed you're participating on behalf of Microsoft. That means you cannot sign the contributor's agreement as an individual. Microsoft has already signed up for the Fetch Workstream so you have to join the relevant GitHub organization (MicrosoftWHATWGContributors) and make your membership thereof public.

annevk · 2025-04-09T11:26:19Z

fetch.bs

@@ -6319,6 +6321,24 @@ optional boolean <var>forceNewConnection</var> (default false), run these steps:
         <li><p>Let <var>codings</var> be the result of <a>extracting header list values</a> given
         `<code>Content-Encoding</code>` and <var>response</var>'s <a for=response>header list</a>.

+         <li><p>Let <var>filteredCoding</var> be "<code>unknown</code>".


Why is this a distinct value from the empty string? It also seems to squat on the value space of the registry, which doesn't seem ideal?

unknown means there is a compression that's not recognized by the browser
empty string means there is no compression.
We would like to distinguish the two. In this discussion thread: w3c/resource-timing#381, nhelfman points out that the two situations need to be distinguishable.

Okay, what about the value space concern?

Also, at the moment "unknown" is also used when the header could not be parsed. Maybe that's okay. Is that tested?

Could you help me understand the "value space concern" problem?

Yes, I made two test cases where the contentEncoding value is filtered to unknown value.
https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/external/wpt/resource-timing/content-encoding.https.html;l=36

Well, in principle someone could register unknown as a content coding and user agents could implement it, but this API would not be able to distinguish it from the unknown case.

Ah I see! I think a solution is to make unknown a reserved word and it cannot be used as contentEncoding in the response header. There is already one reserved word identity.

Do you think I should try to make that happen? I couldn't find how to propose changes to that iana "http parameters". There is a "contact" section on that page but the emails are obviously out of date. (they are @sun.com :) )

Can you email Mark Nottingham [email protected] and copy [email protected] (that's me) with the information stated at https://httpwg.org/specs/rfc9110.html#content.coding.extensibility?

Do you think I should try to make that happen? I couldn't find how to propose changes to that iana "http parameters". There is a "contact" section on that page but the emails are obviously out of date. (they are @sun.com :) )

These are contacts for existing registrations; not the contact point for new registrations. Blame IANA for the confusion.

I sent email to Mark (and copied Anne).

fetch.bs

annevk · 2025-04-09T11:26:53Z

fetch.bs

+         <li><p>Otherwise, if <var>codings</var> contains two strings or more, set <var>filteredCoding</var> to
+         "<code>multiple</code>".


then set*

And this should probably compare with Infra's size concept.

added "then". But I am not sure how to "compare with infra's size concept". I looked at the "infra" section but I didn't see a title related to "size". Would you please help me understand what else needs to be done with this paragraph? Thanks.

https://infra.spec.whatwg.org/#list-size

I see! I made changes accordingly.

fetch.bs

annevk · 2025-04-09T11:29:29Z

fetch.bs

+
+         <li><p>Otherwise, if <var>codings[0]</var> is the empty string, or it is supported by the user agent,
+         and is listed in the <a href="https://www.iana.org/assignments/http-parameters/http-parameters.xhtml#content-coding">
+         content encoding registry on IANA</a>, set <var>filteredCoding</var> to <var>codings[0]</var>.


You want <cite>HTTP Content Coding Registry</cite> inline and then use a reference for the actual URL.

(And also apply the earlier comments.)

This isn't quite done. The casing is incorrect and you didn't move the URL into the references section.

Oh, I see another example of reference, so I did the same: with an inline link reference in place, I also made an entry in the "reference section".

The reference is not in Specref yet, so I submitted this PR:
tobie/specref#860

I cannot verify this yet because the specref PR is not merged.
Does this look correct to you?

It doesn't have to be in specref, you can also modify <pre class=biblio> in this document.

Done. Do you think I should withdraw the PR to the specref? Thanks.

fetch.bs

annevk · 2025-04-09T11:37:07Z

Now with the filtering in place it's probably slightly more reasonable to leave the existing parsing issue unsolved for now, assuming there's adequate test coverage.

guohuideng2024 · 2025-04-10T20:47:51Z

Thanks @annevk! I am working on MicrosoftWHATWGContributors membership right now.

annevk · 2025-04-11T07:02:28Z

fetch.bs

@@ -6319,6 +6321,24 @@ optional boolean <var>forceNewConnection</var> (default false), run these steps:
         <li><p>Let <var>codings</var> be the result of <a>extracting header list values</a> given
         `<code>Content-Encoding</code>` and <var>response</var>'s <a for=response>header list</a>.

+         <li><p>Let <var>filteredCoding</var> be "<code>unknown</code>".


Okay, what about the value space concern?

annevk · 2025-04-11T07:04:06Z

fetch.bs

+
+         <li><p>Otherwise, if <var>codings</var>[0] is the empty string, or it is supported by the user agent,
+         and is listed in the <a href="https://www.iana.org/assignments/http-parameters/http-parameters.xhtml#content-coding">
+         <cite>content encoding registry on IANA</cite></a>, then set <var>filteredCoding</var> to <var>codings</var>[0].


What if the server specifies "GZIP"? Does it get lowercased?

Yes all output should be lowercased and I made changes in text accordingly. Thanks for pointing out.

In the current text, a value is "allowed" as long as it "case insensitive matches" a registered value. And it will be lowercased before exposed. Is it O.K.?

Thanks.

annevk · 2025-04-15T09:48:42Z

fetch.bs

+
+         <li><p>If <var>codings</var> is null, then set <var>filteredCoding</var> to the empty string.
+
+         <li><p>Otherwise, if <var>codings</var>'s <a for=list>size</a> is 2 or more, then set


Suggested change

<li><p>Otherwise, if <var>codings</var>'s <a for=list>size</a> is 2 or more, then set

<li><p>Otherwise, if <var>codings</var>'s <a for=list>size</a> is greater than 1, then set

annevk · 2025-04-15T09:49:40Z

fetch.bs

+
+         <li><p>Set <var>response</var>'s <a for=response>body info</a>'s
+         <a for="response body info">content encoding</a> to the result of
+         <a lt=byte-lowercased>byte-lowercasing</a>  <var>filteredCoding</var>.


Suggested change

<a lt=byte-lowercased>byte-lowercasing</a> <var>filteredCoding</var>.

<a lt=byte-lowercased>byte-lowercasing</a> <var>filteredCoding</var>.

Done. This sentence is modified, and the superfluous space is gone

annevk · 2025-04-15T09:51:00Z

fetch.bs

+         <li><p>Otherwise, if <var>codings</var>[0] is the empty string, or it is supported by the user agent,
+         and is a <a>byte-case-insensitive</a> match for an entry listed in the
+         <a href="https://www.iana.org/assignments/http-parameters/http-parameters.xhtml#content-coding">
+         <cite>HTTP Content Coding Registry</cite></a> of [[!IANA-HTTP-PARAMS]], then set
+         <var>filteredCoding</var> to <var>codings</var>[0].


I think it would be clearer if you did the lowercasing here. There's no reason to lowercase the other branches.

Good idea. Done.

guohuideng2024 added 3 commits December 11, 2024 15:38

Add Content Encoding response header to fetch.

2c939cb

Add more about content-encoding

dc2b573

And some corrections.

Remove a trailing space.

7647a50

guohuideng2024 mentioned this pull request Dec 13, 2024

Expose a filtered content-encoding value to resource timing w3c/resource-timing#411

Open

content-encoding always available

f7bdfa9

guohuideng2024 marked this pull request as ready for review December 17, 2024 19:37

noamr mentioned this pull request Jan 2, 2025

Provide clear guidelines for what Timing-Allow-Origin should expose w3c/webappsec#664

Open

noamr reviewed Jan 7, 2025

View reviewed changes

fetch.bs Outdated Show resolved Hide resolved

fetch.bs Outdated Show resolved Hide resolved

fetch.bs Outdated Show resolved Hide resolved

Fix indentation and the "let" in the middle of sentence.

94ecd4c

chromium-wpt-export-bot mentioned this pull request Jan 16, 2025

Expose contentEncoding in PerformanceResourceTiming web-platform-tests/wpt#50115

Merged

annevk reviewed Jan 17, 2025

View reviewed changes

fetch.bs Outdated Show resolved Hide resolved

fetch.bs Outdated Show resolved Hide resolved

fetch.bs Outdated Show resolved Hide resolved

fetch.bs Outdated Show resolved Hide resolved

guohuideng2024 and others added 2 commits January 21, 2025 15:54

Merge branch 'whatwg:main' into main

c8a0981

Fix a number of issues:

a007614

1) formatting; 2) "gzip, GZIP" is ok for they case-insensitive match. 3) there is a mistake saying that the "contentEncoding" consists of digits; 4) no longer returns "contentEncoding" for data url.

noamr reviewed Jan 22, 2025

View reviewed changes

fetch.bs Outdated Show resolved Hide resolved

fetch.bs Outdated Show resolved Hide resolved

fetch.bs Outdated Show resolved Hide resolved

fetch.bs Outdated Show resolved Hide resolved

Remove spurious new lines and correct a paragraph

46c10a9

That's on the client side getting the reponse header.

noamr reviewed Jan 22, 2025

View reviewed changes

fetch.bs Outdated Show resolved Hide resolved

Simply

a963008

Just add the content encoding to body info.

noamr reviewed Jan 23, 2025

View reviewed changes

fetch.bs Outdated Show resolved Hide resolved

fetch.bs Outdated Show resolved Hide resolved

fetch.bs Outdated Show resolved Hide resolved

Fix: content-encoding not prohibited header.

16f1d1f

restore a new line.

This was referenced Mar 4, 2025

expose contentEncoding in resourceTiming WebKit/standards-positions#467

Open

Expose contentEncoding in resourceTiming mozilla/standards-positions#1189

Open

Expose contentEncoding in resourceTiming w3ctag/design-reviews#1064

Open

Add the text about "filtering Content-Encoding".

3c5eb46

guohuideng2024 marked this pull request as draft March 7, 2025 23:34

Guohui Deng and others added 5 commits March 7, 2025 16:55

Move the filtering text.

f41fa1f

It should be before copying "content-encoding" value to response bodyinfo.

restore a deleted empty line.

17cdde2

Add "Otherwise"

16ac25e

Add an empty line.

af1709e

Change "content-type" in "body info" to a string.

f121d66

guohuideng2024 marked this pull request as ready for review March 8, 2025 23:14

annevk reviewed Apr 9, 2025

View reviewed changes

fix a few errors.

28fd341

annevk reviewed Apr 11, 2025

View reviewed changes

guohuideng2024 mentioned this pull request Apr 14, 2025

Add the iana "http parameters" reference. tobie/specref#860

Merged

Fix a few issues.

7dd1eba

annevk reviewed Apr 15, 2025

View reviewed changes

Guohui Deng added 2 commits April 15, 2025 09:50

Fix a few errors.

c450e00

Add IANA-HTTP-PARAMS reference

8b91820

whatwg deleted a comment from guohuideng2024 Apr 16, 2025

		<li><p>Otherwise, if <var>codings</var> contains two strings or more, set <var>filteredCoding</var> to
		"<code>multiple</code>".


		<li><p>If <var>codings</var> is null, then set <var>filteredCoding</var> to the empty string.

		<li><p>Otherwise, if <var>codings</var>'s <a for=list>size</a> is 2 or more, then set

	<a lt=byte-lowercased>byte-lowercasing</a> <var>filteredCoding</var>.
	<a lt=byte-lowercased>byte-lowercasing</a> <var>filteredCoding</var>.

Return a content-encoding header for resource timing and more #1796

Are you sure you want to change the base?

Return a content-encoding header for resource timing and more #1796

Conversation

guohuideng2024 commented Dec 13, 2024 • edited by pr-preview bot Loading

annevk commented Jan 7, 2025

guohuideng2024 commented Jan 14, 2025

guohuideng2024 commented Jan 22, 2025

guohuideng2024 commented Jan 23, 2025

noamr commented Jan 23, 2025

annevk commented Feb 27, 2025

guohuideng2024 commented Mar 6, 2025

annevk commented Mar 6, 2025

noamr commented Mar 7, 2025

guohuideng2024 commented Mar 8, 2025

guohuideng2024 commented Mar 8, 2025

noamr commented Mar 11, 2025

guohuideng2024 commented Mar 19, 2025

annevk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

guohuideng2024 Apr 14, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

annevk commented Apr 9, 2025

guohuideng2024 commented Apr 10, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Return a `content-encoding` header for resource timing and more #1796

Return a `content-encoding` header for resource timing and more #1796

guohuideng2024 commented Dec 13, 2024 •

edited by pr-preview bot

Loading

guohuideng2024 Apr 14, 2025 •

edited

Loading