Support encoding spans with relative offsets #119302

Mark-Simulacrum · 2023-12-25T16:57:46Z

The relative offset is often smaller than the absolute offset, and with
the LEB128 encoding, this ends up cutting the overall metadata size
considerably (~1.5 megabytes on libcore). We can support both relative
and absolute encodings essentially for free since we already take a full
byte to differentiate between direct and indirect encodings (so an extra
variant is quite cheap).

rustbot · 2023-12-25T16:57:53Z

r? @WaffleLapkin

(rustbot has picked a reviewer for you, use r? to override)

Mark-Simulacrum · 2023-12-25T16:57:55Z

@bors try @rust-timer queue

bors · 2023-12-25T16:59:04Z

⌛ Trying commit 9aec3df with merge a32f9c5...

Encode spans with relative offsets The relative offset is always smaller than the absolute offset, and with the LEB128 encoding, this ends up cutting the overall metadata size considerably (~1.5 megabytes on libcore).

bors · 2023-12-25T18:25:55Z

☀️ Try build successful - checks-actions
Build commit: a32f9c5 (a32f9c5c445f63284f8834ef40bd32c67196a1a1)

Mark-Simulacrum · 2023-12-25T19:07:16Z

Pushed optionally-relative encoding -- it's strictly better than before now from a bytes-on-disk perspective, we can run another try + perf run but my sense is that it's probably not necessary if this one comes back clean.

rust-timer · 2023-12-25T21:05:30Z

Finished benchmarking commit (a32f9c5): comparison URL.

Overall result: ✅ improvements - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.5%	[-0.5%, -0.5%]	1
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.5%	[1.5%, 1.5%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-3.0%	[-3.6%, -2.5%]	2
All ❌✅ (primary)	-	-	0

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.6%	[1.6%, 1.6%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.6%	[1.6%, 1.6%]	1

Binary size

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.3%	[-4.6%, -0.1%]	115
Improvements ✅ (secondary)	-2.0%	[-4.0%, -0.0%]	59
All ❌✅ (primary)	-1.3%	[-4.6%, -0.1%]	115

Bootstrap: 670.163s -> 669.991s (-0.03%)
Artifact size: 312.56 MiB -> 312.44 MiB (-0.04%)

WaffleLapkin

r=me with nits
@rustbot author

WaffleLapkin · 2023-12-27T06:26:32Z

compiler/rustc_metadata/src/rmeta/encoder.rs

+            Entry::Occupied(o) => {
+                // If an offset is smaller than the absolute position, we encode with the offset.
+                // This saves space since smaller numbers encode in less bits.
+                let offset = s.opaque.position() - *o.get();


can this ever underflow? (if it can the code probably needs to account for that, if it can't, it would be nice to have a comment)

No, it shouldn't be possible. We write ~all metadata bytes strictly increasing positions in the buffer, so the current position must be larger than the position we saved into the HashMap.

I'll add a comment.

WaffleLapkin · 2023-12-27T06:28:20Z

compiler/rustc_metadata/src/rmeta/encoder.rs

+            Entry::Occupied(o) => {
+                // If an offset is smaller than the absolute position, we encode with the offset.
+                // This saves space since smaller numbers encode in less bits.
+                let offset = s.opaque.position() - *o.get();


since *o.get() is used multiple times, it would be nicer to have a variable for it like let &addr = o.get();.

The relative offset is often smaller than the absolute offset, and with the LEB128 encoding, this ends up cutting the overall metadata size considerably (~1.5 megabytes on libcore). We can support both relative and absolute encodings essentially for free since we already take a full byte to differentiate between direct and indirect encodings (so an extra variant is quite cheap).

Mark-Simulacrum · 2023-12-27T13:13:11Z

@bors r=WaffleLapkin

bors · 2023-12-27T13:13:13Z

📌 Commit 9c5293c has been approved by WaffleLapkin

It is now in the queue for this repository.

bors · 2023-12-27T13:48:11Z

⌛ Testing commit 9c5293c with merge 2f92fd7...

…ffleLapkin Support encoding spans with relative offsets The relative offset is often smaller than the absolute offset, and with the LEB128 encoding, this ends up cutting the overall metadata size considerably (~1.5 megabytes on libcore). We can support both relative and absolute encodings essentially for free since we already take a full byte to differentiate between direct and indirect encodings (so an extra variant is quite cheap).

rust-log-analyzer · 2023-12-27T14:17:33Z

The job dist-armv7-linux failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

bors · 2023-12-27T14:18:01Z

💔 Test failed - checks-actions

lqd · 2023-12-27T14:54:09Z

failed to get successful HTTP response from 'https://crates.io/api/v1/crates/bitmaps/2.1.0/download' (3.163.158.6), got 408

@bors retry network timeout issue

bors · 2023-12-27T14:55:21Z

⌛ Testing commit 9c5293c with merge 15755f3...

lqd · 2023-12-27T14:58:53Z

There's a crates.io outage, currently in recovery, so we'll see if it works this time.

bors · 2023-12-27T16:57:03Z

☀️ Test successful - checks-actions
Approved by: WaffleLapkin
Pushing 15755f3 to master...

rust-timer · 2023-12-27T18:19:27Z

Finished benchmarking commit (15755f3): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.2%	[1.2%, 1.2%]	1
Improvements ✅ (primary)	-0.7%	[-0.7%, -0.7%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.7%	[-0.7%, -0.7%]	1

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.0%	[0.6%, 1.5%]	3
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.0%	[0.6%, 1.5%]	3

Binary size

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.4%	[-4.9%, -0.1%]	115
Improvements ✅ (secondary)	-2.0%	[-4.4%, -0.0%]	58
All ❌✅ (primary)	-1.4%	[-4.9%, -0.1%]	115

Bootstrap: 671.868s -> 670.827s (-0.15%)
Artifact size: 312.43 MiB -> 312.33 MiB (-0.03%)

rustbot assigned WaffleLapkin Dec 25, 2023

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Dec 25, 2023

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Dec 25, 2023

This comment has been minimized.

Sign in to view

Mark-Simulacrum force-pushed the relative-spans branch from 9aec3df to 4b34a61 Compare December 25, 2023 19:01

Mark-Simulacrum changed the title ~~Encode spans with relative offsets~~ Support encoding spans with relative offsets Dec 25, 2023

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Dec 25, 2023

WaffleLapkin approved these changes Dec 27, 2023

View reviewed changes

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 27, 2023

Mark-Simulacrum force-pushed the relative-spans branch from 4b34a61 to 9c5293c Compare December 27, 2023 12:31

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Dec 27, 2023

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Dec 27, 2023

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 27, 2023

bors added the merged-by-bors This PR was explicitly merged by bors. label Dec 27, 2023

bors merged commit 15755f3 into rust-lang:master Dec 27, 2023

rustbot added this to the 1.77.0 milestone Dec 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support encoding spans with relative offsets #119302

Support encoding spans with relative offsets #119302

Mark-Simulacrum commented Dec 25, 2023 •

edited

Loading

rustbot commented Dec 25, 2023

Mark-Simulacrum commented Dec 25, 2023

This comment has been minimized.

bors commented Dec 25, 2023

bors commented Dec 25, 2023

This comment has been minimized.

Mark-Simulacrum commented Dec 25, 2023

rust-timer commented Dec 25, 2023

WaffleLapkin left a comment

WaffleLapkin Dec 27, 2023

Mark-Simulacrum Dec 27, 2023

WaffleLapkin Dec 27, 2023

Mark-Simulacrum Dec 27, 2023

Mark-Simulacrum commented Dec 27, 2023

bors commented Dec 27, 2023

bors commented Dec 27, 2023

rust-log-analyzer commented Dec 27, 2023

bors commented Dec 27, 2023

lqd commented Dec 27, 2023 •

edited

Loading

bors commented Dec 27, 2023

lqd commented Dec 27, 2023

bors commented Dec 27, 2023

rust-timer commented Dec 27, 2023

Support encoding spans with relative offsets #119302

Support encoding spans with relative offsets #119302

Conversation

Mark-Simulacrum commented Dec 25, 2023 • edited Loading

rustbot commented Dec 25, 2023

Mark-Simulacrum commented Dec 25, 2023

This comment has been minimized.

bors commented Dec 25, 2023

bors commented Dec 25, 2023

This comment has been minimized.

Mark-Simulacrum commented Dec 25, 2023

rust-timer commented Dec 25, 2023

Overall result: ✅ improvements - no action needed

WaffleLapkin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mark-Simulacrum commented Dec 27, 2023

bors commented Dec 27, 2023

bors commented Dec 27, 2023

rust-log-analyzer commented Dec 27, 2023

bors commented Dec 27, 2023

lqd commented Dec 27, 2023 • edited Loading

bors commented Dec 27, 2023

lqd commented Dec 27, 2023

bors commented Dec 27, 2023

rust-timer commented Dec 27, 2023

Overall result: no relevant changes - no action needed

Mark-Simulacrum commented Dec 25, 2023 •

edited

Loading

lqd commented Dec 27, 2023 •

edited

Loading