Skip to content

Commit dccee13

Browse files
authored
Improved CLI 2: explicit CLI flags for compaction settings (#7061)
Before: ```sh $ RERUN_CHUNK_MAX_ROWS=4096 RERUN_CHUNK_MAX_BYTES=1048576 rerun rrd compact /my/recordings/*.rrd -o output.rrd ``` After: ```sh $ rerun rrd compact --max-rows 4096 --max-bytes=1048576 /my/recordings/*.rrd -o output.rrd ``` ```sh $ rerun rrd compact --help Compacts the contents of one or more .rrd/.rbl files and writes the result to a new file. Uses the usual environment variables to control the compaction thresholds: `RERUN_CHUNK_MAX_ROWS`, `RERUN_CHUNK_MAX_ROWS_IF_UNSORTED`, `RERUN_CHUNK_MAX_BYTES`. Unless explicit flags are passed, in which case they will override environment values. Examples: * `RERUN_CHUNK_MAX_ROWS=4096 RERUN_CHUNK_MAX_BYTES=1048576 rerun rrd compact /my/recordings/*.rrd -o output.rrd` * `rerun rrd compact --max-rows 4096 --max-bytes=1048576 /my/recordings/*.rrd -o output.rrd` Usage: rerun rrd compact [OPTIONS] --output <dst.(rrd|rbl)> [PATH_TO_INPUT_RRDS]... Arguments: [PATH_TO_INPUT_RRDS]... Options: -o, --output <dst.(rrd|rbl)> --max-bytes <MAX_BYTES> What is the threshold, in bytes, after which a Chunk cannot be compacted any further? Overrides RERUN_CHUNK_MAX_BYTES if set. --max-rows <MAX_ROWS> What is the threshold, in rows, after which a Chunk cannot be compacted any further? Overrides RERUN_CHUNK_MAX_ROWS if set. --max-rows-if-unsorted <MAX_ROWS_IF_UNSORTED> What is the threshold, in rows, after which a Chunk cannot be compacted any further? This specifically applies to _non_ time-sorted chunks. Overrides RERUN_CHUNK_MAX_ROWS_IF_UNSORTED if set. -h, --help Print help (see a summary with '-h') ``` - Part of #7048
1 parent a07a680 commit dccee13

File tree

2 files changed

+41
-2
lines changed

2 files changed

+41
-2
lines changed

crates/top/rerun/src/commands/rrd/merge_compact.rs

+33
Original file line numberDiff line numberDiff line change
@@ -43,20 +43,53 @@ pub struct CompactCommand {
4343

4444
#[arg(short = 'o', long = "output", value_name = "dst.(rrd|rbl)")]
4545
path_to_output_rrd: String,
46+
47+
/// What is the threshold, in bytes, after which a Chunk cannot be compacted any further?
48+
///
49+
/// Overrides RERUN_CHUNK_MAX_BYTES if set.
50+
#[arg(long = "max-bytes")]
51+
max_bytes: Option<u64>,
52+
53+
/// What is the threshold, in rows, after which a Chunk cannot be compacted any further?
54+
///
55+
/// Overrides RERUN_CHUNK_MAX_ROWS if set.
56+
#[arg(long = "max-rows")]
57+
max_rows: Option<u64>,
58+
59+
/// What is the threshold, in rows, after which a Chunk cannot be compacted any further?
60+
///
61+
/// This specifically applies to _non_ time-sorted chunks.
62+
///
63+
/// Overrides RERUN_CHUNK_MAX_ROWS_IF_UNSORTED if set.
64+
#[arg(long = "max-rows-if-unsorted")]
65+
max_rows_if_unsorted: Option<u64>,
4666
}
4767

4868
impl CompactCommand {
4969
pub fn run(&self) -> anyhow::Result<()> {
5070
let Self {
5171
path_to_input_rrds,
5272
path_to_output_rrd,
73+
max_bytes,
74+
max_rows,
75+
max_rows_if_unsorted,
5376
} = self;
5477

5578
let mut store_config = ChunkStoreConfig::from_env().unwrap_or_default();
5679
// NOTE: We're doing headless processing, there's no point in running subscribers, it will just
5780
// (massively) slow us down.
5881
store_config.enable_changelog = false;
5982

83+
if let Some(max_bytes) = max_bytes {
84+
store_config.chunk_max_bytes = *max_bytes;
85+
}
86+
if let Some(max_rows) = max_rows {
87+
store_config.chunk_max_rows = *max_rows;
88+
}
89+
if let Some(max_rows_if_unsorted) = max_rows_if_unsorted {
90+
store_config.chunk_max_rows_if_unsorted = *max_rows_if_unsorted;
91+
}
92+
6093
merge_and_compact(&store_config, path_to_input_rrds, path_to_output_rrd)
6194
}
6295
}

crates/top/rerun/src/commands/rrd/mod.rs

+8-2
Original file line numberDiff line numberDiff line change
@@ -26,12 +26,18 @@ pub enum RrdCommands {
2626

2727
/// Compacts the contents of one or more .rrd/.rbl files and writes the result to a new file.
2828
///
29-
/// Use the usual environment variables to control the compaction thresholds:
29+
/// Uses the usual environment variables to control the compaction thresholds:
3030
/// `RERUN_CHUNK_MAX_ROWS`,
3131
/// `RERUN_CHUNK_MAX_ROWS_IF_UNSORTED`,
3232
/// `RERUN_CHUNK_MAX_BYTES`.
3333
///
34-
/// Example: `RERUN_CHUNK_MAX_ROWS=4096 RERUN_CHUNK_MAX_BYTES=1048576 rerun rrd compact /my/recordings/*.rrd -o output.rrd`
34+
/// Unless explicit flags are passed, in which case they will override environment values.
35+
///
36+
/// Examples:
37+
///
38+
/// * `RERUN_CHUNK_MAX_ROWS=4096 RERUN_CHUNK_MAX_BYTES=1048576 rerun rrd compact /my/recordings/*.rrd -o output.rrd`
39+
///
40+
/// * `rerun rrd compact --max-rows 4096 --max-bytes=1048576 /my/recordings/*.rrd -o output.rrd`
3541
Compact(CompactCommand),
3642

3743
/// Merges the contents of multiple .rrd/.rbl files, and writes the result to a new file.

0 commit comments

Comments
 (0)