diff --git a/README.md b/README.md index 7c4ba9c..a3b7db3 100644 --- a/README.md +++ b/README.md @@ -1,121 +1,97 @@ # BRRO Compressor -Version: 0.5 Released: 30/11/2023 - -## Major Changes - -### 0.5 - -- Added Polynomial Compressor (with 2 variants) -- Created and Integrated a proper file type (wbro) -- Benchmarks of the different compressors -- Integration testing -- Several fixes and cleanups - ## Description +**BRRO Compressor** is an advanced signal-based compression tool designed to provide better compression ratios than traditional methods. It performs an initial analysis of the input signal to select the best compression technique for each segment. -BRRO Compressor is a compressor that relies on the characteristics of a signal to provide a far greater compression that currently existing ones. BRRO relies on different techniques based on a initial analysis of the signal to use the best suited method for compressor for that specific signal segment. - -For a detailed description on the compressor methods and logic check `BRRO.md`. - -## Getting Started with BRRO Compressor +For detailed information on the compression methods used, see [BRRO.md](paper/BRRO.md). +## Major Changes +### Version 0.5 - Released 30/11/2023 +- Added Polynomial Compressor (with 2 variants). +- Introduced `wbro` file type. +- Added benchmarks for various compressors. +- Integration testing implemented. +- Various fixes and code cleanups. + +## Getting Started ### Prerequisites - -- Ensure you have [Rust](https://www.rust-lang.org/tools/install) and Cargo installed on your system. +- Install Rust and Cargo on your system. ### Installation - 1. Clone the repository: - ```bash - git clone https://github.com/instaclustr/fft-compression - cd fft-compression - ``` - + ```bash + git clone https://github.com/instaclustr/fft-compression + cd fft-compression + ``` 2. Build the project: - ```bash - cargo build --release - ``` - -### Usage + ```bash + cargo build --release + ``` -Currently BRRO relies on have Raw BRRO files generated by our prometheus remote endpoint. This would work as input for the compressor. - -Compressor usage: +## Usage +BRRO Compressor currently requires raw BRRO files generated by a Prometheus remote endpoint as input. +### Command-Line Options ``` Usage: brro-compressor [OPTIONS] Arguments: - input file + Input file Options: ---compressor [default: auto] [possible values: auto, fft, constant, polynomial] --u Uncompresses the input file/directory --h, --help Print help --V, --version Print version + --compressor + Select a compressor, default is auto [default: auto] [possible values: auto, noop, fft, constant, polynomial, idw] + -e, --error + Sets the maximum allowed error for the compressed data, must be between 0 and 50. Default is 5 (5%). 0 is lossless compression 50 will do a median filter on the data. In between will pick optimize for the error [default: 5] + -u + Uncompresses the input file/directory + -c, --compression-selection-sample-level + Samples the input data instead of using all the data for selecting the optimal compressor. Only impacts speed, might or not increased compression ratio. For best results use 0 (default). Only works when compression = Auto. 0 will use all the data (slowest) 6 will sample 128 data points (fastest) [default: 0] + --verbose + Verbose output, dumps everysample in the input file (for compression) and in the ouput file (for decompression) + -h, --help + Print help + -V, --version + Print version ``` -#### Compress a File - +### Compress a File To compress a file using the BRRO Compressor, run: - ```bash brro-compressor ``` -#### Decompress a File - -To decompress a file, use the following command: - +### Decompress a File +To decompress a file, use: ```bash brro-compressor -u ``` -## Programs and description - -This repository contains one main program and other programs that serve different purposes, some are for just testing, others do some actual work. - -### flac-server +## Programs and Descriptions -**NOTE**: Remote read is currently NOT working, as it depends on FLAC files that are no longer generated. +Requires a Prometheus server to collect samples. Supports read and write operations with Prometheus. -Needs a prometheus server. We need it to get our samples out. Supports read and write from prometheus. - -Launch the `flac-server` and set it as your remote endpoint for prometheus, example below. - -```YAML +Example configuration: +```yaml # Remote read and Write remote_write: - - url: "http://localhost:9201/api/write" + - url: "http://localhost:9201/api/write" remote_read: - - url: "http://localhost:9201/api/read" - read_recent: true - name: "flac_server" + - url: "http://localhost:9201/api/read" + read_recent: true + name: "flac_server" ``` - -Make Prometheus server a source of your grafana and check the data. - -### brro_optimizer - -Maybe the most important tool at this point, it picks a WAV file from the datasets described below and optimizes it into a way that we might see a meaningful compression into FLAC. -The tool also has options to dump the output of the file as a single sample per period, instead of the 4 channels. This is good to obtain the data as it was feed into the flac-server. -The code performs optimizations based on file name, so renaming might cause issues. - -Usage (Getting raw samples): `./brro_optimizer infile.wav --dump-raw > file.raw` -Usage (Getting optimized samples): `./brro_optimizer infile.wav --dump-optimized > file.raw` -Usage (Generate a optimized file): `./brro_optimizer -w infile.wav` - -If you set the ENV Variable for Debug it will output what it is doing. - -### Matlab folder - -Exploratory code. Should be removed. +Add the Prometheus server as a data source in Grafana to visualize the data. ## Roadmap +- Update `flac-server` to support reading and writing WBRO/BRO files. +- Implement streaming compression/decompression. +- Introduce automated compressor selection. +- Implement frame expansion (allowing new data to be appended to existing frames). + +## Contributing +Please see [CONTRIBUTING.md](CONTRIBUTING.md) for contribution guidelines. -1. Update `flac-server` to read/write WBRO/BRO files. -2. Streaming compression/decompression -3. Automated compressor selection -4. Frame expansion (Allowing new data to be appended to existing frames) +## License +This project is licensed under the [Apache License](LICENSE). \ No newline at end of file