Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the DRY dynamic N-gram anti-repetition sampler #982

Merged
merged 11 commits into from
Jul 13, 2024
5 changes: 5 additions & 0 deletions common/common.h
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,11 @@ struct gpt_params {
int32_t mirostat = 0; // 0 = disabled, 1 = mirostat, 2 = mirostat 2.0
float mirostat_tau = 5.00f; // target entropy
float mirostat_eta = 0.10f; // learning rate
float dry_multiplier = 0.0f; // penalty multiplier, 0.0 = disabled
float dry_base = 1.75f; // exponential base
int32_t dry_allowed_length = 2; // repeated sequences longer than this are penalized
int32_t dry_penalty_last_n = 0; // how many tokens to scan for repetitions (0 = entire context)
std::vector<std::string> dry_sequence_breakers; // DRY sequence breakers

// DynaTemp!
float dynatemp_range = 0.0f; // enables DynaTemp if greater than 0. dynatemp_min = temperature - dt_range, dynatemp_max = temperature + dt_range
Expand Down
6 changes: 6 additions & 0 deletions expose.h
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ const int stop_token_max = 16;
const int ban_token_max = 16;
const int tensor_split_max = 16;
const int logit_bias_max = 16;
const int dry_seq_break_max = 16;
const int images_max = 4;

// match kobold's sampler list and order
Expand Down Expand Up @@ -83,6 +84,11 @@ struct generation_inputs
const int mirostat = 0;
const float mirostat_eta = 0.0f;
const float mirostat_tau = 0.0f;
const float dry_multiplier = 0.0f;
const float dry_base = 0.0f;
const int dry_allowed_length = 0;
const int dry_penalty_last_n = 0;
const char * dry_sequence_breakers[dry_seq_break_max] = {};
const samplers sampler_order[KCPP_SAMPLER_MAX] = {};
const int sampler_len = 0;
const bool allow_eos_token = false;
Expand Down
Loading