dev: small r1 context for now

zhudotexe · zhudotexe · commit 2e8c1a70156e · 2025-02-17T18:28:33.000-05:00
diff --git a/sandbox/r1-quant.py b/sandbox/r1-quant.py
@@ -26,6 +26,7 @@
     repo_id="unsloth/DeepSeek-R1-GGUF",
     filename="DeepSeek-R1-Q2_K_XS/DeepSeek-R1-Q2_K_XS-00001-of-00005.gguf",
     prompt_pipeline=pipeline,
+    max_context_size=10000,
     model_load_kwargs={"n_gpu_layers": -1, "additional_files": []},
 )
 

Original file line number	Diff line number	Diff line change
`@@ -26,6 +26,7 @@`
`26`	`26`	`repo_id="unsloth/DeepSeek-R1-GGUF",`
`27`	`27`	`filename="DeepSeek-R1-Q2_K_XS/DeepSeek-R1-Q2_K_XS-00001-of-00005.gguf",`
`28`	`28`	`prompt_pipeline=pipeline,`
	`29`	`+ max_context_size=10000,`
`29`	`30`	`model_load_kwargs={"n_gpu_layers": -1, "additional_files": []},`
`30`	`31`	`)`
`31`	`32`