You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Running quantize with a target dtype of F32, F16, or Q8_0 can result in a Q6_K output tensor without --pure (ref #5631 (comment)). This is surprising, as I would expect converting to F32 and then quantizing to F16 to produce similar results to converting directly to F16.
I suggest that the k-quant mixture logic should never attempt to decrease the quality of the output tensor, only increase it.
The text was updated successfully, but these errors were encountered:
Running quantize with a target dtype of F32, F16, or Q8_0 can result in a Q6_K output tensor without --pure (ref #5631 (comment)). This is surprising, as I would expect converting to F32 and then quantizing to F16 to produce similar results to converting directly to F16.
I suggest that the k-quant mixture logic should never attempt to decrease the quality of the output tensor, only increase it.
The text was updated successfully, but these errors were encountered: