Use the generation config to control generation #410

katalinic-gc · 2023-06-07T09:18:35Z

What does this PR do?

Fixes # (issue)

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests?

HuggingFaceDocBuilderDev · 2023-06-07T09:22:46Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

…alidation

kundaMwiza

No complaints

katalinic-gc changed the title ~~Fix generation config~~ Use the generation config to control generation Jun 7, 2023

katalinic-gc added 7 commits June 8, 2023 15:30

Use generation config in generation utils

c9e6a33

Copy generation config when instantiating pipelined models

dfe2260

Style

f4270e9

Only ensure generation steps increase when KV caching is enabled

8ee2f62

Instead of erroring, override use_cache for models that dont support it

a0685b6

Use generation config when creating KV caches, add generation kwarg v…

fef9a7c

…alidation

Fix

4f6d694

katalinic-gc force-pushed the fix_generation_config branch from 9a7f19c to 4f6d694 Compare June 8, 2023 14:31

Fix generation tests

f018086

katalinic-gc merged commit 065d1d9 into huggingface:main Jun 8, 2023

katalinic-gc deleted the fix_generation_config branch June 8, 2023 16:15

kundaMwiza approved these changes Jun 8, 2023

View reviewed changes