Skip to content
GitLab

fix off by one in API

Merged Administrator requested to merge github/fork/lilisierrayu/api_output_issue into main

Created by: lilisierrayu

Patch Description The prompt_len in current code relies on the batch_generator implementation and can easily introduce error. Remove that dependence in this PR.

Testing steps Describe how you tested your changes