Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • M metaseq
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 95
    • Issues 95
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 41
    • Merge requests 41
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Administrator
  • metaseq
  • Merge requests
  • !167

[api] Fix singular positive logit

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Administrator requested to merge fixposlogit into main Jun 21, 2022
  • Overview 5
  • Commits 2
  • Pipelines 0
  • Changes 3

Created by: stephenroller

Patch Description For a few weeks now, the API has consistently assigned the first logit after the prompt to be positive, which makes it an invalid probability.

Digging into it, it's because beam search keeps track of cumulative NLL (which makes sense for beam search). However, the first step of the beam search was being provided logits only for the newest token. As a result, the cumulative logit logic was assigning a "reset to 0" offset on the first one.

While this is a little bit kludgy, adding in a new one-time-only parameter, it's the best way to provide this information to the search algorithm without gutting it all.

This also makes the writing into the scores slightly more compact.

Note that generations do not change compared to previously. Just the bookkeeping of logits.

Testing steps Generations with and without topp; with and without batching. Confirmed generations for greedy stay the same.

Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: fixposlogit