Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • M metaseq
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 95
    • Issues 95
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 41
    • Merge requests 41
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Administrator
  • metaseq
  • Merge requests
  • !455

Add argument for max generated tokens

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Administrator requested to merge add_num_gen_tokens into opt_instruct Oct 26, 2022
  • Overview 2
  • Commits 1
  • Pipelines 0
  • Changes 2

Created by: sriniiyer

Added a new argument to restrict generation to a fixed number of tokens

Tested with

CUDA_VISIBLE_DEVICES=0,1 FSD=/fsx-mudslide/rpasunuru/data/instruct-opt/prompt_data/allbenchmarks_io_streaming_after_dedup_v5_sorted \
python metaseq_internal/scripts/eval/schedule_jobs_few_shot_instruct_opt.py \
-t opti_valid_data_to_text__unified_skg__totto__prompt0 \
-m 1.3B_gptz \
-o /fsx-mudslide/sviyer/tmp_100/ \
--slurm-partition instruct-opt \
--batch-size 8 --override-completed --n-eval-samples 100 --max-gen-tokens 100 --local
  • Confirmed that it is generating 100 tokens for each example with this argument, and without it, it generates > 1800 tokens per argument.
  • Conputed rouge metrics and 100-tokens = 0.0472 2048 tokens = 0.0047

Also putting out a PR for MSI

Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: add_num_gen_tokens