Fix forwarding ARN sequence lengths from QairtPipelinePass to QairtEncapsulationPass#2526
Conversation
…tion QairtEncapsulation uses sequence_lengths from model_attributes to compute max_length in genai_config.json. QairtGenAIBuilder populates this via its own config, but QairtPipelinePass had no equivalent path. Extract the arn list from the genai_builder.transform_options in the YAML recipe and pass it through model_attributes so QairtEncapsulation can pick it up.
…-pipeline-context-lengths
Covers the case where arn is present in genai_builder.transform_options and forwarded as sequence_lengths in model_attributes, and the case where arn is absent and sequence_lengths is not set.
qti-kromero
changed the title
There was a problem hiding this comment.
Pull request overview
This PR ensures QAIRT pipeline recipes that specify AR decode lengths (stages.genai_builder.transform_options.arn) correctly forward that information through QairtPipelinePass via model_attributes["sequence_lengths"], enabling QairtEncapsulation to compute genai_config.json max_length consistently with the QairtGenAIBuilder path.
Changes:
- Forward
arnfrom the pipeline recipe intoQairtModelHandler.model_attributes["sequence_lengths"]inQairtPipelinePass. - Add unit tests verifying
arnis forwarded when present and omitted when absent.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
olive/passes/qairt/pipeline.py |
Extracts arn from recipe and forwards it as sequence_lengths via model_attributes for downstream encapsulation. |
test/passes/qairt/test_pipeline_pass.py |
Adds coverage for forwarding behavior and the no-arn behavior. |
…psulation QairtEncapsulation calls min(sequence_lengths) on the forwarded value, so a malformed YAML arn (string, bare int, empty list, mixed types) would fail with an opaque error. Raise a clear ValueError at the pipeline pass boundary instead.
|
@maintainers - this is ready for human review |
|
@xiaoyu-work would it be possible to get a reviewer assigned to this |
|
@xiaoyu-work it seems like there was an issue with the CI job |
|
hey @xiaoyu-work, sorry to bother you but it seems like there was another timeout failure on this CI job. not sure if this is intermittent or if there is some fix that needs to be made |
Yes there is a CI issue now. I'm trying to fix this. |
QairtEncapsulation uses sequence_lengths from model_attributes to compute max_length in genai_config.json. QairtGenAIBuilder populates this via its own config, but QairtPipelinePass had no equivalent path. Extract the arn list from the genai_builder.transform_options in the YAML recipe and pass it through model_attributes so QairtEncapsulation can pick it up.
Previously caused an issue in execution where context length would be exceeded and some kind of crash would occur
Describe your changes
Checklist before requesting a review
lintrunner -a(Optional) Issue link