Skip to content

clean tokamax gmm and add test#4152

Merged
copybara-service[bot] merged 1 commit into
mainfrom
shuningjin-gmm-clean
Jun 12, 2026
Merged

clean tokamax gmm and add test#4152
copybara-service[bot] merged 1 commit into
mainfrom
shuningjin-gmm-clean

Conversation

@shuningjin

@shuningjin shuningjin commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

Description

  1. Refactor manual_axis_type conditional specification.
  • It was previously introduced by PR#3770 and PR#3869.
  • It requires tokamax > 0.12.0 (unreleased yet) with commit
  • It is used for gmm shard_map with check_vma=True. This is currently only activated for experimental run (deepseek_batchsplit and use_manual_quantization). Tests won't blocked by dependency.
  1. Add unit test for use_tokamax_gmm=True for smoke train. Both bf16 and fp8.
  • Smoke train instead of AOT as blocked by b/489205940
  • Run in github CI/CD as tpu_only
  1. Reset group offset to None: PR#4082 added group_offset to tokamax.ragged_dot and tokamax.ragged_dot_general. However, group_offset is not yet supported by tokamax:

Tests

unit test tokamax_test

Test on v5p-8: log

 pytest -v -m integration_test tests/integration/tokamax_test.py

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run end-to-end tests tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

@codecov

codecov Bot commented Jun 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 58.33333% with 5 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/maxtext/kernels/megablox/ops.py 66.66% 3 Missing ⚠️
src/maxtext/layers/quantizations.py 33.33% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

Comment thread tests/integration/tokamax_gmm_test.py Outdated
@shuningjin shuningjin force-pushed the shuningjin-gmm-clean branch 2 times, most recently from c8af3f1 to 0ebbe64 Compare June 12, 2026 16:31
Comment thread src/maxtext/layers/quantizations.py Outdated
Comment thread src/maxtext/kernels/megablox/ops.py
@shuningjin shuningjin force-pushed the shuningjin-gmm-clean branch 2 times, most recently from 88e561b to f71379c Compare June 12, 2026 19:28
@shuningjin shuningjin force-pushed the shuningjin-gmm-clean branch from f71379c to b88c6ac Compare June 12, 2026 19:46
@copybara-service copybara-service Bot merged commit 912471e into main Jun 12, 2026
56 of 58 checks passed
@copybara-service copybara-service Bot deleted the shuningjin-gmm-clean branch June 12, 2026 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants