[Handoff to @Oseltamivir Claude /loop] [Klaud Cold] Add qwen3.5-fp8-mi300x-sglang-mtp recipe

[functionstackx]

Summary

Adds the qwen3.5-fp8-mi300x-sglang-mtp recipe (yaml + launch script(s) + perf-changelog entry).

Test plan

• YAML loads; bash -n syntax passes on launch script(s).
• full-sweep-enabled sweep finishes green.

🤖 Generated with Claude Code

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://docs.sglang.io/cookbook/intro
• https://recipes.vllm.ai/

If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sglang/tree/main/docs_new

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://docs.sglang.io/cookbook/intro
• https://recipes.vllm.ai/

If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sglang/tree/main/docs_new

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26016444033
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=26016444033

[claude]

🔴 The new qwen3.5_fp8_mi300x_mtp.sh recipe has two copy-paste defects that together prevent it from running: (1) a misplaced backslash at lines 12-13 makes check_env_vars end after RESULT_FILENAME and then tries to execute a command literally named EP_SIZE, leaving $EP_SIZE unvalidated; and (2) a stray EP_SIZE \ at line 72 inside run_benchmark_serving is parsed as a positional argument that hits the function's wildcard case and aborts the benchmark. Fix by moving the backslash up to RESULT_FILENAME\ and dropping it after EP_SIZE (lines 12-13), and by deleting line 72 entirely — see the sibling qwen3.5_fp8_mi355x_mtp.sh for the correct structure.

Extended reasoning...

What is broken

The new script benchmarks/single_node/qwen3.5_fp8_mi300x_mtp.sh contains two distinct EP_SIZE-related copy-paste defects. Both can be seen by comparing against the sibling benchmarks/single_node/qwen3.5_fp8_mi355x_mtp.sh, which has the correct shape.

Defect 1 — broken check_env_vars call (lines 5-13).

In the new file:

check_env_vars \
    MODEL \
    ...
    RESULT_FILENAME       # <-- NO trailing backslash, terminates check_env_vars
    EP_SIZE \             # <-- starts a NEW command, backslash continues to blank line 14

In the (correct) mi355x sibling at lines 12-13:

    RESULT_FILENAME \
    EP_SIZE

Defect 2 — stray positional inside run_benchmark_serving (line 72).

run_benchmark_serving \
    ...
    --result-filename "$RESULT_FILENAME" \
    EP_SIZE \                     # <-- stray literal, no `$`, no flag
    --result-dir /workspace/ \
    --use-chat-template

The mi355x sibling has no such line.

Step-by-step proof of failure

Assume the harness invokes the script with EP_SIZE exported (every yaml entry sets ep: 1, so this is the intended happy path):

1. Line 5-12 — check_env_vars runs with arguments MODEL TP CONC ISL OSL RANDOM_RANGE_RATIO RESULT_FILENAME. All set, so it passes. EP_SIZE is never validated (a real but latent fail-fast regression for the unset-EP_SIZE case).
2. Line 13 — bash now parses EP_SIZE \ as its own statement. The trailing backslash continues onto the blank line 14, so the resulting command is literally EP_SIZE with no arguments. There is no such command on $PATH, so bash prints EP_SIZE: command not found. The script does not set -e (only set +x at the bottom), so execution silently continues.
3. Lines 15-60 — sglang starts and the wait completes (assuming EP_SIZE=1 was exported).
4. Line 62-74 — run_benchmark_serving is invoked. Its argument loop in benchmarks/benchmark_lib.sh (the second one starting around line 187) is a strict while [[ $# -gt 0 ]]; do case $1 in ... *) echo "Unknown parameter: $1"; return 1; ;; esac done. The preceding --result-filename "$RESULT_FILENAME" does a shift 2, so when the loop next sees $1 = "EP_SIZE" (the bare string from line 72), it hits the wildcard at lib.sh:279-282 and returns 1. The benchmark client never runs; no perf data is produced.
5. Lines 76-84 — set -e is not active, so the script continues into run_eval / stop_gpu_monitor and exits 0 from set +x. The CI job appears to succeed but produced no benchmark output.

Why the broken state is exactly the wrong way around

The EP_SIZE \ artifact appears in both locations (line 13 and line 72), strongly suggesting both were introduced by the same bad insertion. The sibling qwen3.5_fp8_mi355x_mtp.sh (lines 12-13 and 60-72) has neither artifact.

Fix

Three edits in benchmarks/single_node/qwen3.5_fp8_mi300x_mtp.sh:

1. Line 12: RESULT_FILENAME → RESULT_FILENAME \
2. Line 13: EP_SIZE \ → EP_SIZE (final argument to check_env_vars)
3. Delete line 72 ( EP_SIZE \) entirely — $EP_SIZE is already consumed by --ep-size $EP_SIZE on line 43 and has no business being passed to the benchmark client.

After these three edits, bash -n continues to pass and the recipe actually has a chance of producing benchmark results on the first sweep run.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26016446435
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=26016446435

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26185781844
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=26185781844