Skip to content

Add Docker infrastructure for running TALES experiments - #38

Open
Marc-Alexandre Côté (MarcCote) wants to merge 8 commits into
mainfrom
macote/docker
Open

Add Docker infrastructure for running TALES experiments#38
Marc-Alexandre Côté (MarcCote) wants to merge 8 commits into
mainfrom
macote/docker

Conversation

@MarcCote

@MarcCote Marc-Alexandre Côté (MarcCote) commented May 8, 2026

Copy link
Copy Markdown
Contributor

Docker images (vLLM + SGLang):

  • docker/Dockerfile: All-in-one image based on vllm/vllm-openai:latest
  • docker/Dockerfile.sglang: Alternative image based on lmsysorg/sglang:latest
  • docker/entrypoint.sh: Unified entrypoint supporting both inference backends (SERVER_TYPE=vllm|sglang), activity-based startup timeout, parallel execution
  • docker/README.md: Comprehensive documentation with env var reference

Local runner:

  • run_experiment.sh: Launch experiments via local Docker with full CLI (--model, --server, --gpu-count, --envs, --wandb, etc.)
  • .dockerignore: Whitelist-based to minimize build context

Bug fixes:

  • agents/reasoning.py: Fix reasoning_effort parsed as string instead of int, add DeepSeek-V4 thinking tag extraction
  • benchmark.py: Handle incomplete WandB runs in continue-from logic
  • tales/token.py: Improve token counting robustness
  • tales/wandb_utils.py: Better handling of missing summary data

@MarcCote
Add Docker infrastructure for running TALES experiments
eed2765 
Docker images (vLLM + SGLang):
- docker/Dockerfile: All-in-one image based on vllm/vllm-openai:latest
- docker/Dockerfile.sglang: Alternative image based on lmsysorg/sglang:latest
- docker/entrypoint.sh: Unified entrypoint supporting both inference backends
  (SERVER_TYPE=vllm|sglang), activity-based startup timeout, parallel execution
- docker/README.md: Comprehensive documentation with env var reference

Local runner:
- run_experiment.sh: Launch experiments via local Docker with full CLI
  (--model, --server, --gpu-count, --envs, --wandb, etc.)
- .dockerignore: Whitelist-based to minimize build context

Bug fixes:
- agents/reasoning.py: Fix reasoning_effort parsed as string instead of int,
  add DeepSeek-V4 thinking tag extraction
- benchmark.py: Handle incomplete WandB runs in continue-from logic
- tales/token.py: Improve token counting robustness
- tales/wandb_utils.py: Better handling of missing summary data

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@MarcCote
Make <think> tag parsing generic for all models
76f0e27 
Replace model-specific checks (Qwen3, DeepSeek) with generic
detection: if <think> or </think> tags are found in the response,
extract thinking content regardless of model name. This fixes
MiniMax-M2.7 and any future model that uses <think> tags.

DeepSeek remains a special case only for the 'disable thinking'
mechanism (custom chat template). All other models use the standard
chat_template_kwargs approach.

Tested end-to-end with Qwen3-0.6B via vLLM: thinking content
correctly extracted to Thinking field, actions clean.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@MarcCote
Add DeepSeek-V4 chat template and fix V4 thinking support
a8577c1 
DeepSeek-V4-Pro ships without a Jinja chat template (uses custom
Python encoding). This caused vLLM to use a fallback template,
resulting in no thinking being triggered (thinking_tokens=0 on WandB).

Changes:
- Add docker/templates/deepseek_v4.jinja2: V4-compatible template
  with enable_thinking kwarg (same mechanism as Qwen3)
- Auto-detect V4 models in entrypoint.sh and pass --chat-template
  to both vLLM and SGLang servers
- Add CHAT_TEMPLATE env var for explicit override
- Move V4 from DeepSeek-R1-specific branch to generic branch in
  reasoning agent (uses chat_template_kwargs instead of custom
  chat template per-request)
- Update .dockerignore to include docker/templates/

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@MarcCote
Apply generic fixes from k8s branch
49d2fbe 
- agents/reasoning.py: enable_thinking=True for open models via vLLM/SGLang
- agents/walkthrough.py: simplify reset logic, return QUIT when exhausted
- docker/entrypoint.sh: remove deprecated --rope-scaling, restructure
  parallel execution (seeds sequential, games parallel within each seed)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@MarcCote
Fix: handle Mistral-native models that use vLLM reasoning parser
d9c365a 
Mistral models (Magistral, etc.) use a native tokenizer that doesn't
support chat_template_kwargs. Instead, thinking is enabled via vLLM's
--enable-reasoning --reasoning-parser mistral flags, which puts
thinking content in the 'reasoning' field of the API response.

Changes:
- Skip chat_template_kwargs for Mistral-native models
- Use non-streaming for Mistral (streaming combine_chunks loses reasoning)
- Extract reasoning from response.json() choices[0].message.reasoning
- Accept responses with reasoning but empty content in retry loop
- Wrap extracted reasoning in <think> tags for uniform downstream handling

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@MarcCote
Fix: handle Mistral-native tokenizer models (Magistral)
a5d4626 
Magistral models use a Mistral native tokenizer that doesn't support
Jinja2 chat_template_kwargs. They produce <think>...</think> tags in
content when the system prompt includes thinking instructions.

Changes:
- Skip chat_template_kwargs for MISTRAL_NATIVE_MODELS
- Add THINKING_INSTRUCTION suffix to system prompt for these models
- No reasoning parser needed — existing think-tag parser handles them
- Update entrypoint.sh to use --reasoning-config for vLLM 0.20.0

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@MarcCote
Fix: thinking extraction for Mistral-native models
694d34f 
- Extract thinking from unclosed <think> blocks (token budget exceeded)
- Keep truncated thinking stub in history so model continues reasoning

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@MarcCote
Sync relevant changes from macote/k8s branch
f833229 
Brings in all non-infrastructure improvements from the k8s branch:

- agents/reasoning.py: Nemotron prompt-based thinking, Gemma4/Qwen3.5
  thinking extraction fixes, generic <think> tag parsing, default
  cot-max-tokens, HF tokenizer fix for vLLM-served models
- docker/entrypoint.sh: Auto-detect reasoning parser, auto-switch to
  reasoning agent when REASONING_EFFORT is set, sanity check before
  benchmark, export SERVER_URL/REASONING_PARSER
- scripts/sanity_check.py: New thinking trace verification script
- tales/token.py: Token counting improvements
- agents/random.py: Minor cleanup
- .dockerignore: Add scripts/ exclusion fix
- MODEL_CANDIDATES.md: Model evaluation notes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8221a5ca-bf6d-4071-9edf-550c8897650f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@christopherzc Christopher Cui (christopherzc)

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@MarcCote @christopherzc