Skip to content

server: accept inline voice_ref in task requests - #1252

Merged
0xShug0 merged 1 commit into
0xShug0:mainfrom
magnus-dev-melin:tasks-inline-voice-ref
Nov 10, 2026
Merged

0xShug0 merged 1 commit into
0xShug0:mainfrom
magnus-dev-melin:tasks-inline-voice-ref

Conversation

@magnus-dev-melin

Copy link
Copy Markdown
Contributor

/v1/tasks/run took voice_ref only as a server-local path, so a client sending a reference voice had to store it on the server first, for example the target speaker of a voice conversion, which may be a person's recording that shouldn't be written to disk. /v1/audio/speech already accepts it inline. Task requests now take the same shapes: a path string (unchanged), {"type": "path", "path": "..."}, or {"type": "base64", "data": "..."} with a WAV or data:audio/wav;base64,... URI of at most 5 MiB, the same limit /v1/audio/speech uses. Invalid values are request errors (HTTP 400). Previously an object voice_ref failed with "json value is not a string".

The change is in the shared request parser (app/cli/request.cpp), so CLI request files and the batch and stream endpoints accept it too; both server runtimes use that parser. Only voice_ref parsing changes: path requests behave as before. Docs in app/server/api.md.

Validation (macOS, M1 Max, Metal, chatterbox_q8_0):

  • cmake -S . -B build/pr -DCMAKE_BUILD_TYPE=Release -DENGINE_BUILD_EXTENDED_TESTS=ON -DAUDIOCPP_MODELS=chatterbox -DGGML_METAL=ON, then cli_request_options_test: the new test_request_json_voice_ref (path string, path object, inline base64, data URI, null, and rejections of a non-string/non-object, unknown or missing type, missing path/data, invalid base64, an empty payload and an over-limit payload) passes; on main it aborts with "json value is not a string".
  • Legacy server (audiocpp_server --config … --backend metal) with chatterbox_q8_0 as "task": "vc": /v1/tasks/run with audio_base64 plus voice_ref as inline base64, data URI, path object and path string all returned HTTP 200 and a 4.7 s 24 kHz WAV; {"type": "url"} and invalid base64 returned HTTP 400 with the messages above.

/v1/tasks/run (and CLI request files, batch and stream requests, which
share the request parser) took voice_ref only as a server-local path, so
a client had to store a reference voice on the server, for example the
target speaker of a voice conversion. voice_ref now takes the shapes
/v1/audio/speech takes: a path string, {"type": "path", "path"}, or
{"type": "base64", "data"} with a WAV or data URI of at most 5 MiB.
Invalid values are request errors (HTTP 400).
@0xShug0
0xShug0 merged commit 066ea98 into 0xShug0:main Nov 10, 2026
12 checks passed
@0xShug0

0xShug0 commented Nov 10, 2026

Copy link
Copy Markdown
Owner

@magnus-dev-melin Thanks for the PR! Merged.

One follow-up item: The PR description and API docs say invalid voice_ref values return HTTP 400. I sent valid Base64 that decoded to non-WAV bytes and got HTTP 500. Please return 400 for this case (preferred) or update the docs to reflect the current behavior.

@magnus-dev-melin

Copy link
Copy Markdown
Contributor Author

Thanks for merging, and good catch. Fixed in #1255: valid Base64 that isn't a WAV now returns HTTP 400 ("voice_ref base64 data is not a readable WAV (…)"), with a unit test that aborts on main, and the docs say exactly which voice_ref problems return 400.

magnus-dev-melin added a commit to stubbedev/conference that referenced this pull request Nov 11, 2026
The inline voice_ref and the conversion cache are merged upstream
(0xShug0/audio.cpp#1252, #1253, #1255), so no patched build is needed;
the cache saves 0.2-0.3 s a sentence on a quiet M1 Max.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants