Repository navigation
Add admin-configurable summary system prompt (SummaryProvider currently hardcodes it) #405
Description
Activity
Hi @khnjrdm,
For the max tokens you can it in the UI it is hidden under advanced input options.
Allowing admins to modify the prompt for the providers is a bit out of scope for this app and would result in hard to troubleshoot configurations. Also the next release is bringing out some more customization options for summarize (#387) and would be incompatible with allowing admins to change the prompt.
Thanks @lukasdotcom, and thanks for the screenshot — that clarifies which UI you meant.
On
max_tokens: the "advanced input options" you're showing is the Assistant Summarize task UI, where a admin is actively invoking summarize and can setmax words/tokensper call. That's distinct from the app-level default I actually had to change to make Talk recording summaries usable, which isAdmin → Connected accounts → Text generation → Max output tokens per request = 8000:
Confirming which of these the automated Talk flow actually reads would be useful either way, but that's not the core of this ticket.
The core of the ticket is the automated Talk recording → summary flow, and I don't think #387 or the advanced input options reach it. When someone finishes recording a Talk call,
spreedqueues acore:text2text:summaryTaskProcessing task from server-side code — there is no human sitting in front of the Assistant Summarize UI at that moment to drag amax words/tokensslider or a #387 complexity/length knob. The task fires with defaults;SummaryProvider::process()reads the app-level config, uses the hardcoded prompt, and writesRecording <ts> - summary.mdinto the recording folder.What the default prompt produces on that automated path (verbatim from
SummaryProvider.php):You are a helpful assistant that summarizes text in the same language as the text. You should only return the summary without any additional information.On a 60-minute meeting transcript (LiteLLM + Qwen 2.5 7B, also verified against GPT-4o-mini) this produces 2–4 sentences of generic narration, "The team discussed several topics including X and Y", with no attendees, no decisions, no action items, no next steps. It's the shape of a book blurb, not meeting minutes. For its actual use (internal recap someone reads the next morning), that's not useful.
What we're running in production after patching
SummaryProvider.php:You are analyzing a transcript, typically of a business meeting, workshop, or client call. Produce a structured summary in Markdown in the SAME LANGUAGE as the source text. Detect the language automatically; do not mention which language you detected. Include these sections when the transcript supports them: **Meeting overview** (2-3 sentences of context); **Attendees** (if identifiable); **Key topics discussed** (bullet points, 1-2 sentences each); **Decisions made** (concrete outcomes); **Action items** (who committed to what, by when if stated); **Open questions** (items requiring follow-up); **Next steps** (planned follow-ups or milestones). Omit sections that have no content. Do not fabricate action items, decisions, or attendees not present in the transcript. For non-meeting content or very short input, produce a proportionally shorter summary without imposing this structure.Same model, same transcript, this produces structured Markdown minutes with real attendees, decisions, and action items. The delta isn't stylistic, it's structural. Complexity + length are shape modifiers (how detailed, how long); they can't request a specific document structure. LLMs don't put attendees / decisions / action items into distinct sections unless the prompt asks for it, and a "detailed, long" summary of a meeting is still narrative rather than minutes.
Three smaller-scope options that sidestep the "admin can rewrite any prompt is a foot-gun" concern, any of these solves the automated Talk flow:
occ-only config, no UI:integration_openai.summary_system_promptviaconfig:app:setonly. Default = today's string; admins who override it have explicitly taken the risk. Same pattern asspreed.call_recording_transcription.- Move meeting-summary out of the generic path: let
spreed(Talk) pass a prompt hint in the TaskProcessing input when the summary source is a call recording. The caller knows it's a meeting so it asks for meeting-shaped output.integration_openaistays generic. - New task subtype:
core:text2text:recording-summarywith a meeting-oriented default in core. Bigger change but solves "generic summary is wrong for meetings" across the stack.
Happy to run comparative output on our stack (NC 33.0.6 + Talk 23.0.8 + LiteLLM + Qwen 2.5) against real recordings if that helps decide. Otherwise fine to close as won't-fix and keep patching locally, just wanted to make sure the ticket wasn't read as "let admins rewrite any prompt they want" when the pain is specifically automated Talk recording summaries.
Hi @khnjrdm ! I’d like to work on this issue.
I checked the current implementation.
SummaryProviderstill has the summary system prompt hardcoded, while max_tokens is already handled by the current Task Processing implementation (SummaryProvider.php).I’m planning to:
- add an admin-configurable
summary_system_promptsetting, using the current hardcoded prompt as the default. - expose it as a multiline textarea in the admin settings.
- use the configured prompt in
SummaryProvider, with a fallback to the default when empty. - preserve the existing format and complexity instructions.
Please let me know if this approach looks good before I start working on the PR.
Reacted by Kim Haverblad- add an admin-configurable
@Abhijeet-035 Yes, from my perspective that would make things easier to deal with and my own patches wouldn't be needed even if they've so far survived updates of Nextcloud.
Describe the feature you'd like to request
apps/integration_openai/lib/TaskProcessing/SummaryProvider.phpcurrently hardcodes the system prompt used for text-summarisation tasks:There is no admin UI or
config:app:setkey to override this prompt. The admin-visible Chat User Instructions for Chat Completions field applies only to the interactive chat-completion path, not to summarisation (which goes through TaskProcessing →SummaryProvider::process()on a separate code path). Themax_tokens = 1000default on this same code path is also very restrictive.Combined, this produces very short generic summaries. Talk call-recording summaries in particular miss most of the value of the transcript — no structure, no attendees, no action items, no decisions, no next steps.
Every deployment that wants better summaries — richer meeting summaries, per-team wording, per-language variants, non-English house style, industry-specific vocabulary — has to patch PHP directly. That patch is clobbered by every app update and has to be manually re-applied. This is fragile across a fleet and is a real papercut for any admin running
integration_openaiagainst a LocalAI-compatible endpoint (LiteLLM, Ollama, self-hosted OpenAI-shape LLMs).Environment where this was observed:
urlconfig points at the LiteLLM endpointDescribe the solution you'd like
Add a Summary system prompt text field to
Admin → Connected accounts → OpenAI and LocalAI integration(or wherever fits the app's existing settings UI). The field should:integration_openaiapp config under a stable key, e.g.summary_system_promptSummaryProvider::process()in place of the hardcoded string; fall back to the current default when the config key is emptySketch of the code change in
SummaryProvider.php:Code Change 2 (related, same file): consider also surfacing
max_tokensfor summarisation as a UI field. It's already tunable viaocc config:app:set integration_openai max_tokens --value=8000, but the pairing (prompt + token budget) is what actually determines summary quality — surfacing both in the same UI section makes them discoverable together.Broader shape (nice-to-have, out of scope for this ticket): the same "hardcoded prompt with no override" pattern likely applies to other
core:text2text:*task types (headline, chat-summary, etc.). A broader "per-task-type prompt templates" epic would be the ideal end state, but even just the Summary field would unblock the most common request today.Describe alternatives you've considered
Patching
SummaryProvider.phpdirectly on each server. Works, but the patch is clobbered on every app upgrade. Not maintainable across a fleet, and every admin who wants a better summary has to re-derive the same patch.Using the existing "Chat User Instructions for Chat Completions" field. That field only applies to the interactive chat path, not to TaskProcessing summarisation. Confirmed by inspecting
SummaryProvider::process()— the chat instructions config is not consulted there.Using a middleware proxy (LiteLLM, LocalAI itself, etc.) to rewrite the system prompt. LiteLLM can inject or replace system prompts on a per-model basis. This works, but pushes app-level configuration into infrastructure, is invisible to Nextcloud admins, and means every deployment needs a proxy in front of the LLM endpoint just for this one setting.
Waiting for a broader "per-task-type prompt templates" feature. Ideal end state, but shipping just the Summary override in the meantime would unblock most of the actual pain today.