Skip to content

ProjectOS_Technical_Breakdown_Revised.md

Latest commit

 

History

History
155 lines (103 loc) · 13 KB

ProjectOS_Technical_Breakdown_Revised.md

File metadata and controls

155 lines (103 loc) · 13 KB
Raw

ProjectOS — Technical Breakdown

A module-by-module plan for a Python team building this at a hackathon. Each section covers: the problem, the solution approach, the concrete tech stack, how it talks to the rest of the system, and what wasn't spelled out in the brief but needs an answer.

0. System shape (read this first)

ProjectOS is really two products sharing one brain:

  1. Project Management Engine — manages requirements, user stories, sprints, meetings, and risks.
  2. Knowledge Hub + Onboarding Assistant — turns a live codebase + its tools into a queryable brain for new devs.

Both are powered by the same core pattern: ingest → chunk/embed → retrieve → generate (LLM) → structure output. This is just RAG (Retrieval-Augmented Generation) applied twice, to different project knowledge sources. Build the RAG pipeline once as a shared service; point it at different data sources for the two halves. This is the single most important architectural decision — don't let two sub-teams build two separate retrieval stacks.

Suggested core stack (shared across everything):

  • Backend: FastAPI (Python) — async, fast to build, good for streaming LLM responses via SSE/websockets.
  • LLM: Claude API (Sonnet for most tasks, Haiku for cheap/fast classification steps).
  • Orchestration: LangChain or LlamaIndex for RAG plumbing, or hand-roll it (often faster to debug at hackathon scale — LlamaIndex/LangChain abstractions can cost you more time than they save in 24-48h).
  • Vector store: Chroma (embedded, zero-ops, perfect for a hackathon) or Qdrant if you want something more production-y but still self-hostable.
  • Embeddings: voyage-code-2 (good for code) or OpenAI text-embedding-3-small for general text; Anthropic doesn't yet serve embeddings directly so you'll pair Claude generation with a separate embedding provider.
  • Relational store: Postgres (projects, users, sprints, tasks, permissions) — SQLAlchemy + Alembic for migrations.
  • Task queue: Celery + Redis, or for a hackathon, just asyncio background tasks / FastAPI BackgroundTasks — full Celery is probably overkill unless ingestion is slow.
  • Frontend: React (or just ship API + a thin Streamlit/Gradio UI if your team is backend-heavy — very common hackathon shortcut).
  • Auth: OAuth2 via GitHub/Microsoft (since you're integrating GitHub + Azure DevOps anyway, reuse their OAuth instead of building your own login).

3. User Stories & Acceptance Criteria Generation

Problem: Translating requirements into well-formed user stories ("As a... I want... so that...") with testable acceptance criteria is repetitive and quality varies by writer.

Solution: For each requirement/work package, prompt the LLM to generate:

  • A set of user stories in standard format.
  • Acceptance criteria per story, in Given/When/Then (Gherkin) format — this is valuable because it's directly testable and pluggable into BDD test frameworks later.
  • INVEST-checklist self-critique pass: have a second LLM call review the generated stories against the INVEST criteria (Independent, Negotiable, Valuable, Estimable, Small, Testable) and flag/fix violations. This "generate then critique" two-pass pattern noticeably improves quality and is cheap to add.

Tech: Claude, structured JSON output again (story id, title, description, acceptance criteria list, story points estimate). Optionally push directly into Azure DevOps / Jira / GitHub Issues via their APIs once generated — this is a great demo moment ("click button, stories appear in your backlog").

Integration: Stories reference the parent work package from step 2. They become the atomic unit Sprint Planning (step 4) allocates into sprints, and what Risk Assessment (step 6) scans for ambiguity/risk flags.

4. Sprint Planning

Problem: Allocating stories into sprints considering velocity, dependencies, and team capacity is a constraint-satisfaction problem usually done by feel in a planning meeting.

Solution: Treat it as a lightweight bin-packing / scheduling problem layered with LLM judgment:

  • Inputs: story list with points, team size, sprint length, known dependencies (LLM can infer simple dependencies — "story X needs auth from story Y" — by reading story text).
  • Greedy bin-packing algorithm allocates stories to sprints respecting velocity capacity, then an LLM pass reorders for dependency sanity and flags risky overcommitment.

Tech:

  • Plain Python for the bin-packing (greedy or simple ILP via pulp if you want to be fancier — greedy is enough for a demo and far less debugging risk).
  • Claude for the dependency-inference and "does this sprint plan make sense" sanity pass.
  • Output: sprint-by-sprint backlog, rendered as a simple Gantt/kanban view in the frontend.

Integration: Consumes stories from step 3, team capacity config from project setup. Output (sprint plan) is what gets pushed to Azure DevOps boards / GitHub Projects, and is the thing Meeting Summarization (step 5) and Risk Assessment (step 6) reference when generating sprint-review content or flagging "sprint N is overloaded."

5. Meeting Documentation / Summarization

Problem: Standups, planning meetings, retros generate decisions and action items that live only in people's memory or scattered chat logs.

Solution: Take a transcript (uploaded audio → transcribed, or pasted text) and generate:

  • Structured summary: decisions made, action items (with owner if mentioned), open questions, risks raised.
  • Auto-link mentioned items to existing stories/tasks (fuzzy match meeting text against story titles in Postgres) so "we decided to descope the reporting widget" actually updates/flags that story.

Tech:

  • Whisper (openai-whisper or faster-whisper, runs locally, no API needed) for audio → text if you want a live demo with mic input — strong demo moment.
  • Claude for summarization with structured JSON output (decisions[], action_items[{text, owner}], risks[]).
  • Simple fuzzy string matching (rapidfuzz) to link summary content back to story IDs.

Integration: Action items flagged as risks feed into Risk Assessment (step 6). Linked story updates write back to the same Postgres tables sprint planning uses — this is what makes the platform feel "alive" rather than a one-shot generator: meetings actually update the plan.

6. Risk Assessment

Problem: Risks (scope creep, unclear requirements, resource gaps, technical risk) are usually noticed too late, after they've already caused delay.

Solution: Risk assessment isn't really a separate "module" — it's a cross-cutting analysis pass that reads everything else (requirements, stories, sprint plan, meeting notes) and looks for risk signals:

  • Ambiguous/contradictory requirements (LLM flags vague language: "TBD", "approximately", conflicting statements).
  • Overloaded sprints (from step 4's capacity math).
  • Action items repeatedly slipping across meetings (from step 5's history).
  • Dependency chains with single points of failure (one story blocking many others).

Tech: A scheduled or on-demand Claude call that's given a digest of the current project state (not the raw firehose — summarize first, since context windows and cost matter) and asked to output a risk register: {risk, likelihood, impact, mitigation suggestion}. This is a great place to use a cheaper/faster model (Haiku) since it runs frequently.

Integration: This module is the one place that genuinely needs read access to every other module's data — design your Postgres schema so risk assessment can be "just another query + LLM call over the existing tables," not its own data silo.

7. Knowledge Hub (the integration layer)

Problem: Project knowledge is scattered across GitHub/Azure DevOps repos, CI/CD pipelines, docs, and tickets. No single place to ask "how does this system work."

Solution: This is a continuous ingestion pipeline, separate from the on-demand planning calls above:

  • Code: clone repo, walk the tree, chunk files by function/class (use tree-sitter to parse syntax-aware chunks rather than naive line-splitting — this dramatically improves retrieval quality for code).
  • Docs: ingest README/wiki/Confluence-style markdown.
  • CI/CD: pull pipeline definitions (YAML) and recent run logs/status.
  • Tickets: pull issue/PR descriptions and comments from GitHub/Azure DevOps APIs.
  • Embed all of it into the shared vector store with metadata (file path, repo, type, last-updated) so retrieval can be filtered ("only search this repo's deployment docs").
  • Incremental indexing: on webhook (push event), re-embed only changed files — don't re-index the whole repo every time. This matters more than it sounds like for a "continuously builds a knowledge hub" claim to be true rather than aspirational.

Tech:

  • PyGithub for GitHub API, azure-devops Python SDK for Azure DevOps.
  • tree-sitter (with tree-sitter-languages package) for syntax-aware code chunking.
  • GitHub/Azure DevOps webhooks → a FastAPI endpoint → re-embed changed files only.
  • Same Chroma/Qdrant vector store as the planning side, but a separate collection/namespace per project (and per repo) for clean filtering.

Integration: This is the data layer the Onboarding Assistant (step 8) queries. It's also a secondary input the Risk Assessment module can use — e.g. flagging that a story references a module with no tests, by checking the code knowledge hub.

8. Developer Onboarding Assistant (NL Q&A over the codebase)

Problem: New devs depend on senior engineers to understand architecture, where things live, and how to deploy — this doesn't scale and bottlenecks the seniors.

Solution: A chat interface backed by RAG over the Knowledge Hub:

  • User asks "how does authentication work in this repo?"
  • Retrieve top-k relevant code chunks + docs + related tickets from the vector store.
  • Claude answers with citations (file path + line range) so the answer is verifiable and the dev can jump straight to the source — don't let it just narrate confidently with no pointer back to ground truth, that's how trust in the tool collapses fast.
  • Bonus "wow" feature for the demo: auto-generate an architecture diagram on demand by having the LLM read the repo's import graph (build it with ast/tree-sitter static analysis) and describe module relationships, rendered as a simple graph (e.g. via graphviz or a mermaid diagram in the chat UI).

Tech: Same RAG stack as above, exposed as a chat endpoint (FastAPI + SSE streaming for token-by-token responses, since that reads as much more polished in a demo than a long wait-then-dump).

Integration: Sits directly on top of the Knowledge Hub's vector store. Conversation history can optionally be logged and fed back into Risk Assessment ("3 devs this week asked how deployment works — docs are probably missing something") — a nice cross-module callback if you have time.

9. Gaps in the brief — things you'll need to decide yourselves

These weren't specified but you'll hit them within the first hour of building:

  • Auth & permissions: who can see what. Minimum viable: OAuth login via GitHub, one role (admin) per project, no fine-grained permissions — don't build RBAC at a hackathon unless a judge explicitly cares.
  • Multi-tenancy / project isolation: every table and every vector-store query needs a project_id filter from day one. Retrofitting this later is painful — bake it into your schema from the first migration.
  • Hallucination control: every generated artifact (proposal, story, risk) should show its source citations (which RFP paragraph, which file). This is cheap to add (you already have retrieval indices) and is the single best way to make the demo look trustworthy rather than "AI vibes."
  • Cost/rate limits: cache LLM calls where the input is identical (e.g. re-running the same RFP extraction). A simple hash-of-input → cached-response table in Postgres or Redis saves real API cost during dev/demo iteration.
  • Evaluation/feedback loop: even a simple thumbs up/down on generated stories, stored in Postgres, is worth having — shows judges you've thought about quality over time, not just generation.
  • Notifications: integrate Slack/Teams webhook for "new risk flagged" or "sprint plan generated" — cheap, high demo impact.
  • Where the architecture diagram for onboarding comes from: static analysis (ast module for Python repos, tree-sitter for polyglot) building an import/dependency graph, rendered via graphviz/mermaid — not generated from vibes, generated from actually parsing the code.

10. Suggested hackathon scope (be realistic)

Building all 8 modules well in a hackathon timeframe is unlikely. Suggested priority order if you need to cut:

  1. Knowledge Hub + Onboarding Q&A (steps 5 & 6) over one sample GitHub repo.
  2. User Stories + Sprint Planning.
  3. Meeting Summarization.
  4. Risk Assessment.

Build the shared RAG service first (ingest → embed → retrieve → generate), since modules 7 and 8 all depend on it. That's the piece worth getting right before splitting up into parallel workstreams.