Discogs SageMaker — Album Cover Recognition

Train a PyTorch album-cover classifier (local or SageMaker), deploy a SageMaker endpoint, and expose it via API Gateway + Lambda for a frontend.

What's in this repo

backend/            # Data pipeline, training, FastAPI, SageMaker entry points
│                   # scripts/ = core CLIs; utils/ = optional helpers
infrastructure/     # CDK: InferenceStack (API Gateway → Lambda → SageMaker)
notebooks/          # SageMaker Studio notebook
scripts/            # Repo helper scripts (upload, verify)
docs/               # Guides
huggingface_space/  # Optional Hugging Face Space app
samples/            # 50 sample covers for manual testing
data/               # Local data (gitignored — build or download yourself)

There is no frontend in this repo. Point a separate Vercel/Next app at the API Gateway /predict URL (see docs/deploy-api.md).

Prerequisites

• Python 3.10+ (3.11 recommended)
• AWS CLI configured (aws sts get-caller-identity works)
• Node.js 18+ (for CDK deploy)
• Optional: Discogs personal access token for enriching metadata

Quick start (local)

cd backend
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Build a small dataset (needs Discogs dump / network for images)
python scripts/build_data.py --count 500

# Train
python -m scripts.train --data-dir ../data --model-dir models

# Serve locally
python main.py
# → http://localhost:8000

See docs/backend.md for full local workflow.

SageMaker path

1. Upload code/data

   ./scripts/prepare_for_studio.sh YOUR_S3_BUCKET us-east-2

2. Train / deploy endpoint — open notebooks/studio_notebook.ipynb in Studio and set BUCKET_NAME.
3. Public API

   cd infrastructure
   npm install
   export CDK_DEFAULT_REGION=us-east-2
   npm run cdk bootstrap aws://YOUR_ACCOUNT_ID/us-east-2   # once
   npm run deploy -- InferenceStack

   Stack default endpoint name: album-classifier (override with --context endpointName=...).

Docs: docs/sagemaker.md · docs/deploy-api.md · docs/README.md

Environment

Copy examples and fill in values (never commit real secrets):

Variable	Where	Purpose
DISCOGS_USER_TOKEN	backend/.env.local	Discogs API for enrichment
BUCKET_NAME	shell / Studio	S3 bucket for data & models
SAGEMAKER_ENDPOINT_NAME	Lambda (CDK)	Target endpoint (default album-classifier)
NEXT_PUBLIC_API_URL	frontend (external)	API Gateway predict URL

Sharing notes

• data/, backend/models/, *.tar.gz, venv/, node_modules/, and cdk.out/ are gitignored.
• IAM policy templates live under infrastructure/iam/ with YOUR_BUCKET_NAME placeholders.
• Do not commit AWS account IDs, bucket names from personal accounts, or access keys.

License

MIT