feat(cli): add code, traced and legacy docker builds to generators

[sf-sahil-jassal]

Description

This PR adds a faster Docker build flow to the monorepos that sl scaffold and sl microservice generate. Before, each full image ran its own npm ci over the whole lockfile. With many services that was slow, and the builds could not run in parallel because the installs used all the agent memory.

Now the pipeline picks one of three modes for each changed service:

• Code build (Dockerfile.code): puts the new host-built dist on top of the last image for the environment. Most builds use this mode.
• Traced full build (Dockerfile.nft, default): starts from one shared deps image (Dockerfile.deps, one npm ci per pipeline run). Then scripts/node-file-tracer.js (@vercel/nft) copies only the files that the app loads.
• Legacy full build (Dockerfile): the in-image build, kept as a fallback with the USE_NFT_BUILD Jenkins parameter.

scripts/get-full-build-packages.sh decides which services need a full build. It reads changes to Dockerfiles, dependencies, the nft config, patches and the lockfile. Its header comment lists the conventions it depends on, e.g. @local/<folder> package names.

Changes

• Scaffold: Dockerfile.deps, .dockerignore, the tracer and the change detection script, plus a docker:build:deps script and a root nft config. The GitHub build workflow now uses the traced build.
• Microservice generator:
  • new Dockerfile.nft and Dockerfile.code, and a rewritten Dockerfile, all with build args and the same runtime layout
  • new scripts: docker:build:full|nft|code, docker:push, docker:tag, docker:retag, helm-update
  • nft.alwaysCopy for each loopback-connector-* dependency
  • removed the old docker:build, docker:build:dev and docker:push:dev scripts, and the unused _appendDockerScript
• Bug fix: the old scripts passed the base service name as SERVICE_NAME. The scripts now use the service folder name.
• Jenkinsfile template:
  • change detection and the code, traced or legacy build split
  • BUILD_ALL_PACKAGES, and image promotion with retag
  • Trivy in place of Snyk
  • the jq and yq downloads pick the agent architecture
• Docs in packages/cli/src/generators/microservice/docs/docker-builds.md: files, change detection, Jenkins parameters, code builds, traced builds (deps image, what a tracer cannot see, nft config, testing), legacy builds, local packages and adoption.
• Sonar and nyc now exclude the new templates.

Type of change

• New feature (non-breaking change which adds functionality)

This changes only files in newly generated projects. Services generated before this change keep their old scripts.

How Has This Been Tested?

• npm run build and npm test in packages/cli (22 passing)
• Rendered all changed templates with EJS. The rendered package.json.tpl parses as JSON.
• Ran get-full-build-packages.sh in a test git repo for 10 cases: first build, no change, source only, version bump, local package deps, Dockerfile.nft change, Dockerfile.code change, lockfile only, root nft change, unknown commit. Each case gave the expected output.
• Ran docker buildx build --check on all Dockerfile templates.
• Not done yet: a real traced build and boot of a newly scaffolded project.

Checklist:

• Performed a self-review of my own code
• npm test passes on your machine
• New tests added or existing tests modified to cover all changes
• Code conforms with the style guide
• API Documentation in code was updated
• Any dependent changes have been merged and published in downstream modules

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[rohit-sourcefuse]

Hi Sahil, nice work on this.

The shared deps image instead of an npm ci per service is a real speedup, and the docs are genuinely useful. Good to see the personal git identity, the Snyk org and the gobinaries | bash step gone too.

I spent some time on the change detection and the build flow, and tried it out in a small two-service monorepo built from these templates (with Docker and lerna 9.0.7). A few cases end up shipping stale or broken images while the build stays green, so I'd like these sorted before merge.

Details and fixes are inline, and I tested each fix in the same repo:

1. A code build on top of a traced image can't load a newly imported module.
2. Full builds triggered by root files (lockfile, patches/, Dockerfile.deps, tracer, root nft) are never pushed, tagged or deployed.
3. The lockfile rule is skipped whenever any package.json changes, including a version bump.
4. The service's nested node_modules is merged into the root one, so shared packages get the service's versions.
5. sudo docker push fails in the generated GitHub workflow.
6. Trivy is installed unpinned, as root.
7. The build gate is hardcoded to dev, and promotion can retag a :dev built from commits the target branch doesn't have.

Smaller things, fine as follow-ups:

• sudo npm install -g lerna@^9 isn't pinned. lerna is already in the root devDependencies, so npx lerna after npm ci would do.
• get-full-build-packages.sh prints an empty list and exits 0 if jq is missing or git diff fails, which means code builds for everything. A command -v jq check and set -e would catch it.
• With docker:build removed, an older Jenkinsfile silently skips newly generated services, and the service README template still mentions it. Maybe keep it as an alias for docker:build:full.
• --helmPath now needs to start with <project>-helm/, otherwise helm-update edits a file outside the helm clone. Worth a line in the docs and the flag help.
• A few tests for the script would be great. It's easy to drive with temp git repos.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[rohit-sourcefuse]

Approving. I re-checked cec8f89. The earlier points are all addressed, the scaffold tests are fixed, and the deps_changed HEAD fix looks good. I agree with keeping cpio, the copyDirs throw, and the public/.env and BUILD_ENV behaviour as they are. The Trivy failure is from sandbox lockfiles this PR doesn't touch.

One follow-up for the next PR, inline on Dockerfile.code.