Call variadic foreign functions through a variadic function type

[dg1sbg]

CMP:FUNCTION-TYPE-CREATE-ON-THE-FLY builds every foreign function type with varargs hardcoded to NIL:

(let ((arg-types ...)
      (varargs nil))
  (llvm-sys:function-type-get ... arg-types varargs))

So a variadic callee is always called through an ordinary function type. That is undefined behaviour in C, and the platforms genuinely differ:

• x86-64 SysV passes variadic arguments in the same registers as fixed ones, so the mistake is invisible.
• Darwin arm64 passes them on the stack. A non-variadic lowering leaves them in registers, the callee reads an unwritten slot, and every variadic argument arrives as zero.

The symptom

A mode or flag that silently does nothing. shm_open(name, flags, 0600) creates its object with st_mode 0, which cannot then be reopened — SBCL on the same machine gives 0600. That is how this was found.

Reproduced without strings or filesystem state, in one process:

FD_CLOEXEC before set: 0
fcntl(F_SETFD, FD_CLOEXEC) rc: 0      ← reports success
FD_CLOEXEC after set:  0              ← the argument never arrived

The change

The foreign-type list gains an optional third element, the number of fixed parameters:

(return-type (arg-types ...))                 ; non-variadic, unchanged
(return-type (arg-types ...) fixed-count)     ; variadic

Backward compatible by construction — the BIR instruction, the translator and the foreign-caller cache all treat the signature as opaque data and needed no change.

%FOREIGN-FUNCALL-VARARGS and %FOREIGN-FUNCALL-POINTER-VARARGS expose it:

;; int fcntl(int fd, int cmd, ...);  -- two fixed parameters
(clasp-ffi:%foreign-funcall-varargs "fcntl" 2 :int fd :int f-setfd :int fd-cloexec :int)

The count must be a literal integer, since it selects the fixed prefix when the type is built; both macros validate it at macroexpansion rather than letting a malformed signature fail later inside codegen with an obscure SUBSEQ error.

%FOREIGN-FUNCALL is unchanged and still builds a non-variadic type, which is correct for callees without an ellipsis.

Prior art

cffi-sbcl.lisp addresses the same platform issue in its own backend by inserting &optional into the alien signature under #+(and darwin arm64), commented "All SBCL platforms would understand this but this is the only one where it's required." Same root cause; this change is unconditional rather than target-gated.

On the CI you will see

This is invisible on x86-64. The four Linux cells will show no change either way — they prove only that nothing regressed. Only the macOS cells carry signal for the fix itself, which is worth knowing before reading the matrix.

Testing

Three tests in defcallback-native, using fcntl rather than shm_open so they need no string marshalling, no filesystem state and no cleanup, and because F_GETFD/F_SETFD/FD_CLOEXEC are identical on Linux and macOS: the variadic argument arrives, the same via the pointer variant, and fixed-argument calls are unaffected.

Deliberately absent: a test asserting the old behaviour. Calling a variadic function through a non-variadic type is UB that happens to work on x86-64, so asserting FD_CLOEXEC = 0 would enshrine UB and fail on Linux.

Regression suite 2029 → 2032, ANSI 21936 with 0 unexpected failures, on macOS arm64 boehmprecise.

Follow-up, not included

CFFI carries the fixed/variadic split down to the backend and falls back to appending the two lists when the backend does not define %FOREIGN-FUNCALL-VARARGS — which clasp's does not, so CFFI users get the broken path today. That fix belongs in cffi-clasp.lisp upstream and needs this PR first, since it calls macros that do not exist in released Clasp. Documented in docs/fli-notes.md §II.3.