protographic: nullable custom scalars lose presence in compileOperationsToProto, and mappings to google.protobuf.Value/Empty get the wrong import

[saikiranjsq]

Component(s)

protographic (operations to proto: compileOperationsToProto). The cli component (wgc) bundles protographic.

Component version

@wundergraph/protographic 0.24.15 (latest). The same behavior occurs in 0.24.3, which wgc 0.129.2 bundles.

wgc version

n/a (protographic is used as a library)

controlplane version

n/a

router version

n/a

What happened?

Description

compileOperationsToProto has two problems with custom scalars:

1. A nullable custom scalar has no presence. A nullable custom scalar (for example Decimal, DateTime) becomes a bare proto3 string. In proto3, an unset string and "" are the same value, so a client cannot send "not set" and cannot see the difference between "absent" and "empty". Built-in scalars do not have this problem: a nullable String becomes google.protobuf.StringValue. The SDL mapper also uses google.protobuf.StringValue for a nullable custom scalar (proto-utils.ts:238), so the SDL mapper and the operations mapper give different results for the same field.

   customScalarMappings does not help. One mapping applies to nullable and non-null fields alike. { Decimal: 'string' } gives a bare string for nullable fields. { Decimal: 'google.protobuf.StringValue' } also wraps non-null fields, so required and optional fields look the same.

2. The generated proto does not compile when a mapping uses a well-known type that is not a wrapper. The header adds import "google/protobuf/wrappers.proto" for every field whose type starts with google.protobuf. (detectWrapperTypeUsage in operations/proto-text-generator.ts). A mapping to google.protobuf.Value (for a JSON scalar), Struct, Empty, Timestamp or Duration gets the wrong import.

Steps to Reproduce

import { compileOperationsToProto } from '@wundergraph/protographic';

const schema = `
  scalar Decimal
  scalar JSON
  type Product { price: Decimal, cost: Decimal!, metadata: JSON }
  type Query { product(maxPrice: Decimal): Product }
`;
const operation = `
  query GetProduct($maxPrice: Decimal) {
    product(maxPrice: $maxPrice) { price cost metadata }
  }
`;

// 1. No mappings
console.log(compileOperationsToProto(operation, schema).proto);

// 2. With mappings
console.log(compileOperationsToProto(operation, schema, {
  customScalarMappings: { Decimal: 'string', JSON: 'google.protobuf.Value' },
}).proto);

Expected Result

Nullability applies to custom scalars in the same way as to built-in scalars, and every well-known type gets its own import:

import "google/protobuf/struct.proto";
import "google/protobuf/wrappers.proto";

message GetProductRequest {
  google.protobuf.StringValue max_price = 1;
}

message GetProductResponse {
  message Product {
    google.protobuf.StringValue price = 1;   // nullable Decimal
    string cost = 2;                         // Decimal!
    google.protobuf.Value metadata = 3;      // JSON -> google.protobuf.Value (a message; has presence)
  }
  Product product = 1;
}

Actual Result

Case 1 (no mappings), 0.24.15:

message GetProductRequest {
  string max_price = 1;
}

message GetProductResponse {
  message Product {
    string price = 1;
    string cost = 2;
    string metadata = 3;
  }
  Product product = 1;
}

Case 2 (with mappings), 0.24.15: nullable Decimal fields are still bare string, and the only import is wrappers.proto:

import "google/protobuf/wrappers.proto";

message GetProductRequest {
  string max_price = 1;
}

message GetProductResponse {
  message Product {
    string price = 1;
    string cost = 2;
    google.protobuf.Value metadata = 3;
  }
  Product product = 1;
}

buf build on a proto from case 2 fails:

service.proto:14:3:cannot find `google.protobuf.Value` in this scope

Proposed fix

We have a fix with tests and can open a PR if the approach is acceptable:

• operations/type-mapper.ts
  • An unmapped nullable custom scalar maps to google.protobuf.StringValue (the same as the SDL mapper). An unmapped non-null custom scalar stays string. With useWrapperTypes: false, the current behavior does not change.
  • A customScalarMappings entry names the base type. A nullable field gets the wrapper of a mapped proto scalar (string -> StringValue, bool -> BoolValue, int64 -> Int64Value, and so on). A mapped message type (for example google.protobuf.Value) is used as-is, because a message already has presence.
• operations/proto-text-generator.ts: the header adds the file that defines each well-known type in use (struct.proto, empty.proto, timestamp.proto, duration.proto, any.proto, field_mask.proto), and wrappers.proto only for wrapper types.
• types.ts: a new PROTO_SCALAR_WRAPPER_TYPE_MAP (proto scalar -> wrapper type).

This is a change in generated output: a nullable custom scalar changes from string to google.protobuf.StringValue (same field number, same JSON form, different proto type). If a default change is not acceptable, the same logic can go behind an option (for example customScalarNullability: 'wrapper').

Tests: 20 new cases in tests/operations/type-mapper.test.ts, proto-text-generator.test.ts and operation-to-proto.test.ts. The full protographic suite passes (633 tests). One existing test, "should fallback to string for unknown custom scalars", asserts the current nullable behavior, so it changes to expect StringValue.

Source diff (against main; the three files are unchanged since 0.24.9)

diff --git a/protographic/src/operations/proto-text-generator.ts b/protographic/src/operations/proto-text-generator.ts
--- a/protographic/src/operations/proto-text-generator.ts
+++ b/protographic/src/operations/proto-text-generator.ts
@@ -102,9 +102,9 @@ function generateHeader(root: protobuf.Root, options?: ProtoTextOptions): string
     imports.add('google/protobuf/descriptor.proto');
   }
 
-  // Only add wrapper types import if actually used
-  if (detectWrapperTypeUsage(root)) {
-    imports.add('google/protobuf/wrappers.proto');
+  // Only add well-known type imports that are actually used
+  for (const imp of collectWellKnownTypeImports(root)) {
+    imports.add(imp);
   }
 
   // Add custom imports
@@ -415,36 +415,50 @@ function formatReservedNumbers(numbers: number[]): string {
 }
 
 /**
- * Detects if any message in the root uses Google Protocol Buffer wrapper types
+ * Well-known types that are not defined in google/protobuf/wrappers.proto,
+ * keyed by fully qualified type name.
  */
-function detectWrapperTypeUsage(root: protobuf.Root): boolean {
+const WELL_KNOWN_TYPE_IMPORTS: Record<string, string> = {
+  'google.protobuf.Value': 'google/protobuf/struct.proto',
+  'google.protobuf.Struct': 'google/protobuf/struct.proto',
+  'google.protobuf.ListValue': 'google/protobuf/struct.proto',
+  'google.protobuf.NullValue': 'google/protobuf/struct.proto',
+  'google.protobuf.Empty': 'google/protobuf/empty.proto',
+  'google.protobuf.Timestamp': 'google/protobuf/timestamp.proto',
+  'google.protobuf.Duration': 'google/protobuf/duration.proto',
+  'google.protobuf.Any': 'google/protobuf/any.proto',
+  'google.protobuf.FieldMask': 'google/protobuf/field_mask.proto',
+};
+
+/**
+ * Collects the imports for the Google Protocol Buffer well-known types that
+ * any message in the root uses
+ */
+function collectWellKnownTypeImports(root: protobuf.Root): Set<string> {
+  const imports = new Set<string>();
   for (const nested of root.nestedArray) {
-    if (nested instanceof protobuf.Type && messageUsesWrapperTypes(nested)) {
-      return true;
+    if (nested instanceof protobuf.Type) {
+      collectMessageWellKnownTypeImports(nested, imports);
     }
   }
-  return false;
+  return imports;
 }
 
 /**
- * Recursively checks if a message or its nested messages use wrapper types
+ * Recursively collects well-known type imports for a message and its nested messages
  */
-function messageUsesWrapperTypes(message: protobuf.Type): boolean {
-  // Check fields in this message
+function collectMessageWellKnownTypeImports(message: protobuf.Type, imports: Set<string>): void {
   for (const field of message.fieldsArray) {
     if (field.type.startsWith('google.protobuf.')) {
-      return true;
+      imports.add(WELL_KNOWN_TYPE_IMPORTS[field.type] ?? 'google/protobuf/wrappers.proto');
     }
   }
 
-  // Check nested messages recursively
   for (const nested of message.nestedArray) {
-    if (nested instanceof protobuf.Type && messageUsesWrapperTypes(nested)) {
-      return true;
+    if (nested instanceof protobuf.Type) {
+      collectMessageWellKnownTypeImports(nested, imports);
     }
   }
-
-  return false;
 }
 
 /**
diff --git a/protographic/src/operations/type-mapper.ts b/protographic/src/operations/type-mapper.ts
--- a/protographic/src/operations/type-mapper.ts
+++ b/protographic/src/operations/type-mapper.ts
@@ -12,7 +12,7 @@ import {
   getNamedType,
   GraphQLScalarType,
 } from 'graphql';
-import { SCALAR_TYPE_MAP, SCALAR_WRAPPER_TYPE_MAP } from '../types.js';
+import { PROTO_SCALAR_WRAPPER_TYPE_MAP, SCALAR_TYPE_MAP, SCALAR_WRAPPER_TYPE_MAP } from '../types.js';
 import { unwrapNonNullType, isNestedListType, calculateNestingLevel } from './list-type-utils.js';
 
 /**
@@ -79,15 +79,13 @@ export function mapGraphQLTypeToProto(type: GraphQLType, options?: TypeMapperOpt
         };
       }
 
-      // Use direct scalar type for non-null fields
-      if (SCALAR_TYPE_MAP[scalarName]) {
-        return {
-          typeName: SCALAR_TYPE_MAP[scalarName],
-          isRepeated: innerInfo.isRepeated,
-          isWrapper: false,
-          isScalar: true,
-        };
-      }
+      // Use direct scalar type for non-null fields; unmapped custom scalars are strings
+      return {
+        typeName: SCALAR_TYPE_MAP[scalarName] ?? 'string',
+        isRepeated: innerInfo.isRepeated,
+        isWrapper: false,
+        isScalar: true,
+      };
     }
 
     return innerInfo;
@@ -100,12 +98,16 @@ export function mapGraphQLTypeToProto(type: GraphQLType, options?: TypeMapperOpt
   if (isScalarType(namedType)) {
     const scalarName = namedType.name;
 
-    // Check custom mappings first
-    if (customScalarMappings[scalarName]) {
+    // Custom mappings name the base type; a nullable field needs the wrapper
+    // of that type to keep presence. Message types (e.g. google.protobuf.Value)
+    // already carry presence and are used as-is.
+    const mappedType = customScalarMappings[scalarName];
+    if (mappedType) {
+      const wrapperType = useWrapperTypes ? PROTO_SCALAR_WRAPPER_TYPE_MAP[mappedType] : undefined;
       return {
-        typeName: customScalarMappings[scalarName],
+        typeName: wrapperType ?? mappedType,
         isRepeated: false,
-        isWrapper: false,
+        isWrapper: wrapperType !== undefined,
         isScalar: true,
       };
     }
@@ -120,6 +122,17 @@ export function mapGraphQLTypeToProto(type: GraphQLType, options?: TypeMapperOpt
       };
     }
 
+    // Unmapped custom scalars fall back to string, like the SDL mapper does,
+    // and a nullable one keeps presence through StringValue.
+    if (useWrapperTypes && !SCALAR_TYPE_MAP[scalarName]) {
+      return {
+        typeName: PROTO_SCALAR_WRAPPER_TYPE_MAP.string,
+        isRepeated: false,
+        isWrapper: true,
+        isScalar: true,
+      };
+    }
+
     // Fallback to direct mapping
     const protoType = SCALAR_TYPE_MAP[scalarName] || 'string';
     return {
diff --git a/protographic/src/types.ts b/protographic/src/types.ts
--- a/protographic/src/types.ts
+++ b/protographic/src/types.ts
@@ -37,6 +37,22 @@ export const SCALAR_WRAPPER_TYPE_MAP: Record<string, string> = {
   Boolean: 'google.protobuf.BoolValue',
 };
 
+/**
+ * Maps Protocol Buffer scalar types to the wrapper type that carries presence
+ * for them. Used when a nullable custom scalar is mapped to a proto scalar.
+ */
+export const PROTO_SCALAR_WRAPPER_TYPE_MAP: Record<string, string> = {
+  string: 'google.protobuf.StringValue',
+  bool: 'google.protobuf.BoolValue',
+  int32: 'google.protobuf.Int32Value',
+  int64: 'google.protobuf.Int64Value',
+  uint32: 'google.protobuf.UInt32Value',
+  uint64: 'google.protobuf.UInt64Value',
+  float: 'google.protobuf.FloatValue',
+  double: 'google.protobuf.DoubleValue',
+  bytes: 'google.protobuf.BytesValue',
+};
+
 /**
  * Protocol Buffer idempotency levels for RPC methods
  * @see https://protobuf.dev/reference/protobuf/google.protobuf/#idempotency-level

Environment information

Environment

OS: macOS 26 (darwin 25.6.0)
Package Manager: npm (Node 22)

Additional context

We generate ConnectRPC services from GraphQL operations, and our schema has about 20 custom scalars. With the current behavior, every nullable custom scalar in a request or response loses presence, and we cannot map JSON-type scalars to google.protobuf.Value without editing the generated imports.

[github-actions]

WunderGraph commits fully to Open Source and we want to make sure that we can help you as fast as possible.
The roadmap is driven by our customers and we have to prioritize issues that are important to them.
You can influence the priority by becoming a customer. Please contact us here.