Skip to content

feat(go): update data collection docs #18646

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
giortzisg wants to merge 1 commit into
base: master
Choose a base branch
from
+203 −15
Open

feat(go): update data collection docs #18646

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
114 changes: 113 additions & 1 deletion docs/platforms/go/common/configuration/options.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,12 +90,124 @@ Grouping in Sentry is different for events with stack traces and without. As a r

<SdkOption name="SendDefaultPII" type="bool" defaultValue="false">

If this flag is enabled, certain personally identifiable information (PII) is added by active integrations. By default, no such data is sent.
<Alert level="warning">
`SendDefaultPII` is deprecated. Use [`DataCollection`](#DataCollection) for
granular control over automatically collected data.
</Alert>

If this flag is enabled, certain personally identifiable information (PII) is added by active integrations.
For backwards compatibility, `SendDefaultPII: true` behaves like enabling all [`DataCollection`](#DataCollection) categories.
If both options are set, `DataCollection` takes precedence.

<Alert>
Passing [`DataCollection`](#DataCollection) opts you into the more permissive `DataCollection` defaults. To preserve the old `SendDefaultPII: false` behavior while using `DataCollection`, opt out of each category explicitly:

```go
sensitiveTerms := []string{"forwarded", "-ip", "remote-", "via", "-user"}

err := sentry.Init(sentry.ClientOptions{
Dsn: "___PUBLIC_DSN___",
DataCollection: &sentry.DataCollection{
UserInfo: sentry.Set(false),
Cookies: &sentry.KeyValueCollectionBehavior{Mode: sentry.CollectionOff},
HTTPBodies: []sentry.BodyType{},
HTTPHeaders: &sentry.HeaderCollectionConfig{
Request: &sentry.KeyValueCollectionBehavior{
Mode: sentry.CollectionDenyList,
Terms: sensitiveTerms,
},
Response: &sentry.KeyValueCollectionBehavior{
Mode: sentry.CollectionDenyList,
Terms: sensitiveTerms,
},
},
QueryParams: &sentry.KeyValueCollectionBehavior{
Mode: sentry.CollectionDenyList,
Terms: sensitiveTerms,
},
},
})
```

Data you set explicitly, for example with `Scope.SetUser`, is always sent regardless of `DataCollection`.

</Alert>

If you enable this option, be sure to manually remove what you don't want to send using our features for managing [_Sensitive Data_](../../data-management/sensitive-data/).

</SdkOption>

<SdkOption name="DataCollection" type="*sentry.DataCollection">

Controls which categories of data the SDK collects automatically. All fields are optional.
When `DataCollection` is configured, the SDK collects rich debugging context and scrubs values whose keys match the built-in sensitive denylist (`auth`, `token`, `password`, and similar).

For backwards compatibility, if `DataCollection` is not configured, the SDK uses `SendDefaultPII` to determine legacy collection behavior.
Set `DataCollection: &sentry.DataCollection{}` to use the new defaults.

Common defaults:

| Configuration | Behavior |
| ---------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| `DataCollection: nil`, `SendDefaultPII: false` | Legacy privacy defaults: no cookies or HTTP bodies, user information disabled, headers and query strings collected with additional privacy filtering. |
| `DataCollection: nil`, `SendDefaultPII: true` | Legacy PII-enabled behavior, equivalent to enabling the `DataCollection` defaults. |
| `DataCollection: &sentry.DataCollection{}` | New `DataCollection` defaults: collect supported categories and scrub sensitive values. |

For more on what data Sentry collects and how to control it, see <PlatformLink to="/data-management/">Data Management</PlatformLink>.

| Field | Type | Default when `DataCollection` is configured | Description |
| ------------- | ------------------------------------ | --------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `UserInfo` | `sentry.Option[bool]` | `sentry.Set(true)` | Populates `user.*` fields from auto-instrumentation. This doesn't affect user data explicitly set with `Scope.SetUser`. |
| `Cookies` | `*sentry.KeyValueCollectionBehavior` | denylist mode | Collects cookies and scrubs sensitive cookie values. |
| `HTTPHeaders` | `*sentry.HeaderCollectionConfig` | request and response headers in denylist mode | Collects HTTP request and response headers independently. |
| `HTTPBodies` | `[]sentry.BodyType` | all body types | Body types to collect: `sentry.BodyIncomingRequest`, `sentry.BodyOutgoingRequest`, and `sentry.BodyIncomingResponse`. Set to an empty slice to disable body collection. |
| `QueryParams` | `*sentry.KeyValueCollectionBehavior` | denylist mode | Collects URL query parameters and scrubs sensitive values. |

`Cookies`, `HTTPHeaders.Request`, `HTTPHeaders.Response`, and `QueryParams` use `KeyValueCollectionBehavior`:

```go
// Zero value: collect keys and scrub values whose keys match the built-in
// sensitive denylist plus any additional Terms.
sentry.KeyValueCollectionBehavior{
Mode: sentry.CollectionDenyList,
Terms: []string{"custom-secret"},
}

// Disable collection for this category.
sentry.KeyValueCollectionBehavior{Mode: sentry.CollectionOff}

// Only listed keys send their real value. Built-in sensitive keys are still scrubbed.
sentry.KeyValueCollectionBehavior{
Mode: sentry.CollectionAllowList,
Terms: []string{"page", "x-request-id"},
}
```

Example:

```go
err := sentry.Init(sentry.ClientOptions{
Dsn: "___PUBLIC_DSN___",
DataCollection: &sentry.DataCollection{
UserInfo: sentry.Set(false),
HTTPBodies: []sentry.BodyType{},
HTTPHeaders: &sentry.HeaderCollectionConfig{
Request: &sentry.KeyValueCollectionBehavior{
Mode: sentry.CollectionAllowList,
Terms: []string{"x-request-id"},
},
Response: &sentry.KeyValueCollectionBehavior{Mode: sentry.CollectionOff},
},
QueryParams: &sentry.KeyValueCollectionBehavior{
Mode: sentry.CollectionDenyList,
Terms: []string{"customer"},
},
},
})
```

</SdkOption>

<SdkOption name="ServerName" type="string">

Supplies a server name. When provided, the name of the server is sent along and persisted in the event. For many integrations, the server name actually corresponds to the device hostname, even in situations where the machine is not actually a server.
Expand Down
96 changes: 85 additions & 11 deletions docs/platforms/go/common/data-management/data-collected.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,32 +6,106 @@ sidebar_order: 1

Sentry takes data privacy very seriously and has default settings in place that prioritize data safety, especially when it comes to personally identifiable information (PII) data. When you add the Sentry SDK to your application, you allow it to collect data and send it to Sentry during the runtime of your application.

The category types and amount of data collected vary, depending on the integrations you've enabled in the Sentry SDK. This page lists data categories that the Sentry Go SDK collects.
The category types and amount of data collected vary, depending on the integrations you've enabled in the Sentry SDK. This page lists data categories that the Sentry Go SDK collects automatically.

For many of the categories listed here it is required to enable the <PlatformLink to="/configuration/options">SendDefaultPII option</PlatformLink>.
## Options to Control Data Collection

You can control many of the categories listed here with the <PlatformLink to="/configuration/options/#DataCollection">`DataCollection` option</PlatformLink>, which lets you opt in or out of each data category individually. The legacy <PlatformLink to="/configuration/options/#SendDefaultPII">`SendDefaultPII` option</PlatformLink> is still supported, but deprecated.

How much data the SDK collects by default depends on which option you use. Without `DataCollection` (and with `SendDefaultPII` unset or `false`), the SDK keeps the legacy privacy defaults. As soon as you pass a `DataCollection` value, categories you don't set explicitly fall back to their `DataCollection` defaults, which collect richer debugging context and scrub values whose keys match the built-in sensitive denylist (`auth`, `token`, `password`, and similar). Setting `SendDefaultPII: true` is equivalent to enabling all `DataCollection` categories. If both are set, `SendDefaultPII` is ignored.

Regardless of these options, you can always scrub any data before it's sent to Sentry. See <PlatformLink to="/data-management/sensitive-data/">Scrubbing Sensitive Data</PlatformLink> for details.

## HTTP Headers

By default, the Sentry SDK doesn't send any HTTP headers. Even when sending HTTP headers is enabled, we have a [denylist](https://github.com/getsentry/sentry-go/blob/master/interfaces.go#L157-163) in place, which filters out any headers that contain sensitive data.
When using `DataCollection`, the Sentry SDK collects HTTP request and response headers by default.

To start sending HTTP headers, set <PlatformLink to="/configuration/options">SendDefaultPII option</PlatformLink> to true.
Use the `DataCollection.HTTPHeaders` option to control this. You can disable it with `sentry.CollectionOff`, or use `sentry.CollectionAllowList` or `sentry.CollectionDenyList` to restrict which header values are sent. Values whose keys match Sentry's built-in sensitive denylist are automatically scrubbed, while the keys are kept.

Without `DataCollection` (and with `SendDefaultPII` unset or `false`), HTTP headers are collected with additional legacy privacy filtering.

## Cookies

By default, the Sentry SDK doesn't send cookies. Sentry tries to remove any cookies that contain sensitive information, such as the Session ID and CSRF Token cookies.
When using `DataCollection`, the SDK collects cookies with sensitive values scrubbed. Opt out by setting `DataCollection.Cookies` to `sentry.CollectionOff`.

If you want to send cookies, set <PlatformLink to="/configuration/options">SendDefaultPII option</PlatformLink> to true.
To collect cookies when not using `DataCollection`, set the deprecated `SendDefaultPII: true` option. You can also restrict which cookie values are sent with `sentry.CollectionAllowList` or `sentry.CollectionDenyList`.

## Users' IP Addresses
## Information About Logged-in User

By default, the Sentry SDK doesn't send the user's IP address. Once enabled, the Sentry backend services will infer the user ip address based on the incoming request, unless certain integrations you can enable override this behavior.
When using `DataCollection`, the SDK automatically populates user information from instrumentation. To disable this, set `DataCollection.UserInfo` to `sentry.Set(false)`.

To enable sending the user's IP address, set <PlatformLink to="/configuration/options">SendDefaultPII option</PlatformLink> to true.
Without `DataCollection` (and with `SendDefaultPII` unset or `false`), user information is not sent automatically. User data that you explicitly set, for example with `Scope.SetUser`, is always sent regardless of `DataCollection`.

## Users' IP Addresses

When using `DataCollection`, the SDK can send request environment information, such as `REMOTE_ADDR`, which Sentry can use to infer users' IP addresses. To disable this, set `DataCollection.UserInfo` to `sentry.Set(false)`.

Without `DataCollection` (and with `SendDefaultPII` unset or `false`), the user's IP address is not sent through request environment information.

Even when this is disabled, IP addresses can still reach Sentry through collected HTTP headers, cookies, or query parameters (for example, the `X-Forwarded-For` header). If you use `DataCollection`, add these terms to the partially matched deny lists for those categories so their values are filtered:

```go
sensitiveTerms := []string{"forwarded", "-ip", "remote-", "via", "-user"}

err := sentry.Init(sentry.ClientOptions{
Dsn: "___PUBLIC_DSN___",
DataCollection: &sentry.DataCollection{
HTTPHeaders: &sentry.HeaderCollectionConfig{
Request: &sentry.KeyValueCollectionBehavior{
Mode: sentry.CollectionDenyList,
Terms: sensitiveTerms,
},
Response: &sentry.KeyValueCollectionBehavior{
Mode: sentry.CollectionDenyList,
Terms: sensitiveTerms,
},
},
Cookies: &sentry.KeyValueCollectionBehavior{
Mode: sentry.CollectionDenyList,
Terms: sensitiveTerms,
},
QueryParams: &sentry.KeyValueCollectionBehavior{
Mode: sentry.CollectionDenyList,
Terms: sensitiveTerms,
},
},
})
```

## Request URL

The full request URL of incoming HTTP requests is **always sent to Sentry**. Depending on your application, this could contain PII data.
The SDK collects request URLs for incoming HTTP requests and outgoing HTTP client spans. Depending on your application, URL paths can contain PII data. For example, a URL like `/users/1234/details`, where `1234` is a user id, may be considered PII.

URL credentials are redacted. Query parameters are controlled separately by `DataCollection.QueryParams`.

## Request Query String

The full request query string of incoming HTTP requests is **always sent to Sentry**. Depending on your application, this could contain PII data.
When using `DataCollection`, the SDK collects query strings from outgoing and incoming HTTP requests by default. Values whose keys match the built-in sensitive denylist are replaced with `[Filtered]`.

Use the `DataCollection.QueryParams` option to control this. Set it to `sentry.CollectionOff` to disable collection entirely, or use `sentry.CollectionAllowList` or `sentry.CollectionDenyList` to filter which values are sent.

Sentry also has some additional [server-side data scrubbing](/security-legal-pii/scrubbing/server-side-scrubbing/) in place to remove sensitive data from the query string.

## Request Body

When using `DataCollection`, supported incoming and outgoing request bodies are collected by default. JSON and form-encoded bodies are parsed and values whose keys match the sensitive denylist are replaced with `[Filtered]`. Opaque body data is replaced entirely with `[Filtered]`.

To disable body collection, set `DataCollection.HTTPBodies` to an empty slice. You can also collect only specific body types by providing a subset, such as `sentry.BodyIncomingRequest` or `sentry.BodyOutgoingRequest`.

```go
err := sentry.Init(sentry.ClientOptions{
Dsn: "___PUBLIC_DSN___",
DataCollection: &sentry.DataCollection{
HTTPBodies: []sentry.BodyType{
sentry.BodyIncomingRequest,
sentry.BodyOutgoingRequest,
},
},
})
```

Without `DataCollection` (and with `SendDefaultPII` unset or `false`), Sentry doesn't send request body content.

## Device, OS, and Runtime Information

By default, the Sentry SDK sends information about the operating system and Go runtime to Sentry. This helps provide context about where an event occurred.
6 changes: 4 additions & 2 deletions docs/platforms/go/common/data-management/sensitive-data/index.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ Ensure that your team is aware of your company's policy around what can and cann

If you _do not_ wish to use the default PII behavior, you can also choose to identify users in a more controlled manner, using our [user identity context](../../enriching-events/identify-user/).

You can also configure automatic data collection in the SDK with <PlatformLink to="/configuration/options/#DataCollection">`DataCollection`</PlatformLink>. This lets you disable categories such as cookies, HTTP headers, query strings, and HTTP bodies, or collect them while scrubbing values whose keys match Sentry's sensitive denylist.

## Scrubbing Data

### <PlatformIdentifier name="before-send" /> & <PlatformIdentifier name="before-send-transaction" />
Expand All @@ -43,8 +45,8 @@ Sensitive data may appear in the following areas:

- Stack-locals → Some SDKs (Python, PHP and Node) will pick up variable values within the stack trace. These can be scrubbed, or this behavior can be disabled altogether if necessary.
- Breadcrumbs → Some SDKs (JavaScript and the Java logging integrations, for example) will pick up previously executed log statements. **Do not log PII** if using this feature and including log statements as breadcrumbs in the event. Some backend SDKs will also record database queries, which may need to be scrubbed. Most SDKs will add the HTTP query string and fragment as a data attribute to the breadcrumb, which may need to be scrubbed.
- User context → Automated behavior is controlled via <PlatformIdentifier name="send-default-pii" />.
- HTTP context → Query strings may be picked up in some frameworks as part of the HTTP request context.
- User context → Automated behavior is controlled via <PlatformLink to="/configuration/options/#DataCollection">`DataCollection`</PlatformLink>.
- HTTP context → Headers, cookies, query strings, and bodies may be picked up by HTTP integrations. Use <PlatformLink to="/configuration/options/#DataCollection">`DataCollection`</PlatformLink> to disable specific categories or scrub sensitive keys before data leaves the SDK.
- Transaction Names → In certain situations, transaction names might contain sensitive data. For example, a browser's pageload transaction might have a raw URL like `/users/1234/details` as its name (where `1234` is a user id, which may be considered PII). In most cases, our SDKs can parameterize URLs and routes successfully, that is, turn `/users/1234/details` into `/users/:userid/details`. However, depending on the framework, your routing configuration, race conditions, and a few other factors, the SDKs might not be able to completely parameterize all of your URLs.
- HTTP Spans → Most SDKs will include the HTTP query string and fragment as a data attribute, which means the HTTP span may need to be scrubbed.

Expand Down
2 changes: 1 addition & 1 deletion docs/platforms/go/common/enriching-events/identify-user/index.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ An alternative, or addition, to the username. Sentry is aware of email addresses
### `ip_address`

The user's IP address. If the user is unauthenticated, Sentry uses the IP address as a unique identifier for the user.
The SDK will attempt to pull the IP address from the HTTP request data on incoming requests (`request.env.REMOTE_ADDR` field in JSON), if available. That requires <PlatformIdentifier name="send-default-pii" /> set to `true` in the SDK options.
The SDK will attempt to pull the IP address from the HTTP request data on incoming requests (`request.env.REMOTE_ADDR` field in JSON), if available. That requires automatic user information collection to be enabled with <PlatformLink to="/configuration/options/#DataCollection">`DataCollection.UserInfo`</PlatformLink>, or the legacy <PlatformIdentifier name="send-default-pii" /> option set to `true`.

If the user's `ip_address` is set to `"{{auto}}"`, Sentry will infer the IP address from the connection between your app and Sentry's server. If the field is omitted, the default value is `null`.

Expand Down
Loading