How can I change the columnDelimiter for the data upsert bulk command ?

[RupertBarrow]

The Salesforce Metadata API for Bulk Upsert v2.0 (https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asynch/walkthrough_upsert.htm) lets you define the following during a job creation (example) :

{
    "object" : "Account",
    "externalIdFieldName" : "customExtIdField__c",
    "contentType" : "CSV",
    "operation" : "upsert",
    "lineEnding" : "LF"
}

The API provides other options (https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asynch/create_job.htm) such as columnDelimiter, assignmentRuleId.

The SF CLI data upsert bulk command has options to set object (-s) and externalIdFieldName(-i).

How can I set lineEnding, columnDelimiteror assignmentRuleIdin the data upsert bulkCLI command ?

[WillieRuemmele]

Hey @RupertBarrow - As you know, this is exactly why we've started using github discussions, so we can find gaps and deliver features users would actually want and use! Hopefully we can add this ability soon!

[VivekMChawla]

@RupertBarrow - Thank you for the feedback!

We're actively working to improve bulk data operations in the CLI.

I've asked @cristiand391 to look into adding support for the settings you've specified.

[cristiand391]

@RupertBarrow

columnDelimiter

This option is for the job results, example using PIPE:

➜  jsforce git:(main) ✗ sf org api 'services/data/v61.0/jobs/query/7506s000005cdAYAAY/results'
"Id"|"Name"
"0016s00000kZs5ZAAS"|"Bulk Account 1723651704871 (Updated)"
"0016s00000kZs5aAAC"|"Bulk Account 1723651704871 (Updated)"
"0016s00000kZs5bAAC"|"Bulk Account 1723651704871 (Updated)"
...

For bulk commands that take a csv file as input, would you expect --column-delimiter to enforce both the input and output format or just output?

I feel like the latter would be more flexible but not sure why would you input with a comma delimiter and require a different one as output.

lineEnding

Same, this applies to the output only.
data bulk commands set this depending on the OS:
https://github.com/salesforcecli/plugin-data/blob/e5e21b902b5945bca58ab5beb60504b1ae245504/src/bulkOperationBase.ts#L114

is this something you require to customize (getting CRLF line endings on non-windows/LF on win)?

assignmentRuleId

we can add it as an optional flag 👍🏼

[RupertBarrow]

@cristiand391

Input and output are different : we are usually constrained by an existing file format as input, so input must be very flexible, and there is no reason that the output format be the same. Personally, I would expect output to the screen to always be the same, whereas output as CSV, if it existed, should require its own options.

I suggest you look at the Unix awk and read -line commands to check for other possible input options and their values (such as RS/ORS line/record separator, eg LF, CR, CR+LF -, and IFS/FS/OFS field separators - such as ' or "). Of course, adding Salesforce header options such as assignmentRuleId and others would be very useful.

[cristiand391]

@RupertBarrow

thanks for the feedback, we discussed this with Vivek and agreed we'll expose all requested features as optional flags in old and new commands 👍🏼

[joanmferreras]

Hi @cristiand391

Is there a ticket I can follow on line-ending being settable from the CLI as a flag?

Thanks

[cristiand391]

there's a PR for adding --line-ending / --column-delimiter:
salesforcecli/plugin-data#1110

I can post here when it's available in the sf nightly channel.

[cristiand391]

this was included in sf v2.69.14:
https://github.com/forcedotcom/cli/tree/main/releasenotes/#26914-december-11-2024-stable