ETL: TOKENIZE

Use the Tokenize expression function in Integrate.io ETL to split a string into tokens by delimiter for parsing CSV-like fields and array transformations.

Description

Generates a bag from delimited strings in string field. If you don’t specify a single-character delimiter, tokenize will use any of the following characters:

• space
• double-quote
• comma
• parenthesis
• star

Syntax1

TOKENIZE(string, delimiter)

Arguments

string - string expression

delimiter - string expression indicating a single character delimiter.

Examples

TOKENIZE('a,b,c', ',') returns a bag containing a, b, c

Return value datatype

Bag of strings

Impact of null value

If input is null, returns null.