ETL: SharePoint Source

How to configure the SharePoint source component in Integrate.io ETL to read document libraries, list items, and metadata into your data pipeline using OAuth.

This article explains how to use the Sharepoint source component and its elements.

Choose input connection

Select a valid Sharepoint connection from the connections list.

Source Properties

• Site - property indicates a Sharepoint site which contains needed files. You can find the current site in your browser’s URL. It should have the following format: https://<company subdomain>.sharepoint.com/sites/<my_site_name>. Only the site name should be specified in the related input. https://<company subdomain>.sharepoint.com was already specified in the Sharepoint connection.

• Source path - property holds a path to a file/files. It can point to a concrete file or a number of files that can be matched with path patterns.

• If a concrete file should be matched, source input must have the following format: /<library>/<nested directories>/<file>.<extension>.

• If there are multiple files that have the same schema they can be merged and loaded using the following special characters:

• * - matches any number of any characters, including none. For example, *.csv matches all files ending in .csv. Any part of file path might be replaced with *.

• ** - matches any number of directories recursively. For example if file is located in /Shared Library/dir1/dir2/dir3/data.csv it can be matched like /Share Library/**/data.csv or **/data.csv.

• Record type - defines a file type that should be loaded.

• Delimited values - the files like csv, tsv or with any custom delimiter

• Excel - Microsoft excel files with specified sheet name

• Json Object - files with json data

• Raw - any format, but should be further parsed with Integrate.io available functions

• Load Type - indicates how to load records:

• Full Load - loads all the data that source files have

• Incremental Load - loads files newer/older than specified date

Schema

Represents a view of sample data and properties parsed from the specified files. You can select needed properties and data types for further processing.