ETL: SugarCRM Source

How to configure the SugarCRM component to read CRM data (Accounts, Contacts, Leads, and custom modules) in your Integrate.io ETL pipeline.

Use the SugarCRM source component to read records from any module in your SugarCRM instance, including standard modules (Accounts, Contacts, Leads, Opportunities, Cases, Tasks, Calls, Meetings, etc.) and any custom modules your tenant has, and ingest them into your Integrate.io ETL pipeline. The connector authenticates against the SugarCRM REST API v11_24 using your SugarCRM username and password, fetches an OAuth token internally on every job, and supports both full and incremental loads.

Connection Setup

Create a SugarCRM connection from Connections → New connection → SugarCRM.

Field	Description
Name	Display name for the connection inside Integrate.io
Instance URL	The base URL of your SugarCRM instance, e.g. https://yourcompany.sugarondemand.com. Do not include /rest/...; Integrate.io appends the API path automatically.
Username	The SugarCRM user the connector authenticates as. This user’s role determines which modules and records the pipeline can read.
Password	The password for that SugarCRM user. Integrate.io uses this once per pipeline run to obtain a short-lived OAuth token from /rest/v11_24/oauth2/token; the password itself is never sent to SugarCRM modules.

Use a dedicated integration user with read-only access to the modules you want to extract, rather than a regular user account. This isolates pipeline activity in Sugar’s audit log and avoids breakage when a real user changes their password or leaves.

After filling the form, click Test connection. A successful test confirms that:

1. The instance URL resolves and /rest/v11_24/oauth2/token accepted the username + password
2. The user has at least read access to the me endpoint

If the test fails with invalid_grant, double-check the password and confirm the user can log in to SugarCRM’s web UI directly. SugarCRM’s password-grant OAuth flow uses the same credentials.

Source Properties

The source component is configured in Step 02 of the component editor.

Source Table (Object)

The dropdown lists every SugarCRM module the authenticated user can see. This includes:

• Curated standard modules. Exposed by default with rich datetime field metadata so incremental loading works out of the box
• Discovered custom modules. Anything else the user can see on the Sugar instance, queried from /rest/v11_24/metadata?type_filter=full_module_list once per pipeline run and merged into the picker

Curated standard modules

Object	SugarCRM module	Datetime fields available for incremental load
accounts	Accounts	date_entered, date_modified
contacts	Contacts	date_entered, date_modified
leads	Leads	date_entered, date_modified
opportunities	Opportunities	date_entered, date_modified, date_closed
cases	Cases	date_entered, date_modified
notes	Notes	date_entered, date_modified
tasks	Tasks	date_entered, date_modified, date_start, date_due
calls	Calls	date_entered, date_modified, date_start, date_end
meetings	Meetings	date_entered, date_modified, date_start, date_end
emails	Emails	date_entered, date_modified, date_sent
campaigns	Campaigns	date_entered, date_modified, start_date, end_date
users	Users	date_entered, date_modified
documents	Documents	date_entered, date_modified
quotes	Quotes	date_entered, date_modified, date_quote_expected_closed
contracts	Contracts	date_entered, date_modified, start_date, end_date
bugs	Bugs	date_entered, date_modified
prospects	Prospects	date_entered, date_modified

Custom modules

If your SugarCRM tenant has custom modules (e.g. Subsc_Subscriptions, My_Custom_Module), they appear in the dropdown under the lowercased module name (subsc_subscriptions, my_custom_module). The connector hits the SugarCRM REST endpoint at /rest/v11_24/{ModuleName} and exposes date_entered and date_modified as the default incremental fields. These are inherited from SugarBean, the base class every Sugar module extends, so they’re present on every record by default.

Click the Refresh button next to the dropdown to re-query SugarCRM’s module list. This is useful after an admin adds a new custom module or grants the integration user access to additional modules.

Load Type

Select how records are loaded on each pipeline run:

• Full Load. Fetches all records for the selected object on every run.
• Incremental Load. Fetches only records created or updated after a reference date. Recommended for scheduled pipelines once the initial historical load is complete, so you don’t re-process records that haven’t changed.

Incremental Load Settings

When Incremental Load is selected, three additional fields appear:

Field	Description
Sync date field	The datetime field used to filter records server-side. Defaults to date_modified for most modules. You can pick any datetime field listed for the object (for example date_closed on Opportunities or date_start on Meetings), and SugarCRM will filter on that field instead. The connector substitutes the picked field into the API filter filter[0][{field}][$gte]=..., so jobs work consistently regardless of which datetime field you choose.
Load records	The filter direction. newer than ( > ) fetches records with {field} >= reference date. older than ( < ) fetches records with {field} <= reference date.
Reference date	The source of the date value. Last successful run auto-fills incremental_load_date with the $_PACKAGE_LAST_SUCCESSFUL_JOB_SUBMISSION_TIMESTAMP system variable, so each scheduled run picks up only what changed since the previous run finished. This is the recommended option for scheduled pipelines. Fixed Date uses a specific calendar date, useful for one-off backfills. Variable uses a custom package variable, for cases where you need to drive the start date from a value other than the last successful run timestamp.

SugarCRM expects ISO 8601 dates. The connector passes the date value directly to Sugar as provided. If you use a system variable like $package_last_successful_job_submission_timestamp, Integrate.io fills it in with a UTC ISO 8601 timestamp automatically.

Field Flattening

The SugarCRM REST API returns nested JSON for some fields, including relate links, address objects, and custom field groups. The connector automatically flattens these into individual columns using underscore-separated naming so each leaf value lands in its own column instead of a serialized JSON blob.

For example, a relate field that the API returns as:

{
  "assigned_user_link": {
    "name": "Jane Doe",
    "id": "abc-123"
  }
}

is flattened into two columns: assigned_user_link_name and assigned_user_link_id. This lets you map the columns directly to your destination schema without writing a downstream transformation to parse JSON.

Schema

After configuring the source properties, the Schema section (Step 03) displays all available fields with their detected data types.

Common SugarCRM field types and how they map:

SugarCRM type	Integrate.io type	Notes
varchar / text / name / phone / email	string	All string-shaped fields
int / id	string	SugarCRM IDs are 36-char UUIDs returned as strings
datetime / date	string	ISO 8601 strings (e.g. 2026-04-28T18:01:12+10:00); cast to datetime in a downstream Select transformation if your destination needs typed timestamps
bool	boolean
decimal / float / currency	double
enum / multienum	string	Multi-select enums are returned as comma-separated strings
relate (assigned_user_id, account_id, etc.)	string	The ID of the related record. The display name is in the corresponding *_link_name field after flattening.

Use the field selector to choose which columns to include in your pipeline. You can rename fields with aliases and override the detected data type as needed.

Best Practices

• Use a dedicated integration user with the minimum module-level role needed. Avoid using a real person’s account. If their password changes, every pipeline breaks at the next run.
• Start with full load, switch to incremental once the destination is seeded with historical data. Use date_modified as the sync field for most modules, as it covers both inserts and updates. Only switch to date_entered if you specifically want inserts only.
• Pick the right datetime field for the module’s semantics. date_modified catches every change. date_closed on Opportunities only fires when the opportunity stage moves to Closed Won/Lost, which is useful for daily revenue reporting but skips opportunities still in flight. date_start on Calls/Meetings reflects when the activity is scheduled, not when it was logged.
• Custom module datetime fields: discovered custom modules default to date_entered and date_modified only. If your custom module has additional date fields you want to filter on (e.g. effective_date_c), pick the field manually in the dropdown. The connector will still build a valid filter.
• Refresh the object list after Sugar admins add or rename modules. The list is cached for 1 hour per connection; the Refresh button bypasses the cache.

FAQ

Q: Why is my custom module not in the dropdown?

The connector lists modules using the SugarCRM /rest/v11_24/metadata?type_filter=full_module_list endpoint, which only returns modules the authenticated user can see. If the integration user doesn’t have a role granting access to that module, it won’t appear. Have a SugarCRM admin grant the user a role with read access to the module, then click Refresh in the source dropdown.

Q: I picked date_entered for incremental load, but the pipeline returns data older than my reference date. What’s wrong?

This was a bug in earlier versions of the connector. The filter was hard-coded for date_modified, so when you picked a different field the filter was silently dropped and SugarCRM returned all records. The current connector substitutes the picked field into the filter automatically, so the issue should be resolved. If you’re still seeing it, check the pipeline log for the line Incremental date filter for API: and confirm it shows filter[0][date_entered][$gte]=... (not date_entered=...). If the wrapper is missing, your job is running against an older Pig JAR. Contact support to upgrade.

Q: What SugarCRM API version does the connector use?

v11_24. This applies to both the OAuth token endpoint (/rest/v11_24/oauth2/token) and all data endpoints (/rest/v11_24/{Module}). SugarCRM’s v11 is an alias for the latest minor, but the connector pins to a specific version for stability across SugarCRM tenant upgrades.

Q: How does pagination work?

The connector uses SugarCRM’s offset-based pagination (offset= + max_num=1000 per page). Up to 10,000 pages are followed per object per run, which translates to 10 million records, enough for any single full load. The schema-discovery and data-preview steps cap at 50 records to keep the UI responsive.

Q: Is there a rate limit?

The connector throttles outbound requests to 15 requests per second by default and retries up to 3 times on HTTP 429 / 503 with exponential backoff capped at 60 seconds. SugarCRM’s per-tenant API quotas are typically much higher, so this rarely matters in practice, but if you run many concurrent pipelines against the same tenant you may want to stagger them.

Related

REST API Source

API Endpoints with Pagination Support

Defining Connections

Sources Overview