# Marketo ## Pipeline Concepts Before setting up the Pipeline, learn about Pipeline concepts [here](https://docs.sprinkledata.com/product/ingesting-your-data/pipelines) ## Step by Step Guide ### STEP-1: Configure Connection To learn about Connection, refer [here](https://docs.sprinkledata.com/product/ingesting-your-data/pipelines) * Log into Sprinkle application * Navigate to Ingest -> Connections Tab -> New Connection -> * Select Marketo * Provide all the mandatory details To get the details: Login to your Marketo account and in the admin section, click on “Users and Roles” on the left panel. Create a new role. Reveal the list of role permissions by clicking Access API. Post this, an API-only user needs to be created and associated (On clicking the checkbox) with the API role created in the previous step. Create a custom application, Admin -> Launch point -> Create a new service. On providing the display name, service type, description and email. On clicking “view details” [Client ID](https://developers.marketo.com/blog/quick-start-guide-for-marketo-rest-api/) and [Client Secret](https://developers.marketo.com/blog/quick-start-guide-for-marketo-rest-api/) are obtained. Marketo Admin -> Web Services Panel -> [Endpoint URL](https://developers.marketo.com/rest-api/base-url/) and below that [Identity URL](https://developers.marketo.com/rest-api/base-url/) can also be found. * *Name*: Name to identify this connection * *Client Id* * *Client Secret* * *Endpoint Url* * *Identity Url* * Test Connection * Create ### STEP-2: Configure Pipeline To learn about Pipeline, refer [here](https://docs.sprinkledata.com/product/ingesting-your-data/pipelines) * Navigate to Ingest -> Pipeline Tab -> Add -> * Select Marketo * Provide the name -> Create * **Connection Tab**: * From the drop-down, select the name of connection created in STEP-2 * Update ### STEP-3: Create Dataset **Datasets Tab**: To learn about Dataset, refer [here](https://docs.sprinkledata.com/product/ingesting-your-data/pipelines). Add Dataset for each report/dataset that you want to integrate, providing following details * *Report Type* (Required): Select from drop-down: * *Activity\_Types* * *Custom\_Activity\_Types* * *Campaigns* * *Smart\_Campaigns* * *Channels* * *Custom\_Objects* * *Emails* * *Lists* * *Smart\_Lists* * *Segmentations* * *Programs* * *Tag\_Types* * Flatten Level(Required) : Select One Level or Multi Level. In one level, flattening will not be applied on complex type. They will be stored as string. In multi level, flattening will be applied in complex level till they become simple type. * *Destination Schema* (Required) : Data warehouse schema where the table will be ingested into * *Destination Table name* (Required) : It is the table name to be created on the warehouse. If not given, sprinkle will create like ds\_\\_\ * *Destination Create Table Clause*: Provide additional clauses to warehouse-create table queries such as clustering, partitioning, and more, useful for optimizing DML statements. [Learn more](https://docs.sprinkledata.com/product/ingesting-your-data/pipelines/databases/features/destination-create-table-clause) on how to use this field. * Create ### STEP-4: Run and schedule Ingestion In the **Ingestion Jobs** tab**:** * Trigger the Job, using Run button * To schedule, enable Auto-Run. Change the frequency if needed ### Dataset Fields All the datasets can have the following fields
emails * id * name * description * createdat * updatedat * url * subject\_type * subject\_value * fromname\_type * fromname\_value * fromemail\_type * fromemail\_value * replyemail\_type * replyemail\_value * folder\_type * folder\_value * folder\_foldername * operational * textonly * publishtomsi * webview * status * template * workspace * isopentrackingdisabled * version * autocopytotext
activity_types * id name * description * primaryattribute\_name * primaryattribute\_datatype * attributes\_0\_\_name * attributes\_0\_\_datatype * attributes\_1\_\_name * attributes\_1\_\_datatype * attributes\_2\_\_name * attributes\_2\_\_datatype * attributes\_3\_\_name * attributes\_3\_\_datatype * attributes\_4\_\_name * attributes\_4\_\_datatype * attributes\_5\_\_name * attributes\_5\_\_datatype * attributes\_6\_\_name * attributes\_6\_\_datatype * attributes\_7\_\_name * attributes\_7\_\_datatype * attributes\_8\_\_name * attributes\_8\_\_datatype * attributes\_9\_\_name * attributes\_9\_\_datatype * attributes\_10\_\_name * attributes\_10\_\_datatype * attributes\_11\_\_name * attributes\_11\_\_datatype * attributes\_12\_\_name * attributes\_12\_\_datatype * attributes\_13\_\_name * attributes\_13\_\_datatype
programs * id * name * description * createdat * updatedat * url * type * channel * folder\_type * folder\_value * folder\_foldername * status * workspace * exheadstart
smart_campaigns * id * name * description * createdat * updatedat * status * type * issystem * isactive * isrequestable * iscommunicationlimitenabled * recurrence\_weekdayonly * qualificationruletype * qualificationruleinterval * qualificationruleunit * workspace * smartlistid * flowid * ecomputedurl
smart_lists * id * name * createdat * updatedat * url * folder\_id * folder\_type * workspace * description
segmentations * id * name * description * createdat * updatedat * url * folder\_type * folder\_value * status * workspace
lists * id * name * workspaceid * workspacename * createdat * updatedat
channels * id * name * applicableprogramtype * progressionstatuses\_0\_\_name * progressionstatuses\_0\_\_step * progressionstatuses\_0\_\_type * progressionstatuses\_0\_\_success * progressionstatuses\_0\_\_hidden * progressionstatuses\_1\_\_name * progressionstatuses\_1\_\_step * progressionstatuses\_1\_\_type * progressionstatuses\_1\_\_success * progressionstatuses\_1\_\_hidden * progressionstatuses\_2\_\_name * progressionstatuses\_2\_\_step * progressionstatuses\_2\_\_type * progressionstatuses\_2\_\_success * progressionstatuses\_2\_\_hidden * createdat * updatedat * progressionstatuses\_3\_\_name * progressionstatuses\_3\_\_step * progressionstatuses\_3\_\_type * progressionstatuses\_3\_\_success * progressionstatuses\_3\_\_hidden * progressionstatuses\_4\_\_name * progressionstatuses\_4\_\_step * progressionstatuses\_4\_\_type * progressionstatuses\_4\_\_success * progressionstatuses\_4\_\_hidden * progressionstatuses\_5\_\_name * progressionstatuses\_5\_\_step * progressionstatuses\_5\_\_type * progressionstatuses\_5\_\_success * progressionstatuses\_5\_\_hidden * progressionstatuses\_6\_\_name * progressionstatuses\_6\_\_step * progressionstatuses\_6\_\_type * progressionstatuses\_6\_\_success * progressionstatuses\_6\_\_hidden * progressionstatuses\_7\_\_name * progressionstatuses\_7\_\_step * progressionstatuses\_7\_\_type * progressionstatuses\_7\_\_success * progressionstatuses\_7\_\_hidden * progressionstatuses\_8\_\_name * progressionstatuses\_8\_\_step * progressionstatuses\_8\_\_type * progressionstatuses\_8\_\_success * progressionstatuses\_8\_\_hidden
describe_leads * id * displayname * datatype * length * rest\_name * rest\_readonly * soap\_name * soap\_readonly
custom_activity_types * apiName * attributes * createdAt * description * filterName * id * name * primaryAttribute * status * triggerName * updatedAt
custom_objects * createdAt * dedupeFields * description * displayName * pluralName * fields * idField * apiName * relationships * searchableFields * updatedAt * state * version
campaigns * id * name * active * createdAt * description * programId * programName * type * updatedAt * workspaceName
tag_types * applicableProgramTypes * required * tagType