Sprinkle Docs
Search…
REST API Datasource
Guide to ingest data using your application's REST APIs
With REST API datasource, you can pull data from any application which exposes REST APIs, and ingest into the data warehouse, without any coding.

Datasource Concepts

Before setting up the datasource, learn about datasource concepts here​

Step by Step Guide

STEP-1: Configure REST API Connection

To learn about Connection, refer here​
  • Log into Sprinkle application
  • Navigate to Datasources -> Connections Tab -> New Connection ->
  • Select REST API
  • Provide all the mandatory details
    • Name: Name to identify this connection
  • Create

STEP-2: Configure REST API datasource

To learn about datasource, refer here​
  • Navigate to Datasources -> Datasources Tab -> Add ->
  • Select REST API
  • Provide the name -> Create
  • Connection Tab:
    • From the drop-down, select the name of connection created in STEP-2
    • Update

STEP-3: Create Dataset

Datasets Tab: To learn about Dataset, refer here. Add Dataset for each API that you want to ingest data from, providing following details
  • URL with Params (Required) : Give base url (with parameters if any). example - https://base_url_example?params_key1=params_value1ΒΆms_key2=params_value2
  • Request Method (Required)
    • GET
    • POST
      • Body: Raw Data
        • Content Type: Select from the drop-down
        • Raw Data
  • Headers (Optional)
  • Data Root (Optional) : Give the json path from which data should be extracted. ex - {country:[{state:st1,city:abc},{state:st2,city:xyz}],offset:1} if data_root is country, then {state:st1, city:abc},{state:st2,city:xyz} will be stored in two different rows in the warehouse table. Otherwise whole json will be flattened and stored in single row. For complex type give keys with dot(.) separated. ex - {book:{writer:[{name:abc},{name,xyz}]},offset:1} for book.writer it will give {name:abc} and {name,xyz} in two separate rows.
  • Flatten Level (Required): Select from One Level or Multi Level. In one level, flattening will not be applied on complex type. They will be stored as string. In multi level, flattening will be applied in complex level till they become simple type.
  • Destination Schema (Required) : Data warehouse schema where the table will be ingested into
  • Destination Table name (Required) : It is the table name to be created on the warehouse. If not given, sprinkle will create like ds_<datasourcename>_<tablename>
  • Create

STEP-4: Run and schedule Ingestion

In the Ingestion Jobs tab:
  • Trigger the Job, using Run button
  • To schedule, enable Auto-Run. Change the frequency if required
Copy link
On this page
Datasource Concepts
Step by Step Guide
STEP-1: Configure REST API Connection
STEP-2: Configure REST API datasource
STEP-3: Create Dataset
STEP-4: Run and schedule Ingestion