Sprinkle Docs
  • What is Sprinkle?
  • Quick Start
  • Analysing your data
    • πŸ”­Analytics Overview
    • πŸ’ Data Models
      • *️Variables
      • 🌲Hierarchies
      • 🀿Column Mask
    • πŸŽ‰Switch to New Reports & Dashboards
    • πŸ†•Reports
      • Overview
      • Build Using Tables
        • Create a new Report
        • Layout and options
        • Build and Format - Overview
        • Apply Row Limits
        • Identify Date Columns
        • Filter your data
        • Visualizations
          • Table
          • Pivot
          • Line Chart
          • Bar Chart
          • Column Chart
          • Area Chart
          • Combo Chart
          • Scatter & Bubble Plot
          • Pie Chart
          • Funnel Chart
          • Stat Card
          • Point Map
          • Heat Map
          • Radial gauge chart
        • Advanced Features
          • Custom Analysis
          • Variables
          • Table & Quick Calculations
          • Drill - Hierarchical & Date
          • Break Out
          • RLS in Table reports
          • Scheduled Exports
          • Embedding Table Reports
      • Build Using Models
        • Create a new report
        • Layout and options
        • Visualizations
        • Advanced Features
      • Build SQL Reports
        • Create a new Report
        • Layout and options
        • Writing a SQL Code on Editor
        • Visualizations
        • Variables in SQL Reports
    • πŸ†•Dashboards
      • πŸŒ€Filters
      • πŸ‘†Click Behaviour
      • ⏰Data Alerts
      • πŸ—“οΈDate Drill
      • πŸ“€Scheduled Exports
      • πŸ”—Embed link
      • πŸ–₯️Dashboard layout
      • πŸ“±Mobile Dashboards
  • Transforming your data
    • πŸ”°SQL Transform
    • πŸ““Python Notebooks
  • Integrating your data
    • ☁️Destination Warehouses
      • AWS Athena
        • Manage storage of Flow tables
      • AWS Redshift
      • Azure Synapse
      • Databricks
      • Google BigQuery
      • MySQL
      • Postgres
      • Snowflake
      • SQL Server
      • K8 Setup
        • AWS EKS
        • Google GKE
        • Azure AKS
    • βš™οΈWarehouse & Storage Setup
  • Ingesting your data
    • β˜„οΈData Imports
      • Databases
        • Azure Cosmos DB
        • Azure Table Storage
        • Google BigQuery
        • Mongo DB
        • MySQL DB
        • Oracle DB
        • Postgres DB
        • SQL Server DB
        • Features
          • Ingestion Modes
          • Add Multiple Datasets
          • CDC Setup
            • CDC setup in Mysql
            • CDC setup in Postgres
            • CDC setup in Mongo
            • CDC setup in SQL Server
          • Destination Create Table Clause
          • SSH Tunnel Setup
      • Files
        • AWS S3
        • AWS S3 External
        • Azure Blob
        • FTP
        • Google Cloud Storage
        • Google Sheet
        • SFTP
      • Applications
        • Apple Search Ads
        • Appsflyer
        • Branch
        • Clevertap
        • Facebook Ads
        • Freshdesk
        • Freshsales
        • Google Ads
        • Google Ads V2
        • Google Analytics
        • Google Analytics 4
        • Google Analytics MCF
        • Google Search Console
        • Hubspot
        • Impact Ads
        • Intercom
        • Klaviyo
        • Leadsquared
        • LinkedIn Ads
        • Magento
        • Mailchimp
        • Marketo
        • Mixpanel
        • MoEngage
        • Rocketlane
        • Salesforce
        • SAP S4
        • Shopify
        • Snapchat Marketing
        • TikTok Ads
        • WooCommerce
        • Zendesk Chat
        • Zendesk Support
        • Zoho Analytics
        • Zoho Books
        • Zoho CRM
        • Zoho Desk
        • Zoho Invoice
        • Zoho Subscription
      • Events
        • Apache Kafka
        • AWS Kinesis
        • Azure EventHub
    • πŸ“€File Uploads
    • πŸ€–API Pulls
    • πŸ•ΈοΈWebhooks
  • Collaborating on data
    • πŸ“€Sharing
    • πŸ’¬Comments
    • ⚑Activity
    • 🏷️Labels
  • Managing Schedules and Data Refreshes
    • ⏱️Schedules
    • πŸ””Notifications
  • User Management
    • πŸ”‘Access Management
    • πŸ§‘β€πŸ€β€πŸ§‘Groups
    • πŸ“‚Folders
    • πŸ”„Syncing users, groups and RLS
    • πŸ“§Azure AD Integration
  • Data Security & Privacy
    • πŸ”Security at Sprinkle
    • πŸ“„GDPR
    • πŸ“„Privacy Policy
  • Release Notes
    • πŸ“’Release Notes
      • πŸ—’οΈRelease Notes - v12.1 (New)
      • πŸ—’οΈRelease Notes - v12.0
      • πŸ—’οΈRelease Notes - v11.0
      • πŸ—’οΈRelease Notes - v10.8
      • πŸ—’οΈRelease Notes - v10.7
      • πŸ—’οΈRelease Notes - v10.6
      • πŸ—’οΈRelease Notes - v10.5
      • πŸ—’οΈRelease Notes - v10.4
      • πŸ—’οΈRelease Notes - v10.3
      • πŸ—’οΈRelease Notes - v10.2
      • πŸ—’οΈRelease Notes - v10.1
      • πŸ—’οΈRelease Notes - v10.0
      • πŸ—’οΈRelease Notes - v9.31
      • πŸ—’οΈRelease Notes - v9.30
      • πŸ—’οΈRelease Notes - v9.29
      • πŸ—’οΈRelease Notes - v9.28
      • πŸ—’οΈRelease Notes - v9.27
      • πŸ—’οΈRelease Notes - v9.25
      • πŸ—’οΈRelease Notes - v9.24
      • πŸ—’οΈRelease Notes - v9.23
      • πŸ—’οΈRelease Notes - v9.22
      • πŸ—’οΈRelease Notes - v9.21
      • πŸ—’οΈRelease Notes - v9.20
      • πŸ—’οΈRelease Notes - v9.19
      • πŸ—’οΈRelease Notes - v9.18
      • πŸ—’οΈRelease Notes - v9.17
      • πŸ—’οΈRelease Notes - v9.16
      • πŸ—’οΈRelease Notes - v9.14
      • πŸ—’οΈRelease Notes - v9.13
      • πŸ—’οΈRelease Notes - v9.12
      • πŸ—’οΈRelease Notes -v9.8
      • πŸ—’οΈRelease Notes - v9.7
      • πŸ—’οΈRelease Notes - v9.6
      • πŸ—’οΈRelease Notes - v9.5
      • πŸ—’οΈRelease Notes - v9.4
      • πŸ—’οΈRelease Notes - v9.3
      • πŸ—’οΈRelease Notes - v9.2
      • πŸ—’οΈRelease Notes - v9.1
      • πŸ—’οΈRelease Notes - v9.0 (Major)
      • πŸ—’οΈRelease Notes - v7.23
      • πŸ—’οΈRelease Notes - v7.21
      • πŸ—’οΈRelease Notes - v7.20
      • πŸ—’οΈRelease Notes - v7.15
      • πŸ—’οΈRelease Notes - v7.14
      • πŸ—’οΈRelease Notes - v7.13
Powered by GitBook
On this page
  • Overview
  • Watch Video
  • Feature Walkthrough
  • Create Python Notebook
  • Using Sprinkle SDK
  • How to work on Spark session operations?
  1. Transforming your data

Python Notebooks

PreviousSQL TransformNextDestination Warehouses

Last updated 2 years ago

Overview

You can use your favourite Notebooks on Sprinkle. Through the navigation panel click on Notebooks.

The Notebook is an open-source web application that allows you to create and share documents that contain live code, narrative text, equations, and visualizations.

Use notebooks for data cleaning, transformations, numerical simulation, statistical modeling, data visualization & machine learning.

Watch Video

Create Python Notebook

  • Provide a name for the Python Notebook.

  • Select Kernel (Optional): Select python3 from the dropdown.

  • Select VM Size (Optional): Select from the below options of CPUs and Virtual Machine Memory Size from the dropdown for the Python Notebook.

    • Option 1 - 1 CPU & 1700 Mi (Mebibyte) Virtual Storage Memory.

    • Option 2 - 2 CPUs & 1800 Mi (Mebibyte) Virtual Storage Memory.

  • User API Key and User API Secret are optional to fill in this form. In case you want to use the Sprinkle SDK functions, it is mandatory to provide the API Key and API Secret. In the settings, these can also be provided after the Python Notebook is created.

Using Sprinkle SDK

Sprinkle SDK enables you to Import your data from sprinkle’s SQL Explore and Reports to be used in the notebook

  • Import Sprinkle SDK

from sprinkleSdk import SprinkleSdk as sp
  • Read Report

Reads data from the mentioned report into a data frame

df = sp.readReport('<report_id>')
  • Read SQL Explore

Reads data from the mentioned SQL Explore into a data frame

df = sp.readExplore('<explore_id>')

Once data is imported, you can run all kinds of analyses using these data in your Notebook

  • Create a table or update an existing table in the warehouse using a data frame

sp.createOrUpdateTable('<dataImportName>','<destinationTableName>', df)

Multiple tables can be created in a single Data Import. The data Import created using the above function can be seen in the Ingest -> File Uploads.

  • Drop the table from the warehouse

sp.dropTable('<table_name>')

How to work on Spark session operations?

  • Get spark session with default configurations

spark = sp.getOrCreate()
  • Change the spark app name while creating the default spark session

spark = sp.appName('some-name').getOrCreate()
  • Get a spark session where the user can customize the configuration

spark = sp.sparkBuilder()
appName('some-name')
.config("spark.some.config.option1", "some-value")
.config("spark.some.config.option2", "some-value")
.getOrCreate()

Feature Walkthrough

Click on Transform -> Python Notebooks on the left navigation pane, to start using the Python Notebooks feature on Sprinkle. The listing page lists all the Python Notebooks that have been created.

Click on Create New Notebook on the top right corner of the page to create a new Python Notebook.

To generate API Key and Secret, click on your user icon on the top right, then Account -> API Keys. Click on Generate new, to create a new API Key and Secret for yourself.

πŸ““
🚢
πŸ–±οΈ
πŸ–±οΈ
πŸ–±οΈ
πŸ–±οΈ
πŸ“”
πŸ“Š
πŸ’Ή
πŸ“Ί
Python Notebooks : Explanation & Feature Walkthrough