Sprinkle Docs
  • What is Sprinkle?
  • Quick Start
  • Analysing your data
    • 🔭Analytics Overview
    • 💠Data Models
      • *️Variables
      • 🌲Hierarchies
      • 🤿Column Mask
    • 🎉Switch to New Reports & Dashboards
    • 🆕Reports
      • Overview
      • Build Using Tables
        • Create a new Report
        • Layout and options
        • Build and Format - Overview
        • Apply Row Limits
        • Identify Date Columns
        • Filter your data
        • Visualizations
          • Table
          • Pivot
          • Line Chart
          • Bar Chart
          • Column Chart
          • Area Chart
          • Combo Chart
          • Scatter & Bubble Plot
          • Pie Chart
          • Funnel Chart
          • Stat Card
          • Point Map
          • Heat Map
          • Radial gauge chart
        • Advanced Features
          • Custom Analysis
          • Variables
          • Table & Quick Calculations
          • Drill - Hierarchical & Date
          • Break Out
          • RLS in Table reports
          • Scheduled Exports
          • Embedding Table Reports
      • Build Using Models
        • Create a new report
        • Layout and options
        • Visualizations
        • Advanced Features
      • Build SQL Reports
        • Create a new Report
        • Layout and options
        • Writing a SQL Code on Editor
        • Visualizations
        • Variables in SQL Reports
    • 🆕Dashboards
      • 🌀Filters
      • 👆Click Behaviour
      • ⏰Data Alerts
      • 🗓️Date Drill
      • 📤Scheduled Exports
      • 🔗Embed link
      • 🖥️Dashboard layout
      • 📱Mobile Dashboards
  • Transforming your data
    • 🔰SQL Transform
    • 📓Python Notebooks
  • Integrating your data
    • ☁️Destination Warehouses
      • AWS Athena
        • Manage storage of Flow tables
      • AWS Redshift
      • Azure Synapse
      • Databricks
      • Google BigQuery
      • MySQL
      • Postgres
      • Snowflake
      • SQL Server
      • K8 Setup
        • AWS EKS
        • Google GKE
        • Azure AKS
    • ⚙️Warehouse & Storage Setup
  • Ingesting your data
    • ☄️Data Imports
      • Databases
        • Azure Cosmos DB
        • Azure Table Storage
        • Google BigQuery
        • Mongo DB
        • MySQL DB
        • Oracle DB
        • Postgres DB
        • SQL Server DB
        • Features
          • Ingestion Modes
          • Add Multiple Datasets
          • CDC Setup
            • CDC setup in Mysql
            • CDC setup in Postgres
            • CDC setup in Mongo
            • CDC setup in SQL Server
          • Destination Create Table Clause
          • SSH Tunnel Setup
      • Files
        • AWS S3
        • AWS S3 External
        • Azure Blob
        • FTP
        • Google Cloud Storage
        • Google Sheet
        • SFTP
      • Applications
        • Apple Search Ads
        • Appsflyer
        • Branch
        • Clevertap
        • Facebook Ads
        • Freshdesk
        • Freshsales
        • Google Ads
        • Google Ads V2
        • Google Analytics
        • Google Analytics 4
        • Google Analytics MCF
        • Google Search Console
        • Hubspot
        • Impact Ads
        • Intercom
        • Klaviyo
        • Leadsquared
        • LinkedIn Ads
        • Magento
        • Mailchimp
        • Marketo
        • Mixpanel
        • MoEngage
        • Rocketlane
        • Salesforce
        • SAP S4
        • Shopify
        • Snapchat Marketing
        • TikTok Ads
        • WooCommerce
        • Zendesk Chat
        • Zendesk Support
        • Zoho Analytics
        • Zoho Books
        • Zoho CRM
        • Zoho Desk
        • Zoho Invoice
        • Zoho Subscription
      • Events
        • Apache Kafka
        • AWS Kinesis
        • Azure EventHub
    • 📤File Uploads
    • 🤖API Pulls
    • 🕸️Webhooks
  • Collaborating on data
    • 📤Sharing
    • 💬Comments
    • ⚡Activity
    • 🏷️Labels
  • Managing Schedules and Data Refreshes
    • ⏱️Schedules
    • 🔔Notifications
  • User Management
    • 🔑Access Management
    • 🧑‍🤝‍🧑Groups
    • 📂Folders
    • 🔄Syncing users, groups and RLS
    • 📧Azure AD Integration
  • Data Security & Privacy
    • 🔐Security at Sprinkle
    • 📄GDPR
    • 📄Privacy Policy
  • Release Notes
    • 📢Release Notes
      • 🗒️Release Notes - v12.1 (New)
      • 🗒️Release Notes - v12.0
      • 🗒️Release Notes - v11.0
      • 🗒️Release Notes - v10.8
      • 🗒️Release Notes - v10.7
      • 🗒️Release Notes - v10.6
      • 🗒️Release Notes - v10.5
      • 🗒️Release Notes - v10.4
      • 🗒️Release Notes - v10.3
      • 🗒️Release Notes - v10.2
      • 🗒️Release Notes - v10.1
      • 🗒️Release Notes - v10.0
      • 🗒️Release Notes - v9.31
      • 🗒️Release Notes - v9.30
      • 🗒️Release Notes - v9.29
      • 🗒️Release Notes - v9.28
      • 🗒️Release Notes - v9.27
      • 🗒️Release Notes - v9.25
      • 🗒️Release Notes - v9.24
      • 🗒️Release Notes - v9.23
      • 🗒️Release Notes - v9.22
      • 🗒️Release Notes - v9.21
      • 🗒️Release Notes - v9.20
      • 🗒️Release Notes - v9.19
      • 🗒️Release Notes - v9.18
      • 🗒️Release Notes - v9.17
      • 🗒️Release Notes - v9.16
      • 🗒️Release Notes - v9.14
      • 🗒️Release Notes - v9.13
      • 🗒️Release Notes - v9.12
      • 🗒️Release Notes -v9.8
      • 🗒️Release Notes - v9.7
      • 🗒️Release Notes - v9.6
      • 🗒️Release Notes - v9.5
      • 🗒️Release Notes - v9.4
      • 🗒️Release Notes - v9.3
      • 🗒️Release Notes - v9.2
      • 🗒️Release Notes - v9.1
      • 🗒️Release Notes - v9.0 (Major)
      • 🗒️Release Notes - v7.23
      • 🗒️Release Notes - v7.21
      • 🗒️Release Notes - v7.20
      • 🗒️Release Notes - v7.15
      • 🗒️Release Notes - v7.14
      • 🗒️Release Notes - v7.13
Powered by GitBook
On this page
  • STEP-1: Create GKE Cluster
  • STEP-2: Generate user token
  • STEP-3: Configure GKE connection
  • Checking Sprinkle Job Logs
  1. Integrating your data
  2. Destination Warehouses
  3. K8 Setup

Google GKE

Guide to create GKE cluster and connect to Sprinkle

Sprinkle requires GKE cluster for data ingestion and processing. This is required to process all your data locally within your Google private network.

Follow the below steps to create and configure GKE cluster:

STEP-1: Create GKE Cluster

Create a GKE Standard cluster with all default settings with following configuration:

Networking

  • Select Public cluster

  • Enable control plane authorized networks: Provide any name like Sprinkle network and put the CIDR range (to whitelist the Sprinkle IPs), as 34.93.254.126/32

  • Default Node Pool: 2 nodes

  • Node Type: n1-standard-1

STEP-2: Generate user token

Sprinkle authenticates to GKE cluster using kubernetes user token. Follow below steps to generate User token:

  • Install kubectl and gcloud CLI

  • Generate ~/.kube/config file:

gcloud container clusters get-credentials  --project <project_id>  --zone <zone> <cluster_name>
  • To verify the setup, run kubectl command to fetch running nodes:

kubectl get nodes
  • Create namespace

kubectl create namespace sprinkle
  • Create Admin User In kubernetes: Create file service-account-create.yml:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: sprinkle-admin-user
  namespace: sprinkle
kubectl apply -f service-account-create.yml
  • Create ClusterRoleBinding: create a file role-binding.yml:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: sprinkle-admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: sprinkle-admin-user
  namespace: sprinkle
kubectl apply -f role-binding.yml
  • To create a long-lived API token for a ServiceAccount, you create a new secret file sprinkle-admin-secret.yml with a special annotation, kubernetes.io/service-account.name:

apiVersion: v1
kind: Secret
metadata:
  name: sprinkle-admin-secret
  namespace: sprinkle
  annotations:
    kubernetes.io/service-account.name: sprinkle-admin-user
type: kubernetes.io/service-account-token
kubectl apply -f sprinkle-admin-secret.yml
  • User token Token will be printed by this command, note down the generated token:

kubectl describe secrets/sprinkle-admin-secret -n sprinkle

STEP-3: Configure GKE connection

  • Log into Sprinkle application

  • Navigate to Admin -> Drivers -> Create Compute

  • Select GKE

  • Provide all the mandatory details

    • Distinct Name: Any name to identify the connection

    • Cluster Url: Provide the url of the GKE created in the format https://<ENDPOINT>

    • Cluster CA Certificate: Provide Cluster CA certificate of the GKE cluster

    • Is Certificate Encoded: No

    • User Token: Paste the User Token generated in STEP-2 above

    • Deploy namespace: sprinkle

    • Supported Kernels : Enter name of supported kernels. For more than one kernel, separate them by comma

    • CPU and VM size for Notebook : No of CPU and size of each VM. If more than one entry then separate them by comma.

    • CPU and VM size for Ingestion : No of CPU and size of each VM.

    • Node group labels : Key value pair separated by comma, ex:- key1: val1, key2: val2, key3: val3. If label is configured then all pods will be launched only on the node group having the same label.

    • Advance Settings : Yes

    • Notebook Idle Timeout : Time in minutes

    • Notebook: (If you want to use a Notebook for transform, make Notebook enable.) Yes or No

  • Test Connection

  • Create

Checking Sprinkle Job Logs

  • Search for "Logging".

  • On the left panel click on Logs Explorer.

  • From the Query panel

    • Click on Resource -> Select Kubernetes Container -> Choose cluster name -> Choose Namespace

    • In the Search container name box, put the sprinkle job id. (To find the sprinkle job id, click on "Show details" link in the sprinkle UI for that job)

  • Run Query.

  • Checks for logs.

PreviousAWS EKSNextAzure AKS

Last updated 7 months ago

Notebook Docker Container url : Url of the Jupyter Notebook docker container, available on dockerhub. See available images

Open google cloud platform ->

☁️
here
https://console.cloud.google.com/