Add Multiple Datasets
How to use the Add Multiple Datasets feature?
Last updated
How to use the Add Multiple Datasets feature?
Last updated
When adding a large number of datasets, the process becomes tedious and repetitive.
With the Add multiple datasets feature, you can select multiple datasets from a source database at once. Through the multi-select form, you can control which source tables are to be ingested, and how and what data is ingested from those source tables. You can configure options like exclude/mask columns, flatten JSON, complete and incremental fetch, etc. from the Add multiple datasets form.
This feature is live ⚡ for MySQL DB and Postgres DB sources in Data Imports
Create a Data Import of the above-mentioned source types. Establish a connection and move to the next stage of the process to Select Datasets for Ingestion.
There are two ways by which you can select the datasets (tables) from the source. Add Dataset - lets you select datasets one by one. Add multiple datasets - this enables you to add multiple datasets to the list in a single go. 🖱️Click on Add multiple datasets
Select the Database name from where the source tables are to be selected. On selecting the Database, Sprinkle fetches the source table details and prepares the form for you. In the example below we have selected the "classicmodels" database. The tabular form gets generated which shows all the tables in the database.
🖱️Check the Source tables that you want to be ingested into your warehouse. The checkbox on the top can be used to select all. Below we have selected all the tables from "classicmodels" database except the employee and office tables.
Make edits for the selected source tables to configure features like Exclude Columns, Mask Columns, and Mode while ingesting.
Change the Mode to Incremental if required. 📖 Read about the Ingestion modes supported in Sprinkle. On changing the mode to incremental, the detailed form opens up to enable you to configure the incremental ingestion settings.
Click on Create, once you are done with configuring the datasets. On successful addition of the dataset, you are directed to the listing page.
The bulk edit is available on the Datasets listing page. The already selected dataset's configuration can be edited in a single go, just like in the case of adding multiple datasets.
⚒️How to use Bulk Edit?
Click on Bulk Edit
Select the Database - All the already selected datasets from the particular database with their related settings appear in a tabular form. Make necessary edits and click on Save.
Select the datasets on the listing page that you want to reset. The reset action deletes the warehouse table and storage data, but the table and storage data will be recreated upon the next ingestion schedule.
Select the datasets on the listing page that you want to delete. The delete action deletes the warehouse table and storage data.
