Import from BigQuery

If you have data stored online in Google BigQuery, you can import this data easily to the Peltarion Platform and start training models on it.

In the Datasets view, click on the BigQuery button. A small dialog will open to let you authenticate inside your warehouse, and select the table you want to use as a dataset.

  • We only use your authentication credentials to read the table you want to import, and we don’t store them after the import has finished.

BigQuery import

BigQuery import lets you retrieve data stored in a Google BigQuery table, and turns it into a dataset inside your Peltarion Platform project.
This allows you to easily train models on large datasets available to you or your company.

To get started, click on the BigQuery Import button and follow these 3 steps:

  1. Authentication
    Select the Google Account you want to use, and click Allow when asked to allow to view your data in Google BigQuery.
    If you are not already signed in this account, Google will ask you to identify yourself.

    • If you have several Google Accounts, select the one which has access to the BigQuery table you want to use.

    • Peltarion does not store the access permission.
      This means that Peltarion loses all access to your BigQuery information and data as soon as you close the import dialog. This also means that you need to authenticate every time you open the import dialog.

  2. Source
    Google BigQuery charges a fee when you retrieve data from their storage.

    1. Select the Billing project that should be charged for the data transfer from BigQuery to the Peltarion Platform. The account you authenticate with needs to have the permission for the selected Billing project.

    2. Then find your table by selecting the Project ID, Dataset, and Table from the ones visible to your account. The platform creates one dataset from a single, entire BigQuery table.
      Note that you cannot import from BigQuery external tables, that is, where the data is not stored in BigQuery storage.
      You might need to create a new table in your BigQuery project before importing it on the platform, especially:

      • If you need data spread across several BigQuery tables

      • If an existing table has more data (columns or rows) than you need

  3. Preview
    The preview lets you check that the table contains the data you expect before importing it (there is no transfer cost incurred from showing the preview).

When everything looks good click on Create, and a new dataset will be created from the BigQuery table, ready to be used in your project’s experiments!

Was this page helpful?
Yes No