Documentation

Write to Google BigQuery

To write data to Google BigQuery with Flux:

  1. Import the sql package.

  2. Pipe-forward data into sql.to() and provide the following parameters:

    • driverName: bigquery
    • dataSourceName: See data source name
    • table: Table to write to
    • batchSize: Number of parameters or columns that can be queued within each call to Exec (default is 10000)
import "sql"

data
    |> sql.to(
        driverName: "bigquery",
        dataSourceName: "bigquery://projectid/?apiKey=mySuP3r5ecR3tAP1K3y",
        table: "exampleTable",
    )
  • Copy
  • Fill window

BigQuery data source name

The bigquery driver uses the following DSN syntaxes (also known as a connection string):

bigquery://projectid/?param1=value&param2=value
bigquery://projectid/location?param1=value&param2=value
  • Copy
  • Fill window

Common BigQuery URL parameters

  • dataset - BigQuery dataset ID. When set, you can use unqualified table names in queries.

BigQuery authentication parameters

The Flux BigQuery implementation uses the Google Cloud Go SDK. Provide your authentication credentials using one of the following methods:

  • Set the GOOGLE_APPLICATION_CREDENTIALS environment variable to identify the location of your credential JSON file.

  • Provide your base-64 encoded service account, refresh token, or JSON credentials using the credentials URL parameter in your BigQuery DSN.

    Example credentials URL parameter
    bigquery://projectid/?credentials=eyJ0eXBlIjoiYXV0...
    
    • Copy
    • Fill window

Flux to BigQuery data type conversion

sql.to() converts Flux data types to BigQuery data types.

Flux data typeBigQuery data type
intINT64
floatFLOAT64
stringSTRING
boolBOOL
timeTIMESTAMP

Was this page helpful?

Thank you for your feedback!


The future of Flux

Flux is going into maintenance mode. You can continue using it as you currently are without any changes to your code.

Read more

Now Generally Available

InfluxDB 3 Core and Enterprise

Start fast. Scale faster.

Get the Updates

InfluxDB 3 Core is an open source, high-speed, recent-data engine that collects and processes data in real-time and persists it to local disk or object storage. InfluxDB 3 Enterprise builds on Core’s foundation, adding high availability, read replicas, enhanced security, and data compaction for faster queries and optimized storage. A free tier of InfluxDB 3 Enterprise is available for non-commercial at-home or hobbyist use.

For more information, check out: