Java Flight SQL package

Apache Arrow Flight SQL for Java integrates with Java applications to query and retrieve data from Flight database servers using RPC and SQL.

Use InfluxDB v3 client libraries

We recommend using the influxdb3-java Go client library for integrating InfluxDB v3 with your Java application code.

InfluxDB v3 client libraries wrap Apache Arrow Flight clients and provide convenient methods for writing, querying, and processing data stored in InfluxDB Clustered. Client libraries can query using SQL or InfluxQL.

Get started using the Java Flight SQL client to query InfluxDB
Troubleshoot Arrow Flight requests

Get started using the Java Flight SQL client to query InfluxDB

Write a Java class for a Flight SQL client that connects to InfluxDB Clustered, executes an SQL query, and retrieves data stored in an InfluxDB Clustered database.

The example uses the Apache Arrow Java implementation (org.apache.arrow) for interacting with Flight database servers like InfluxDB v3.

org.apache.arrow: Provides classes and methods for integrating Java applications with Apache Arrow data and protocols.
org.apache.arrow.flight.sql: Provides classes and methods for interacting with Flight database servers using Arrow Flight RPC and Flight SQL.

Set up InfluxDB
Install prerequisites
Create the FlightQuery class
Create a query client
Execute a query
Retrieve and process Arrow data

To clone or download the example application that you can run with Docker, see the InfluxCommunity/ArrowFlightClient_Query_Examples repository on GitHub.

Set up InfluxDB

To configure the application for querying InfluxDB Clustered, you’ll need the following InfluxDB resources:

InfluxDB Clustered database
InfluxDB Clustered database token with read permission to the database

If you don’t already have a database token and a database, see how to set up InfluxDB. If you don’t already have data to query, see how to write data to a database.

Install prerequisites

The following uses Docker and Maven to build and run the Java application and avoid platform-specific dependency problems.

The example Dockerfile installs compatible versions of Maven and Java JDK in the Docker container, and then runs the Maven commands to download dependencies and compile the application.

Follow the instructions to download and install Docker for your system:

macOS: Install Docker for macOS
Linux: Install Docker for Linux

View the Dockerfile

View the Maven pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.influxdb</groupId>
  <artifactId>examples</artifactId>
  <version>1.0-SNAPSHOT</version>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <version>3.2.0</version>
        <configuration>
          <archive>
            <manifest>
              <addClasspath>true</addClasspath>
              <classpathPrefix>lib/</classpathPrefix>
              <mainClass>com.influxdb.examples.FlightExamples</mainClass>
            </manifest>
          </archive>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-help-plugin</artifactId>
        <version>3.2.0</version>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-shade-plugin</artifactId>
        <version>3.4.1</version>
        <executions>
          <execution>
            <goals>
              <goal>shade</goal>
            </goals>
            <configuration>
              <shadedArtifactAttached>true</shadedArtifactAttached>
            </configuration>
          </execution>
        </executions>
        <configuration>
          <minimizeJar>false</minimizeJar>
          <filters>
            <filter>
              <artifact>*:*</artifact>
              <excludes>
                <exclude>META-INF/*.SF</exclude>
                <exclude>META-INF/*.DSA</exclude>
                <exclude>META-INF/*.RSA</exclude>
              </excludes>
            </filter>
          </filters>
        </configuration>
      </plugin>
    </plugins>
  </build>

  <properties>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.apache.arrow</groupId>
      <artifactId>flight-sql</artifactId>
      <version>11.0.0</version>
    </dependency>
    <dependency>
      <groupId>io.netty</groupId>
      <artifactId>netty-all</artifactId>
      <version>4.1.74.Final</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-simple</artifactId>
      <version>1.7.30</version>
    </dependency>
  </dependencies>

</project>

Create the FlightQuery class

View FlightQuery.java

package com.influxdb.examples;

import org.apache.arrow.flight.auth2.BearerCredentialWriter;
import org.apache.arrow.flight.CallHeaders;
import org.apache.arrow.flight.CallStatus;
import org.apache.arrow.flight.grpc.CredentialCallOption;
import org.apache.arrow.flight.Location;
import org.apache.arrow.flight.FlightClient;
import org.apache.arrow.flight.FlightClientMiddleware;
import org.apache.arrow.flight.FlightInfo;
import org.apache.arrow.flight.FlightStream;
import org.apache.arrow.flight.sql.FlightSqlClient;
import org.apache.arrow.flight.Ticket;
import org.apache.arrow.memory.BufferAllocator;
import org.apache.arrow.memory.RootAllocator;
import org.apache.arrow.vector.VectorSchemaRoot;

public class FlightQuery {

    /* Get server credentials from environment variables */
    public static final String DATABASE_NAME = System.getenv("DATABASE_NAME");
    public static final String HOST = System.getenv("HOST");
    public static final String TOKEN = System.getenv("TOKEN");

    public static void main() {

        System.out.println("Query InfluxDB with the Java Flight SQL Client");

        // Create an interceptor that injects header metadata (database name) in every request.
        FlightClientMiddleware.Factory f = info -> new FlightClientMiddleware() {
            @Override
            public void onBeforeSendingHeaders(CallHeaders outgoingHeaders) {
                outgoingHeaders.insert("database", DATABASE_NAME);
            }

            @Override
            public void onHeadersReceived(CallHeaders incomingHeaders) {

            }

            @Override
            public void onCallCompleted(CallStatus status) {

            }
        };

        // Create a gRPC+TLS channel URI with HOST and port 443.
        Location location = Location.forGrpcTls(HOST, 443);

        // Set the allowed memory.
        BufferAllocator allocator = new RootAllocator(Long.MAX_VALUE);

        // Create a client with the allocator and gRPC channel.
        FlightClient client = FlightClient.builder(allocator, location)
                .intercept(f)
                .build();
        System.out.println("client" + client);

        FlightSqlClient sqlClient = new FlightSqlClient(client);
        System.out.println("sqlClient: " + sqlClient);

        // Define the SQL query to execute.
        String query = "SELECT * FROM home";
        
        /*  Construct a bearer credential using TOKEN.
            Construct a credentials option using the bearer credential.
        */
        CredentialCallOption auth = new CredentialCallOption(new BearerCredentialWriter(TOKEN));

        /*  Execute the query.
            If successful, execute returns a FlightInfo object that contains metadata
            and an endpoints list.
            Each endpoint contains the following:
                - A list of addresses where you can retrieve the data.
                - A `ticket` value that identifies the data to retrieve.
        */
        FlightInfo flightInfo = sqlClient.execute(query, auth);

        // Extract the Flight ticket from the response.
        Ticket ticket = flightInfo.getEndpoints().get(0).getTicket();
        
        // Pass the ticket to request the Arrow stream data from the endpoint.
        final FlightStream stream = sqlClient.getStream(ticket, auth);

        // Process all the Arrow stream data.
        while (stream.next()) {
            try {
                // Get the current vector data from the stream.
                final VectorSchemaRoot root = stream.getRoot();
                System.out.println(root.contentToTSVString());
            } catch (Exception e) {
                // Handle exceptions.
                System.out.println("Error executing FlightSqlClient: " + e.getMessage());
            }
        }
        try {
            // Close the stream and release resources.
            stream.close();
        } catch (Exception e) {
            // Handle exceptions.
            System.out.println("Error closing stream: " + e.getMessage());
        }

        try {
            // Close the client
            sqlClient.close();
        } catch (Exception e) {
            // Handle exceptions.
            System.out.println("Error closing client: " + e.getMessage());
        }
    }
}

In your <PROJECT_ROOT>/src/main/java directory, create the com/influxdb/examples subdirectories for the com.influxdb.examples package.

In the examples directory from the preceding step, create the FlightQuery.java class file. You should have the following directory structure:

PROJECT_ROOT
└──src
   └──main
      └──java
         └──com
            └──influxdb
               └──examples
                  └──FlightQuery.java

In FlightQuery.java:
1. Add the package name:
```
package com.influxdb.examples;
```
2. Add import statements for the following packages. You’ll use classes and methods from these packages in the remaining steps:
  - org.apache.arrow.flight.auth2.BearerCredentialWriter
  - org.apache.arrow.flight.CallHeaders
  - org.apache.arrow.flight.CallStatus
  - org.apache.arrow.flight.grpc.CredentialCallOption
  - org.apache.arrow.flight.Location
  - org.apache.arrow.flight.FlightClient
  - org.apache.arrow.flight.FlightClientMiddleware
  - org.apache.arrow.flight.FlightInfo
  - org.apache.arrow.flight.FlightStream
  - org.apache.arrow.flight.sql.FlightSqlClient
  - org.apache.arrow.flight.Ticket
  - org.apache.arrow.memory.BufferAllocator
  - org.apache.arrow.memory.RootAllocator
  - org.apache.arrow.vector.VectorSchemaRoot
3. Create a FlightQuery class.
4. In the FlightQuery class:
  1. Define constants for server credentials.
    - DATABASE_NAME
    - HOST
    - TOKEN
    The example Dockerfile defines environment variables for these credentials.
  2. Create a main() method.

Create a query client

In the FlightQuery.main() method, do the following to create an SQL client that can connect to HOST and DATABASE_NAME:

Construct a gRPC+TLS channel URI with HOST and port 443 for communicating with a gRPC server over TLS.
Instantiate FlightClientMiddleware and define an event callback that inserts the following Flight request metadata header property:
```
"database": "DATABASE_NAME"
```
Instantiate a BufferAllocator that sets the memory allowed for the client.
Create a FlightClient with the allocator and gRPC channel.
Instantiate a FlightSqlClient that wraps the FlightClient instance.

Execute a query

In the FlightQuery.main method:

Instantiate a CredentialCallOption with TOKEN as a bearer credential. The result is a credential object that you’ll pass in each request to the server.
Define a string that contains the SQL query to execute–for example:
```
String query = "SELECT * FROM home";
```
Call the FlightSqlClient.execute method with the SQL query and the CredentialCallOption.
If successful, the FlightSqlClient.execute method responds with a FlightInfo object that contains metadata and an endpoints: [...] list. Each endpoint contains the following:
- A list of addresses where you can retrieve the data.
- A ticket value that identifies the data to retrieve.
Extract the ticket from the response.

Retrieve and process Arrow data

In the FlightQuery.main() method, do the following to retrieve the data stream described in the FlightInfo response:

Call the FlightSqlClient.getStream method with the ticket and the CredentialCallOption to fetch the Arrow stream.
Call the FlightStream.getRoot method to get the current vector data from the stream.
Process the data and handle exceptions. The example converts the vector data into tab-separated values and prints the result to System.out.
For more examples using Java to work with Arrow data, see the Apache Arrow Java Cookbook.
Finally, close the stream and client.

Run the application

Follow these steps to build and run the application using Docker:

Copy the Dockerfile and pom.xml to your project root directory.
Open a terminal in your project root directory.
In your terminal, run the docker build command and pass --build-arg flags for the server credentials:
- DATABASE_NAME: your InfluxDB Clustered database
- HOST: your [InfluxDB cluster hostname (URL without the “https://”)
- TOKEN: your InfluxDB Clustered database token with read permission to the database
```
docker build \
--build-arg DATABASE_NAME=INFLUX_DATABASE \
--build-arg HOST=cluster-host.com \
--build-arg TOKEN=INFLUX_TOKEN \
-t javaflight .
```
The command builds a Docker image named javaflight.
To run the application in a new Docker container, enter the following command:
```
docker run javaflight
```
The output is the query data in TSV-format.

Troubleshoot Arrow Flight requests

For the list of Arrow Flight error response codes, see the Arrow Flight RPC documentation.

Was this page helpful?

Thank you for your feedback!

Support and feedback

Thank you for being part of our community! We welcome and encourage your feedback and bug reports for InfluxDB and this documentation. To find support, use the following resources:

Customers with an annual or support contract can contact InfluxData Support.

Edit this page Submit docs issue Submit InfluxDB issue

Java Flight SQL package

Use InfluxDB v3 client libraries

Get started using the Java Flight SQL client to query InfluxDB

Set up InfluxDB

Install prerequisites

Create the FlightQuery class

Create a query client

Execute a query

Retrieve and process Arrow data

Run the application

Troubleshoot Arrow Flight requests

Support and feedback

The future of Flux

InfluxDB v3 enhancements and InfluxDB Clustered is now generally available

InfluxDB v3 performance and features

InfluxDB Clustered general availability

Java Flight SQL package

Use InfluxDB v3 client libraries

Get started using the Java Flight SQL client to query InfluxDB

Set up InfluxDB

Install prerequisites

Create the FlightQuery class

Create a query client

Execute a query

Retrieve and process Arrow data

Run the application

Troubleshoot Arrow Flight requests

Related

Support and feedback

What is your InfluxDB cluster URL?

Enter cluster URL

Thank you for your feedback!

The future of Flux

InfluxDB v3 enhancements and InfluxDB Clustered is now generally available

InfluxDB v3 performance and features

InfluxDB Clustered general availability