Configuring data-sources: Google BigQuery

If you have data in Google BigQuery, you can explore this data as a graph using Linkurious Enterprise.

To achieve this, you must first define a property graph schema in BigQuery, which defines a projection of the data in your tables as a graph.

Prerequisites

This section defines the technical requirements to use BigQuery with Linkurious Enterprise.

Graph Schema

Make sure that your graph schema defines at least one KEY property for each node and edge in the graph schema. Make sure all KEY columns are part of the graph node/edge properties. The Supported key types are INT64 and STRING.

Nodes with multiple labels are not supported, please assign a single label for each table. Each label must unambiguously reference a single table, make sure a given label does not correspond to two different SQL tables.

For edges, we recommend using:

• For one-to-many edge types: the key property of the target table row (e.g. for Person (1) -OWNS_BIKE-> (*) Bike, a good key for the OWNS_BIKE edge is the key property of the Bike node-table).
• For many-to-many edge types, when multiples edges of that type is possible between two nodes: the key property of the join-table itself (e.g. for Account (*) <-TRANSACTION-> (*) Account a good key for the TRANSACTION edge is the key of the TRANSACTION edge-table).
• For many-to-many edge types, when only one edge of that type is possible between two nodes: a combination of the key property of the source and target table rows (e.g. for Person (*) <-KNOWS-> (*) Person, a good key for the KNOWS edge is (sourcePersonId, targetPersonId)).

Full-text search

To use this feature, you must have at least one node property in your graph that is indexed.

For more details, see how to configure Google BigQuery Search.

Configuration

To edit the BigQuery data-source configuration, you can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json.

Example configuration:

{
  "dataSources": [
    {
      "graphdb": {
        "vendor": "bigQuery",
        "url": "https://cloud.google.com/bigquery",
        "projectId": "dev-big-query",
        "datasetId": "big_query_dev_dataset",
        "graphName": "big_query_dev_graph",
        "keyFileContent": "<secret>"
      },
      "index": {
        "vendor": "bigQuerySearch"
      }
    }
  ]
}

Supported graphdb options with BigQuery:

• vendor (required): must be bigQuery;
• url (required): must be https://cloud.google.com/bigquery.
• projectId (required): Your Google Cloud project ID.
• datasetId (required): Your Google BigQuery dataset ID.
• graphName (required): Your property graph's name.
• keyFileContent: If you are not running Linkurious Enterprise on GCP, you can create a service account key file and copy the full content of the file in this field;
• keyFile (used only if keyFileContent is not set): If you are running Linkurious Enterprise on GCP, you can set up application credentials and set this field to $ENV:GOOGLE_APPLICATION_CREDENTIALS (this will read the value of this environment variable);

Supported index options with BigQuery:

• vendor (required): must be bigQuerySearch;
• retrievalLimit (optional, default: 10000): Limit how many results to fetch and rank in each table when executing a search.

For details, see BigQuery's documentation about full-text search.

Authentication options

There are 3 ways to authenticate to Google Spanner.

Using keyFileContent:

• When to use: if you are not running on GCP.
• How to use:
  • Create a service account key file
  • Copy the full content of the file into keyFileContent
  • Leave keyFile unset.

Using a keyFile:

• When to use: if you are on GCP
• How to use:
  • Set up application credentials
  • Set keyFile field to $ENV:GOOGLE_APPLICATION_CREDENTIALS (this will read the value of this environment variable)
  • Leave keyFileContent unset.

Using a Service Account:

• When to use: if you are on GCP and want to use a Service Account
• How to use:
  • Leave both keyFile and keyFileContent unset
  • create a Service Account (make sure its scope includes cloud-platform)
  • Assign rights to the Service Account
  • Attach the Service Account to your server.

Importing data

See Google BigQuery's documentation about importing data.