Configuring data-sources: Neo4j

Please check for supported Neo4j versions in our compatibility matrix.

Configuration

To edit the Neo4j data-source configuration, you can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json.

Example configuration:

{
  "dataSources": [
    {
      "graphdb": {
        "vendor": "neo4j",
        "url": "neo4j://127.0.0.1:7687/",
        "user": "myNeo4jUser",
        "password": "myNeo4jPassword"
      },
      "index": {
        "vendor": "neo4jSearch"
      }
    }
  ]
}

Example configuration using TLS:

{
  "dataSources": [
    {
      "graphdb": {
        "vendor": "neo4j",
        "url": "neo4j+s://127.0.0.1:7687/",
        "user": "myNeo4jUser",
        "password": "myNeo4jPassword"
      },
      "index": {
        "vendor": "neo4jSearch"
      }
    }
  ]
}

Learn more on how to configure SSL from the Neo4j documentation.

Linkurious connects to Neo4j via the Bolt protocol. To do so, you need to enable the protocol in your Neo4j configuration file. If an HTTP/S URL is configured, Linkurious will automatically upgrade the connection to Bolt.

Supported graphdb options with Neo4j:

• url (required): URL of the Neo4j server (http, https, bolt, bolt+s, bolt+ssc, neo4j, neo4j+s, neo4j+ssc)
• user (optional): Neo4j user (if credentials are enabled, see Neo4j credentials)
• password (optional): Neo4j password (if credentials are enabled)
• proxy (optional): URL of the HTTP proxy to use to connect to Neo4j (only used when url is HTTP/S)
• alternativeNodeId (optional): Name of the node property to use as reference in visualizations (see alternative IDs)
• alternativeEdgeId (optional): Name of the edge property to use as reference in visualizations
• latitudeProperty (optional): Name of the node property to use for latitude (used in geo mode)
• longitudeProperty (optional): Name of the node property to use for longitude (used in geo mode)
• allowSelfSigned (optional, default false): Whether to allow self-signed certificates
• databaseName (optional): Name of the database to be connected
• allowVirtualEntities (optional, default true): Whether to allow virtual nodes and virtual edges
• alternativeURLs (optional): Linkurious accepts a string array consisting of alternative Neo4j Bolt URLs for high availability
• ignoredEdgeTypePrefixes (optional): List of edge type prefixes to be excluded in the schema sampling

Neo4j Aura

Linkurious Enterprise allows using Neo4j instances running on Neo4j Aura as data-sources.

Neo4j Aura is only supported for the Neo4j Aura instances running Neo4j engine v4.0 and later.

Search with Neo4j

In order to have full-text search, you can choose among the following options:

• Configure a search index in Neo4j.
• Configure a search index in Elasticsearch.

Neo4j credentials

If you just installed Neo4j, these steps will help you create credentials:

1. Launch the Neo4j server
2. Open your Web browser at http://127.0.0.1:7474
3. Follow the instructions to create a new username and password

Alternatively, you can disable credentials in Neo4j by editing the Neo4j configuration at neo4j/conf/neo4j.conf by uncommenting the following line:

dbms.security.auth_enabled=false

Neo4j access rights

Note that configuring access rights is unnecessary on the Neo4j Community edition or if Neo4j credentials are disabled.

In Neo4j, access rights are managed using role-based access control. You can either:

• Create a custom role with individually defined privileges and assign this custom role to your Neo4j user.
• Assign a built-in role to your Neo4j user.

Required privileges for graph exploration

In order to connect to a data-source, Linkurious requires a Neo4j user with the ACCESS, EXECUTE PROCEDURE and EXECUTE FUNCTION privileges. These privileges only allow Linkurious to connect to the data-source. Additional privileges must be granted in order to interact with the graph database. The Neo4j built-in PUBLIC role has these privileges on the default database. These privileges can be granted on a database $name to a custom role $role by running the following Cypher commands:

GRANT ACCESS ON DATABASE $name TO $role;
GRANT EXECUTE FUNCTION * ON DBMS TO $role;
GRANT EXECUTE PROCEDURE * ON DBMS TO $role;

If the data-source is read-only, the MATCH privilege must be granted on part or all of the graph, so that Linkurious can fetch nodes and edges. The Neo4j built-in reader role has this privilege on all databases, except the system one. This privilege can be granted by running the following Cypher commands:

GRANT MATCH {*} ON GRAPH $name TO $role;

If the data-source is read-write and the schema is in strict mode, the WRITE privilege must also be granted on the graph, so that linkurious can create nodes and edges. This corresponds to the Neo4j built-in editor role.

GRANT WRITE ON GRAPH $name TO $role;

Additionally, if the schema is not in strict mode, the NAME MANAGEMENT privilege must also be granted on the graph, so that linkurious can alter the schema. This corresponds to the Neo4j built-in publisher role.

GRANT NAME MANAGEMENT ON DATABASE $name TO $role;

Additional privileges required for full-text search

Neo4j search requires the INDEX MANAGEMENT privilege. This corresponds to the Neo4j built-in architect role.

GRANT INDEX MANAGEMENT ON DATABASE $name TO $role;

Elasticsearch doesn't require any specific privilege if incremental indexing is not used.

However, enabling incremental indexing on Elasticsearch requires the INDEX MANAGEMENT privilege. On Neo4j 5.0 and onward, the EXECUTE ADMIN PROCEDURE privilege is also needed in order to manage Neo4j triggers. And between Neo4j 5.0 and 5.3, the SHOW SERVER privilege is needed in check if the Neo4j server is a standalone instance or a cluster. These privileges correspond to the Neo4j built-in admin role.

GRANT INDEX MANAGEMENT ON DATABASE $name TO $role;
GRANT EXECUTE ADMIN PROCEDURES ON DBMS TO $role; // Only on Neo4j 5.0 and onward
GRANT SHOW SERVER ON DBMS TO $role; // Only between Neo4j 5.0 and 5.3

Configure Alternative Ids indices

Configuring alternative IDs indices is recommended.

The first step is to:

• Decide a unique name for the newly created indices, one for nodes and one for relationships. In the following example query we picked myAlternativeNodeIdIndex for nodes and myAlternativeEdgeIdIndex for relationships.
• Identify the list of node and relationship types in the Neo4j database. In the following example: Company, Person and City are the node categories while WORKS_FOR and LIVES_IN are the relationship types.
• Have the 2 property keys, one for nodes and one for relationships, that identify an item uniquely in the database. In the following example we picked myUniqueNodeId for nodes and myUniqueEdgeId for relationships.

Once we have this information we can create the indices with the following Cypher queries:

CREATE FULLTEXT INDEX `myAlternativeNodeIdIndex` FOR (n:`Company`|`Person`|`City`) ON EACH [n.`myUniqueNodeId`] OPTIONS { indexConfig: { `fulltext.analyzer`: 'keyword' } }

CREATE FULLTEXT INDEX `myAlternativeEdgeIdIndex` FOR ()-[r:`WORKS_FOR`|`LIVES_IN`]-() ON EACH [r.`myUniqueEdgeId`] OPTIONS { indexConfig: { `fulltext.analyzer`: 'keyword' } }

If you are running a version of Neo4j older than v4.3 you may need to use the old syntax:

call db.index.fulltext.createNodeIndex('myAlternativeNodeIdIndex', ['Company', 'Person', 'City'], ['myUniqueNodeId'], {analyzer: 'keyword'})

call db.index.fulltext.createRelationshipIndex('myAlternativeEdgeIdIndex', ['WORKS_FOR', 'LIVES_IN'], ['myUniqueEdgeId'], {analyzer: 'keyword'})

If new node labels or edge types are added to Neo4j, it's necessary to recreate these indices with the full list of categories.

Once the indices are created, we can configure them:

Example configuration:

{
  "dataSources": [
    {
      "graphdb": {
        "vendor": "neo4j",
        "url": "neo4j://127.0.0.1:7687/",
        "user": "myNeo4jUser",
        "password": "nyNeo4jPassword",
        "alternativeNodeId": "myUniqueNodeId",
        "alternativeNodeIdIndex": "myAlternativeNodeIdIndex",
        "alternativeEdgeId": "myUniqueEdgeId",
        "alternativeEdgeIdIndex": "myAlternativeEdgeIdIndex"
      },
      "index": {
        "vendor": "neo4jSearch"
      }
    }
  ]
}

Neo4j credentials mapping

It is possible to map user-groups in Linkurious Enterprise to users in Neo4j.

For example, if an administrator wants to make sure that all graph queries executed by members of the "investigators" user-group in Linkurious Enterprise are executed in Neo4j using the "invest-user" user, then the follopwing data-sources configuration should be used:

{
  "dataSource": [
    {
      "name": "my neo4j database",
      "graphdb": {
        "vendor": "neo4j",
        "user": "my-default-user",
        "password": "mySecret1",
        // omitted
        "credentialsMappings": [
          // the "investigators" group has the ID #12 (see group-management page)
          {"groupIds": [12], "neo4jUser": "invest-user", "neo4jPassword": "mySecret2"}
        ]
      },
      "index": {
        "vendor": "neo4jSearch"
      }
    }
  ]
}

ℹ️ The credentialsMappings field cannot be edited via the Web interface and can only be edited by editing the configuration file with a file editor.

The rules to map a user-group in Linkurious Enterprise to a user in Neo4j are the following:

• If the current user has no groups in common with any mapping rule, the default user is used;
• If the current user has groups in common with several mapping rules, the rule with most groups in common is chosen;
• If the current user has the same number of groups in common with several rules, the first one found is used.

Some examples to illustrate credential mapping rules

Input credential mapping:

"user": "my-default-user",
"password": "mySecretPassword1",
"credentialsMappings": [
  {"groupIds": [1], "neo4jUser": "user1", "neo4jPassword": "password1"},
  {"groupIds": [1, 2], "neo4jUser": "user2", "neo4jPassword": "password2"},
  {"groupIds": [1, 2, 3], "neo4jUser": "user3", "neo4jPassword": "password3"},
]

Resulting mapping:

• Current user groups: [1] => User "user1", because it is the first mapping that contains all the groups.
• Current user groups: [1, 2] => User "user2", because it is the first mapping that contains all the groups.
• Current user groups: [1, 2, 3] => User "user3", because it is the first mapping that contains all the groups.
• Current user groups: [1, 2, 3, 4] => User "user3", because it is the mapping with most groups in common.
• Current user groups: [2] => User "user3", because it is the first mapping with most groups in common.
• Current user groups: [4] => User "my-default-user" because no mapping has any groups in common.

Migrating to Neo4j block storage

Neo4j introduced a new "block" storage format. This new storage format was introduced in Neo4j 5.14 and is the default since Neo4j 5.17. Migrating the storage format of a Neo4j database will change the nodes and edges IDs, and thus break references in Linkurious Enterprise's user-data store. As a consequence, migrating to block format without following the proper steps will cause all existing visualizations and alert cases to become either empty or filled with the wrong nodes and edges.

To migrate to block format without breaking references in Linkurious Enterprise, the neo4jBlockFormatMigration script is available to migrate the content of Linkurious Enterprise's user-data store.

The neo4jBlockFormatMigration script has two commands:

• prepare, that must be executed before migrating Neo4j to block storage format
• complete, that must be executed after migrating Neo4j to block storage format.

You should be able to see the documentation of the neo4jBlockFormatMigration script by running it without any argument:

• On Docker

  docker run --rm --mount type=bind,src=/path/to/my/data/folder,dst=/data linkurious:4.2.3 node /usr/src/linkurious/system/scripts/neo4jBlockFormatMigration` (don't forget to replace `/path/to/my/data/folder` to your data folder, or you can use any other volume configuration that mounts the data folder)
  

• On Linux

  • From the Linkurious Enterprise installation directory ./neo4jBlockFormatMigration.sh

• On macOS

  • From the Linkurious Enterprise installation directory ./neo4jBlockFormatMigration.sh.command

• On Windows

  • From the Linkurious Enterprise installation directory ./neo4jBlockFormatMigration.bat

• You should see: "Usage: neo4jBlockFormatMigration [command] [options] (...)" and more documentation about the script.

Please follow these steps to perform a migration to block format (the database is called my-database here):

1. Make sure the Neo4j database is not already using a "block" storage format by running SHOW DATABASES YIELD name, store in the Neo4j console.

   • If "store" starts with record-aligned, it means that the database is not yet migrated to block format
   • If "store" starts with block-block, it means that the database was already migrated to block format

2. Stop Linkurious Enterprise

3. Run the neo4jBlockFormatMigration script with the prepare command (add prepare as the end of the neo4jBlockFormatMigration script documented above).

   • ⚠️ You should see this message: "The preparation of the Neo4j block format migration has been done successfully". If you do not see this message, please make sure you ran the command properly and contact support if needed.

4. Migrate Neo4j to block storage:

   • Read the script documentation
   • Stop the database by running the following Cypher command: STOP DATABASE my-database WAIT;
   • On your Neo4j instance, run neo4j-admin database migrate --to-format="block" my-database
   • Re-start your database by running the following Cypher command: START DATABASE my-database WAIT;

5. Run the neo4jBlockFormatMigration script with the complete command (add complete as the end of the neo4jBlockFormatMigration script documented above).

   • ⚠️ You should see this message: "The completion of the Neo4j block format migration has been done successfully". If you do not see this message, please make sure you ran the command properly and contact support if needed.

6. Start Linkurious Enterprise