GraphSchema library - Tutorial 2 : set up a simple Schema (Classes, Properties)¶

and perform a data import (Data Nodes and relationships among them)¶

If you're new to Graph Databases, you can think of "Classes" and "Properties" along the lines of, respectively, "Table names" and "Table field lists".

If you need to first clear out your test database, one of the cells below (currently commented out) will conveniently let you do it

Background Article on Schema in Graph Databases ¶

In [1]:

import os
import sys
import getpass
import pandas as pd

from brainannex import GraphAccess, GraphSchema
# In case of problems, try a sys.path.append(directory) , where directory is your project's root directory

In [ ]:

Connect to the database,¶

using the `GraphAccess` library¶

You can use a free local install of the Neo4j database, or a remote one on a virtual machine under your control, or a hosted solution, or simply the FREE "Sandbox" : instructions here ¶

In [2]:

# Save your credentials here - or use the prompts given by the next cell
host = "bolt://localhost"         # EXAMPLES:  bolt://123.456.789.012   OR   bolt://localhost
                                  # (CAUTION: do NOT include the port number!)
password = ""

In [ ]:

In [3]:

db = GraphAccess(host=host,
                 credentials=("neo4j", password), debug=False)   # Notice the debug option being OFF

Connection to Neo4j database established.

In [4]:

print("Version of the Neo4j driver: ", db.version())

Version of the Neo4j driver:  4.4.13

In [ ]:

Import of data from a Pandas data frame using the `GraphSchema` library¶

Initial setup¶

In [5]:

# CLEAR OUT THE DATABASE
#db.empty_dbase()       # UNCOMMENT IF DESIRED   *****************  WARNING: USE WITH CAUTION!!!  ************************

In [6]:

GraphSchema.set_database(db)

Create the Schema¶

In [7]:

# Create a "City" Class node - together with its Properties, based on the data to import

GraphSchema.create_class_with_properties(name="City", properties=["City ID", "name"])

Out[7]:

(3408, 'schema-1')

In [8]:

# Likewise for a "State" Class node - together with its Properties, based on the data to import

GraphSchema.create_class_with_properties(name="State", properties=["State ID", "name", "2-letter abbr"])  

Out[8]:

(3411, 'schema-4')

In [9]:

# Now add a relationship named "IS_IN", from the "City" Class to the "State" Class

GraphSchema.create_class_relationship(from_class="City", to_class="State", rel_name="IS_IN")

In [ ]:

Now import some data¶

We'll pass our data as Pandas data frames; those could easily be read in from CSV files, for example

In [10]:

city_df = pd.DataFrame({"City ID": [1, 2, 3, 4], "name": ["Berkeley", "Chicago", "San Francisco", "New York City"]})
city_df

Out[10]:

	City ID	name
0	1	Berkeley
1	2	Chicago
2	3	San Francisco
3	4	New York City

In [11]:

state_df = pd.DataFrame({"State ID": [1, 2, 3], "name": ["California", "Illinois", "New York"], "2-letter abbr": ["CA", "IL", "NY"]})
state_df

Out[11]:

	State ID	name	2-letter abbr
0	1	California	CA
1	2	Illinois	IL
2	3	New York	NY

In [12]:

# In this example, we assume a separate table ("join table") with the data about the relationships;
# this would always be the case for many-to-many relationships; 
# 1-to-many relationships, like we have here, could also be stored differently
state_city_links_df = pd.DataFrame({"State ID": [1, 1, 2, 3], "City ID": [1, 3, 2, 4]})
state_city_links_df

Out[12]:

	State ID	City ID
0	1	1
1	1	3
2	2	2
3	3	4

Note: those dataframes would often be read in from CSV files, with instruction such as¶

city_df = pd.read_csv('D:\my_path\city_data_file.csv', encoding = "ISO-8859-1")¶

In [ ]:

Ingesting the Data Frames into the graph database is quite easy:¶

In [13]:

GraphSchema.import_pandas_nodes(df=city_df, class_name="City")     # Import the 4 cities

import_pandas_nodes(): importing 4 records in 1 batch(es) of size up to 1000...

   Importing batch # 1 : 4 row(s)
     Interim status: at the end of this batch, imported a grand total of 4 record(s), and created a grand total of 4 new node(s)
    FINISHED importing 4 record(s), and created 4 new node(s) in the process

Out[13]:

{'number_nodes_created': 4, 'affected_nodes_ids': [3415, 3416, 3417, 3418]}

In [14]:

GraphSchema.import_pandas_nodes(df=state_df, class_name="State")    # Import the 3 states

import_pandas_nodes(): importing 3 records in 1 batch(es) of size up to 1000...

   Importing batch # 1 : 3 row(s)
     Interim status: at the end of this batch, imported a grand total of 3 record(s), and created a grand total of 3 new node(s)
    FINISHED importing 3 record(s), and created 3 new node(s) in the process

Out[14]:

{'number_nodes_created': 3, 'affected_nodes_ids': [3419, 3420, 3421]}

In [15]:

# Finally, link up the cities to the states, using links named "IS_IN"
GraphSchema.import_pandas_links(df=state_city_links_df,
                                class_from="City", class_to="State", 
                                col_from="City ID", col_to="State ID",                            
                                link_name="IS_IN")

import_pandas_links(): importing 4 links in 1 batch(es) of max size 1000...

    Result of running batch :  {'_contains_updates': True, 'relationships_created': 4, 'returned_data': [{'link_id': 1954}, {'link_id': 1955}, {'link_id': 1956}, {'link_id': 1957}]}
    FINISHED importing a total of 4 links

Out[15]:

[1954, 1955, 1956, 1957]

In [ ]:

IMPORTANT UPDATE: in new versions of BrainAnnexm the `SCHEMA` relationships have been replaced by `_CLASS` properties on the Data Nodes¶

(the diagram below reflects older versions, and still holds conceptually)¶

This is what we have created with our import:

Schema

Notice that the data from that "join table" (with "State ID" and "City ID") that was used for the import, to link up states and cities, is now stored as RELATIONSHIPS - nativety represented in the graph database¶

In [ ]:

GraphSchema library - Tutorial 2 : set up a simple Schema (Classes, Properties)¶

and perform a data import (Data Nodes and relationships among them)¶

Background Article on Schema in Graph Databases¶

Connect to the database,¶

using the GraphAccess library¶

You can use a free local install of the Neo4j database, or a remote one on a virtual machine under your control, or a hosted solution, or simply the FREE "Sandbox" : instructions here¶

Import of data from a Pandas data frame using the GraphSchema library¶

Initial setup¶

Create the Schema¶

Now import some data¶

Note: those dataframes would often be read in from CSV files, with instruction such as¶

city_df = pd.read_csv('D:\my_path\city_data_file.csv', encoding = "ISO-8859-1")¶

Ingesting the Data Frames into the graph database is quite easy:¶

IMPORTANT UPDATE: in new versions of BrainAnnexm the SCHEMA relationships have been replaced by _CLASS properties on the Data Nodes¶

(the diagram below reflects older versions, and still holds conceptually)¶

Notice that the data from that "join table" (with "State ID" and "City ID") that was used for the import, to link up states and cities, is now stored as RELATIONSHIPS - nativety represented in the graph database¶

Background Article on Schema in Graph Databases ¶

using the `GraphAccess` library¶

You can use a free local install of the Neo4j database, or a remote one on a virtual machine under your control, or a hosted solution, or simply the FREE "Sandbox" : instructions here ¶

Import of data from a Pandas data frame using the `GraphSchema` library¶

IMPORTANT UPDATE: in new versions of BrainAnnexm the `SCHEMA` relationships have been replaced by `_CLASS` properties on the Data Nodes¶