About Michael Hunger

Posts by Michael Hunger:

 
0

Importing Mapping Metaphor into Neo4j

on Sep 30, 2017 in Uncategorized

I came across this tweet, which sounded really interesting.

@MappingMetaphor: Metaphor Map now complete! Remaining data now up, and showing nearly 12,000 metaphorical connections: http://mappingmetaphor.arts.gla.ac.uk/

Mapping Metaphor

The Metaphor Map of English shows the metaphorical links which have been identified between different areas of meaning. These links can be from the Anglo-Saxon period right up to the present day so the map covers 1300 years of the English language. This allows us the opportunity to track metaphorical ways of thinking and expressing ourselves over more than a millennium; see the Metaphor in English section for more information.

The Metaphor Map was built as part of the Mapping Metaphor with the Historical Thesaurus project. This was completed by a team in English Language at the University of Glasgow and funded by the Arts and Humanities Research Council from 2012 to early 2015. The Metaphor Map is based on the Historical Thesaurus of English, which was published in 2009 by Oxford University Press as the Historical Thesaurus of the Oxford English Dictionary.

The site is really nice and fun to explore, with an interesting data visualization of the metaphoric connections between areas of language and thought:

metaphors

When most people think of metaphor, they cast their minds back to school and remember examples from poetry and drama, such as Shakespeare’s “Juliet is the sun”. This is unsurprising; metaphor is usually described as a literary phenomenon used to create arresting images in the mind of the reader. However, linguists would argue that metaphor is far more pervasive within our language and indeed within thought itself.

Useful natural language correlation network are always fun to work with, so let’s have a look at it in a graph database.

Install Neo4j & APOC

  1. Download and install Neo4j-Desktop from http://neo4j.com/download/other-releases

  2. Create a database and add the APOC procedure library.

  3. I also installed Neo4j Graph Algorithms to use later.

  4. Start the database.

Download Data

All the data is available from:
Mapping Metaphor with the Historical Thesaurus. 2015. Metaphor Map of English Glasgow: University of Glasgow. http://mappingmetaphor.arts.gla.ac.uk.
  1. select “Advanced Search”,

  2. select all categories (you’re interested in)

  3. select “Connections between selected sections and all other sections”

  4. Metaphor Strength: “Both”

  5. Click “Search”

  6. Select “View results as a table”

  7. Click the “Download” icon in the left box

The downloaded file “metaphor.csv” should contain almost 12k lines of metaphors:

Copy metaphor.csv into the import folder of your database (“Open Folder”) or in an http-accessible location to load via an http-url.

Run Import

Our data model is really simple, we have

  1. :Category nodes with id and name.

  2. :Strong or :Weak relationships between them with the start property for the start era and examples for the example words.

A more elaborate model could model the Metaphor as node, with the Words too and Era too and connect them.
I was just not sure, what to name the metaphor, that information was missing in the data.
But for this demonstration the simpler model is good enough.

For good measure.

create constraint on (c:Category) assert c.id is unique;

Run this Cypher statement to import in a few seconds

// load csv as individual lines keyed with header names
LOAD CSV WITH HEADERS FROM "file:///metaphor.csv" AS line

// get-or-create first category (note typo in name header)
merge (c1:Category {id:line.`Category 1 ID`}) ON CREATE SET c1.name=line.`Categroy 1 Name`
// get-or-create second category
merge (c2:Category {id:line.`Category 2 ID`}) ON CREATE SET c2.name=line.`Category 2 Name`

// depending on direction flip order of c1,c2
with line, case line.Direction when '>' then [c1,c2] else [c2,c1] end as cat,

// split words on ';' and remove last empty entry
     apoc.coll.toSet(split(line.`Examples of metaphor`,';'))[0..-1] as words

// create relatiosnship with dynamic type, set era & words as relatiosnship properties
call apoc.create.relationship(cat[0],line.Strength,{start:line.`Start Era`, examples:words},cat[1]) yield rel

// return rows processed
return count(*)

I rendered the category nodes pretty large so that you can read the names, and have the “Strong” links display their “words” instead.

metaphors

For finding categories quickly

create index on :Category(name);

Run graph algorithms.

Degree distribution

╒════════╤═══════════╤═══════╤═════╤═════╤═════╤═════╤═════╤══════╤═════╤═════╤═════════════════╕
│"type"  │"direction"│"total"│"p50"│"p75"│"p90"│"p95"│"p99"│"p999"│"max"│"min"│"mean"           │
╞════════╪═══════════╪═══════╪═════╪═════╪═════╪═════╪═════╪══════╪═════╪═════╪═════════════════╡
│"Weak"  │"OUTGOING" │7908   │11   │31   │48   │61   │84   │100   │100  │0    │19.10144927536232│
│"Strong"│"OUTGOING" │3974   │3    │12   │28   │37   │86   │107   │107  │0    │9.599033816425122│
└────────┴───────────┴───────┴─────┴─────┴─────┴─────┴─────┴──────┴─────┴─────┴─────────────────┘

Top 10 Categories by in-degree:

MATCH (c:Category)
WITH c,size( (c)-->()) as out,size( (c)<--()) as in
RETURN c.id, c.name,in, out
ORDER BY in DESC LIMIT 10;

╒══════╤═════════════════════════╤════╤═════╕
│"c.id"│"c.name"                 │"in"│"out"│
╞══════╪═════════════════════════╪════╪═════╡
│"2D06"│"Emotional suffering"    │119 │7    │
├──────┼─────────────────────────┼────┼─────┤
│"2C02"│"Bad"                    │119 │7    │
├──────┼─────────────────────────┼────┼─────┤
│"3M06"│"Literature"             │116 │29   │
├──────┼─────────────────────────┼────┼─────┤
│"1O22"│"Behaviour and conduct"  │109 │10   │
├──────┼─────────────────────────┼────┼─────┤
│"3L02"│"Money"                  │106 │44   │
├──────┼─────────────────────────┼────┼─────┤
│"2C01"│"Good"                   │105 │2    │
├──────┼─────────────────────────┼────┼─────┤
│"1P28"│"Greatness and intensity"│104 │2    │
├──────┼─────────────────────────┼────┼─────┤
│"2A22"│"Truth and falsity"      │104 │5    │
├──────┼─────────────────────────┼────┼─────┤
│"2D08"│"Love and friendship"    │100 │17   │
├──────┼─────────────────────────┼────┼─────┤
│"2A18"│"Intelligibility"        │99  │5    │
└──────┴─────────────────────────┴────┴─────┘

Outgoing Page-Rank of Categories

call algo.pageRank.stream(null,null) yield node, score
with node, toInt(score*10) as score order by score desc limit 10
return node.name, score/10.0 as score;

╒══════════════════════════════════════╤═══════╕
│"node.name"                           │"score"│
╞══════════════════════════════════════╪═══════╡
│"Greatness and intensity"             │5.6    │
├──────────────────────────────────────┼───────┤
│"Colour "                             │3.5    │
├──────────────────────────────────────┼───────┤
│"Unimportance"                        │3.5    │
├──────────────────────────────────────┼───────┤
│"Importance"                          │3.4    │
├──────────────────────────────────────┼───────┤
│"Hatred and hostility"                │3.4    │
├──────────────────────────────────────┼───────┤
│"Plants"                              │2.9    │
├──────────────────────────────────────┼───────┤
│"Good"                                │2.9    │
├──────────────────────────────────────┼───────┤
│"Age"                                 │2.8    │
├──────────────────────────────────────┼───────┤
│"Love and friendship"                 │2.7    │
├──────────────────────────────────────┼───────┤
│"Memory, commemoration and revocation"│2.6    │
└──────────────────────────────────────┴───────┘

Funny that both importance and unimportance have such a high rank.

call algo.pageRank.stream(null,null,{direction:'INCOMNG'}) yield node, score
with node, toInt(score*10) as score order by score desc limit 10
return node.name, score/10.0 as score;

Betweeness Centrality

Which categories connect others:

call algo.betweenness.stream('Category','Strong') yield nodeId, centrality as score
match (node) where id(node) = nodeId
with node, toInt(score) as score order by score desc limit 10
return node.id, node.name, score;

╒═════════╤═══════════════════════════════════════════╤═══════╕
│"node.id"│"node.name"                                │"score"│
╞═════════╪═══════════════════════════════════════════╪═══════╡
│"2C01"   │"Good"                                     │165912 │
├─────────┼───────────────────────────────────────────┼───────┤
│"1E02"   │"Animal categories, habitats and behaviour"│131109 │
├─────────┼───────────────────────────────────────────┼───────┤
│"3D05"   │"Authority, rebellion and freedom"         │108292 │
├─────────┼───────────────────────────────────────────┼───────┤
│"2D06"   │"Emotional suffering"                      │87551  │
├─────────┼───────────────────────────────────────────┼───────┤
│"1J34"   │"Colour "                                  │83595  │
├─────────┼───────────────────────────────────────────┼───────┤
│"1E05"   │"Insects and other invertebrates"          │77171  │
├─────────┼───────────────────────────────────────────┼───────┤
│"3D01"   │"Command and control"                      │71873  │
├─────────┼───────────────────────────────────────────┼───────┤
│"1O20"   │"Vigorous action and degrees of violence"  │65028  │
├─────────┼───────────────────────────────────────────┼───────┤
│"1C03"   │"Mental health"                            │64567  │
├─────────┼───────────────────────────────────────────┼───────┤
│"1F01"   │"Plants"                                   │59444  │
└─────────┴───────────────────────────────────────────┴───────┘

There are many other explorative queries and insights we can draw from this.

Let me know in the comments what you’d be interested in.

 
0

Fullstack JavaScript – Neo4j Script Procedures

on Apr 1, 2017 in cypher, neo4j

Imagine, being a fullstack JavaScript developer and not just using the language in the frontend, middleware or backend but also to create your user-defined procedures and functions in the database.

Several other databases support a similar approach for views and user defined extensions, and now you can do it with Neo4j too.

Already early last year, Neo4j’s [...]

 
1

Creating a Neo4j Example Graph with the Arrows Tool

on Mar 21, 2017 in cypher, import

Some years ago my colleague Alistair Jones created a neat little tool in JavaScript to edit and render example graphs in a consistent way.

It is aptly named Arrows and you can find it here: http://www.apcjones.com/arrows

We mostly use it for presentations, but also to show data models for Neo4j GraphGists and Neo4j Browser Guides.
Because it also [...]

 
0

Academy Awards (Oscars) from Kaggle to Neo4j

on Mar 9, 2017 in cypher, import, neo4j

pre {
font-style:normal;
font-size:75%;
line-height: normal;
}
table {
margin: 10px;
}
th, td {
border-bottom: 1px solid black;
}
th {
background-color: #ddd;
}
img {
max-width:100%;
}

Table of Contents

Install Neo4j
Find the data as CSV
Importing the Data
Some Fun Queries
Part 2: Full Import from Kaggle
More Queries

Most Oscars
Most different Awards
Which Countries got the most [...]

 
1

5 Tips & Tricks for Fast Batched Updates of Graph Structures with Neo4j and Cypher

on Mar 2, 2017 in Uncategorized, cypher, import, neo4j

Inefficient Solutions
Better Approach

UNWIND to the Rescue
Overall Syntax Structure

Examples

Create node with properties
MERGE node with properties
Node lookup and MERGE/CREATE relationship between with properties
Lookup by id, or even list of ids

Faster, Better, Further: All the tricks

Update of existing nodes by id
Update of existing relationships by id

Conditional Data Creation
Utilizing APOC Procedures

Creating Nodes and Relationships dynamically
Batched Transactions
Creating / Updating Maps [...]

 
0

The Reddit Meme Graph with Neo4j

on Feb 25, 2017 in cypher, import

pre {
font-style:normal;
font-size:75%;
line-height: normal;
}
img {
max-width:100%;
}

Saturday night after not enough drinks, I came across these tweets by @LeFloatingGhost.

This definitely looks like a meme graph.

We can do that too

 
0

User Defined Functions in Neo4j 3.1.0-M10

on Oct 6, 2016 in apoc, cypher

Neo4j 3.1 brings some really neat improvements in Cypher alongside other cool features

I already demonstrated the – GraphQL inspired – map projections and pattern comprehensions in my last blog post.

User Defined Procedures

In the 3.0 release my personal favorite was user defined procedures which can be implemented using Neo4j’s Java API and called directly from Cypher.
You [...]

 
0

Neo4j 3.0 Stored Procedures

on Feb 29, 2016 in cypher, java

One of the many exciting features of Neo4j 3.0 are “Stored Procedures” that, unlike the existing Neo4j-Server extensions are directly callable from Cypher.

At the time of this writing it is only possible to call them in a stand-alone statement with CALL package.procedure(params)
but the plan is to make them a fully integrated part of Cypher statements.
Either [...]

 
0

Using XRebel 2 with Neo4j

on May 5, 2015 in neo4j

At Spring.IO in Barcelona I met my pal Oleg from ZeroTurnaround and we looked at how the new XRebel 2
integrates with Neo4j, especially with the remote access using the transactional Cypher http-endpoint.

As you probably know, Neo4j currently offers a remoting API based on HTTP requests (a new binary protocol is in development).

Our JDBC driver utilizes [...]

 
1

Neo4j Server Extension for Single Page Experiments

on Apr 24, 2015 in neo4j, server

Sometimes you have a nice dataset in Neo4j and you’d want to provide a self-contained way of quickly exposing it to the outside world without a multi-tier setup.

So for experiments and proofs of concepts it would be helpful to be able to extend Neo4j Browser to accomodate new types of frames and commands.
Unfortunately we’re not [...]

Copyright © 2007-2017 Better Software Development All rights reserved.
Multi v1.4.5 a child of the Desk Mess Mirrored v1.4.6 theme from BuyNowShop.com.