Introducing tidygraph
July 7, 2017
I’m very pleased to announce that my new package tidygraph
is now
available on CRAN. As the name
suggests, tidygraph
is an entry into the tidyverse that provides a tidy
framework for all things relational (networks/graphs, trees, etc.). tidygraph
is a relatively big package in terms of exported functions (280 exported
symbols) so all functions will not be covered in this release note. I will
however provide an overview of all the areas that tidygraph
touches upon so you
should have a pretty good grasp on what the package can do for you.
Tidy network data?
There’s a discrepancy between relational data and the tidy data idea, in that relational data cannot in any meaningful way be encoded as a single tidy data frame. On the other hand, both node and edge data by itself fits very well within the tidy concept as each node and edge is, in a sense, a single observation. Thus, a close approximation of tidyness for relational data is two tidy data frames, one describing the node data and one describing the edge data.
The tbl_graph object
Underneath the hood of tidygraph
lies the well-oiled machinery of igraph,
ensuring efficient graph manipulation. Rather than keeping the node and edge
data in a list and creating igraph
objects on the fly when needed, tidygraph
subclasses igraph
with the tbl_graph
class and simply exposes it in a tidy
manner. This ensures that all your beloved algorithms that expects igraph
objects still works with tbl_graph
objects. Further, tidygraph
is very
careful not to override any of igraph
s exports so the two packages can coexist
quite happily.
To underline the tidyness of the tbl_graph
class the print method shows the
object as two tibbles along with additional network information.
tbl_graph
objects can be created directly using the tbl_graph()
function
that takes a node data.frame and an edge data.frame. On top of that, tidygraph
also provides coercion from a huge amount of relational data structures. The
following list gives the packages/classes that can currently be converted to
tbl_graph
s, using the as_tbl_graph
function:
data.frame
,list
,matrix
frombase
igraph
fromigraph
network
fromnetwork
dendrogram
andhclust
fromstats
Node
fromdata.tree
phylo
andevonet
fromape
graphNEL
,graphAM
,graphBAM
fromgraph
(in Bioconductor)
For all of the coercions you can expect that data on the nodes and edges are
kept and available after conversion to tbl_graph
:
Lastly, tidygraph
also wraps the multitude of graph constructors available in
igraph
and exports them under the create_*()
family of functions for
deterministic constructors (e.g. the call to create_ring(10)
above) and the
play_*()
family for constructors that incorporate sampling (e.g.
play_erdos_renyi()
for creating graphs with a fixed edge probability). All of
these functions provide a consistent argument naming scheme to make them easier
to use and understand.
Meet a new verb…
There are many ways a multitable setup could fit into the tidyverse. There could
be an added qualifier to the verbs such as mutate_nodes()
and filter_edges()
or each verb could take an additional argument specifying what is targeted e.g.
arrange(..., target = 'nodes')
. Both of these approachable are viable but
would require a huge amount of typing as well as being taxing to support down
the line.
The approach used by tidygraph
is to let the data object itself carry around
a pointer to the active data frame that should be the target of manipulation.
This pointer is changed using the activate()
verb, which, on top of changing
which part of the data is being worked on, also changes the print output to show
the currently active data on top:
As can be seen, activate()
takes a single argument specifying the part of the
data that should be targeted for subsequent operations as an unquoted symbol.
tidygraph
continues the naming conventions from ggraph
using nodes
and
edges
to denote the entities and their connections respectively, but
vertices
and links
are allowed synonyms inside activate()
.
The current active data can always be extracted as a tibble using as_tibble()
The dplyr verbs
Using activate()
it is possible to use the well known dplyr
verbs as one
would expect without much hassle:
In the above the .N()
function is used to gain access to the node data while
manipulating the edge data. Similarly .E()
will give you the edge data and
.G()
will give you the tbl_graph
object itself.
Some verbs have effects outside of the currently active data.
filter()
/slice()
on node data will remove the edges terminating at the
removed nodes and arrange()
on nodes will change the indexes of the to
and
from
column in the edge data.
While one might expect all of dplyr
s verbs to be supported in that manner,
there is a clear limitation in the relational data structure that requires rows
to maintain their identity. Thus, summarise()
and do()
are not allowed as
there is no clear interpretation of how alterations on the node and edge data
with these verbs should be interpreted. If these operations are required I
suggest applying them to a tibble representation and then joining the result
back in.
Speaking of joining, all joins from dplyr
are supported. Nodes and edges are
added and removed as required by the join. New edge data to be joined in must
have a to
and from
column referencing valid nodes in the existing graph.
Expanding the vocabulary
On top of what has been showed so far, tidygraph
provides an assortment of
graph specific verbs that can be used to power your analysis and manipulation.
Analogous to bind_rows()
, tidygraph
provides three functions to expand your
data: bind_nodes()
and bind_edges()
append nodes and edges to the graph
respectively. As with the join functions bind_edges()
must contain valid
from
and to
columns. bind_graphs()
allows you to combine multiple graphs
in the same graph structure resulting in each original graph to become a
component in the returned graph.
While bind_graphs()
cannot be used to create edges between the merged graphs
graph_join()
can do just that. It merges nodes using a full_join()
semantic
and keeps the individual edges from both graphs:
The standard dplyr
verbs protects the to
and from
columns in the edge data
in order to avoid accidental modification of the graph topology. If changing of
the terminal nodes are necessary the reroute()
verb will come in handy:
As can be seen, reroute works pretty much as a to
and from
specific
mutate()
, with the added benefit of incorporating a subset operator if only a
few edges should be changed.
Making the most of graphs
While being able to use the dplyr
verbs on relational data is nice and all,
one of the reasons we are dealing with graph data in the first place is because
we need some graph-based algorithms for solving our problem at hand. If we need
to break out of the tidy workflow every time this was needed we wouldn’t have
gained much. Because of this tidygraph
has wrapped more or less all of
igraph
s algorithms in different ways, ensuring a consistent syntax as well as
output that fits into the tidy workflow. In the following we’re going to take a
look at these.
Central to all of these functions is that they know about which graph is being
computed on (in the same way that n()
knows about which tibble is currently in
scope). Furthermore they always return results matching the node or edge
position so they can be used directly in mutate()
calls.
Node and edge types
On the top of our list of things we might be interested to know about is whether
nodes or edges are of specific types such as leaf, sink, loop,
etc. All of these functions return a logical vector indicating if the node or
edge belong to the specified group. To easily find functions that queries types,
all functions are prefixed with node_is_*
/edge_is_*
.
Another example could be to remove loop edges using filter(!edge_is_loop())
.
Centrality
One of the simplest concepts when computing graph based values is that of
centrality, i.e. how central is a node or edge in the graph. As this
definition is inherently vague, a lot of different centrality scores exists that
all treat the concept of central a bit different. One of the famous ones is
the pagerank algorithm that was powering Google Search in the beginning.
tidygraph
currently has 11 different centrality measures and all of these are
prefixed with centrality_*
for easy discoverability. All of them returns a
numeric vector matching the nodes (or edges in the case of
centrality_edge_betweenness()
).
It is quite difficult to a priori decide which centrality measure makes most sense for a problem at hand so having easy access to a large range of them in a common syntax is a boon.
Clustering
Another common operation is to group nodes based on the graph topology,
sometimes referred to as community detection based on its commonality in social
network analysis. All clustering algorithms from igraph
is available in
tidygraph
using the group_*
prefix. All of these functions return an integer
vector with nodes (or edges) sharing the same integer being grouped together.
Node pairs
Some statistics are a measure between two nodes, such as distance or similarity
between nodes. In a tidy context one of the ends must always be the node defined
by the row, while the other can be any other node. All of the node pair
functions are prefixed with node_*
and ends with _from
/_to
if the measure
is not symmetric and _with
if it is; e.g. there’s both a node_max_flow_to()
and node_max_flow_from()
function while only a single node_cocitation_with()
function. The other part of the node pair can be specified as an integer vector
that will get recycled if needed, or a logical vector which will get recycled
and converted to indexes with which()
. This means that output from node type
functions can be used directly in the calls, e.g.
Searches
An integral type of operation on graphs is to perform a search, that is, start
from one node and then traverse the edges until all nodes has been visited. The
most common approaches are either breath first search where all neighbors of
a node is visited before moving on to the next node, or depth first serch
where you move along to the next node immediately and only backtracks and visit
other neighbors when you’ve hit a dead end. Different statistics from these
searches are available in tidygraph
through the bfs_*()
and dfs_*()
family
of functions e.g. the distance to the start node along the search can be
obtained with bfs_dist()
/dfs_dist()
. The root node can be specified in the
same way as with node pairs. Sorting based on a search from the node with
highest centrality can thus be done with:
Local measures
Often we find ourselves interested in the local neighborhood of a node for
various reasons. We might want to know the average degree around a node or the
number of triangles each node participate in. The local_*()
family of
functions provide access to a range of node measures that are dependent on the
local neighborhood of each node.
All the rest
While an ontology of graph operations has been attempted in the different
functions above, there are some that falls outside. These have been lumped
together under the node_*()
and edge_*()
umbrellas and include things such
as topological ordering and Burt’s constraint among others. All of these
functions ensures a mutate-compatible output.
Graph measures
Along with computations on the individual nodes and edges it is sometimes
necessary to get summary statistics on the graph itself. These can be simple
measures such as the number of nodes and edges as well as more involved measures
such as assortativity (the propensity of similar nodes to be connected). All of
these measures can be calculated through the graph_*()
function family and
they will all return a scalar.
Mapping over nodes
Just to spice it all up a bit tidygraph
pulls purrr
into the mix and
provides some additional graph-centric takes on the familiar map*()
. More
specifically tidygraph
provides functionality to apply a function over nodes
as a breath or depth first search is carried out, while getting access to the
result of the computations coming before, as well as mapping over the local
neighborhood of each node. All of these function returns a list in their
bare bone form, but as with purrr
versions exists that ensures the output is of
a certain type (e.g. map_bfs_dbl()
).
Mapping over searches
The search maps comes in two flavors. Either the nodes are mapped in the order of the search, or they are mapped in the reverse order. In the first version, each call will have access to the statistics and map results of all the nodes that lies between itself and the root. In the second version each call will have access to the results and statistics of all its offspring. Furthermore the mapping function is passed the graph itself as well as all the search statistics of the node currently being mapped over. An example would be to propagate the species value in our iris clustering upwards as long as theirs agreement between the children. For this to work, we will need the reverse version of a breath first search to make sure that all children have been evaluated prior to mapping over a node:
Mapping over neighborhoods
The neighborhood map is exposed through map_local()
as well as its type safe
versions. The mapping function has a much simpler format as it simply gets
passed a subgraph representing the local neighborhood as well as the index of
the node in the original graph being mapped over. E.g. to get the number of
edges in the local neighborhood around each node, one would simply do:
One last thing…
While the functions discussed above makes it easy to make slight changes to your
network topology it is less straightforward to make radical changes. Even more
so if the radical changes are only needed temporarily for the sake of a few
computations. This is where the new morph()
verb comes in handy (along with
the accompanying unmorph()
and crystallise()
verbs). In essence, morph()
lets you set up a temporary alternative version of your graph, make computations
on it using the standard dplyr
verbs, and then merge the changes back in
using unmorph()
. The types of alternative representations are varied and can
be extended by the user. Nodes can be converted to edges and the other way
around, both nodes and edges can be combined, and the alternate representation
does not need to cover the full original graph. Instead of trying to describe it
in words, let’s see how it plays out in use:
As can be seen, the morph syntax both handles multiple graphs, collapsed nodes
and changing edges to nodes, without any change in the mental model of the
operations. All morphing functions are prefixed with to_*
for easy discovery
and includes minimum spanning trees, complement graph, dominator tree etc. In
the case where you are interested to continue working with the morphed
representation as a proper tbl_graph
you can use the cystallise()
verbs that
removes any link to the original graph and returns a tibble with a row per graph
in the morphed representation (as a morph can result in multiple graphs):
Wrapping it all up
I hope I have given you a small glimpse of what tidygraph
is all about. If
working with network data in the past has felt intimidating and strange
tidygraph
might feel more at home, but even if you’re a seasoned pro within
network analysis the package should provide a powerful but streamlined interface
to many operations.
Roadmap
The next goal of my quest to revamp relational data analysis in R will be to
“rebuild” ggraph
around tidygraph
. This is not to say that the two are
incompatible at the moment — there’s full support through ggraph
s support for
igraph
— rather I want to only support tbl_graph
in the future as all
relevant data structures can be converted to this common format through
as_tbl_graph()
.
For tidygraph
itself, I have some more ideas I want to explore. Currently
missing from the whole package is any notion of modelling and it will be
interesting to see how this can fit in. Further, I have this wild idea about
providing a tidygraph
link to graph databases such as Neo4J in the same way
as dbplyr
provides an interface to SQL databases. Lastly, the current focus
has been on supporting the algorithms provided by igraph
. While an extensive
package, igraph
does not implement everything and there might be stuff lacking
that should be added down the line.
Take care…