Exposing hidden networks
Renaud Lambiotte, Somerville fellow and tutor in Applied Mathematics, uses maths to understand and map networks in the real world, ranging from epidemics to social media. Here, he discusses his most recent breakthrough, which has shed new light on the world of hidden networks.
For most of my mathematical career, I have been trying to understand how the inner structure of systems – seen as nodes connected by pairwise edges – affects their function. I then try to develop methods to extract information from large social or brain networks through community detection to identify densely connected groups or communities of nodes.
But about 3 years ago, I had a sort of intellectual midlife crisis, if you will. I started to question how useful networks are and how researchers can use them in situations when their edges cannot be observed. This may sound surprising, but there are many situations where you would expect a relationship to exist between objects but we cannot observe one directly.
Take time series in functional Magnetic Resonance Imaging (fMRI), in meteorology, or in finance. In each case, there exist dependencies between the objects, but we can only observe the effects of their hidden network, rather than the network itself.
The standard approach in my field was to first infer the underlying network, and then to apply network method in order to uncover the salient features of the system. This two-step procedure is unsatisfactory for many reasons, from a lack of elegance to computational limitations.
Our objective was to provide the result of a network analysis without even constructing the network.
In collaboration with researchers from Imperial College, Université Catholique de Louvain, and Spotify, we were able to develop an end-to-end algorithm that would do just this, skipping the unnecessary intermediary steps.
We put the new algorithm to the test using climate data from 1429 U.S. cities. The Köppen-Geiger classification system, which was originally created in 1884, puts cities in 5 different categories based on average temperature and precipitation.
We found that our model revealed communities of cities in which the temperature and precipitation varied similarly with much more detail than the manmade Köppen-Geiger method. You can see the results for yourself in the diagram below comparing the two methods.
This is a significant conceptual advance in the field of network theory. Our next aim will be to generalise this for other network algorithms.