Applications of Graph Neural Networks
Graphs and their study have received a lot of attention since ages due to their ability of representing the real world in a fashion that can be analysed objectively. Indeed, graphs can be used to represent a lot of useful, real world datasets such as social networks, web link data, molecular structures, geographical maps, etc. Apart from these cases which have a natural structure to them, non-structured data such as images and text can also be modelled in the form of graphs in order to perform graph analysis on them. Due to the expressiveness of graphs and a tremendous increase in the available computational power in recent times, a good amount of attention has been directed towards the machine learning way of analysing graphs, i.e. Graph Neural Networks.
图表和他们的研究自古以来就受到了很多关注，因为他们能够以一种可以客观分析的方式表现现实世界。 实际上，图表可用于表示许多有用的真实世界数据集，例如社交网络，网络链接数据，分子结构，地理地图等。除了这些具有自然结构的情况，非结构化数据，如 因为图像和文本也可以以图形的形式建模，以便对它们进行图形分析。 由于图的表现力和近来可用计算能力的巨大增加，已经将大量注意力集中在分析图的机器学习方法上，即图形神经网络。
So, what are Graph Neural Networks (GNNs)?
According to this paper, Graph neural networks (GNNs) are connectionist models that capture the dependence of graphs via message passing between the nodes of graphs. They are extensions of the neural network model to capture the information represented as graphs. However, unlike the standard neural nets, GNNs maintain state information to capture the neighbouurhood properties of the nodes. These states of the nodes of a graph can then be used to produce output labels such as a classification of the node or an arbitrary function value computed by the node. The network tries to learn these encodings(hv) for each of the nodes through a mutual sharing of data among the nodes’ neighbours in an iterative manner until convergence.
图形神经网络（GNN）是连接模型，通过图形节点之间的消息传递捕获图形的依赖性。 它们是神经网络模型的扩展，用于捕获以图形表示的信息。 然而，与标准神经网络不同，GNN维护状态信息以捕获节点的邻近属性。 然后，可以使用图的节点的这些状态来产生输出标签，例如节点的分类或由节点计算的任意函数值。 网络试图通过迭代方式在节点的邻居之间相互共享数据来学习每个节点的这些编码（hv），直到收敛为止。
Given the wide range of domains in which GNNs can be applied, most of them could either be classified as structural applications or as non-structural applications based on the type of data. The structural case represents data that has an inherent structure to it such social interaction networks or molecular networks. Consequently, the non-structural scenarios represent cases where the structure in the data is not explicit such as text and images. In these non-structural scenarios then, the general approach is to try to transform the data to a structured format and then apply GNN. There are a few other applications as well that do not fall into the purview of either of the classes mentioned above. We will discuss some interesting applications from each of these categories here.
鉴于可以应用GNN的广泛领域，其中大多数可以被分类为结构应用程序或基于数据类型的非结构应用程序。 结构案例表示具有其固有结构的数据，例如社交互动网络或分子网络。 因此，非结构性场景表示数据中的结构不明确的情况，例如文本和图像。 在这些非结构性场景中，一般方法是尝试将数据转换为结构化格式，然后应用GNN。 还有一些其他应用程序也不属于上述任何类别的范围。 我们将在这里讨论这些类别中的一些有趣的应用程序。
Physical Systems
The human brain is said to aid its reasoning process through the creation of graphs learned from daily experiences of the real world. Hence great importance is being assigned to the modelling of real-world entities and their interactions as graphs. This is deemed as a first steps towards human-like artificial intelligence. By considering objects as nodes and the relations or interactions between them as edges, simple physical systems can be modelled as graphs and can be effectively analysed using GNN.
Interaction networks can be trained to reason about the interactions of objects in a complex physical system. It can make predictions and inferences about various system properties in domains such as collision dynamics (rigid and non-rigid). It simulates these systems using object and relation centric reasonings using deep neural networks on graphs. These GNNs have been shown to predict the trajectories of different objects thousands of time steps into the future.
据说人类大脑通过创建从现实世界的日常经验中学习的图表来帮助其推理过程。因此，重要的是分配给现实世界实体的建模及其作为图形的交互。这被认为是迈向人类人工智能的第一步。通过将对象视为节点以及它们之间的关系或交互作为边缘，可以将简单的物理系统建模为图形，并且可以使用GNN进行有效分析。
可以训练交互网络来推断复杂物理系统中对象的交互。它可以对域中的各种系统属性进行预测和推断，例如碰撞动力学（刚性和非刚性）。它使用图上的深度神经网络，使用对象和关系中心推理来模拟这些系统。已经证明这些GNN可以预测未来数千个时间步长的不同物体的轨迹。
Visual interaction networks make use of the above interaction network to go a step further and learn about the dynamics of a system from just its raw visual observation or to put it simply, with as little as six video frames of the system in action. Apart from predicting the future trajectories of a range of physical systems just like its parent network, this model can also infer the mass and future states of unknown objects from their influence on the visible ones! This is achieved through the co-training of a perceptual front-end convolutional neural network that parses a visual scene to provide a set of object node representations to a back-end interaction network.
视觉互动网络利用上述互动网络 更进一步，从原始的视觉观察中了解系统的动态，或简单地说，系统中只有六个视频帧在运行。 除了像其父网络一样预测一系列物理系统的未来轨迹之外，该模型还可以根据它们对可见物体的影响来推断未知物体的质量和未来状态！ 这是通过对感知前端卷积神经网络的协同训练来实现的，该神经网络解析视觉场景以向后端交互网络提供一组对象节点表示。
Chemistry
The nano-scale molecules have an inherent graph like structure with the ions or the atoms being the nodes and the bonds between them, edges. GNNs can be applied in both scenarios: learning about existing molecular structures as well as discovering new chemical structures. This has had a significant impact in computer aided drug design.
纳米级分子具有固有的图形结构，其中离子或原子是节点，它们之间是键，边缘。 GNN可以应用于两种情况：了解现有的分子结构以及发现新的化学结构。 这对计算机辅助药物设计产生了重大影响。
Molecular fingerprints are feature vectors that represent molecules. Machine learning models to predict the properties of a new molecule by learning from example molecules use fixed length fingerprints as inputs. GNNs can replace the traditional means that give a fixed encoding of the molecule to allow the generation of differentiable fingerprints adapted to the task for which they are required. Furthermore, these molecular fingerprints learned by the GNN from the graph structure of the molecule need not be as large as the fixed fingerprints which must encode all possible substructures in a single feature vector. The differentiable fingerprints can be optimised to encode only relevant features, reducing downstream computation and regularisation requirements.
分子指纹是代表分子的特征向量。 通过学习示例分子来预测新分子特性的机器学习模型使用固定长度指纹作为输入。 GNN可以取代传统方法，该方法提供分子的固定编码，以允许产生适合于它们所需任务的可区分指纹。 此外，GNN从分子的图结构中学习的这些分子指纹不需要与必须在单个特征向量中编码所有可能的子结构的固定指纹一样大。 可以优化可微分指纹以仅编码相关特征，减少下游计算和正则化要求。
Image classification
Image classification, a classic computer vision problem, has outstanding solutions from a number of state-of-the-art machine learning mechanisms, the most popular being convolutional neural networks (CNN). GNN, which drive their motivation out of CNN, have also been applied in this domain. Most of these models, including GNN, do provide attractive results when given a huge training set of labelled classes. The focus now is towards getting these models to perform well on zero-shot and few-shot learning tasks. Zero shot learning (ZSL) refers to trying to learn to recognise classes that the model has not encountered in its training. ZSL recognition relies on the existence of a labelled training set of seen classes and the knowledge about how each unseen class is semantically related to the seen ones. One approach is to leverage structural information, in the form of graphs, in ZSL image classification. GNN, consequently, appear quite appealing in this respect. Knowledge graphs can provide the necessary information to guide the ZSL task. Different approaches differ in the kind of information they represent in the knowledge graph. These graphs may be based on the similarities between the images themselves or those of the objects in the images extracted through object detection. The graphs may also incorporate semantic information from word embeddings of the category labels of the images. GNNs can then be applied to this structured data to aid the ZSL image classification-recognition task.
图像分类是一种经典的计算机视觉问题，具有许多最先进的机器学习机制的最佳解决方案，最流行的是卷积神经网络（CNN）。从CNN中推动其动力的GNN也已应用于此领域。大多数这些模型，包括GNN，在提供大量标记类训练时确实提供了有吸引力的结果。现在的重点是让这些模型在零射击和少射击学习任务中表现良好。零射击学习（ZSL）是指尝试学习识别模型在训练中未遇到的类。 ZSL识别依赖于所看到的类的标记训练集的存在以及关于每个看不见的类如何与所看到的类在语义上相关的知识。一种方法是在ZSL图像分类中以图形的形式利用结构信息。因此，GNN在这方面看起来非常吸引人。知识图可以提供指导ZSL任务的必要信息。不同的方法在知识图中表示的信息类型不同。这些图可以基于图像本身之间的相似性或通过对象检测提取的图像中的对象的相似性。图形还可以包含来自图像的类别标签的字嵌入的语义信息。然后可以将GNN应用于该结构化数据以辅助ZSL图像分类识别任务。
Text
Like images, the relation within text is also not explicit, i.e. text cannot be considered structured data to which GNN can be applied directly. However, there are ways to convert a text document into structured data such as a graph of words or of sentences and then use graph convolution to convolve the word graphs. Another approach to structure the text data is to use Sentence LSTM which views an entire sentence as a single state (node) consisting of sub-states i.e. words. Document citation relations can also be used to construct graphs with documents as the nodes. Text GNNs can then be used to learn embeddings for words and documents. These approaches can be used for various NLP tasks such as text classification, sequence labelling, machine translation, relation and event extraction.
与图像一样，文本内的关系也不明确，即文本不能被视为可以直接应用GNN的结构化数据。 但是，有一些方法可以将文本文档转换为结构化数据，例如单词或句子图形，然后使用图形卷积来拼写单词图形。 构造文本数据的另一种方法是使用Sentence LSTM，其将整个句子视为由子状态即单词组成的单个状态（节点）。 文档引用关系也可用于构建以文档作为节点的图形。 然后可以使用文本GNN来学习单词和文档的嵌入。 这些方法可用于各种NLP任务，例如文本分类，序列标记，机器翻译，关系和事件提取。
One interesting area of application is reading comprehension; given a text passage, the model must be able to answer questions based on it by consolidating information in the passage.
Reading comprehension is one of the complex reasoning tasks performed by humans; the answer to a question may not be located in any single part of the extract but may need global inferencing. Representing the passage in the form of a graph using a Sentence LSTM helps in better connecting the global evidences.
一个有趣的应用领域是阅读理解; 如果给出文本段落，模型必须能够通过合并段落中的信息来回答基于它的问题。
阅读理解是人类执行的复杂推理任务之一; 问题的答案可能不在提取的任何单个部分中，但可能需要全局推理。 使用Sentence LSTM以图形的形式表示段落有助于更好地连接全局证据。
Combinatorial Optimization
Combinatorial optimization problems over graphs are a set of NP-hard problems. Some of these can be solved by heuristic methods. In recent times, attempt is being made to solve them using deep neural networks. Consequently, GNNs are also being leveraged to operate over these graph structured datasets.
图上的组合优化问题是一组NP难问题。 其中一些可以通过启发式方法解决。 最近，正在尝试使用深度神经网络来解决它们。 因此，GNN也被用于对这些图形结构化数据集进行操作。
Interestingly, a general GNN based framework can be applied to a number of optimisation problems over graphs such the minimum vertex cover problem, maximum cut, the travelling salesman problem, minimum spanning tree, etc. A few approaches combine GNNs with reinforcement learning to iteratively learn a solution given an input graph. One of the instances in which GNNs outperform traditional methods is the Quadratic Assignment Problem. It aims to measure the similarity of two graphs. The comparison of the graphs using GNN-learned node embeddings offers better performance than standard relaxation-based techniques.
有趣的是，一般的基于GNN的框架可以应用于图形上的许多优化问题，例如最小顶点覆盖问题，最大切割，旅行商问题，最小生成树等。一些方法将GNN与强化学习相结合以迭代学习 给出输入图的解决方案。 GNN优于传统方法的一个例子是二次分配问题。 它旨在测量两个图的相似性。 使用GNN学习节点嵌入的图形比较提供了比基于标准松弛的技术更好的性能。
Conclusion
Graph Neural Networks are increasingly gaining popularity, given their expressive power and explicit representation of graphical data. Hence, they have a wide range of applications in domains that can harness graph structures out of their data. Presented above is just the tip of the iceberg. As newer architectures continue to crop up, GNNs will continue to foray into diverse domains.
图形神经网络越来越受欢迎，因为它们具有表现力和图形数据的明确表示。 因此，它们在域中具有广泛的应用，可以利用其数据中的图形结构。 上面提到的只是冰山一角。 随着新架构的不断涌现，GNN将继续涉足多个领域。