EXPLORING GRAPH GENERATIVE MODELS: TECHNIQUES, APPLICATIONS, AND FUTURE DIRECTIONS
Keywords:
Graph Generative Models, Node Embeddings, Generative Adversarial Networks (GANs), Drug Discovery, Social Network AnalysisAbstract
Graphs are one of the most elegant ways to store data that shows complex connections and interactions across many entities. Recent progress in deep learning has led to the creation of strong graph-generative models that can learn and create graph-structured data with myriad applications. This article gives an overview of graph-generative models, focusing on the techniques they use, the things they can be used for, and where the field is headed. We talk about well-known methods like Graph Variational Autoencoders (Graph-VAEs), Graph Generative Adversarial Networks (Graph-GANs), and GraphRNN, as well as their variations and enhancements. We also talk about the usefulness of graph-generative models in areas such as drug discovery, studying social networks, finding scams, and studying biological networks. Lastly, we talk about open problems and possible directions for future study in this field that is changing very quickly
References
M. E. J. Newman, "The structure and function of complex networks," SIAM Review, vol. 45, no. 2, pp. 167-256, 2003.
J. Ugander, B. Karrer, L. Backstrom, and C. Marlow, "The anatomy of the Facebook social graph," arXiv preprint arXiv:1111.4503, 2011.
J. You, R. Ying, X. Ren, W. L. Hamilton, and J. Leskovec, "GraphRNN: Generating realistic graphs with deep auto-regressive models," in Proceedings of the 35th International Conference on Machine Learning, 2018, pp. 5708-5717.
N. Joshi, P. Dixit, and N. R. Pal, "A survey on graph generative models," IEEE Access, vol. 9, pp. 88675-88699, 2021.
N. De Cao and T. Kipf, "MolGAN: An implicit generative model for small molecular graphs," arXiv preprint arXiv:1805.11973, 2018.
A. Bojchevski, O. Shchur, D. Zügner, and S. Günnemann, "NetGAN: Generating graphs via random walks," in Proceedings of the 35th International Conference on Machine Learning, 2018, pp. 610-619.
T. N. Kipf and M. Welling, "Variational graph auto-encoders," arXiv preprint arXiv:1611.07308, 2016.
H. Wang, J. Wang, J. Wang, M. Zhao, W. Zhang, F. Zhang, X. Xie, and M. Guo, "GraphGAN: Graph representation learning with generative adversarial nets," in Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018, pp. 2508-2515.
J. You, R. Ying, X. Ren, W. L. Hamilton, and J. Leskovec, "GraphRNN: Generating realistic graphs with deep auto-regressive models," in Proceedings of the 35th International Conference on Machine Learning, 2018, pp. 5708-5717.
Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and P. S. Yu, "A comprehensive survey on graph neural networks," IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 1, pp. 4-24, 2021.
H. Kwak, C. Lee, H. Park, and S. Moon, "What is Twitter, a social network or a news media?" in Proceedings of the 19th International Conference on World Wide Web, 2010, pp. 591-600.
J. Ugander, B. Karrer, L. Backstrom, and C. Marlow, "The anatomy of the Facebook social graph," arXiv preprint arXiv:1111.4503, 2011.
D. J. Price, "Networks of scientific papers," Science, vol. 149, no. 3683, pp. 510-515, 1965.
S. Wasserman and K. Faust, Social Network Analysis: Methods and Applications. Cambridge University Press, 1994.
A. Barrat, M. Barthélemy, R. Pastor-Satorras, and A. Vespignani, "The architecture of complex weighted networks," Proceedings of the National Academy of Sciences, vol. 101, no. 11, pp. 3747-3752, 2004.
E. Bullmore and O. Sporns, "Complex brain networks: Graph theoretical analysis of structural and functional systems," Nature Reviews Neuroscience, vol. 10, no. 3, pp. 186-198, 2009.
S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak, and Z. Ives, "DBpedia: A nucleus for a web of open data," in The Semantic Web. Springer, 2007, pp. 722-735.
T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms. MIT Press, 2009.
J. Leskovec, A. Rajaraman, and J. D. Ullman, Mining of Massive Datasets. Cambridge University Press, 2014.
T. N. Kipf and M. Welling, "Semi-supervised classification with graph convolutional networks," in Proceedings of the 5th International Conference on Learning Representations (ICLR), 2017.
A. Grover and J. Leskovec, "node2vec: Scalable feature learning for networks," in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 855-864.
B. Perozzi, R. Al-Rfou, and S. Skiena, "DeepWalk: Online learning of social representations," in Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2014, pp. 701-710.
W. L. Hamilton, R. Ying, and J. Leskovec, "Inductive representation learning on large graphs," in Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS), 2017, pp. 1025-1035.
J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, and Q. Mei, "LINE: Large-scale information network embedding," in Proceedings of the 24th International Conference on World Wide Web (WWW), 2015, pp. 1067-1077.
K. Xu, W. Hu, J. Leskovec, and S. Jegelka, "How powerful are graph neural networks?" in Proceedings of the 7th International Conference on Learning Representations (ICLR), 2019.
T. N. Kipf and M. Welling, "Variational graph auto-encoders," arXiv preprint arXiv:1611.07308, 2016.
G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, vol. 313, no. 5786, pp. 504-507, 2006.
T. N. Kipf and M. Welling, "Semi-supervised classification with graph convolutional networks," in Proceedings of the 5th International Conference on Learning Representations (ICLR), 2017.
W. Jin, R. Barzilay, and T. Jaakkola, "Junction tree variational autoencoder for molecular graph generation," in Proceedings of the 35th International Conference on Machine Learning (ICML), 2018, pp. 2323-2332.
T. N. Kipf and M. Welling, "Variational graph auto-encoders," arXiv preprint arXiv:1611.07308, 2016.
A. Grover, A. Zweig, and S. Ermon, "Graphite: Iterative generative modeling of graphs," in Proceedings of the 36th International Conference on Machine Learning (ICML), 2019, pp. 2434-2444.
T. N. Kipf and M. Welling, "Variational graph auto-encoders," arXiv preprint arXiv:1611.07308, 2016.
S. Pan, R. Hu, G. Long, J. Jiang, L. Yao, and C. Zhang, "Adversarially regularized graph autoencoder for graph embedding," in Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI), 2018, pp. 2609-2615.
N. De Cao, T. Kipf, and M. Welling, "MolGAN: An implicit generative model for small molecular graphs," arXiv preprint arXiv:1805.11973, 2018.
M. Simonovsky and N. Komodakis, "GraphVAE: Towards generation of small graphs using variational autoencoders," in Proceedings of the 27th International Conference on Artificial Neural Networks (ICANN), 2018, pp. 412-422.
A. Bojchevski, O. Shchur, D. Zügner, and S. Günnemann, "NetGAN: Generating graphs via random walks," in Proceedings of the 35th International Conference on Machine Learning (ICML), 2018, pp. 610-619.
J. You, R. Ying, X. Ren, W. L. Hamilton, and J. Leskovec, "GraphRNN: Generating realistic graphs with deep auto-regressive models," in Proceedings of the 35th International Conference on Machine Learning (ICML), 2018, pp. 5708-5717.
W. Jin, R. Barzilay, and T. Jaakkola, "Junction tree variational autoencoder for molecular graph generation," in Proceedings of the 35th International Conference on Machine Learning (ICML), 2018, pp. 2323-2332.
H. Zeng, H. Zhou, A. Srivastava, R. Kannan, and V. Prasanna, "GraphSAINT: Graph sampling based inductive learning method," in Proceedings of the 8th International Conference on Learning Representations (ICLR), 2020.
N. Brown, M. Fiscato, M. H. Segler, and A. C. Vaucher, "GuacaMol: Benchmarking models for de novo molecular design," Journal of Chemical Information and Modeling, vol. 59, no. 3, pp. 1096-1108, 2019.
D. Neil, M. Segler, L. Guasch, M. Ahmed, D. Plumbley, M. Sellwood, and N. Brown, "Exploring deep recurrent models with reinforcement learning for molecule design," in Proceedings of the 6th International Conference on Learning Representations (ICLR), 2018.
A. Bojchevski, O. Shchur, D. Zügner, and S. Günnemann, "NetGAN: Generating graphs via random walks," in Proceedings of the 35th International Conference on Machine Learning (ICML), 2018, pp. 610-619.
D. Dou, L. Sun, H. Peng, and P. S. Yu, "Robust spammer detection by Nash reinforcement learning," in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2020, pp. 924-933.
A. Ching, S. Zhu, and X. Li, "Modeling and generation of gene regulatory networks with random graph models," in Proceedings of the 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 2019, pp. 584-589.
H. Wang, N. Wang, and D.-Y. Yeung, "Collaborative deep learning for recommender systems," in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2015, pp. 1235-1244.
A. Tsitsulin, D. Mottin, P. Karras, A. Bronstein, and E. Müller, "NetLSD: Hearing the shape of a graph," in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2018, pp. 2347-2356.
M. Zitnik, M. Agrawal, and J. Leskovec, "Modeling polypharmacy side effects with graph convolutional networks," Bioinformatics, vol. 34, no. 13, pp. i457-i466, 2018.
Y. Sun, S. Wang, X. Li, S. Ding, Y. Xia, and Y. Xia, "Adversarial attacks and defenses on graphs: A survey and empirical study," arXiv preprint arXiv:2003.00653, 2020.
B. Yang, W.-t. Yih, X. He, J. Gao, and L. Deng, "Embedding entities and relations for learning and inference in knowledge bases," in Proceedings of the 3rd International Conference on Learning Representations (ICLR), 2015.
Y. Rong, W. Huang, T. Xu, and J. Huang, "DropEdge: Towards deep graph convolutional networks on node classification," in Proceedings of the 8th International Conference on Learning Representations (ICLR), 2020.
