Keynotes
Jie Tang is a Professor and the Associate Chair of the Department of Computer Science at Tsinghua University. He is a Fellow of the IEEE. His interests include artificial intelligence, data mining, social networks, and machine learning. He served as General Co-Chair of WWW’23, and PC Co-Chair of WWW’21, CIKM’16, WSDM’15, and EiC of IEEE T. on Big Data and AI Open J. He leads the project AMiner.org, an AI-enabled research network analysis system, which has attracted more than 20 million users from 220 countries/regions in the world. He was honored with the SIGKDD Test-of-Time Award, UK Royal Society-Newton Advanced Fellowship Award, NSFC for Distinguished Young Scholar, and KDD’18 Service Award.
WuDao: Pretrain the World
Large-scale pretrained model on web texts have substantially advanced the state of the art in various AI tasks, such as natural language understanding and text generation, and image processing, multimodal modeling. The downstream task performances have also constantly increased in the past few years. In this talk, I will first go through three families: augoregressive models (e.g., GPT), autoencoding models (e.g., BERT), and encoder-decoder models. Then, I will introduce China’s first homegrown super-scale intelligent model system, with the goal of building an ultra-large-scale cognitive-oriented pretraining model to focus on essential problems in general artificial intelligence from a cognitive perspective. In particular, as an example, I will elaborate a novel pretraining framework GLM (General Language Model) to address this challenge. GLM has three major benefits: (1) it performs well on classification, unconditional generation, and conditional generation tasks with one single pretrained model; (2) it outperforms BERT-like models on classification due to improved pretrain-finetune consistency; (3) it naturally handles variable-length blank filling which is crucial for many downstream tasks. Empirically, GLM substantially outperforms BERT on the SuperGLUE natural language understanding benchmark with the same amount of pre-training data.
HAMED ZAMANI is an Assistant Professor in the Manning College of Information and Computer Sciences at the University of Massachusetts Amherst (UMass), where he also serves as the Associate Director of the Center for Intelligent Information Retrieval (CIIR), one of the top academic research labs in Information Retrieval worldwide. Prior to UMass, he was a Researcher at Microsoft. His research focuses on designing and evaluating statistical and machine learning models with applications to (interactive) information access systems, including search engines, recommender systems, and question answering. He is mostly known for his recent work in the areas of neural information retrieval and conversational information seeking. His work has led to over 70 refereed publications in the field, including a few Best Paper and Honorable Mentions, in addition to a number of open-source research tools.
Recent Advancements and Current Challenges in Neural Information Retrieval
Deep learning models for information retrieval (IR), also called Neural IR, have been studied for almost a decade. Neural IR research has observed a few paradigm shifts in the last few years. For example, the use of pre-trained large language models has recently become a popular recipe for achieving state-of-the-art retrieval performance. In this talk, I will first summarize the recent key findings by the Neural IR research community. In more detail, I will highlight recent neural network architectures and optimization methodologies for fundamental IR problems. I will further discuss recent Neural IR methods for standalone retrieval of documents from large collections. I will finally highlight the current research challenges in this area and discuss potential solutions to address these issues.
