| AMiner

课程简介

近年随着数据复杂度的不断提高，经典机器学习算法已经很难满足实际需求，适应于不同复杂环境、不同领域的学习机制和学习方法不断涌现，针对复杂数据对象、复杂任务和模拟人脑思维过程的机器学习理论逐渐成为数据挖掘和智能信息处理的主流。掌握并熟练应用这些新型机器学习理论和相关算法成为计算机学科研究生必备的技能之一。

本课程基于经典机器学习算法，重点介绍近年流行的机器学习理论和算法，并通过提供的统一实验验证平台进行实践，可以使学生不仅能够熟练掌握先进的机器学习理论知识，并能学习将其应用于新一代信息智能处理的系统方法。

课程大纲（syllabus）

课程主要介绍机器学习的理论和相关算法；课程内容涵盖基础篇：经典机器学习方法的回顾、概率图生成模型（Generative learning）和概率图判别模型（Discriminative learning），进阶篇：介绍最近流行的深度学习、表示学习、增强学习、自动机器学习以及对抗学习，高级篇：介绍当前热点的预训练、自学习以及认知图谱等。具体内容包括：

====================================================基础篇==================================================

名称	一句话	术语表	参考文献	作业或备注
机器学习简介 [课件下载] 建议课时：1	为什么学机器学习	[术语表 ]	From System 1 deep learning to system 2 deep learning	作业1： challenge (Coronavirus infection prediction)
有监督学习 [课件1下载] [课件2下载] 建议课时：5	定义、正则化、最大化边界、感知机、决策树、PAC学习理论、SVM、DNN等基础算法	[术语表 ]	A Tutorial on Support Vector Machines for Pattern Recognition, An Equivalence between the Lasso and Support Vector Machines, The Entire Regularization Path for the Support Vector Machine, Large Linear Classiﬁcation When Data Cannot Fit In Memory, Making large-scale svm learning practical, 1-norm Support Vector Machines
无监督学习 [课件1下载] [课件2下载] 建议课时：8	基础聚类算法、降维、PCA/SVD、谱聚类、Mixture模型、EM算法、pLSI、LDA（吉布斯采样、变分）	[术语表 ]	Reducing the sampling complexity of topic models, ParallelInferenceforLatentDirichletAllocationon GraphicsProcessingUnits, LightLDA: Big Topic Models on Modest Compute Clusters , Efﬁcient Methods for Topic Model Inference on Streaming Document Collections	作业2： Gibbs sampling for LDA and application to text.
矩阵分解与推荐 [课件下载] 建议课时：3	以推荐为例介绍矩阵分解及其应用。SVD、Probabilistic Interpretation、Sparse Coding、Factorization Machines、Neural Models	[术语表 ]	The BellKor Solution to the Netﬂix Grand Prize, Recommender, Probabilistic interpretation of population codes	Project （二选一） Prj1: to design new ML algs. to real problems Prj2: to solve novel problems using ML algs.
非参贝叶斯模型 [课件下载] 建议课时：1.5	非参贝叶斯模型、Gaussian Process、Dirichlet Process	[术语表 ]	Learning Time-Intensity Profiles of Human Activity using Non-Parametric Bayesian Models, Multi-HDP: a non parametric Bayesian model for tensor factorization
概率图模型 [课件下载] 建议课时：3	HMM、MEMM、CRF、Factor Graph以及这些概率图模型的应用案例，最后会介绍概率图模型和深度学习模型结合的最新进展	[术语表 ]	Semi-Supervised Aﬃnity Propagation with Instance-Level Constraints, Factor graphs and the sum-product algorithm, Markov Random Fields in Statistics, An Introduction to Conditional Random Fields, Graphical Models, Exponential Families, and Variational Inference, TheHammersley-Clifford TheoremanditsImpacton ModernStatistics
图模型近似推断算法 [课件下载] 建议课时：1	MCMC、Mean Field	[术语表 ]	Variational Mean Field Variational Mean Field for Graphical Models, Correctness of belief propagation in Gaussian graphical models of arbitrary topology, Understanding Belief Propagationandits Generalizations, Tutorial on variational approximation methods
集成学习 [课件下载] 建议课时：2	主要讲解Boosting、Bagging、XGboost等知识，并结合Toutiao Byte Cup讲解有关应用	[术语表 ]	Bagging and Boosting, Can we make dumb learners smart?, Boosting, Ensemble methods. Baggingand Boosting	作业3: 理论推导

====================================================进阶篇==================================================

名称	一句话	术语表	参考文献	作业或备注
卷积神经网络 (CNN) [课件下载] 建议课时：2	Basic CNN, Network in Network, ResNet, ResNext, Inception, VGG, DenseNet, SeNet, etc.	[术语表 ]	A Convolutional Neural Network for Modelling Sentences, Extracting and composing robust features with denoising autoencoders, Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis , Contractive Auto-Encoders: Explicit Invariance During Feature Extraction, Gradient-Based Learning Applied to Document Recognition, Deep Residual Learning for Image Recognition, Face recognition: a convolutional neural-network approach
循环神经网络 (RNN) [课件下载] 建议课时：2	Bidirectional RNNs, Encoder-Decoder Architectures, Long-Term Dependencies, Pre-training of sequence model	[术语表 ]	Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation, Structural-RNN: Deep Learning on Spatio-Temporal Graphs, Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
自编码 (AutoEncoder) [课件下载] 建议课时：2	Sparse AE, Denoising AE, Contractive AE, Stacked AE, Convolutional AE, Variational AE	[术语表 ]	A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines, Discriminative Recurrent Sparse Auto-Encoders, Generalized Denoising Auto-Encoders as Generative Models, Contractive Auto-Encoders: Explicit Invariance During Feature Extraction, Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimization, Variational Autoencoder for Deep Learning of Images, Labels and Captions
表示学习 [课件下载] 建议课时：3	介绍表示学习的基本内容以及NLP中表示学习的方法Neural Language Model, Word2Vec, Doc2Vec,KnowledgeBase2Vec	[术语表 ]	A Convolutional Neural Network for Modelling Sentences, A Deep Architecture for Matching Short Texts, Zero-Shot Learning Through Cross-Modal Transfer, word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method, Reasoning With Neural Tensor Networks for Knowledge Base Completion
网络表示学习与图神经网络 [课件下载] 建议课时：3	介绍网络的特点以及如何在网络中进行机器学习Node Embeddings,Graph Neural Network	[术语表 ]	GraRep: Learning Graph Representations with Global Structural Information, DeepWalk: Online Learning of Social Representations, Learning multi-faceted representations of individuals from heterogeneous evidence using neural networks, Diffusion-Convolutional Neural Networks, Semi-supervised Convolutional Neural Networks for Text Categorization via Region Embedding, Network representation learning with rich text information, LINE: Large-scale Information Network Embedding
自动机器学习 (AutoML) [课件下载] 建议课时：1	Hyper-parameter Optimization, Neural Architecture Search, Meta-Learning	[术语表 ]	DARTS: Differentiable Architecture Search, Once for All: Train One Network and Specialize it for Efficient Deployment, MnasNet: Platform-Aware Neural Architecture Search for Mobile, Efficient Neural Architecture Search via Parameter Sharing	作业4： Make a Survey on a Topic
对抗学习 [课件下载] 建议课时：1	White-box Attack, Black-box Attack, Graph Attack	[术语表 ]	Adversarial Learning of Structure-Aware Fully Convolutional Networks for Landmark Localization, Semantic Adversarial Attacks: Parametric Transformations That Fool Deep Classifiers, Towards Deep Learning Models Resistant to Adversarial Attacks
生成对抗网络 (GAN) [课件下载] 建议课时：2	GAN及其变型	[术语表 ]	Generative Adversarial Networks, Deep learning, GP-GAN: Towards Realistic High-Resolution Image Blending, The relativistic discriminator: a key element missing from standard GAN
在线学习 (Online Learning) [课件下载] 建议课时：2	Bandit Strategy (UCB, Thompson Sampling) contextual bandit 以及一些最近的算法	[术语表 ]	Nonparametric Bandits with Covariates， Analysis of Thompson Sampling for the multi-armed bandit problem， Thompson Sampling for Contextual Bandits with Linear Payoffs ， Thompson Sampling for Complex Online Problems
增强学习 (RL) [课件下载] 建议课时：2	基本概念，马尔科夫决策过程（MDPs), 求解MDPs的迭代算法（Dynamic Programming, Monte-Carlo evaluation, Temporal Difference learning), value-based RL, Policy-based RL, Multi-Agent Reinforcement Learning	[术语表 ]	Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables, Using a Logarithmic Mapping to Enable Lower Discount Factors in Reinforcement Learning, Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models, Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansion

====================================================高级篇==================================================

名称	一句话	术语表	参考文献
预训练 (Pre-Training) [课件下载] 建议课时：2	BERT, BERTology, XLNet, ERNIE, ALBERT, ELECTRA	[术语表 ]	ALBERT: A Lite BERT for Self-supervised Learning of Language Representations, Incorporating BERT into Neural Machine Translation, XLNet: Generalized Autoregressive Pretraining for Language Understanding
Reasoning [课件下载] 建议课时：2	认知推理相关原理算法介绍Knowledge Graph, Cognitive Reasoning, Logic Expression	[术语表 ]	Cognitive Graph for Multi-Hop Reading Comprehension at Scale, A cognition graph approach for insights generation from event sequences, Neural-Symbolic Cognitive Reasoning
图神经网络GNN [课件下载] 建议课时：2	关于图神经网络的一些最新工作	[术语表 ]	Capsule Graph Neural Network, Graph Neural Network Based Coarse-Grained Mapping Prediction, Self-supervised edge features for improved Graph Neural Network training, Extensions of recurrent neural network language model

名称

一句话

术语表

参考文献

作业或备注

预训练 (Pre-Training)
[课件下载]

建议课时：2

BERT, BERTology, XLNet, ERNIE, ALBERT, ELECTRA

[术语表 ]

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations, Incorporating BERT into Neural Machine Translation, XLNet: Generalized Autoregressive Pretraining for Language Understanding

Reasoning
[课件下载]

建议课时：2

认知推理相关原理算法介绍Knowledge Graph, Cognitive Reasoning, Logic Expression

[术语表 ]

Cognitive Graph for Multi-Hop Reading Comprehension at Scale, A cognition graph approach for insights generation from event sequences, Neural-Symbolic Cognitive Reasoning

图神经网络GNN
[课件下载]

建议课时：2

关于图神经网络的一些最新工作

[术语表 ]

Capsule Graph Neural Network, Graph Neural Network Based Coarse-Grained Mapping Prediction, Self-supervised edge features for improved Graph Neural Network training, Extensions of recurrent neural network language model

特邀报告

2020年5月13日

Jian Tang is an assistant professor at Mila (Quebec AI Institute founded by Yoshua Bengio) and HEC Montreal since December, 2017. His research interests focus on deep graph representation learning with a variety of applications including knowledge graphs, drug discovery and recommender systems.

Relational Reasoning with Graph Neural Networks

Graphs are very general and flexible data structures to capture the relationships between different objects, which are ubiquitous in the real world. Meanwhile, graphs can be an important bridge between System I and II reasoning by capturing the relationships between high-level semantic variables. Many of these problems are essentially conducting relational reasoning on graphs. In this talk, I will introduce our recent work on relational reasoning by combining traditional statistical relational learning and recent graph neural networks. If time allows, I will also give an introduction to our work on graph representation learning for drug discovery (including some efforts on COVID-19).

2020年4月29日

李沐，师从机器学习大师 Alex Smola ，现在领导 AWS AI 科学团队，是深度学习框架 mxnet (https://mxnet.io) 和深度学习编译器 tvm (https://tvm.ai) 的主要作者之一。他的团队发布了AutoML工具AutoGluon： https://autogluon.mxnet.io/

Introduction to AutoML

主要包括超参数优化（Hyper-parameter Optimization）和神经网络架构搜索（Neural-network Architecture Search）等自动机器学习方法。

2020年4月29日

李磊，字节跳动人工智能实验室总监、资深科学家，原百度美国深度学习实验室少帅科学家。

Learning Deep Latent Models for Text Sequences

The emergence of new social media platforms calls for fundamentally more effective ways to create information. In this talk, I will present our themed efforts on developing interpretable machine learning models to represent complex data and scalable algorithms that utilize this stream of formalisms to automatically generate text-based information and to build data-efficient machine translation systems. I will describe general formalisms to learn deep latent representations for text and data. The general idea leads to three successful models for: a) learning syntactic and semantic representations from sentences without labeled corpus, b) learning to generate text from table-structured data, and c) learning to translate and compose sentences for two languages simultaneously. These algorithms have been successfully deployed for advertisement design and creation in a large online advertising system, serving a hundred thousand enterprises. Finally, I will demonstrate a writing robot that published over 600,000 articles on social medias covering a variety of subjects including 2018 FIFA World Cup.

2020年4月15日

杨强，微众银行首席人工智能官，香港科技大学讲席教授， AAAI/ACM/CAAI/IEEE/IAPR/AAAS Fellow，《IEEE Transactions on Big Data》和《ACM Transactions on Intelligent Systems and Technology》开创主编。

联邦学习前沿的研究与应用

当下，AI在算法研发方面突飞猛进，但AI离在企业落地实施却有着不小的距离。 AI所面临的一个挑战是标注数据的严重不足，而数据往往分布在不同的机构和个人形成的数据孤岛处。使用和聚合这些数据都会受到用户隐私、商业安全的挑战。在这一讲座中，我将介绍如何利用“联邦学习”（ Federated Learning）来连接数据孤岛的数据，形成合力，以得到更有效的机器学习模型，同时保护用户隐私和商业秘密。我将给大家一个领域概述，同时介绍几个联邦学习的应用。

2020年4月8日

李航，字节跳动科技有限公司人工智能实验室总监，北京大学、南京大学客座教授，IEEE 会士，ACM 杰出科学家，CCF 杰出会员。研究方向包括信息检索，自然语言处理，统计机器学习，及数据挖掘。

Sequence to Sequence Models

Sequence to sequence learning is in some sense the most important deep learning technologies for natural language processing. In this lecture I will introduce the fundamental models of sequence to sequence learning, including the Transformer model. I will also explain its extensions to language representation models such as GPT and BERT.