Scheduling Hyperparameters to Improve Generalization: From Centralized SGD to Asynchronous SGD

Computer Science

• Stochastic Gradient Descent 100%
• Leaning Parameter 100%
• Learning Rate 20%
• Single Machine 20%
• Unified Framework 10%
• Distributed Environment 10%
• Stochastic Differential 10%
• Automated Machine Learning 10%
• Parameter Server 10%
• Generalization Error 10%
• Centralized Environment 10%

Keyphrases

• Asynchronous Stochastic Gradient Descent 100%
• Distributed SGD 33%
• Distributed Stochastic Gradient Descent 16%
• Asynchronous Iterations 16%
• PAC-Bayesian 16%
• Rigorous Justification 16%
• Momentum Factor 16%
• Quasi-hyperbolic 16%
• Large Learning Rate 16%
• Centralized Environment 16%

Mathematics

• Stochastics 100%
• SHB 30%
• Single Machine 20%
• Bayesian 10%
• Trial and error 10%
• Stochastic Differential Equation 10%
• Distributed Environment 10%