Stochastic Gradients: Optimization, Simulation, Randomization, and Sensitivity Analysis

Michael C. Fu; Jiaqiao Hu; Katya Scheinberg

doi:10.1080/24725854.2025.2469839

Stochastic Gradients: Optimization, Simulation, Randomization, and Sensitivity Analysis

Michael C. Fu
, Jiaqiao Hu
, Katya Scheinberg

Applied Mathematics and Statistics

Stony Brook University

University of Maryland, College Park
Georgia Institute of Technology

Research output: Contribution to journal › Article › peer-review

2 Scopus citations

Abstract

Big data and high-dimensional optimization problems in operations research (OR) and artificial intelligence (AI) have brought stochastic gradients to the forefront. This article provides a view of research and applications in stochastic gradient estimation from multiple perspectives, as seminal advances have come from diverse and disparate research fields, including operations research/management science (OR/MS), industrial/systems engineering (ISE), optimal/stochastic control, statistics, and more recently from the computer science (CS) AI machine learning (ML) community.

Original language	English
Pages (from-to)	240-256
Number of pages	17
Journal	IISE Transactions
Volume	58
Issue number	2
DOIs	https://doi.org/10.1080/24725854.2025.2469839
State	Published - 2026

Keywords

Stochastic gradients, perturbation analysis, automatic differentiation, likelihood ratio method
optimization
sensitivity analysis
stochastic approximation, stochastic optimization, simulation optimization
stochastic gradient descent

Access to Document

10.1080/24725854.2025.2469839

Cite this

@article{58b53e6d5b1c4a9c8aa6ea573fc06aeb,

title = "Stochastic Gradients: Optimization, Simulation, Randomization, and Sensitivity Analysis",

abstract = "Big data and high-dimensional optimization problems in operations research (OR) and artificial intelligence (AI) have brought stochastic gradients to the forefront. This article provides a view of research and applications in stochastic gradient estimation from multiple perspectives, as seminal advances have come from diverse and disparate research fields, including operations research/management science (OR/MS), industrial/systems engineering (ISE), optimal/stochastic control, statistics, and more recently from the computer science (CS) AI machine learning (ML) community.",

keywords = "Stochastic gradients, perturbation analysis, automatic differentiation, likelihood ratio method, optimization, sensitivity analysis, stochastic approximation, stochastic optimization, simulation optimization, stochastic gradient descent",

author = "Fu, \{Michael C.\} and Jiaqiao Hu and Katya Scheinberg",

note = "Publisher Copyright: {\textcopyright} Copyright {\textcopyright} 2025 {\textquotedblleft}IISE{\textquotedblright}.",

year = "2026",

doi = "10.1080/24725854.2025.2469839",

language = "English",

volume = "58",

pages = "240--256",

journal = "IISE Transactions",

issn = "2472-5854",

publisher = "Taylor and Francis Ltd.",

number = "2",

}

TY - JOUR

T1 - Stochastic Gradients

T2 - Optimization, Simulation, Randomization, and Sensitivity Analysis

AU - Fu, Michael C.

AU - Hu, Jiaqiao

AU - Scheinberg, Katya

PY - 2026

Y1 - 2026

N2 - Big data and high-dimensional optimization problems in operations research (OR) and artificial intelligence (AI) have brought stochastic gradients to the forefront. This article provides a view of research and applications in stochastic gradient estimation from multiple perspectives, as seminal advances have come from diverse and disparate research fields, including operations research/management science (OR/MS), industrial/systems engineering (ISE), optimal/stochastic control, statistics, and more recently from the computer science (CS) AI machine learning (ML) community.

AB - Big data and high-dimensional optimization problems in operations research (OR) and artificial intelligence (AI) have brought stochastic gradients to the forefront. This article provides a view of research and applications in stochastic gradient estimation from multiple perspectives, as seminal advances have come from diverse and disparate research fields, including operations research/management science (OR/MS), industrial/systems engineering (ISE), optimal/stochastic control, statistics, and more recently from the computer science (CS) AI machine learning (ML) community.

KW - Stochastic gradients, perturbation analysis, automatic differentiation, likelihood ratio method

KW - optimization

KW - sensitivity analysis

KW - stochastic approximation, stochastic optimization, simulation optimization

KW - stochastic gradient descent

UR - https://www.scopus.com/pages/publications/105002980693

U2 - 10.1080/24725854.2025.2469839

DO - 10.1080/24725854.2025.2469839

M3 - Article

SN - 2472-5854

VL - 58

SP - 240

EP - 256

JO - IISE Transactions

JF - IISE Transactions

IS - 2

ER -

Stochastic Gradients: Optimization, Simulation, Randomization, and Sensitivity Analysis

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this