TractableStructuredNatural-GradientDescentUsingLocalParameterizationsWuLin1FrankNielsen2MohammadEmtiyazKhan3MarkSchmidt14AbstractFinally,manyrobustorglobaloptimizationtechniquesem-ployq(w)tosmootho...
StochasticSignDescentMethods:NewAlgorithmsandBetterTheoryMherSafaryan1PeterRichtárik12Abstracthencethetrainingdataistypicallysplitandstoredacrossanumberofcomputenodescapableofworkinginparallel.Var...
KernelSteinDiscrepancyDescentAnnaKorba1Pierre-CyrilAubin-Frankowski2SzymonMajewski3PierreAblin4AbstractbySimon-Gabriel(2018),classicaldissimilaritiesincludeAmongdissimilaritiesbetweenprobabilitydis...
DecentralizedRiemannianGradientDescentontheStiefelManifoldShixiangChen1AlfredoGarcia1MingyiHong2ShahinShahrampour1AbstractagentoptimizationproblemWeconsiderdistributednon-convexoptimization1nwherea...
AgnosticLearningofHalfspaceswithGradientDescentviaSoftMarginsSpencerFrei1YuanCao2QuanquanGu2AbstractminimizethesurrogateriskWeanalyzethepropertiesofgradientDescentonF(w):=E(x,y)∼D(ywx).(1)convexsu...
AZeroth-OrderBlockCoordinateDescentAlgorithmforHuge-ScaleBlack-BoxOptimizationHanQinCai1YuchenLou2DanielMckenzie1WotaoYin3Abstractandonlinemarketing(Flaxmanetal.,2005).Lately,algo-rithmsforzeroth-o...
ARiemannianBlockCoordinateDescentMethodforComputingtheProjectionRobustWassersteinDistanceMinhuiHuang1ShiqianMa2LifengLai1Abstractother.TocalculatetheWassersteindistance,oneisrequiredtosolveanoptima...
ANovelSequentialCoresetMethodforGradientDescentAlgorithmsJiaweiHuang1RuominHuang2WenjieLiu1NikolaosM.Freris1HuDing1AbstractoriginalsetP;thatis,wecanreplacePbyP˜whenrunningAwiderangeofoptimizationp...
VarianceReducedCoordinateDescentwithAcceleration:NewMethodWithaSurprisingApplicationtoFinite-SumProblemsFilipHanzely1DmitryKovalev1PeterRichta´rik1Abstractcontrast,ifψisnotseparable,thecorrespond...
Randomextrapolationforprimal-dualcoordinateDescentAhmetAlacaoglu1OlivierFercoq2VolkanCevher1Abstractmization(Shalev-Shwartz&Zhang,2013;Zhang&Xiao,2017),optimizationwithlargenumberofconstraints(Fer-...
OnlinemirrorDescentanddualaveraging:keepingpaceinthedynamiccaseHuangFang1NicholasJ.A.Harvey1VictorS.Portella1MichaelP.Friedlander1Abstractthebenefitofhindsight.LettingTdenotethenumberofdecisions,th...
OntheNoisyGradientDescentthatGeneralizesasSGDJingfengWu1WenqingHu2HaoyiXiong3JunHuan4VladimirBraverman1ZhanxingZhu5Abstractetal.,2017;Keskaretal.,2017).Togainintuitions,onecancomparethegeneralizati...
OnGradientDescentAscentforNonconvex-ConcaveMinimaxProblemsTianyiLin1ChiJin2Michael.I.Jordan3Abstractincludinggenerativeadversarialnetworks(GANs)(Good-fellowetal.,2014),statistics(Xuetal.,2009;Abade...
Multi-TaskLearningwithUserPreferences:GradientDescentwithControlledAscentinParetoOptimizationDebabrataMahapatra1VaibhavRajan2Abstract2019a),naturallanguageprocessing(Liuetal.,2019b)andbioinformatic...
High-dimensionalRobustMeanEstimationviaGradientDescentYuCheng1IliasDiakonikolas2RongGe3MahdiSoltanolkotabi4Abstractasmallconstantfractionofarbitraryoutliers.Westudytheproblemofhigh-dimensionalro-Th...
EfficientlySolvingMDPswithStochasticMirrorDescentYujiaJin1AaronSidford1AbstractanMDPgivenonlyrestrictedaccesstothemodel.Inpar-ticular,weconsidertheproblemofcomputingan-optimalInthispaperwepresentau...
DualMirrorDescentforOnlineAllocationProblemsSantiagoBalseiro12HaihaoLu2VahabMirrokni2Abstracttherequest(Talluri&vanRyzin,2004).Insearchadvertis-ing,eachtimeausermakesasearch,thesearchenginehasWecon...
DoubleTroubleinDoubleDescent:BiasandVariance(s)intheLazyRegimeSte´phaned’Ascoli1MariaRefinetti1GiulioBiroli1FlorentKrzakala1Abstractrecognition(Hintonetal.,2012),andautomatictransla-tion(Sutskeve...
AdaptiveGradientDescentwithoutDescentYuraMalitsky1KonstantinMishchenko2Abstractwheref:Rd→Risadifferentiablefunction.Throughoutthepaperweassumethat(1)hasasolutionandwedenoteWepresentastrikinglysimp...
AccelerationforCompressedGradientDescentinDistributedandFederatedOptimizationZhizeLi1DmitryKovalev1XunQian1PeterRichta´rik1Abstract1.IntroductionDuetothehighcommunicationcostindistributedWiththepr...