[PDF] Primal Dual Proximal Optimization Algorithms With Bregman Divergences eBook

Author : Xin Jiang
Publisher :
Page : 0 pages
File Size : 21,65 MB
Release : 2022
Category :
ISBN :

GET BOOK

Proximal methods are an important class of algorithms for solving nonsmooth, constrained, large-scale or distributed optimization problems. Because of their flexibility and scalability, they are widely used in current applications in engineering, machine learning, and data science. The key idea of proximal algorithms is the decomposition of a large-scale optimization problem into several smaller, simpler problems, in which the basic operation is the evaluation of the proximal operator of a function. The proximal operator minimizes the function regularized by a squared Euclidean distance, and it generalizes the Euclidean projection onto a closed convex set. Since the cost of the evaluation of proximal operators often dominates the per-iteration complexity in a proximal algorithm, efficient evaluation of proximal operators is critical. To this end, generalized Bregman proximal operators based on non-Euclidean distances have been proposed and incorporated in many algorithms and applications. In the first part of this dissertation, we present primal-dual proximal splitting methods for convex optimization, in which generalized Bregman distances are used to define the primal and dual update steps. The proposed algorithms can be viewed as Bregman extensions of many well- known proximal methods. For these algorithms, we analyze the theoretical convergence and develop techniques to improve practical implementation. In the second part of the dissertation, we apply the Bregman proximal splitting algorithms to the centering problem in large-scale semidefinite programming with sparse coefficient matrices. The logarithmic barrier function for the cone of positive semidefinite completable sparse matrices is used as the distance-generating kernel. For this distance, the complexity of evaluating the Bregman proximal operator is shown to be roughly proportional to the cost of a sparse Cholesky factorization. This is much cheaper than the standard proximal operator with Euclidean distances, which requires an eigenvalue decomposition. Therefore, the proposed Bregman proximal algorithms can handle sparse matrix constraints with sizes that are orders of magnitude larger than the problems solved by standard interior-point methods and proximal methods.

Author : Ke Chen
Publisher : Springer Nature
Page : 1981 pages
File Size : 12,93 MB
Release : 2023-02-24
Category : Mathematics
ISBN : 3030986616

GET BOOK

This handbook gathers together the state of the art on mathematical models and algorithms for imaging and vision. Its emphasis lies on rigorous mathematical methods, which represent the optimal solutions to a class of imaging and vision problems, and on effective algorithms, which are necessary for the methods to be translated to practical use in various applications. Viewing discrete images as data sampled from functional surfaces enables the use of advanced tools from calculus, functions and calculus of variations, and nonlinear optimization, and provides the basis of high-resolution imaging through geometry and variational models. Besides, optimization naturally connects traditional model-driven approaches to the emerging data-driven approaches of machine and deep learning. No other framework can provide comparable accuracy and precision to imaging and vision. Written by leading researchers in imaging and vision, the chapters in this handbook all start with gentle introductions, which make this work accessible to graduate students. For newcomers to the field, the book provides a comprehensive and fast-track introduction to the content, to save time and get on with tackling new and emerging challenges. For researchers, exposure to the state of the art of research works leads to an overall view of the entire field so as to guide new research directions and avoid pitfalls in moving the field forward and looking into the next decades of imaging and information services. This work can greatly benefit graduate students, researchers, and practitioners in imaging and vision; applied mathematicians; medical imagers; engineers; and computer scientists.

Author : Antonio Silveti Falls
Publisher :
Page : 0 pages
File Size : 35,29 MB
Release : 2021
Category :
ISBN :

GET BOOK

In this work we develop and examine two novel first-order splitting algorithms for solving large-scale composite optimization problems in infinite-dimensional spaces. Such problems are ubiquitous in many areas of science and engineering, particularly in data science and imaging sciences. Our work is focused on relaxing the Lipschitz-smoothness assumptions generally required by first-order splitting algorithms by replacing the Euclidean energy with a Bregman divergence. These developments allow one to solve problems having more exotic geometry than that of the usual Euclidean setting. One algorithm is hybridization of the conditional gradient algorithm, making use of a linear minimization oracle at each iteration, with an augmented Lagrangian algorithm, allowing for affine constraints. The other algorithm is a primal-dual splitting algorithm incorporating Bregman divergences for computing the associated proximal operators. For both of these algorithms, our analysis shows convergence of the Lagrangian values, subsequential weak convergence of the iterates to solutions, and rates of convergence. In addition to these novel deterministic algorithms, we introduce and study also the stochastic extensions of these algorithms through a perturbation perspective. Our results in this part include almost sure convergence results for all the same quantities as in the deterministic setting, with rates as well. Finally, we tackle new problems that are only accessible through the relaxed assumptions our algorithms allow. We demonstrate numerical efficiency and verify our theoretical results on problems like low rank, sparse matrix completion, inverse problems on the simplex, and entropically regularized Wasserstein inverse problems.

Author : Neal Parikh
Publisher : Now Pub
Page : 130 pages
File Size : 16,58 MB
Release : 2013-11
Category : Mathematics
ISBN : 9781601987167

GET BOOK

Proximal Algorithms discusses proximal operators and proximal algorithms, and illustrates their applicability to standard and distributed convex optimization in general and many applications of recent interest in particular. Much like Newton's method is a standard tool for solving unconstrained smooth optimization problems of modest size, proximal algorithms can be viewed as an analogous tool for nonsmooth, constrained, large-scale, or distributed versions of these problems. They are very generally applicable, but are especially well-suited to problems of substantial recent interest involving large or high-dimensional datasets. Proximal methods sit at a higher level of abstraction than classical algorithms like Newton's method: the base operation is evaluating the proximal operator of a function, which itself involves solving a small convex optimization problem. These subproblems, which generalize the problem of projecting a point onto a convex set, often admit closed-form solutions or can be solved very quickly with standard or simple specialized methods. Proximal Algorithms discusses different interpretations of proximal operators and algorithms, looks at their connections to many other topics in optimization and applied mathematics, surveys some popular algorithms, and provides a large number of examples of proximal operators that commonly arise in practice.

Author : Xue-Cheng Tai
Publisher : Springer
Page : 516 pages
File Size : 38,59 MB
Release : 2015-01-07
Category : Computers
ISBN : 3319146122

GET BOOK

This volume constitutes the refereed proceedings of the 10th International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition, EMMCVPR 2015, held in Hong Kong, China, in January 2015. The 36 revised full papers were carefully reviewed and selected from 45 submissions. The papers are organized in topical sections on discrete and continuous optimization; image restoration and inpainting; segmentation; PDE and variational methods; motion, tracking and multiview reconstruction; statistical methods and learning; and medical image analysis.

Author : Amir Beck
Publisher : SIAM
Page : 476 pages
File Size : 21,12 MB
Release : 2017-10-02
Category : Mathematics
ISBN : 1611974984

GET BOOK

The primary goal of this book is to provide a self-contained, comprehensive study of the main ?rst-order methods that are frequently used in solving large-scale problems. First-order methods exploit information on values and gradients/subgradients (but not Hessians) of the functions composing the model under consideration. With the increase in the number of applications that can be modeled as large or even huge-scale optimization problems, there has been a revived interest in using simple methods that require low iteration cost as well as low memory storage. The author has gathered, reorganized, and synthesized (in a unified manner) many results that are currently scattered throughout the literature, many of which cannot be typically found in optimization books. First-Order Methods in Optimization offers comprehensive study of first-order methods with the theoretical foundations; provides plentiful examples and illustrations; emphasizes rates of convergence and complexity analysis of the main first-order methods used to solve large-scale problems; and covers both variables and functional decomposition methods.

Author : Milojica Jaćimović
Publisher : Springer Nature
Page : 515 pages
File Size : 33,61 MB
Release : 2020-01-08
Category : Computers
ISBN : 3030386031

GET BOOK

This book constitutes the refereed proceedings of the 10th International Conference on Optimization and Applications, OPTIMA 2019, held in Petrovac, Montenegro, in September-October 2019. The 35 revised full papers presented were carefully reviewed and selected from 117 submissions. The papers cover such topics as optimization, operations research, optimal control, game theory, and their numerous applications in practical problems of operations research, data analysis, and software development.

Author : Sébastien Bubeck
Publisher : Foundations and Trends (R) in Machine Learning
Page : 142 pages
File Size : 21,32 MB
Release : 2015-11-12
Category : Convex domains
ISBN : 9781601988607

GET BOOK

This monograph presents the main complexity theorems in convex optimization and their corresponding algorithms. It begins with the fundamental theory of black-box optimization and proceeds to guide the reader through recent advances in structural optimization and stochastic optimization. The presentation of black-box optimization, strongly influenced by the seminal book by Nesterov, includes the analysis of cutting plane methods, as well as (accelerated) gradient descent schemes. Special attention is also given to non-Euclidean settings (relevant algorithms include Frank-Wolfe, mirror descent, and dual averaging), and discussing their relevance in machine learning. The text provides a gentle introduction to structural optimization with FISTA (to optimize a sum of a smooth and a simple non-smooth term), saddle-point mirror prox (Nemirovski's alternative to Nesterov's smoothing), and a concise description of interior point methods. In stochastic optimization it discusses stochastic gradient descent, mini-batches, random coordinate descent, and sublinear algorithms. It also briefly touches upon convex relaxation of combinatorial problems and the use of randomness to round solutions, as well as random walks based methods.