Probably certifiably correct algorithms

This post is based on two papers (one and two). The task is to quickly solve typical instances of a given problem, and to quickly produce a certificate of that solution. Generally, problems of interest are NP-hard, and so we consider a random distribution on problem instances with the philosophy that real-world instances might mimic this distribution. In my community, it is common to consider NP-hard optimization problems:

minimize f(x) subject to x\in S.     (1)

In some cases, f is convex but S is not, and so one might relax accordingly:

minimize f(x) subject to x\in T,     (2)

where T\supseteq S is some convex set. If the minimizer of (2) happens to be a member of S, then it’s also a minimizer of (1) — when this happens, we say the relaxation is tight. For some problems (and distributions on instances), the relaxation is typically tight, which means that (1) can be typically solved by instead solving (2); for example, this phenomenon occurs in phase retrieval, in community detection, and in geometric clustering. Importantly, strong duality ensures that solving the dual of the convex relaxation provides a certificate of optimality.

Continue reading Probably certifiably correct algorithms

Frames and Combinatorial & Algebraic Geometry

This last week, I attended a workshop at Universitat Bremen on Frames and Combinatorial & Algebraic Geometry. The organizers (Emily King and Eva Maria Feichtner) did a phenomenal job, and I was impressed by the quality of the talks (see this page for the slides). Below are a few of the things that we discussed throughout the workshop:

— Strohmer’s Problem —

At this year’s SampTA conference, Thomas Strohmer posed an interesting frame theory problem during his plenary talk. On the first day of the workshop, Ole Christensen suggested that we study the problem, and it ended up being one of our main focuses. Here’s a technical statement of the problem sans frame-theoretic jargon:

Strohmer’s Problem. Let A be a known n\times d matrix, let D be an unknown n\times n diagonal matrix, and let X be an unknown d\times m matrix. Under what conditions does y=DAX uniquely determine D and/or X?

This problem is motivated by the real-world problem of self-calibration (see for example this blog entry). In particular, the columns of X represent unknown test signals, whereas D represents unknown parameters of a linear sensor DA. The goal of self-calibration is to calibrate your sensor without requiring knowledge of the input (this is particularly important for applications like deep-space imaging).

Continue reading Frames and Combinatorial & Algebraic Geometry

The 4M-4 Conjecture is False!

Congratulations to Cynthia Vinzant for disproving the 4M-4 conjecture! The main result of her 4-page paper is the existence of a 4\times 11 matrix \Phi such that x\mapsto |\Phi x|^2 is injective modulo a global phase factor (indeed, 11<12=4(4)-4). This is not Cynthia’s first contribution to this problem—her recent paper with Conca, Edidin and Hering proves that the conjecture holds for infinitely many M.

cvinzant2

I wanted to briefly highlight the main idea behind this paper: She provides an algorithm that, on input of a 4\times 11 matrix \Phi with complex rational entries, either outputs “not known to be injective” or outputs “injective” along with a certificate of injectivity. The algorithm is fundamentally based on the following characterization of injectivity:

Continue reading The 4M-4 Conjecture is False!

Sparse Representations, Numerical Linear Algebra, and Optimization Workshop

A couple of weeks ago, I attended the “Sparse Representations, Numerical Linear Algebra, and Optimization Workshop.” It was my first time at Banff, and I was thoroughly impressed by the weather, the facility, and the workshop organization. A few of the talks were recorded and are available here. Check out this good-looking group of participants:

groupphoto.sized

I wanted to briefly outline some of the problems that were identified throughout the workshop.

Continue reading Sparse Representations, Numerical Linear Algebra, and Optimization Workshop

Compressed sensing: Variations on a theme

A couple of months ago, I attended a workshop at Oberwolfach (my first!) called “Mathematical Physics meets Sparse Recovery.” I had a great time. I was asked to give the first talk of the week to get everyone on the same page with respect to sparse recovery. Here are the slides from my talk. What follows is an extended abstract (I added hyperlinks throughout for easy navigation):

Compressed sensing has been an exciting subject of research over the last decade, and the purpose of this talk was to provide a brief overview of the subject. First, we considered certain related topics (namely image compression and denoising) which led up to the rise of compressed sensing. In particular, wavelets provide a useful model for images, as natural images tend to be approximated by linear combinations of particularly few wavelets. This sparsity model has enabled JPEG2000 to provide particularly efficient image compression with negligible distortion. Additionally, this model has been leveraged to remove random noise from natural images.

Considering natural images enjoy such a useful model, one may ask whether the model can be leveraged to decrease the number of measurements necessary to completely determine an image. For example, an MRI scan might require up to 2 hours of exposure time, and then the image might be compressed with JPEG2000 after the fact, meaning most of the measurements can be effectively ignored. So is it possible to simply measure the important parts of the image and not waste time in the image acquisition process? This is the main idea underlying compressed sensing, as introduced by Candes, Romberg and Tao and Donoho.

Continue reading Compressed sensing: Variations on a theme

Phase retrieval from coded diffraction patterns

In a previous post, I described a paper I wrote with Afonso and Yutong about how to design O(\log n) masked illuminations that enable efficient phase retrieval of n-dimensional signals for X-ray crystallography and related applications. This masked-illumination methodology was originally proposed in this paper, and our phase retrieval guarantee was based on a recovery method known as polarization. This week, the following paper was posted online, and it gives the first guarantee of this kind for a more popular recovery method called PhaseLift:

Phase retrieval from coded diffraction patterns

Emmanuel J. Candes, Xiaodong Li, Mahdi Soltanolkotabi

In particular, this paper shows that O(\log^4 n) masked illuminations enable PhaseLift recovery, and their result actually holds for a wide assortment of masks. Since this paper is so related to my research, I decided to interview one of the authors (Mahdi Soltanolkotabi). I’ve lightly edited his responses for formatting and hyperlinks:

Continue reading Phase retrieval from coded diffraction patterns

Phase retrieval from very few measurements

Before the AIM meeting last month, I posted a paper on the arXiv that I wrote with Matt Fickus, Aaron Nelson and Yang Wang. This paper provides some very interesting results for phase retrieval in the setting where you wish to use as few intensity measurements as possible. There are really three different types of results in this paper, and they are partitioned into three different sections accordingly.

— 4M-4 injective intensity measurements —

Recall that Bodmann and Hammen provide a construction of 4M-4 vectors in \mathbb{C}^M which yield injective intensity measurements. Such an explicit construction is rather interesting because 4M-4 is conjectured to be the smallest for which this is even possible. From my perspective, any insight into the structure of such ensembles could lead to a deeper understanding of what it takes to be injective in the complex case, and so such constructions are particularly important.

In our paper, we provide a second construction of 4M-4 injective intensity measurements, and our method for proving injectivity was very different from the Bodmann-Hammen approach. Specifically, it is not clear if the Bodmann-Hammen construction has an efficient reconstruction algorithm, whereas for our construction, injectivity is essentially proved by applying a particular reconstruction algorithm that we designed in concert with the ensemble. We used several key ideas in our design, and I’ll briefly discuss them here.

Continue reading Phase retrieval from very few measurements