Comments on: The Dantzig selector: Statistical estimation when p is much larger than n http://terrytao.wordpress.com/2008/03/22/the-dantzig-selector-statistical-estimation-when-p-is-much-larger-than-n/ Updates on my research and expository papers, discussion of open problems, and other maths-related topics. By Terence Tao Wed, 17 Mar 2010 19:34:11 +0000 http://wordpress.com/ hourly 1 By: Terence Tao http://terrytao.wordpress.com/2008/03/22/the-dantzig-selector-statistical-estimation-when-p-is-much-larger-than-n/#comment-30761 Terence Tao Fri, 04 Jul 2008 15:51:19 +0000 http://terrytao.wordpress.com/?p=290#comment-30761 Dear anonymous: thanks for the correction! Dear anonymous: thanks for the correction!

]]> By: Anonymous http://terrytao.wordpress.com/2008/03/22/the-dantzig-selector-statistical-estimation-when-p-is-much-larger-than-n/#comment-30760 Anonymous Fri, 04 Jul 2008 14:24:38 +0000 http://terrytao.wordpress.com/?p=290#comment-30760 "The journal issue (Vol. 35, No. 6, 1997)" you mean 2007, of course. “The journal issue (Vol. 35, No. 6, 1997)” you mean 2007, of course.

]]> By: David Molnar http://terrytao.wordpress.com/2008/03/22/the-dantzig-selector-statistical-estimation-when-p-is-much-larger-than-n/#comment-29586 David Molnar Wed, 07 May 2008 17:56:23 +0000 http://terrytao.wordpress.com/?p=290#comment-29586 Gareth James also has a more recent paper with P. Radchenko on "A Generalized Dantzig Selector with Shrinkage Tuning" listed on his web page. He also includes code for the method in R, in case anyone wants to try it: http://www-rcf.usc.edu/~gareth/research/gdscode Gareth James also has a more recent paper with P. Radchenko on “A Generalized Dantzig Selector with Shrinkage Tuning” listed on his web page. He also includes code for the method in R, in case anyone wants to try it:

http://www-rcf.usc.edu/~gareth/research/gdscode

]]> By: Terence Tao http://terrytao.wordpress.com/2008/03/22/the-dantzig-selector-statistical-estimation-when-p-is-much-larger-than-n/#comment-27757 Terence Tao Sun, 30 Mar 2008 02:00:32 +0000 http://terrytao.wordpress.com/?p=290#comment-27757 Thanks for the reference, which I was unaware of! It now seems, in view of recent literature, that the Dantzig selector and LASSO selector are in fact extremely close cousins of each other in many respects. Thanks for the reference, which I was unaware of! It now seems, in view of recent literature, that the Dantzig selector and LASSO selector are in fact extremely close cousins of each other in many respects.

]]> By: Zhou Fang http://terrytao.wordpress.com/2008/03/22/the-dantzig-selector-statistical-estimation-when-p-is-much-larger-than-n/#comment-27750 Zhou Fang Sat, 29 Mar 2008 22:05:52 +0000 http://terrytao.wordpress.com/?p=290#comment-27750 Are you aware of the work by Gareth James on his 'DASSO' algorithm? It's a LARS modification, so substantially faster for full-solution path calculation. (e.g . for CV) Though it's somewhat slower than lasso/lars (since the Dantzig path seems to have more break points), it looks pretty close to optimal. Are you aware of the work by Gareth James on his ‘DASSO’ algorithm? It’s a LARS modification, so substantially faster for full-solution path calculation. (e.g . for CV) Though it’s somewhat slower than lasso/lars (since the Dantzig path seems to have more break points), it looks pretty close to optimal.

]]> By: Terence Tao http://terrytao.wordpress.com/2008/03/22/the-dantzig-selector-statistical-estimation-when-p-is-much-larger-than-n/#comment-27599 Terence Tao Tue, 25 Mar 2008 00:29:07 +0000 http://terrytao.wordpress.com/?p=290#comment-27599 Thanks for the correction! Thanks for the correction!

]]> By: Gabriel Peyré http://terrytao.wordpress.com/2008/03/22/the-dantzig-selector-statistical-estimation-when-p-is-much-larger-than-n/#comment-27595 Gabriel Peyré Tue, 25 Mar 2008 00:01:31 +0000 http://terrytao.wordpress.com/?p=290#comment-27595 Small typo : "Hard thresholding" should be replaced by "Soft thresholding". Small typo : “Hard thresholding” should be replaced by “Soft thresholding”.

]]> By: John Sidles http://terrytao.wordpress.com/2008/03/22/the-dantzig-selector-statistical-estimation-when-p-is-much-larger-than-n/#comment-27573 John Sidles Mon, 24 Mar 2008 14:21:40 +0000 http://terrytao.wordpress.com/?p=290#comment-27573 Many of the topics on the (wonder-full) weblog -- like parameter estimation, compressive sampling, and even Ricci flow -- share the common theme of order reduction (the reduction or approximation of complex systems as simpler systems). That is why this is one of my favorite blogs ... order reduction is the essence of engineering. It can happen, however, the underlying mathematical state-space seems either too restrictive (the linear framework of compressive sampling) or too general (the coordinate-free manifolds of Ricci flow). In practice, the state-spaces that we engineers *really* like tend to be the the nonlinear yet computationally tractable state-spaces of algebraic geometry. For us, these state-spaces are "the porridge that is not too hot and not too cold." Speaking only for myself, it is quite challenging to develop an appreciation for what is known, and what is not yet known, in this mathematical area. That is why posts in this area are always *very* welcome. And, thanks to all who contribute to this fine weblog. Many of the topics on the (wonder-full) weblog — like parameter estimation, compressive sampling, and even Ricci flow — share the common theme of order reduction (the reduction or approximation of complex systems as simpler systems). That is why this is one of my favorite blogs … order reduction is the essence of engineering.

It can happen, however, the underlying mathematical state-space seems either too restrictive (the linear framework of compressive sampling) or too general (the coordinate-free manifolds of Ricci flow).

In practice, the state-spaces that we engineers *really* like tend to be the the nonlinear yet computationally tractable state-spaces of algebraic geometry. For us, these state-spaces are “the porridge that is not too hot and not too cold.”

Speaking only for myself, it is quite challenging to develop an appreciation for what is known, and what is not yet known, in this mathematical area. That is why posts in this area are always *very* welcome.

And, thanks to all who contribute to this fine weblog.

]]> By: Terence Tao http://terrytao.wordpress.com/2008/03/22/the-dantzig-selector-statistical-estimation-when-p-is-much-larger-than-n/#comment-27529 Terence Tao Sat, 22 Mar 2008 23:03:15 +0000 http://terrytao.wordpress.com/?p=290#comment-27529 Oops... these articles should be available within about 24 hours (Sun 8pm EST, to be precise). I guess I posted too soon... On the other hand, the articles are available from Project Euclid (if your institution has a subscription), at http://projecteuclid.org/DPubS?service=UI&version=1.0&verb=Display&handle=euclid.aos/1201012955 Oops… these articles should be available within about 24 hours (Sun 8pm EST, to be precise). I guess I posted too soon…

On the other hand, the articles are available from Project Euclid (if your institution has a subscription), at

http://projecteuclid.org/DPubS?service=UI&version=1.0&verb=Display&handle=euclid.aos/1201012955

]]> By: Jed Harris http://terrytao.wordpress.com/2008/03/22/the-dantzig-selector-statistical-estimation-when-p-is-much-larger-than-n/#comment-27528 Jed Harris Sat, 22 Mar 2008 22:56:23 +0000 http://terrytao.wordpress.com/?p=290#comment-27528 I'd like to read the articles you reference at arXiv, but all the commentary and your own response are embargoed. I don't know if there is anything you can do about that. I’d like to read the articles you reference at arXiv, but all the commentary and your own response are embargoed. I don’t know if there is anything you can do about that.

]]>