Deep learning: a sensational theorem and its evolutionary implications

 I recently stumbled upon this article in the New York Times: Scientists See Promise in Deep-Learning Programs These sentences initially caught my eye:

"But what is new in recent months is the growing speed and accuracy of deep-learning programs, often called artificial neural networks or just “neural nets” for their resemblance to the neural connections in the brain

'There has been a number of stunning new results with deep-learning methods,” said Yann LeCun, a computer scientist at New York University who did pioneering research in handwriting recognition at Bell Laboratories. “The kind of jump we are seeing in the accuracy of these systems is very rare indeed.' "

Intrigued, I followed the link to Geoffry Hinton's home page. There are links to many papers there, including: A Fast Learning Algorithm for Deep Belief Nets From the summary:

"...we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time..."

Folks, this is an astounding result!  One would imagine that training a neural net with N layers would require adjusting weights of all N layers simultaneously.  If instead, training can be done one layer at a time the computational complexity is reduced by a factor of N.

Furthermore, it is natural to conjecture that all, or almost all, evolutionary processes are in essence greedy algorithms.  Something like this algorithm likely explains how evolution has any chance of creating brains!

The diseases of the Republican Party

I rejoice that Barack Obama has won re-election.  It is terrifying that the Republican Party, as presently constituted, should control the White House.

I have stated many times that I believe that the United States needs two healthy political parties.  There is plenty of room for disagreement about a host of important topics.  This will always be so.

However, the Republican Party is in deep trouble.  It would be better for America, and better for the Republican Party itself, to rid itself of the following diseases:

1.  The Republican Party tolerates denialism.  It somehow feels free to deny scientific fact whenever those facts conflict with the financial interests of its wealthiest members.

2. The Republican Party is based on religious conservatism.  As a militant atheist, I regard any religious basis for political power as revolting opportunism, but basing a political agenda on the religious doctrines of a small minority of Americans is a recipe for permanent minority status.

3. The Republican Party is a party of fiscal and monetary chicken hawks.  Its leaders talk about reducing the deficit, but are, in fact, in favor of budget-busting programs such as the prescriptions drug benefit and the F-35 fighter.  John Mauldin, a prominent financial analyst, eloquently writes about the dangers of large deficits on the one hand, while on the other hand he counsels his rich clients about how to avoid taxes.  How can he not see the contradiction?

4. The Republican Party somehow believes that the wealthiest people in the United States should be exempt from shared sacrifice, and that the rich should be specifically exempted from higher taxes.  This is an absolute failure of political, economic and moral leadership.

5. The Republican Party embraces, rather than repudiates, the contemptible practices of Fox News.  It has become a party of 24/7 propaganda and self deception.  A healthy party would welcome real debate; the present diseased party tolerates the shouting down of all who would dare disagree with it.

To summarize, the Republican Party asks, "What can government do for me?" (less taxes, more contracts, more religious conformity), rather than, "What can government do for all its citizens?".  The Republican Party seems unable to confront the world as it is: a world that is mostly poor, mostly non-white, and deeply in danger of ecological calamity.  It is a party of the insulated and isolated privileged and of the religious fringes.  It can, and must, do better.

Why I am a miltant atheist

The short answer is, because all the facts and evidence demand that conclusion.  We can break down the argument into several parts:

1.  The theory of evolution disproves all super-natural theories of creation.

2.  The is absolutely no evidence for any of the world's religions.

Here are the major influences on my thinking:

1.  Stephen J. Gould. His wonderful popular books about paleontology, evolutionary theory and the history of science showed me how revolutionary evolutionary theory is.  They also showed me how beautiful and powerful science is.

2. Sam Harris, and especially his book, The End of Faith.  This book exposes the absolute lack of evidence upon which all religions are based, and confronts the real danger that religious ideas pose to the world.

3. The scientific journals, Nature and Science. These are the most exciting, the most informative, and the most important journals in the world today.

Once I saw the true beauty and power of scientific thought, it became impossible to believe the drivel that we know as religion and most (not all) forms of spirituality.  It became clear that religion is a crude form of political manipulation, with fear of eternal damnation as its only real weapon. There is no evidence whatsoever for such fear.  To give up religion is to give up the fear of death.

Seen in this light, the new atheists could be said to be stating the obvious.  However, there is a lot more that can be said on the topic of replacing religion with science.  I recommend the following YouTube videos as a starting point.

Richard Dawkins: Militant Atheism

Sam Harris vs an evangelical (The full debate is here.)

Sam Harris: Death and the present moment

Sam Harris: Science can answer moral questions

Harris and Dawkins are both respected scientists.  I recommend all of Dawkins's books.

How the oceans might collapse

Here is yet another refutation of the denialist assertion that all the world needs is more growth.  There is hope for the oceans, but only if catches are reduced to levels substantially below present catches, and substantially below so-called maximum sustainable yields (MSY).

From Science 26 October 2012: Vol. 338 no. 6106 pp. 474-475

The Risks of Overfishing

On page 517 of this issue, Costello et al. paint a dismal picture of the state of the world's fisheries. The authors report that globally, the vast majority of exploited fish populations have been depleted to abundance levels well below those recommended by conventional management guidance. Of even greater concern, most species are on a continuing trajectory of decline. These insights were gleaned from analyses of data from previously unassessed fish populations. These poorly understood fisheries, which represent about 80% of the world's fish catch, are in much worse shape than the relatively well-studied fisheries on which previous global status reviews have relied...

On the basis of these and other observations, the Lenfest Forage Fish Task Force recommended that catches of many forage fish species be cut in half relative to conventional guidance, and that no new forage fish fisheries should be instituted in low-information circumstances.

Dayton argued for a shift in the burden of proof for fisheries management decisions, in line with that applied in other natural resource and human health and safety policy arenas. This shift in the burden of proof would require demonstration of no serious impact before fishing could proceed. It is justified not least because the risks of continuing fishing when it results in serious negative consequences are generally much greater than the risks of curtailing fishing when it does not have a deleterious impact.

 

The Limits to Growth, and Thinking in Systems

 I've just responded to a typical denialist dismissal of The Limits to Growth: The 30-Year Update

The short summary:

1. The authors of the limits to growth computer models are much more nuanced and critical of their models than are their detractors.

2. The book, Thinking in Systems: A Primer, discusses the fundamental aspects of the model, and how such models change our view of the world.  If there is one book that I wish everyone would read, including every scientist, politician, business leader and educator, it would be Thinking in Systems: A Primer.

Edward 

A simple refutation of a silly idea

Somehow people seem to think that we might be living in some kind computer simulation.  See, for example, http://en.wikipedia.org/wiki/Simulated_reality

In fact, evolutionary theory proves that there is no intelligent designer of the world. Thus, the idea that we are living in a *designed* simulation is incompatible with the theory of evolution, and can be dismissed without further ado.

I am well aware that bad ideas never die, and that there will be those who somehow want to dismiss this argument.  They are, quite simply, wrong.

Edward

Optimism: a common non-sequitor

Psychological factors and dispositions such as optimism and pessimism have little to do with public policy, but the typical response to people saying, "these policies are unsustainable" is simply to say, "don't be so pessimistic". This response is nonsense.

For a brilliant discussion of the "this time it's different" syndrome in the financial realm, see the book,
This Time It's Different: Eight Centuries of Financial Folly.

Alas, we are living in times of political, religious and ecological folly as well.

The Righteous Mind

Just finished The Righteous Mind, subtitled, Why Good People Are Divided by Politics and Religion.

There is a lot to like in this book, and a lot that I personally found infuriating.

I'd like to become friends with the author, Jonathan Haidt. Making friends on the internet is dicey at best. I'll keep you posted.

Edward

Leo 4.10 final released

Leo 4.10 final is now available here

Leo is a text editor, data organizer, project manager and much more.

Leo 4.10 contains 9 months of intense work on Leo. Several very important features are subtle; you could almost call them Easter Eggs, so please read the following notes carefully.

The highlights of Leo 4.10:

* Dozens of new and improved features and commands, including...
- Tab completion now shows all @command & @button nodes.
- Leo tabs may be detached from the main window.
- The Open With menu now works.
- The leoInspect module answers questions about Python code.
- Leo can highlight the pane containing the focus.
- The bigdash plugin searches across multiple files.
- Improved abbreviation capabilities.
- Improved handling of URL's.
- Improved editing of non-Leo files.
- Improvements create "weightless" unit testing.
- Improved Leo's home page.
* Easier installation on MacOS.
* Fixed almost 70 bugs.

The Easter Eggs

1. Tab completion now shows all @command & @button nodes.

Put all your common scripts in @command nodes in myLeoSettings.leo. Typing <Alt-X>@c <Tab> will remind you of the names of these scripts. You can execute the scripts by name without the "@command-" prefix.

2. Improved abbreviation capabilities.

Virtually any kind of abbreviation is possible. For example, ~a to ã.

3. Improved handling of URL's.

URL's can be used as links to other Leo outlines.

4 Weightless/waitless unit testing.

The mantra is edit, alt-4 (run-marked-unit-tests-externally), edit, alt-4,... Several seemingly innocuous changes made this work without "friction". The result is a remarkable increase in productivity.

Links

Leo

Forum

Download

Leo 4.10 b1 released

Leo 4.10 b1 March 21, 2012

Leo 4.10 b1 is now available at: http://sourceforge.net/projects/leo/files/

Leo is a text editor, data organizer, project manager and much more.
http://webpages.charter.net/edreamleo/intro.html

Leo 4.10 contains 9 months of intense work on Leo. Several very important
features are subtle; you could almost call them Easter Eggs, so please read
the following notes carefully.

The highlights of Leo 4.10:
---------------------------

* Dozens of new and improved features and commands, including...
- Tab completion now shows all @command & @button nodes.
- Leo tabs may be detached from the main window.
- The Open With menu now works.
- The leoInspect module answers questions about Python code.
- Leo can highlight the pane containing the focus.
- The bigdash plugin searches across multiple files.
- Improved abbreviation capabilities.
- Improved handling of URL's.
- Improved editing of non-Leo files.
- Improvements create "weightless" unit testing.
* Easier installation on MacOS.
* Fixed almost 70 bugs.

The Easter Eggs
---------------

1. Tab completion now shows all @command & @button nodes.

Put all your common scripts in @command nodes in myLeoSettings.leo.
Typing @c will remind you of the names of these scripts.
You can execute the scripts by name without the "@command-" prefix.

2. Improved abbreviation capabilities.

Virtually any kind of abbreviation is possible. For example, ~a to ã.

3. Improved handling of URL's.

URL's can link to other Leo outlines. Ctrl-Click on nodes or URL's
in body text to activate the URL.

4 Weightless unit testing.

The mantra is edit, alt-4 (run-marked-unit-tests-externally), edit,
alt-4,... Several seemingly innocuous changes made this work without an
"friction". The result is a remarkable increase in productivity.

Links:
------
Leo: http://webpages.charter.net/edreamleo/front.html
Forum: http://groups.google.com/group/leo-editor
Download: http://sourceforge.net/projects/leo/files/
Bzr: http://code.launchpad.net/leo-editor/
Quotes: http://webpages.charter.net/edreamleo/testimonials.html

An open letter to Americans Elect

Here is a letter I just sent to Americans Elect.

Absent a multiple-winner voting system, why, oh why, would I want to support a third party?

I prefer President Obama to the present cretinous Republican candidates. In that case, unless I truly believe that your candidate will win, I should prefer that your candidate be more like the *Republican* candidates than the Mr. Obama, because all votes for your candidate will detract from the mainstream candidate most like yours.

Your candidate is more like Obama than any of the Republican candidates, so I conclude that your group will help the Republicans far more than the President. In fact, I wouldn't be surprised if I learned that your group is simply one more Republican dirty trick.

Any comments?