Why does deep and cheap learning work so well?

Very interesting.

the morning paper

Why does deep and cheap learning work so well Lin & Tegmark 2016

Deep learning works remarkably well, and has helped dramatically improve the state-of-the-art in areas ranging from speech recognition, translation, and visual object recognition to drug discovery, genomics, and automatic game playing. However, it is still not fully understood why deep learning works so well.

So begins a fascinating paper looking at connections between machine learning and the laws of physics – showing us how properties of the real world help to make many machine learning tasks much more tractable than they otherwise would be, and giving us insights into why depth is important in networks. It’s a paper I enjoyed reading, but my abilities stop at appreciating the form and outline of the authors’ arguments – for the proofs and finer details I refer you to the full paper.

A paradox

How do neural networks with comparatively…

View original post 1,545 more words

Methods of Proof — Diagonalization

Very clear presentation on the uncountability of the real numbers, and the halting problem.

Further keywords: Cantor, natural numbers, real numbers, diagonalization, bijection, Turing halting problem, proof by contradiction.

Math ∩ Programming

A while back we featured a post about why learning mathematics can be hard for programmers, and I claimed a major issue was not understanding the basic methods of proof (the lingua franca between intuition and rigorous mathematics). I boiled these down to the “basic four,” direct implication, contrapositive, contradiction, and induction. But in mathematics there is an ever growing supply of proof methods. There are books written about the “probabilistic method,” and I recently went to a lecture where the “linear algebra method” was displayed. There has been recent talk of a “quantum method” for proving theorems unrelated to quantum mechanics, and many more.

So in continuing our series of methods of proof, we’ll move up to some of the more advanced methods of proof. And in keeping with the spirit of the series, we’ll spend most of our time discussing the structural form of the proofs…

View original post 1,841 more words

On Differential Forms

Abstract. This article will give a very simple definition of k-forms or differential forms. It just requires basic knowledge about matrices and determinants. Furthermore a very simple proof will be given for the proposition that the double outer differentiation of k-forms vanishes.

MSC 2010: 58A10

1. Basic definitions.

We denote the submatrix of A=(a_{ij})\in R^{m\times n} consisting of the rows i_1,\ldots,i_k and the columns j_1,\ldots,j_k with

\displaystyle{          [A]{\textstyle\!{\scriptstyle j_1\atop\scriptstyle i_1}          \!{\scriptstyle\ldots\atop\scriptstyle\ldots}          \!{\scriptstyle j_k\atop\scriptstyle i_k}} := \begin{pmatrix}                  a_{i_1j_1} & \ldots & a_{i_1j_k}\\                  \vdots     & \ddots & \vdots\\                  a_{i_kj_1} & \ldots & a_{i_kj_k}\\          \end{pmatrix}  }

Continue reading

Announcement: 11th International Conference on Parallel Programming and Applied Mathematics

The conference will start in September 6-9, 2015, Krakow, Poland.

Parallel Programming and Applied Mathematics, PPAM for short, is a biennial conference started in 1994, with the proceedings published by Springer in the Lecture Notes in Computer Sciences series, see PPAM. It is sponsored by IBM, Intel, Springer, AMD, RogueWave, and HP. The last conference had a fee of 420 EUR.

It is held in conjunction with 6th Workshop on Language based Parallel Programming.

Prominent speakers are:

Continue reading

Day 2, Workshop Programming of Heterogeneous Systems in Physics

Day 2 of the conference had below talks. Prof. Dr. Bernd Brügmann gave a short introduction. He pointed out that Jena is number 10 in Physics in Germany, has ca. 100.000 inhabitants, and 20.000 students.

  1. Dr. Karl Rupp, Vienna, Lessons Learned in Developing the Linear Algebra Library ViennaCL. Notes: C++ operator overloading normally uses temporary, special trickery necessary to circumvent this, ViennaCL not callable from Fortran due to C++/operator overloading, eigen.tuxfamily.org, Karl Rupp’s slides, with CUDA 5+6 OpenCL and CUDA are more or less on par,
  2. Prof. Dr. Rainer Heintzmann, Jena, CudaMat – a toolbox for Cuda computations. Continue reading

Day 1, Workshop Programming of Heterogeneous Systems in Physics

As announced in Workshop Programming of Heterogeneous Systems in Physics, July 2014, I attended this two-day conference. Below are speakers and pictures with my personal notes.

  1. Dipl.-Ing. Hans Pabst from Intel, Programming for the future: scaling forward with cores and vectors. Hans Pabst Continue reading

Simple Exercises for a C Programming Language Course

I sometimes teach a C programming language course. I use the following simple exercises for the students to solve on their own. Most solutions are no longer than 10-20 lines of C code.

  1. Exercising simple loop and printf(): Print a table of square root values for arguments 1 to 30.
  2. Exercising printf(), scanf(), arrays, if-statement, flag-handling: Read a list of numbers into an array, and then sort the array with Bubblesort.
  3. Continue reading