I've worked with basic feed forward neural networks with backpropagation. But I'm having difficulty understanding convolutional neural networks.
I'd like to try making a small character recognition program.
Here's a picture I was annoyed by the absence of when I read about the topic. The idea of a convolutional net is to have each neuron only look at the neurons spatially close to it in the layer above it, and for the neurons of each layer only to be of a small-ish number of types which share the same receiving parameters. I think other approaches are more popular for character image recognition now, but apparently you can make a mean go bot with them.
>>7951793
That's not really how it works.
I'd check this out:
http://cs231n.github.io/convolutional-networks/
>>7951866
Ignoring the boring stuff like sampling it says what I said. I pushed the neuron analogy a little further though. In its language each filter corresponds to a different kind of neuron in my language, and the connecting parameters for each kind of neuron corresponds to the matrix entries in the filter.
>>7951885
>In its language each filter corresponds to a different kind of neuron in my language
Okay but then what you said earlier
>The idea of a convolutional net is to have each neuron only look at the neurons spatially close to it in the layer above it
is wrong. Each filter/neuron looks at the entire input volume above it, outputting a single slice of the volume to be processed by the following layer.
>>7951920
I said that filters correspond to different kinds of neurons (as labeled by letters in the picture) not individual neurons. The neurons of each variety of letter together have the effect of making one slice of the output volume, like how each filter makes a slice of the output volume. I swear it's all equivalent.
OP here...
>>7951793
What other approaches are more popular for OCR?
>>7951866
Thanks. That source helps quite a bit. I'm still not sure where convolution comes into play though.
>>7952125
My mistake, it seems this is one of the most powerful approaches to ocr.
>convolution
The shift-multiply-add thing that goes on with filters is exactly convolution. It's a multidimensional sort of convolution though.