As one who has an engineering/scientific background, but no knowledge of neurology, I found the summation box rather terse. Please either give a citation in your description so that the reader can see where the notation comes from or use a standard mathematical form such as .
Question The scope is too general? There are only 42 images in that category and no sub-categories! It would be the same, but would you think that calling the scope "Perceptron units in the context of Artificial neural networks" would be better?
@Martinvl: I've added the complete function which is described by the graphical notation. Is it ok now? (Although this notation is used very often, I'm not aware of a paper which introduces it.) --MartinThoma (talk) 14:39, 30 May 2016 (UTC)[reply]
Since making my comments, I have looked at a number of papers on perceptrons. Firstly, you should make it clear that the Σ sign applies to the set of products xi.wi. Secondly, what exactly is φ? I think that it is the activation output function. If you visit this file amd go to page 4 (Title = "Single Processing Unit") you will see what I have in mind. Rather than the symbol φ, the image shoudl be modified to show the band-pass filter similar to the one shown in the citation.
A suitable scope for this image would, in my opinion, be "Single Perceptron unit".
I'm thinking about adjusting the middle node with the summation symbol and φ so that it becomes more clear that things are first summed and then the activation function φ is applied, similar to the file you linked.
Yes, φ is the activation function as I wrote in the text of the image.
Showing the plot of a sigmoid function is not good. There are many possible activation functions and not all look like the sigmoid function / thanh. For example, ReLU (rectified linear unit; φ(x) = x for x > 0, 0 otherwise). See https://martin-thoma.com/neuronale-netze-vorlesung/#aktivierungsfunktionen for more functions. I will not change that. However, I could add a couple of typical functions in the text, if you think that is necessary? --MartinThoma (talk) 05:14, 1 June 2016 (UTC)[reply]
edit: Ok, I've played a bit with the visualization to make it look more similar to what you have linked. However, I think my current visualization is better. A perceptron unit is nowadays not used as a single unit, but as a building block for more complex multilayer Perceptrons. Those are typically visualized like that: . I remember that I was confused by the visualizations which looked similar to what you linked. The step from the single perceptron to a multilayer Perceptron was bigger with the other visualizations than with my current visualization. --MartinThoma (talk) 11:10, 1 June 2016 (UTC)[reply]
Thank you for the changes. The scope is OK except that it should link to a category - I suggest "Single Perceptron unit".
The diagram itself could do with some fine-tuning:
If you want consistency with normal mathematical/IT conventions, the inputs should either run from (1 ... n) or from (0 ... n-1) (Assuming that there are "n" inputs).
You do not show exactly what is being added in the diagram - I suggest either replacing "wi" in the diagram with "× wi for all values of "i"." or expanding the Σ as discussed earlier.
I suggest breaking the green circle into two boxes, one with the Σ symbol and the next with the φ symbol. This will; indicate that the meaning of φ is described elsewhere (such as in the description). This will also emphasise that the two operations are separate.
has a special role. It is usually set to constant 1 (). So one has features and a bias. The bias is not treated different; it just has to be there. I'm also not the only one using this notation (e.g. [1], [2], [3], [4], [5]), but most seem to go from 1 to .
I've added the explanation with the bias to the text.
Yes, the diagram does not show exactly what is added / multiplied. That is why there is a description. No diagram will ever show this as well as the simple formula . The value of this image is that it helps to give an intuitive understanding on where the learned parameters are (on the edges from input to the neuron), where the nonlinearity happens (in the neuron), how many inputs you can have (as many as you want), how many outputs you can have (only one).
Yes, many use the splitted notation you suggest. But I think it makes the step from going from a single Perceptron to a MLP much harder. I will not change that, because there are many images like that and this is what I think is much better in my image than in those.
I checked the citrations that you gave. In all of them the indicies of the "real" inputs had the range (1 ... n). The bias (which was always explicity set to the value of "1") was either identified at x0 or as xn+1 - ie we had either x0 or xn+1 but not both. I also noticed that four of the five references split the summation and the activation functions into two as I suggested. I suggest therefore that you remove xn+1 and write "x0 = 1" in the input box (or vice-versa). I woudl still like ot see the summatiomn and activation boxes separated out.Martinvl (talk) 12:43, 3 June 2016 (UTC)[reply]
