$$A = \frac{1}{k}\sum_{i=1}^{k}x_i$$
It is astonishing how many people use this definition, without having the slightest idea of how it works. You usually hear "Add all the elements together and divide by the number of elements - that's the average." Ask the person selling you that explanation where does the relationship stem from, you'll probably get a wall of silence, or even worse, more explanations which use the very notion they're trying to explain.
One representing the many
Let's explore this organically, which means our logic will drive us into the wall before reaching the correct answer. Suppose we have an array of a few rather random (accidentally odd) numbers:
$$R = [3, 5, 11]$$
They could be anything. Money, grades in school, irrelevant. The statement of the problem is: "How do we find out a number $n$ which relates to all inputs as close as possible?" What does it mean to lie as close as possible? It means that the absolute difference between the input and a supposed value $n$ is as low as possible. Let's formalize such a statement by trying to estimate how $n=6$ relates to all of our values:
$$\Delta x_1 = | 3 - 6 |= 3 $$
$$\Delta x_2 = | 5 - 6 |= 1 $$
$$\Delta x_3 = | 11 - 6 |= 5 $$
$$\Delta[3,1,5]$$
We can see the how far $n$ is from every input. We want minimal numbers here, as small as the inputs permit (because they modulate the final output). We could spend all day searching for the smallest possible $\Delta$-difference array which is given as a function of $n$, or we could do something clever. We want the number whose distance from every input is as small as possible. Well, we could add up all the differences:
$$ d(n) = \sum_{i=1}^{k}( | x_i - n | )$$
Where $k$ is the number of inputs. In the previous example, that gives us a total miss of $d(6) = 9$. We want $d(n)$ to be as small as possible which simply means to be maximally close to all of the inputs. You could track the possible solution as the lowest point which the function $d(n)$ reaches, but as the function is both linear and limited to positive values means it doesn't have a continuous derivative at the point of reflection, which is exactly the point we're looking for. Well, the only reason we used absolute values was to denote that we desire positive values, because negative distances don't make sense. What else can give us positive values, preserve order and relieve our problem of a non-continuous first derivative? Raise the equations to the power of $2$!
$$ d(n) = \sum_{i=1}^{k}(x_i - n )^2$$
Well, that was quick. What have we gained? We now have our distances squared and summed, but positive. And guess what, we now have a unique peak value:
$$\frac{d}{dn}d(n) = 0$$
So, let's first expand our $d(n)$ before deriving and extrapolating the $n$ at $d'(n) = 0$, here are the steps:
$$ d(n) = kn^2 - 2n\sum_{i=1}^{k}(x_i) + \sum_{i=1}^{k}(x_i^2) $$
Differentiating:
$$ \frac{d}{dn}d(n) = 2kn - 2\sum_{i=1}^{k}(x_i) $$
And the winner is:
$$ 2kn - 2\sum_{i=1}^{k}(x_i) = 0 \iff n = \frac{2\sum_{i=1}^{k}(x_i)}{2k} = \frac{1}{k}\sum_{i=1}^{k}(x_i) = A $$
Could it be? Oh, yes. That's the elusive, proper definition of an arithmetic mean. Nobody woke up one day and simply wrote out the final equation. It was an important issue to resolve, to find what is the value around which all inputs tend to be accumulating.
And that's the whole point of it, when you try to reduce the sum of accumulated squared distances by respecting all the inputs included, you will find that the value drops around the most populated areas of the number line. Also, when you have only two values, that's simply resolves down to the middle between two points on the numberline. The more inputs you have, the more precisely can you define the central tendency.