Dtyjlui

It's well known that we can classify finitely generated abelian groups. Let $M$ be a finitely generated abelian group, in principle we can decide the group structure of $G=Aut(M)$ from $M$. What about the group cohomology? In other words, what are the group structures of $H^i(G,M)$ ($i geq 0$)? More modestly, what are the orders of $H^i(G,M)$? If we can't do things precisely, is there any bound?

Since still no one has answered this question or given you any input whatsoever, I am writing an extended comment about how difficult this question sounds (very difficult). Please keep in mind that I am a novice in group cohomology, and probably don't know anymore than you do. I do think it is unlikely you will get an answer to this question in any generality. It seems to me there are many cases which could be their own unanswered question.

Torsion-free case. A subquestion is to compute the group cohomology of $GL_n(mathbb Z)$ with coefficients in the standard representation $M = mathbb Z^{oplus n}$. You can maybe hope to compute the cohomology with the trivial representation, see some discussion at this overflow question: How to compute the cohomology of the general linear group with integral entries . I'm not sure if this is known. At least you know $H^0(GL_nmathbb Z, M) = M^{GL_nmathbb Z} = 0$ for free.

Torsion case. Thankfully this case splits into a separate problem for each prime, but it still includes as a subproblem computing group cohomology of $GL_n(mathbb F_p)$ and $GL_n(mathbb Z_{p^k})$ with coefficients in the standard representation. I'm not sure whether this is known. At least it can be computed, as indicated in a comment on overflow using Magma, but I don't use Magma so I can't advise you on that.

Still, even for a $p$-group, with mixed exponents it seems like the structure of the automorphism groups could get pretty complicated. It seems unlikely to me anyone would try to compute the group cohomology in nontrivial coefficients of such a thing.

Mixed case. If we write $A = A_{tors}oplus mathbb Z^r$, then $Aut(A)$ is block matrices:
$$begin{pmatrix}Aut(A_{tors}) & Hom(mathbb Z^r, A_{tors})\0&GL_r(mathbb Z)end{pmatrix}$$ in this case the automorphism group is a split extension $$0 to A_{tors}^{oplus r} to A to Aut(A_{tors})times GL(A_{free}) to 1$$ i.e. a semidirect product. For trivial coefficients $B(Grtimes H)$ is a fibration, and there is a spectral sequence to compute its cohomology in terms of $G$ and $H$. There is also a Kunneth theorem for nontrivial coefficients on a product $Gtimes H$, e.g. at this MO question, but that assumes the coefficients is a tensor product of modules for $G$ and $H$.

In general it seems like a challenging problem to deal with groups like $GL_n(mathbb Z_{p^k})$ and $GL_n(mathbb Z)$ alone. But in the case that you are facing of automorphisms groups which are extensions of products of these groups, but for which the coefficient module is not broken down into such pieces, I haven't seen anything to help simplify the problem.

Even in the easiest case of a cyclic group I couldn't look up an answer anywhere. Presumably someone has done that though. Good luck!

I agree with Ben that this is totally intractable (Quillen wrote some complicated and beautiful papers on the case of $GL_n(Bbb F_p)$ with trivial coefficients, instead of $Bbb F_p^n$ coefficients; making the coefficients nontrivial is unlikely to simplify the problem much if at all).

It seemed fun to say something about the cyclic case though, since this is doable. I hope the bottom shows that even the simplest case is somewhat nontrivial. There's probably an easier argument (eg arguing that only $2$-torsion matters because multiplication by $-1$ is one of your automorphisms), I just thought this through and wrote down what came to mind.

There are a few simplifying tricks we begin with. First, $Bbb Z/n cong bigoplus_{i=1}^k Bbb Z/{p_i}^{ell_i}$ as groups for some sequence of distinct primes $p_i$ and integers $ell_i > 0$. Using the Chinese remainder theorem decomposition above, we find that there are surjective "reduction-by-$p$" maps $r_i: text{Aut}(Bbb Z/n) to text{Aut}(Bbb Z/p_i^{ell_i})$. Because $text{Aut}(Bbb Z/n)$ is well-known to be cyclic of order $varphi(n)$, where $varphi$ is Euler's totient function, we see that there are sections $s_i$ of this reduction map.

Now observe that the action of $text{Aut}(Bbb Z/n)$ on $Bbb Z/n$ respects the Chinese remainder theorem splitting, and the action on the component $Bbb Z/p_i^{ell_i}$ is the composite of $r_i$ with the natural action of $text{Aut}(Bbb Z/p_i^{ell_i})$. In particular,

$$H^*(text{Aut}(Bbb Z/n); Bbb Z/n) cong bigoplus_{i=1}^k H^*(text{Aut}(Bbb Z/n); Bbb Z/p_i^{ell_i}).$$

We may make a further simplification: the map $$r_i^*: H^*(text{Aut}(Bbb Z/p_i^{ell_i}); Bbb Z/p_i^{ell_i}) to H^*(text{Aut}(Bbb Z/n); Bbb Z/p_i^{ell_i})$$ induced by $r_i$ is an isomorphism.

The easiest way I see how to say this is the Lyndon-Hochschild-Serre spectral sequence (sorry for the technology, there should be a way in terms of 'transfers' but I didn't see one immediately). In the notation on Wikipedia, for us $G = text{Aut}(Bbb Z/n)$, while $G/N = text{Aut}(Bbb Z/p_i^{ell_i})$, and $N = text{ker}(r_i)$. Then what we know is that there is a 'spectral sequence' (whatever that is) $$H^p(G/N, H^q(N, Bbb Z/p_i^{ell_i}) implies H^{p+q}(G, Bbb Z/p_i^{ell_i}).$$ This data is organized in a plane, and the rule of thumb is that the map given by collapsing first to the "bottom row" $q=0$ and then mapping to $H^*(G, Bbb Z/p_i^{ell_i})$ is the induced map by $r_i^*$. This in mind, if we can show that $H^q(N, Bbb Z/p_i^{ell_i}) = 0$ for $q > 0$, and $$H^0(N, Bbb Z/p_i^{ell_i}) cong Bbb Z/p_i^{ell_i}$$ as a $G/N$-module, then we obtain the desired result.

Now this isn't too bad, because $text{gcd}(|N|, p_i) = 1$, and the group cohomology of any group with coefficients of order prime to that of the group is trivial in positive degrees (this does follow from a transfer argument), and $H^0(N, Bbb Z/p_i^{ell_i}) = Bbb Z/p_i^{ell_i}$, because $N$ (by definition) acts trivially on the coefficients; and $G/N$ acts on the coefficients by the usual action, as expected.

So we have reduced to calculating $H^*(text{Aut}(Bbb Z/p^{ell}); Bbb Z/p^{ell})$ for $p$ prime and $ell > 0$. Let's carry out this calculation, following what is given here.

First, $p = 2, ell = 1$ is a special case, because $text{Aut}(Bbb Z/2) = 1$ is the trivial group; the group cohomology is trivial except in degree zero, where it is $Bbb Z/2$.

Now the operator $N$ in their post is identical to $$sum_{substack{q in Bbb Z/p^ell \ gcd(p, q) = 1}} q,$$ which a little arithmetic shows is $p^{2ell - 1}(p-1)/2$. If $(p,ell) neq (2,1)$, this operator is easily seen to be identically zero.

The fixed point space of $Bbb Z/p^ell$ is zero if $p$ is odd, and equal to $2^{ell - 1} Bbb Z/2^{ell} cong Bbb Z/2$ when $p$ is even: to see this, observe that for $p$ odd, $2$ is prime to $p$, and $2x = x$ implies $x = 0$, so there are no nontrivial fixed points. When $p$ is even, $3$ is invertible, so at least if $x$ is fixed we have $2x = 0$; these are precisely the terms ${0, 2^{ell - 1}}$, and these are fixed by multiplication by any odd integer.

A choice of generator $sigma in text{Aut}(Bbb Z/p^ell)$ always reduces to a generator of $text{Aut}(Bbb Z/p)$, and in particular $sigma neq 1 pmod p$ if $p>2$. Therefore, multiplication by $(sigma - 1)$ is an isomorphism so long as $p>2$. When $p = 2$, instead all we can conclude is that if $ell > 1$ we have $sigma equiv 3 pmod 4$, and so $(sigma - 1)$ is surjective onto $2^{ell - 1} Bbb Z/2^ell$.

Therefore, the group $text{ker}(N) / (sigma - 1)A = 0$ for all $(p, ell)$, and so there is no odd-degree cohomology. The set $(Bbb Z/p^ell)^{text{Fix}}/NA$ is zero for $p$ odd or $ell = 1$, but isomorphic to $Bbb Z/2$ for $p = 2$ and $ell > 1$. This is the even-degree cohomology in positive degrees.

Putting this all together, we have $$H^k(text{Aut}(Bbb Z/n); Bbb Z/n) = begin{cases}0 & n text{ odd},\
Bbb Z/2 & text{gcd}(n, 4) = 2, ; k = 0,\
Bbb Z/2 & 4 mid n, ; 2 mid k,\
0 & text{else}.
end{cases}
$$

OK, inspired by my comment on my previous answer here is some partial progress. I am posting as a separate answer because it is sufficiently independent (and to me, interesting) from the cyclic case. This still leaves open the infinite case and the case $p = 2$ (both difficult and interesting), as well as some stuff with products (not as difficult or interesting).

For $p$ odd and $n > 0$, we have that the cohomology groups $$H^*(GL_n(Bbb Z/p^ell); (Bbb Z/p^ell)^n) = 0.$$

I claim that because $-1 in Zleft(GL_n(Bbb Z/p^ell)right)$, the answer must be 2-torsion; see here. (That this element is central follows because it is a diagonal matrix.) But because multiplication by $p^ell$ is the zero map $(Bbb Z/p^ell)^n to (Bbb Z/p^ell)^n$, the same is true on cohomology, so we also see that the cohomology groups are $p^ell$-torsion. Because $gcd(p^ell, 2) = 1$, we see that the cohomology groups are in fact 1-torsion, aka, zero.