Abstract

Let $\alpha $ be an automorphism of a finite group $G$. For a positive integer $n$, let $E_{G,n}(\alpha )$ be the subgroup generated by all commutators $[...[[x,\alpha ], \alpha ],\dots ,\alpha ]$ in the semidirect product $G\langle\alpha \rangle$ over $x\in G$, where $\alpha $ is repeated $n$ times. By Baer's theorem, if $E_{G,n}(\alpha )=1$, then the commutator subgroup $[G,\alpha ]$ is nilpotent. We generalize this theorem in terms of certain length parameters of $E_{G,n}(\alpha )$. For soluble $G$ we prove that if, for some $n$, the Fitting height of $E_{G,n}(\alpha )$ is equal to $k$, then the Fitting height of $[G,\alpha ]$ is at most $k+1$. For nonsoluble $G$ the results are in terms of the nonsoluble length and generalized Fitting height. The generalized Fitting height $h^*(H)$ of a finite group $H$ is the least number $h$ such that $F^*_h(H)=H$, where $F^*_0(H)=1$, and $F^*_{i+1}(H)$ is the inverse image of the generalized Fitting subgroup $F^*(H/F^*_{i}(H))$. Let $m$ be the number of prime factors of the order $|\alpha |$ counting multiplicities. It is proved that if, for some $n$, the generalized Fitting height of $E_{G,n}(\alpha )$ is equal to $k$, then the generalized Fitting height of $[G,\alpha ]$ is bounded in terms of $k$ and $m$.

The nonsoluble length~$\lambda (H)$ of a finite group~$H$ is defined as the minimum number of nonsoluble factors in a normal series each of whose factors either is soluble or is a direct product of nonabelian simple groups. It is proved that if $\lambda (E_{G,n}(\alpha ))=k$, then the nonsoluble length of $[G,\alpha ]$ is bounded in terms of $k$ and $m$.

We also state conjectures of stronger results independent of $m$ and show that these conjectures reduce to a certain question about automorphisms of direct products of finite simple groups.