They are, by definition, Markov chains.
Even in your attempt to be pedantic here (in which the universe and everything within it is a Markov chain), no, it's not. The hardware state is Markovian but the linguistic processing is a Nth order autoregressive process; it depends on the N previous states. Also, your argument is akin to saying "a Boeing 747 is just an arrangement of quarks, so don't get hung up on aerodynamics." it entirely ignores the relevant architectural details, and instead substitutes a model that blows up exponentially explodes in size within a small number of states.
If you tried to build a Markov model to do what LLMs do, and could store one probability in every unit of Planck space across every unit of Planck time, it couldn't handle a prompt longer than about 2/3rds of the first sentence to A Tale of Two Cities.