The primary effect of LSD is that it breaks down the brains ability to perceive and evaluate those perceptions. This is not experienced as a loss of ability (internally) as many of the processes involved in perception are inhibitory in nature. If you switch off the negative signals about possible perceptions that do not match the incoming data from the environment then suddenly the brain sees a lots more hits, and there is a massive spike in reinforcement - everything feels cool as fuck and makes perfect sense because your brain is awash in the neurotransmitters that reward observing patterns. Of course the brains spends a lot of time observing and evaluating itself in relation to its observations of the world, and so the same rush of positive associations will occur about "deep personal development".
What is really happening? Hard to say: my guess is that our brains are constantly searching for equilibrium and taking a psychedelic causes a massive batch of noise in the search process. It does seem to cause to long-term changes in people's attitudes towards themselves, and the people around them. I've not seen any evidence that those changes are consistent across people - the only consistent pattern is that it changes their relationship to the world. I would speculate that it is just random noise, kicking a vast chunk of their learned behaviour into a different equilibrium. The perception that the change is accessing "a higher state of consciousness" is just another form of buying into some bullshit.
My take on it is that LSD provides access to a type of experience that is unavailable to most people: psychosis. The experience of un-evaluated perception of reality. Whether or not that experience has any value does not seem to have a universal answer, and depends largely on where people are in their lives, what they take into that experience, and what they hope to gain from it. Interpreting a measurement of one property of a brain that may correlate with a level of consciousness in some forms of test is simply reckless.