This is my take on Jodorowsky, his book and his approach to tarot.
Jodorowsky is a man of prodigious talents whose creative vision informs disciplines of many kinds. His book on tarot (one of the many he has written) is richly layered, with the aim of describing and eliciting relationships between the cards as they represent lived aspects of the human psyche. Jodorowsky's understanding of tarot is based on a nearly lifelong process of visionary (indeed, shamanic) interaction with the characters depicted in the cards, and as such the book represents not so much a system, as a flexible scaffolding for understanding the levels of meaning within and between the cards.
Jodorowsky is very much in the magic-realism tradition, in which the imaginal realm contains truth beyond facts. This approach is reflected in everything he does - filmmaking, his work as a psychotherapist, autobiographer, philosopher, tarotist. Nothing he says is set in stone: he intends his work to be a jumping-off place for many points of view.
No less a tarot teacher than Yoav Ben Dov was a student of Jodorowsky's for several years, and they maintained their mentor-mentee throughout Yoav's too-brief career. In his book, The Open Reading, Yoav frequently refers to Jodorowsky's insights into the cards - the only thing he didn't care for was the redesign of Jodorowsky's deck. Jodorowsky at one point told Yoav that he would be glad to discuss the meanings of the cards, with the understanding that his viewpoint might change between one day and the next (tarot being an expression of dynamic processes).
With reference to The Hermit "walking backward," I believe that Jodorowsky means retreat into oneself, a going within after long periods of outward expression; to me, this is very much like the action of a retrograde planet, which of course only seems to move backward in space.