How about you compile your audiobooks into .m4b files and listen to them in iBooks.
There are tools for that and I’m sure Linux will also let you do that in some command-line way
But yeah, you definitely will need iTunes to get files onto your phone into the supplied apps.
Meanwhile, you can download and listen to the fantastic Moby Dick version in the podcast section read by tens of well known speakers and actors.