Audiobooks & AI

[Graham44]

Reading through this thread I can see people debating whether or not AI can do something better than humans (who aren't professional narrators at least) but shouldn't we be discussing whether it's a good idea developing a tool (AI) that could ultimately lead to people being put out of work? Do we really want to encourage industries to rely on a program (that's ultimately probably owned by a big tech company or one organisation) to do something humans can already do?

People get excited over what AI can do - gosh it sounds so realistic- well you could have the real thing, an audio book narrated by a (professional) human! Why re-invent the wheel?

Before long we will be getting products that have had no human input - an e-reader built in a factory by robots run by AI, your ebooks, written by AI, bought from a website run and maintained by AI, not a single human involved in the chain, fantastic!

Ok perhaps I'm being a bit facetious, but it certainly seems like that's the road we could be starting out on - to me at least *

* or perhaps Im just paranoid

[DiapDealer]

People who are waiting for AI to fade away are going to be sorely disappointed. "Hating AI" (regardless if it's with a passion or not) is pointless. It's like hating numbers. Or rain. What you hate, is what humans are doing with it. The list of fantastic things that humans use to do stupid stuff with is legion.

[DNSB]

Originally Posted by salamanderjuice

description = "Laura clamly reads a book to her child. The recording is of very high quality, with the speaker's voice sounding clear and very close up."

The description is specifying a specific voice and not much else.

How do you clamly read?

Originally Posted by salamanderjuice

I've heard much worse from real people.

And so have I. OTOH, professional voice actors definitely do a better job compared to AI.

One author I know was looking into using AI for audiobooks with a lower cost than hiring professionals. What was found was that to get a decent result from AI, it was pretty much required to rewrite the book to add the pitch, tone, pace, etc. indicator elements that a voice actor did not need. Then there was the issue with having invented words pronounced correctly. For humans, they were told the pronunciation. For AI, the entire book was converted to use the IPA though most of that work was automated and about 5% of the words needed to be manually converted (basically look for words that had * around them to say they weren't found in the dictionary). It turned out to be a small group of words so the manual work wasn't that bad, mostly character and place names.

They also looked at using ssml and pronunciation lexicons but that was a lot more work since it required multiple spans and you need to add the ipa bits to the header though it did give an ebook that could be read in English.

<p><span ssml:alphabet="ipa" ssmlh="ðə ˈkʌrᵊnt steɪt ɒv ði ɑːt ɪz nɒt ʌp tuː ðæt ˈlɛvᵊl ɒv səˌfɪstɪˈkeɪʃᵊn.">The current state of the art is not up to that level of sophistication.</span></p>

An sample paragraph using ssml and IPA.

[Quoth]

It won't fade away, but economic reality will mean there will be less hype and more realistic expectations.

[Graham44]

Originally Posted by DiapDealer

People who are waiting for AI to fade away are going to be sorely disappointed. "Hating AI" (regardless if it's with a passion or not) is pointless. It's like hating numbers. Or rain. What you hate, is what humans are doing with it. The list of fantastic things that humans use to do stupid stuff with is legion.

Hating AI may not be logical, but it isn't pointless, I never claimed to be logical

I certainly can't disagree about humans doing stupid things with new ideas/ stuff, as a race we have a talent for that!

[Quoth]

Originally Posted by DNSB

One author I know was looking into using AI for audiobooks with a lower cost than hiring professionals. What was found was that to get a decent result from AI, it was pretty much required to rewrite the book to add the pitch, tone, pace, etc. indicator elements that a voice actor did not need. Then there was the issue with having invented words pronounced correctly. For humans, they were told the pronunciation. For AI, the entire book was converted to use the IPA though most of that work was automated and about 5% of the words needed to be manually converted (basically look for words that had * around them to say they weren't found in the dictionary). It turned out to be a small group of words so the manual work wasn't that bad, mostly character and place names.

Yes, this is what I'm saying.

Also the further the novel is from realistic everyday American English set in a real American place today the more work it is. Forget 1920s cozy mysteries or even Westerns. Don't even think about Fantasy and SF. Also humans often can tell from context what the pronunciation will be (Galway vs Paris vs London settings written by an author in British English.) even without a guide. An Irish person that knows no Irish will get places and names in Ireland and Scotland right in a novel written in English. Some can get Welsh, English and French places right too.

The comparison isn't with teenagers failing English Lit, but people that that have read books out loud for years, or acted (theatre), or worked on radio, TV or Movies, or narrate audio books for a living (a profession over 125 years old).

Edit: Curiously the Google "tool" offered had choice of voices (with British, Australian accents, not just "American"*) but ignored proper language tags in the ebook. It assumed the work was all US English. That can certainly be fixed, but why offer something unfinished? They claim the result can be sold as audio books on the Playstore.

[* I have no idea about Australia, but the USA and English accents I have encountered are diverse. There isn't really any such thing as a British accent and I suspect there is no such thing as an American accent, based on talking with people in different parts of NY, Ohio, Pennsylvania, New Jersey and Boston as well as Texans, and others from diverse US States I've met elsewhere.]

[DNSB]

What!!! You can tell a Boston Brahmin accent from a Southern drawl, a New York accent or a New Orleans Cajun accent without a guide?

[salamanderjuice]

Originally Posted by Quoth

Yes, this is what I'm saying.

Also the further the novel is from realistic everyday American English set in a real American place today the more work it is. Forget 1920s cozy mysteries or even Westerns. Don't even think about Fantasy and SF. Also humans often can tell from context what the pronunciation will be (Galway vs Paris vs London settings written by an author in British English.) even without a guide. An Irish person that knows no Irish will get places and names in Ireland and Scotland right in a novel written in English. Some can get Welsh, English and French places right too.

The comparison isn't with teenagers failing English Lit, but people that that have read books out loud for years, or acted (theatre), or worked on radio, TV or Movies, or narrate audio books for a living (a profession over 125 years old).

Edit: Curiously the Google "tool" offered had choice of voices (with British, Australian accents, not just "American"*) but ignored proper language tags in the ebook. It assumed the work was all US English. That can certainly be fixed, but why offer something unfinished? They claim the result can be sold as audio books on the Playstore.

[* I have no idea about Australia, but the USA and English accents I have encountered are diverse. There isn't really any such thing as a British accent and I suspect there is no such thing as an American accent, based on talking with people in different parts of NY, Ohio, Pennsylvania, New Jersey and Boston as well as Texans, and others from diverse US States I've met elsewhere.]

You have to tell human voice actors how to pronounce made up and obscure words and names too. I bet as tools mature unfamiliar words or phrases will be pointed out and easily addressed.

I think AI narrating is more geared for books where it just doesn't make economic sense to get a professional voice actor in the first place. It wouldn't make sense to spend thousands on a real voice actor, a good studio setup, engineer, etc. for an audiobook that will be lucky to sell 500 copies. But maybe it makes sense to spend to a few hours and a couple hundred dollars on an AI service.

A handspun, handwoven, handsewn, tailored garment has the potential to be much nicer than a factory spun, factory woven, factory sewn one size fits all shirt. But it's also going to be magnitudes more expensive. Same deal here.

[DNSB]

Originally Posted by salamanderjuice

You have to tell human voice actors how to pronounce made up and obscure words and names too. I bet as tools mature unfamiliar words or phrases will be pointed out and easily addressed.

You can hand them a list of words that have non-obvious pronunciation and they have the smarts to stop and ask if they run into a word that isn't on their list.

Originally Posted by salamanderjuice

I think AI narrating is more geared for books where it just doesn't make economic sense to get a professional voice actor in the first place. It wouldn't make sense to spend thousands on a real voice actor, a good studio setup, engineer, etc. for an audiobook that will be lucky to sell 500 copies. But maybe it makes sense to spend to a few hours and a couple hundred dollars on an AI service.

That's the reasoning behind most AI narrated books. I've seen several audiobooks now with an ensemble cast compared to one or two voices commonly used.

Originally Posted by salamanderjuice

A handspun, handwoven, handsewn, tailored garment has the potential to be much nicer than a factory spun, factory woven, factory sewn one size fits all shirt. But it's also going to be magnitudes more expensive. Same deal here.

I think it's closer to comparing a bespoke suit cut and sewn by an experienced tailor to a suit sewn by a person who has never before used a sewing machine and is learning on the fly from YouTube videos.

[Dopedangel]

Originally Posted by Graham44

Reading through this thread I can see people debating whether or not AI can do something better than humans (who aren't professional narrators at least) but shouldn't we be discussing whether it's a good idea developing a tool (AI) that could ultimately lead to people being put out of work? Do we really want to encourage industries to rely on a program (that's ultimately probably owned by a big tech company or one organisation) to do something humans can already do?

People get excited over what AI can do - gosh it sounds so realistic- well you could have the real thing, an audio book narrated by a (professional) human! Why re-invent the wheel?

Before long we will be getting products that have had no human input - an e-reader built in a factory by robots run by AI, your ebooks, written by AI, bought from a website run and maintained by AI, not a single human involved in the chain, fantastic!

Ok perhaps I'm being a bit facetious, but it certainly seems like that's the road we could be starting out on - to me at least *

* or perhaps Im just paranoid

You are not in my opinion the world's current economic system are not going to work when most labor is done by AI owned by capital owners. The other thing that is coming practically free energy to run those AI because of solar and batteries. Lcoe cost of solar and battery system in China is already now less than $0.05-.0.06/kWh when they take the equipment life as 10 years actually life if more 20+ years.

[Quoth]

Originally Posted by salamanderjuice

I think AI narrating is more geared for books where it just doesn't make economic sense to get a professional voice actor in the first place. It wouldn't make sense to spend thousands on a real voice actor, a good studio setup, engineer, etc. for an audiobook that will be lucky to sell 500 copies. But maybe it makes sense to spend to a few hours and a couple hundred dollars on an AI service.

No, it doesn't because it's cheating the customer, who would spend much less buying the ebook and that sort of "AI" audio book production isn't enough better than the customer's own TTS. Pocketbook app on Android using Google's offline speech engine stuff is far better than Amazon's on DXG, K3 or PW3. The premium "cloud" services are not enough better to justify selling that as a product.

However my almost blind friend much prefers human narrated, but finds the TTS on his phone OK for short stories, or a few chapters at a time, not the lenght of time he used to read. He can't afford audio books. I must see if he can navigate our local library on Android as it's free and only has real audio books.

[salamanderjuice]

Originally Posted by Quoth

No, it doesn't because it's cheating the customer, who would spend much less buying the ebook and that sort of "AI" audio book production isn't enough better than the customer's own TTS. Pocketbook app on Android using Google's offline speech engine stuff is far better than Amazon's on DXG, K3 or PW3. The premium "cloud" services are not enough better to justify selling that as a product.

However my almost blind friend much prefers human narrated, but finds the TTS on his phone OK for short stories, or a few chapters at a time, not the lenght of time he used to read. He can't afford audio books. I must see if he can navigate our local library on Android as it's free and only has real audio books.

It's not cheating the customer unless they're lying about it being an AI voice. Should they be priced the same as the human read audiobooks? Probably not, but if it's only a couple dollars extra over the eBook or print book I could see there being some value to some people.

[John F]

Originally Posted by salamanderjuice

Did you listen to example I linked? ...

Do you have a link to so I can enter a sentence and see what it sounds like? I tried some of your links quickly, and they all looked like prepared recordings.

[salamanderjuice]

Originally Posted by John F

Do you have a link to so I can enter a sentence and see what it sounds like? I tried some of your links quickly, and they all looked like prepared recordings.

I used parler-tts locally on my computer to generate that recording, so no link. The online demo seems broken at the moment. I basically just followed the instructions there and changed the prompt and description.

There are other online AI TTS services you can try (e.g., https://huggingface.co/spaces/coqui/xtts) . They'll be a bit different because they're different models trained on different datasets.

[shalym]

Originally Posted by salamanderjuice

There are models that can be annotated with emotions.

Does AI surpass the best human readers? No. But they are leagues above the old TTS and honestly a lot of people if my high school English class was remotely typical.

All I want is them to sound natural and not flub words.

IMO I could listen to whole book if it sounded like this.

Sorry, but if I had paid money for an audiobook that sounded like that I would demand a refund.

Shari