Embedding audio/video in epub files

jonesyb · 07-29-2021, 02:21 AM

Has anyone done this? I have been searching around for a solution. Just wondering if anyone here does it? I posted in the Sigil forum as that is what I primarily use. Apologies if it should go somewhere else.

I read this: https://ebookflightdeck.com/handbook/media - Seems interesting.

Also I have never created an epub 3 file yet so it is a little daunting.

What is your process for this, if you have done it? Would be interesting to hear your thoughts. Thank you

KevinH · 07-29-2021, 08:32 AM

Yes epub3 officially supports some specific audio and video file types. Adding media files is quite easy in Sigil using the Add Existing menu. You can also use the Insert Link menu item. Standard html5 audio and video tags work. There are sample epubs that have been posted to Sigil in the past. Search the Sigil and epub forums here.

The real question is what e-readers support it.

KevinH · 07-29-2021, 01:10 PM

I just found that Video mp4 is very broken on the latest Sigil builds because our custom url handler does not properly handle byte ranges that only get used for audio and video when loaded via a custom scheme:.

Here is a related bug report:

Quote:

When serving media files over custom resource handlers Chromium's media system (E.g. the <audio> tag) requests partial ranges of the target files. The exact partial ranges that it requests depends on the media file and the scheme that is being served over.
For example when using a custom HTTPS handler (E.g. https://resource/myfile.ogg) the media system treats the media file as a streaming object and requests the superfluous range of Range: bytes=0-, and when using a custom scheme handler (E.g. resource://localhost/myfile.ogg) the media file is treated as a non-streamed object and the media system requests more exact partial ranges such as Range: bytes=294912-.
In both cases FFmpeg errors are emitted to the log along the lines of following when attempting to play the media files:
MediaEvent: MEDIA_ERROR_LOG_ENTRY {"error":"FFmpegDemuxer: open context failed"}
MediaEvent: PIPELINE_ERROR DEMUXER_ERROR_COULD_NOT_OPEN
or
MediaEvent: MEDIA_ERROR_LOG_ENTRY {"error":"FFmpegDemuxer: data source error"}
MediaEvent: PIPELINE_ERROR PIPELINE_ERROR_READ
The root cause of these errors is due to CEF's StreamReaderURLLoader class not properly handling the partial ranges which sends incorrect data upstream to Chromium's media internals. In the case of Range: bytes=0- CEF incorrectly sets the expected_content_length in StreamReaderURLLoader::OnReaderSkipCompleted() to 0 instead of -1 which prevents the handler from writing the real content length to the response object in later code. I have not tested other possible ranges in detail as they have not yet shown up in my particular use case.
All of my testing was on a customized 3945 build on macOS but according to several forum threads it looks like it also affects other platforms and branches as wel

I will look into adding some sort of byte range streaming for our sigil scheme. Silly really since we offer up the entire file in a buffer but obviously needed with later Qt versions given our change to a url scheme handler.

KevinH · 07-29-2021, 01:57 PM

Okay that is a Qt QWebEngine/Chromium bug that impacts only audio and video that uses proprietary codecs and a custom scheme handler.

I have added a workaround our URLSchemeHandler to redirect them as file: scheme urls to prevent the partial byte range requests as there appears no way to handle them inside QtWebengine.

This has been pushed to master so mp4 videos now work again in Sigil's Preview in the next release. WebM videos should work already.

MicroDrie · 08-01-2021, 06:58 AM

Quote:

Has anyone done this?

Yes

Quote:

I read this: https://ebookflightdeck.com/handbook/media - Seems interesting.

Be aware that it are the rules for HTML5, while EPUB format is not completely the same. You can use it to change it to the following code:

Code:

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
    <head>
        <title>EPUB Video Example</title>

        <style>
            /* fixed-ratio-resize basic responsive class */
            .frr {
                max-width: 100%;
                height: auto;
                width: auto\9; /* IE8 */
            }
        </style>
    </head>

    <body>
	<p>This is a working EPUB video that could work in an EPUB reader</p>		
        <video id="video01" class="frr" controls="controls" autoplay="" poster="../Images/PosterCC.jpg">
            <!-- MP4 must be first for iPad! -->
            <source src="../Video/shared-culture.mp4" type="video/mp4" />
            <!-- Needed for EPUB reader -->
            <source src="../Video/shared-culture.webm" type="video/webm" />
            <!-- Will automatically add sub-titling button to controls -->
            <!-- Only vtt is supported -->
            <track label="Nederlands" kind="subtitles" srclang="nl" src="../Video/cc-en.vtt" />
            <track label="Frans" kind="subtitles" srclang="fr" src="../Video/cc-fr.vtt" />
            There is video content at this location that is not currently supported on your device. Please visit our website, <a href="http://www.example.com">www.example.com</a>, to watch the videos on your computer.
        </video>
        <p>You can refer in both the source and track elements to a web location.</p>
    </body>
</html>

Adapt the location for the video and subtitling files to your situation. This will work in:

Sigil preview screen;
PageEdit;
The free GitDen reader (tested on Android only)

So far, there are standard options. The big challenge arises when you want to embellish your video player with bells and whistles. After trying a dozen HTML5 examples, I can tell you that they all break in one way or another.

Let me know if this is enough or if you want more like me. I think I've found the difference between the HTML5 and EPUB standard and happen to be working on a solution today. Despite many searches on the Internet, I haven't actually found a working EPUB solution to adjust Video views in an EPUB.

Update: I forgot to mention that you can set the size of the video's width and height as a kind of maximum width and height on a large screen. Up to those dimensions, the frr class adjusts the width and height to the physical screen dimensions.

KevinH · 08-01-2021, 10:41 AM

.mp4 is one of the official video resource types that any epub3 must/should support according to the spec. But of course notall readers do. At one point in time there was a multi-media epub3 test epub that was used to show epub3 compliance among readers.

Sigil should support all of those media types but that does not mean e-readers do.

MicroDrie · 08-03-2021, 11:18 AM

Quote:

Originally Posted by KevinH

Sigil should support all of those media types but that does not mean e-readers do.

The EPUB standard has many open ends. And even if the video/audio player is supported, the full screen support for a video is also a challenge. The free GitDen reader for Android and IOS loses most keys in full screen mode under Android OS.

Unfortunately, the free GitDen reader is not being developed further. As a result, it no longer meets the current Android requirements. As a result, it can no longer be loaded for people with the latest Android IOS version.

Does anyone know of another EPUB reader for Android that can display videos under the current Android OS version?

BeckyEbook · 08-03-2021, 11:35 AM

Quote:

Originally Posted by MicroDrie

Does anyone know of another EPUB reader for Android that can display videos under the current Android OS version?

I did not search too closely, but I checked the epub3 video in two applications available on Google Play:
com.flyersoft.moonreader
com.s2apps.reader

Of course, their imperfections can be annoying as there is no perfect application for reading epub files.

PeterT · 08-03-2021, 12:33 PM

Libera claims to support Audio/Video in epub3 books. Do you have a test book I can try?

Sent from my Pixel 4a using Tapatalk

Doitsu · 08-03-2021, 01:29 PM

Quote:

Originally Posted by PeterT

Libera claims to support Audio/Video in epub3 books. Do you have a test book I can try?

EPUBTEST 0100 - Reflowable Content Tests

Quoth · 08-05-2021, 07:00 AM

An app is usually a better solution as iOS, Mac, Android, Windows or Linux is needed anyway to have video + audio. There are free frameworks and once it's done once the next time is easy. Almost everyone would have to download an App even if epub3 did what what you want.

People have been making multimedia since 1980s and commonly from 1990s. The idea of epub3 competing with multimedia apps and HTML5 websites was a crazy design decision.

KevinH · 08-05-2021, 08:43 AM

I disagree. Epub3 can easily handle video and audio and javascript. Bad reader implementations are the issue.

DiapDealer · 08-05-2021, 08:57 AM

Yep. The willful ignoring of (and/or deviation from) epub specifications is the main issue. Otherwise, Audio/Video would just work. The same holds true for epub reading app devs: iOS, Mac, Android, Windows or Linux is needed for their reading app anyway, so why not just make your EPUB3 reading app work like it's supposed to to begin with?

JSWolf · 08-05-2021, 06:02 PM

I've not yet tried it, but Bluefire Reader for iOS is using the latest RMSDK. That might work for this multimedia ePub 3 eBook.

Hitch · 08-05-2021, 06:03 PM

Quote:

Originally Posted by MicroDrie

Yes

FYI, eBookflightDeck has been dead for what, 7 years now?

Hitch

07-29-2021, 02:21 AM	#1
jonesyb Enthusiast Posts: 34 Karma: 82258 Join Date: May 2021 Device: kindle	Embedding audio/video in epub files Has anyone done this? I have been searching around for a solution. Just wondering if anyone here does it? I posted in the Sigil forum as that is what I primarily use. Apologies if it should go somewhere else. I read this: https://ebookflightdeck.com/handbook/media - Seems interesting. Also I have never created an epub 3 file yet so it is a little daunting. What is your process for this, if you have done it? Would be interesting to hear your thoughts. Thank you

07-29-2021, 08:32 AM	#2
KevinH Sigil Developer Posts: 7,629 Karma: 5433388 Join Date: Nov 2009 Device: many	Yes epub3 officially supports some specific audio and video file types. Adding media files is quite easy in Sigil using the Add Existing menu. You can also use the Insert Link menu item. Standard html5 audio and video tags work. There are sample epubs that have been posted to Sigil in the past. Search the Sigil and epub forums here. The real question is what e-readers support it. Last edited by KevinH; 07-29-2021 at 09:38 AM.

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Embedding metadata into epub files	steve1977	Editor	36	06-23-2021 05:48 PM
Audio and video files support in Calibre	shijo	Calibre	3	07-21-2017 12:19 PM
Plugin for video, audio, and image files?	woodr2011	Plugins	0	09-24-2015 07:51 AM
Audio or Video files into an eBook using calibre	Khadar	Calibre	27	09-22-2014 02:43 AM

07-29-2021, 01:57 PM	#4
KevinH Sigil Developer Posts: 7,629 Karma: 5433388 Join Date: Nov 2009 Device: many	Okay that is a Qt QWebEngine/Chromium bug that impacts only audio and video that uses proprietary codecs and a custom scheme handler. I have added a workaround our URLSchemeHandler to redirect them as file: scheme urls to prevent the partial byte range requests as there appears no way to handle them inside QtWebengine. This has been pushed to master so mp4 videos now work again in Sigil's Preview in the next release. WebM videos should work already.

08-01-2021, 10:41 AM	#6
KevinH Sigil Developer Posts: 7,629 Karma: 5433388 Join Date: Nov 2009 Device: many	.mp4 is one of the official video resource types that any epub3 must/should support according to the spec. But of course notall readers do. At one point in time there was a multi-media epub3 test epub that was used to show epub3 compliance among readers. Sigil should support all of those media types but that does not mean e-readers do.

08-03-2021, 12:33 PM	#9
PeterT Grand Sorcerer Posts: 12,155 Karma: 73448616 Join Date: Nov 2007 Location: Toronto Device: Nexus 7, Clara, Touch, Tolino EPOS	Libera claims to support Audio/Video in epub3 books. Do you have a test book I can try? Sent from my Pixel 4a using Tapatalk

08-05-2021, 07:00 AM	#11
Quoth the rook, bossing Never. Posts: 11,062 Karma: 85874891 Join Date: Jun 2017 Location: Ireland Device: All 4 Kinds: epub eink, Kindle, android eink, NxtPaper11	An app is usually a better solution as iOS, Mac, Android, Windows or Linux is needed anyway to have video + audio. There are free frameworks and once it's done once the next time is easy. Almost everyone would have to download an App even if epub3 did what what you want. People have been making multimedia since 1980s and commonly from 1990s. The idea of epub3 competing with multimedia apps and HTML5 websites was a crazy design decision.

08-05-2021, 08:43 AM	#12
KevinH Sigil Developer Posts: 7,629 Karma: 5433388 Join Date: Nov 2009 Device: many	I disagree. Epub3 can easily handle video and audio and javascript. Bad reader implementations are the issue.

08-05-2021, 08:57 AM	#13
DiapDealer Grand Sorcerer Posts: 27,545 Karma: 193191846 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	Yep. The willful ignoring of (and/or deviation from) epub specifications is the main issue. Otherwise, Audio/Video would just work. The same holds true for epub reading app devs: iOS, Mac, Android, Windows or Linux is needed for their reading app anyway, so why not just make your EPUB3 reading app work like it's supposed to to begin with?

08-05-2021, 06:02 PM	#14
JSWolf Resident Curmudgeon Posts: 73,887 Karma: 128597114 Join Date: Nov 2006 Location: Roslindale, Massachusetts Device: Kobo Libra 2, Kobo Aura H2O, PRS-650, PRS-T1, nook STR, PW3	I've not yet tried it, but Bluefire Reader for iOS is using the latest RMSDK. That might work for this multimedia ePub 3 eBook.

Advert

Advert