Embedding audio/video in epub files

[MicroDrie]

Originally Posted by JSWolf

I've not yet tried it, but Bluefire Reader for iOS is using the latest RMSDK. That might work for this multimedia ePub 3 eBook.

The challenge is not so much displaying a video on an EPUB page, but in the fact that switching to full screen only leaves the standard play, pause, time scrollable and back to normal screen buttons. Or self-defined keys expire in full screen mode. So the button to turn subtitles on or off has also disappeared.
The EPUB readers I know don't allow debug mode. So I can't see if the EPUB reader uses a measure of shielding the HTML/CSS code in the shadow DOM or if the EPUB reader in full screen mode falls back to OS screen routines.

[Quoth]

Originally Posted by KevinH

I disagree. Epub3 can easily handle video and audio and javascript. Bad reader implementations are the issue.

But then the user needs to install a compatible app. If you want a single download to work, then an App always beats any attempt to use epub3 with Audio & Video, i.e. Multimedia.

By not doing an app for multimedia and assuming the epub3 will work for the customer is worse than expecting someone to install fonts on a Kobo or Kindle (mostly they won't).

Unless you test every ereader and App you won't even know which ones work with epub3 used to do multimedia.

[Doitsu]

Originally Posted by Quoth

But then the user needs to install a compatible app. If you want a single download to work, then an App always beats any attempt to use epub3 with Audio & Video, i.e. Multimedia.

But then you'd need to create and maintain apps for different operating systems.

Originally Posted by Quoth

Unless you test every ereader and App you won't even know which ones work with epub3 used to do multimedia.

Many eInk readers have indeed problems with embedded multimedia content, but most epub3 apps for tablets and PCs support them just fine.

[Hitch]

Originally Posted by Doitsu

But then you'd need to create and maintain apps for different operating systems.

Many eInk readers have indeed problems with embedded multimedia content, but most epub3 apps for tablets and PCs support them just fine.

Problems? Hell, that's the kindest word I can come up with for multimedia on eInks. :-) Catastrophe is the next-best word.

Hitch

[Quoth]

Originally Posted by Doitsu

But then you'd need to create and maintain apps for different operating systems.

Many eInk readers have indeed problems with embedded multimedia content, but most epub3 apps for tablets and PCs support them just fine.

Most people already have ZERO programs or apps on PC, Mac, Linux, iOS or Android that support full multimedia epub3 and that is a huge more number of apps/programs to test and support than OSes.

It's possible to use ONE FW for Linux/Mac/Windows multimedia, one for iOS and one for Android. A lot would be in common and once it's made you can use same content for all three for updates and new titles only needs new content. People have been doing this for over 30 years on desktop and over 12 years for iOs and Android. There are many excellent free tools and frameworks.

Epub3 multimedia is limited compared to full HTML5 + code on a web server which is clunky and limited compared to a multimedia app. Even the Web Multimedia has to be tested with different browsers. The epub3 is much worse because by default no-one has a an epub3 that supports multimedia.

[KevinH]

Huh?

All epub3 support multimedia. Only some epub3 readers are broken.

Instead of supporting you own apps (which are a pain) simply provide a list of known good epub3 readers. And that includes almost all modern browsers and any epub3 reader that uses those engines. Many are completely free to download and work on all major platforms including iOS and Android.

Make it clear that your epub3 uses multimedia so that broken e-ink users can self select away

Readium and Thorium are good examples of Browser based epub3 readers that work well on multiple platforms.

[MicroDrie]

Originally Posted by KevinH

Huh?

All epub3 support multimedia. Only some epub3 readers are broken.

Instead of supporting you own apps (which are a pain) simply provide a list of known good epub3 readers. And that includes almost all modern browsers and any epub3 reader that uses those engines. Many are completely free to download and work on all major platforms, including iOS and Android.

Make it clear that your epub3 uses multimedia so that broken e-ink users can self select away

Readium and Thorium are good examples of Browser based epub3 readers that work well on multiple platforms.

The only challenge is that objective comparison of what an EPUB reader can and cannot do is impossible to find. With the EPUB code below, I can see an HTML test score in an IFRAME.

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html>

<html xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops">
<head>
  <title></title>
  <style>
.container {
    position: relative;
     width: 100%;
     height: 100%;
}
.frr {
	max-width: 100%;
	width:400px;
	height: 600px;
	width: auto\9; /* IE8 */
}
 .iframe {
     width: 100%;
     height: 500px;
}

</style>
</head>

<body>
  <div class="frr">
    <iframe src="https://html5test.com" title="Iframe Example" class="frr"></iframe>
  </div>
</body>
</html>

And what's even more interesting, you can also see what is partially supported. When you click on the circle, you will see what is not supported!

These are different test results:
Firefox version 90: 513
Sigil preview window: 523
Gitden Reader: 506

PageEdit asks for permission to open the website in the browser. So I'm not sure whether the Gitden app itself displays the website or whether it also displays the website in a redirected browser view.

Since none of the EPUB readers have an inspection mode except for IOS, it is very difficult to investigate erroneous displays in an EPUB reader.

As for the Sigil preview window, a big compliment from me for the high degree of support for HTML mode!

[DiapDealer]

Happy you're happy. But the browser score is sort of pointless in an EPUB. It's testing for things that aren't based on EPUB specifications. There are probably many things Sigil's WebEngine browser will support that EPUB doesn't.

[MicroDrie]

Originally Posted by DiapDealer

Happy you're happy. But the browser score is sort of pointless in an EPUB. It's testing for things that aren't based on EPUB specifications. There are probably many things Sigil's WebEngine browser will support that EPUB doesn't.

Back to my multimedia EPUB Video challenge in the EPUB readers. It's actually a chicken and egg problem. If you don't use video, you won't run into any problems. If it appears that there are no problems, the problems are not solved.

If you, as one of the Epub readers market party, limit the size of an EPUB and the EPUB standard requires both an MP4 and a WEBM file, yes, then it will be a very short and therefore pointless video.

If none of the EPUB readers support debug facility, then how can you fix EPUB reader errors?

It is a misconception to think that an EPUB reader is a black box that does not reveal how that black box works. A valid combination of HTML, CSS and JavaScript in an EPUB file can be rendered cleanly and successfully. The EPUB reader is built around a standard rendering solution for HTML, CSS and JavaScript. On top of that render engine solution, the specific EPUB rules are implemented. As long as you respect the specific EPUB rules, you can just run HTML, CSS and JavaScript code in your EPUB.

The version of the render engine used in the EPUB reader determines what can and cannot be done within the specific EPUB rules. In addition, one EPUB reader adheres to or has implemented the EPUB rules better than the other EPUB reader. That is why you can successfully run a bootstrap program in an EPUB file with some adjustments.

The irony is that in the early days of HTML5, not everything worked either. Those were reported as bugs and those bugs have been resolved over the years. The knowledge to bypass these HTML5 bugs with all kinds of "hacks" and best practices is slowly disappearing from the Internet. That's a shame because sometimes they can also be used in an EPUB file.

In the EPUB market divided by dominant market parties, openness that can contribute to solving bugs is "hard to find". Bookworms who are addicted to a paper book with the turning of a paper page with page number, you don't get that until reading an EPUB book. Even digitizing a page turn doesn't help with that.
On the other hand, the question is whether the current generation that has grown up with interactive multimedia will buy an EPUB reader to read chunks of text supplemented with a still picture with maybe audio text.
The first market party to understand this has taken the first step to enable debugging in its EPUB reader.

I'm happy, well I can test the HTML, CSS, and JavaScript in Sigil and then hope the EPUB reader will do the same. A low score also makes it clear what is or is not supported. And that gives the opportunity to try whether it will work with an old HTML5 hack or whether it is better to forget about certain HTML code.

[DiapDealer]

Originally Posted by MicroDrie

The version of the render engine used in the EPUB reader determines what can and cannot be done within the specific EPUB rules.

Actually the version of the render engine has very little to do with what EPUB rules they will support. An EPUB3 rendering engine should (and can) support all of the EPUB3 specs. Same for an EPUB2 rendering engine. Devs don't choose to defy EPUB specifications in their rendering engines because doing so would be hard to do. They defy EPUB specifications because they (or their employers) simply don't care about complying with all of it.

Just remember that Sigil is not an EPUB rendering engine. It's a browser that we use to approximate what an epub might look like on an ereading system. We try to support everything that the epub specifications say should be supported. But it's not to be trusted implicitly. It will also probably support many things that very few mainstream rendering engines will. The devices people will be reading your work on should always be what's used for final testing. It's better to fail gracefully than spectacularly where ebooks are concerned. Unsupported CSS that creates unflattering-looking text is still much, much better than unsupported CSS that creates something unreadable.

[DNSB]

Not to mention that the various generations of the ePUB specs are strewn with MUST and SHOULD. Treating any SHOULD item as a null operation makes an ePUB renderer compliant. See 2.2 EPUB Reading Systems from the EPUB 3.2 standard documentation.

For instance:

It is recommended that Reading Systems support at least one of the H.264 [H264] and VP8 [RFC6386] video codecs, but this is not a conformance requirement – a Reading System might support other video codecs, or none at all.

[MicroDrie]

Originally Posted by DiapDealer

Devs don't choose to defy EPUB specifications in their rendering engines because doing so would be hard to do. They defy EPUB specifications because they (or their employers) simply don't care about complying with all of it.

I totally agree with DiapDealer, but when Philips decided not to release certain content on the superior VCC recorders, the market chose the lower quality brand that did release that content. Why would a simple digital copy of a paper book suffice without the additional digital options, giving the digital reader a completely different experience?

In a culture of sixes at school, it is enough to pass with only sixes instead of tens. As a buyer of a six culture-based dominant EPUB reader, there is no way of knowing beforehand how well or how badly that new model follows the EPUB standard.

If you want to create content for an EPUB, you have no idea what to avoid (well video is definitely, but probably not the only thing) which is asking for trouble, in today's sixes culture for EPUB readers.

There is simply no website that provides an objective comparison for EPUB readers and applications. The introduction date also says nothing about the (im)possibilities here. What I like is that the developers of Sigil are better and probably more likely to release a new bug fix release than the Firefox developer community.

[DiapDealer]

Sorry, you're losing me with your various analogies. If you want to create epub content: simple is usually best. If you're not sure something will work: ask. Someone here (or elsewhere) will know. That it works in Sigil's Preview is not a very definitive measuring stick for experimental epub content.

[KevinH]

Yes, I could not agree more. The MUST, SHOULD, and MAY verbiage of the spec really sets an extremely low bar (too low imho) to be called a "compliant" epub3 reader.

Given the eventual epub spec convergence with living html standard, my guess is that most future epub3 readers will use browser engines as their underlying engine which will make SHOULD level very easy to obtain and raise the bar for what a epub3 reader should handle. Both Readium and Thorium already do that as does EpubJS, and Bibi as well.

As for file size limits, hosted remote resources are one of those SHOULD items that really should have been a MUST.

Originally Posted by DNSB

Not to mention that the various generations of the ePUB specs are strewn with MUST and SHOULD. Treating any SHOULD item as a null operation makes an ePUB renderer compliant. See 2.2 EPUB Reading Systems from the EPUB 3.2 standard documentation.

For instance:

[MicroDrie]

Originally Posted by KevinH

Yes, I could not agree more. The MUST, SHOULD, and MAY verbiage of the spec really sets an extremely low bar (too low imho) to be called a "compliant" epub3 reader.

It seems that the manufacturers have not tried very hard to raise the EPUB standard to a higher level. If no one is going to complain, the discipline of the market will not work either. In the longer term it is also not in the interest of the manufacturers if their customers turn away from EPUB.

My video problem is that there is actually very little information to be found. With some EPUB readers you can disable the standard video keys and also replace them with your own keys as long as you don't touch the key full screen. At that moment you lose your own keys and the video player reverts to hard-coded keys.

What is noticeable if not native English speaking is that the subtitle key shown initially is not shown in full screen mode. Great chance that at that moment the shadow DOM is used that the manufacturer uses to shield its incomplete default full screen video setting. In itself, with a correct implementation, there is, but with a faltering application, this is less to live with. Firstly, I don't believe that there are no deaf people in English-speaking countries and secondly, if there are, they probably want to be able to turn the subtitles on or off in full screen.

Does anyone have experience how to use the video player shadow DOM in an EPUB?