Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Formats > Workshop

Notices

Reply
 
Thread Tools Search this Thread
Old 08-09-2009, 02:06 PM   #1
ahi
Wizard
ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.
 
Posts: 1,792
Karma: 507333
Join Date: May 2009
Device: none
53,432 page PDF

I thought this might be of interest even to people other than Sony PRS-505 owners and individuals who want Wikipedia as an eBook... so I'm starting this thread here, with much the same information I posted in an earlier thread.

I have just successfully loaded a PDF with the following attributes:

- 53,432 pages
- 133.2 MB
- at least 15,000 items in external table of contents (2540 first level items, and each of those has 4 - 15+ subitems)
- all pages basically full of text (content is the complete Summa Theologica repeated 5 times over)

When loading it from my old-ish SD card, I've observed the following:

- There is a one-time 7-8 minute loading time.
- Page turning is slower than usual... about 3 seconds, but regardless whether one is going to the next page, or 30,000 pages further in the text.

When loading it from main memory:

- There is a one-time 4 minute loading time.
- Page turning seems consistently about 3 seconds still.

Warning: Don't ever try to actually use an *external* table of contents that has over a hundred thousand entries. My PRS-505 spent several "hours" formatting before I was somehow able to get it to reset. (Initially it was even ignoring my poking the reset button in the back... though otherwise it seemed like it was actually working/processing, not frozen.

---------------

I should mention that, in contrast, a 10,000 page PDF has less than 1 minute one-time loading time and pretty snappy page-turning.

Also, the reason I went for a 50,000 page or so PDF is because LaTeX memory limits (with all settings maxed out) still seem to kick in a few hundred pages before 60,000--albeit this isn't a page limit, but a text (i.e.: string) limit.

If anyone manages to generate a similarly content-heavy PDF with over 100,000 pages, please post what you did.

- Ahi
ahi is offline   Reply With Quote
Old 08-10-2009, 05:15 AM   #2
Jellby
frumious Bandersnatch
Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.
 
Jellby's Avatar
 
Posts: 5,803
Karma: 4027751
Join Date: Jan 2008
Location: Spaniard in Sweden
Device: Cybook Orizon
Quote:
Originally Posted by ahi View Post
anyone manages to generate a similarly content-heavy PDF with over 100,000 pages, please post what you did.
You could maybe try appending the same file several times with pdftk or similar.
Jellby is offline   Reply With Quote
 
Enthusiast
Old 08-10-2009, 07:11 AM   #3
RobbieClarken
Addict
RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.
 
Posts: 372
Karma: 1002274
Join Date: Mar 2008
Location: Australia
Device: Kindle
How big is the LaTeX source? If it is not too big, would you mind uploading it or emailing it to me? I wouldn't mind generating the PDF to see how it performs on the DR1000S.
RobbieClarken is offline   Reply With Quote
Old 08-10-2009, 07:46 AM   #4
ahi
Wizard
ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.
 
Posts: 1,792
Karma: 507333
Join Date: May 2009
Device: none
Quote:
Originally Posted by Jellby View Post
You could maybe try appending the same file several times with pdftk or similar.
That's true, actually. I might do so just to so...

The reason I did not try it is because (even if different files are appended) each segment would be necessarily self-contained with no ability to cross-link into other sections.

Still... it is a reasonable way to see at what point it becomes virtually unusable.

Quote:
Originally Posted by RobbieClarken View Post
How big is the LaTeX source? If it is not too big, would you mind uploading it or emailing it to me? I wouldn't mind generating the PDF to see how it performs on the DR1000S.
I will upload it (somewhere else) tonight. It shouldn't be too big once zipped. It will be nice to hear about how such a large file performs on other devices.

- Ahi

Last edited by ahi; 08-10-2009 at 02:04 PM.
ahi is offline   Reply With Quote
Old 08-10-2009, 09:07 AM   #5
ahi
Wizard
ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.
 
Posts: 1,792
Karma: 507333
Join Date: May 2009
Device: none
Actually, RobbieClarken, given that the LaTeX sources require XeLaTeX and the memory settings maxed out, you might have an easier time with the PDF file itself. (Which is only a 130 MB, as stated above.)

I'll post both the source and the PDF though.

- Ahi
ahi is offline   Reply With Quote
Old 08-10-2009, 10:04 PM   #6
RobbieClarken
Addict
RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.RobbieClarken ought to be getting tired of karma fortunes by now.
 
Posts: 372
Karma: 1002274
Join Date: Mar 2008
Location: Australia
Device: Kindle
Quote:
Originally Posted by ahi View Post
Actually, RobbieClarken, given that the LaTeX sources require XeLaTeX and the memory settings maxed out, you might have an easier time with the PDF file itself. (Which is only a 130 MB, as stated above.
Sure thing, as long as you don't mind uploading 130 MB.
RobbieClarken is offline   Reply With Quote
Old 08-11-2009, 11:46 PM   #7
ahi
Wizard
ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.
 
Posts: 1,792
Karma: 507333
Join Date: May 2009
Device: none
Quote:
Originally Posted by RobbieClarken View Post
Sure thing, as long as you don't mind uploading 130 MB.
I don't, but I haven't manage to do it yet.

In the meantime, the LaTeX source (just click on "Ingyenes", then after on "Letöltés indítása"):
http://data.hu/get/1628766/summa_theologica.zip.html

I'll post again once I have the PDF up somewhere too.

- Ahi
ahi is offline   Reply With Quote
Old 08-12-2009, 06:19 AM   #8
Jellby
frumious Bandersnatch
Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.
 
Jellby's Avatar
 
Posts: 5,803
Karma: 4027751
Join Date: Jan 2008
Location: Spaniard in Sweden
Device: Cybook Orizon
Do you actually need to explicitly duplicate the text? Can't you have a single copy of the text and \input it several times.
Jellby is offline   Reply With Quote
Old 08-12-2009, 06:49 AM   #9
Jellby
frumious Bandersnatch
Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.
 
Jellby's Avatar
 
Posts: 5,803
Karma: 4027751
Join Date: Jan 2008
Location: Spaniard in Sweden
Device: Cybook Orizon
By \input-ing the same text seven times and increasing some settings I could create this PDF (86830 pages, 170MB) with pdflatex. It could be larger, I guess. (source, 3.4MB)
Jellby is offline   Reply With Quote
Old 08-12-2009, 07:56 AM   #10
ahi
Wizard
ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.
 
Posts: 1,792
Karma: 507333
Join Date: May 2009
Device: none
You are obviously right... I'm not sure why that didn't occur to me at the time.

At what page limit does PDFLaTeX start to crap out on you? Perhaps that's what I should be working with for such large files... maybe XeLaTeX (unintuitively enough) has a lower maximum memory limit that PDFLaTeX.

And how does the 86830 page monstrum work on your Cybook Gen3?

- Ahi
ahi is offline   Reply With Quote
Old 08-12-2009, 08:44 AM   #11
Jellby
frumious Bandersnatch
Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.
 
Jellby's Avatar
 
Posts: 5,803
Karma: 4027751
Join Date: Jan 2008
Location: Spaniard in Sweden
Device: Cybook Orizon
Quote:
Originally Posted by ahi View Post
At what page limit does PDFLaTeX start to crap out on you?
I don't know... and I'm not sure I want to try I had to raise max_strings, pool_size and save_size in /usr/share/texmf/web2c/texmf.cnf (I'm using tetex, it may be different in other TeX distributions), and it seems the .out file (for pdf bookmarks) was the most problematic part.

Quote:
And how does the 86830 page monstrum work on your Cybook Gen3?
I don't know either, but I'm afraid it could crash (reboot), it does that with another largish (and not particularly complex) PDF I created. It could be different with the new firmware when it comes out
Jellby is offline   Reply With Quote
Old 08-12-2009, 08:59 AM   #12
ahi
Wizard
ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.
 
Posts: 1,792
Karma: 507333
Join Date: May 2009
Device: none
Hmmm.... are you saying you generated the 80,000 page PDF with the default memory settings?

I only got about 5,000 pages with those... can't even generate the 10,000 page (single Summa Theologica) file without raising those limits.

Can you tell me more about the .out file stuff? Also, do you know how to prevent PDFLaTeX from generating any PDF bookmarks? For the Sony, with PDF links inside the content working, the PDF bookmarks are unnecessary and (I think) mainly responsible for the minutes of one-time initial load time.

- Ahi
ahi is offline   Reply With Quote
Old 08-12-2009, 09:22 AM   #13
Jellby
frumious Bandersnatch
Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.Jellby ought to be getting tired of karma fortunes by now.
 
Jellby's Avatar
 
Posts: 5,803
Karma: 4027751
Join Date: Jan 2008
Location: Spaniard in Sweden
Device: Cybook Orizon
Quote:
Originally Posted by ahi View Post
Hmmm.... are you saying you generated the 80,000 page PDF with the default memory settings?

I only got about 5,000 pages with those... can't even generate the 10,000 page (single Summa Theologica) file without raising those limits.
Hmm... it depends what you mean with "memory settings". I had to change what I said above to generate the 80,000 page PDF, otherwise it aborts at around 60,000 pages, but nothing else, I didn't have to re-create formats or anything else. It may be that I had already modified something a couple of years ago, but I don't think so, this system's tetex should be pretty much out-of-the-box.

Quote:
Can you tell me more about the .out file stuff? Also, do you know how to prevent PDFLaTeX from generating any PDF bookmarks?
The .out file is where hyperref stores the information for the bookmarks. I noticed that when pdflatex aborts, if I try to recompile, it then aborts again right at the beginning, when reading the .out file, that's why I think the stuff in the .out file might be one of the culprits.

To disable the bookmarks try passing bookmarks=false to the hyperref package.
Jellby is offline   Reply With Quote
Old 08-12-2009, 09:35 AM   #14
ahi
Wizard
ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.
 
Posts: 1,792
Karma: 507333
Join Date: May 2009
Device: none
Quote:
Originally Posted by Jellby View Post
Hmm... it depends what you mean with "memory settings". I had to change what I said above to generate the 80,000 page PDF, otherwise it aborts at around 60,000 pages, but nothing else, I didn't have to re-create formats or anything else. It may be that I had already modified something a couple of years ago, but I don't think so, this system's tetex should be pretty much out-of-the-box.
Oh, sorry. I misread what you had written.

Quote:
Originally Posted by Jellby View Post
The .out file is where hyperref stores the information for the bookmarks. I noticed that when pdflatex aborts, if I try to recompile, it then aborts again right at the beginning, when reading the .out file, that's why I think the stuff in the .out file might be one of the culprits.

To disable the bookmarks try passing bookmarks=false to the hyperref package.
Thanks for that!

I don't suppose you want to go for the gold and try to make a 200,000 page version?

If not, I'll try my hand at it in the next day or two.

- Ahi
ahi is offline   Reply With Quote
Old 08-12-2009, 09:41 AM   #15
ahi
Wizard
ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.ahi ought to be getting tired of karma fortunes by now.
 
Posts: 1,792
Karma: 507333
Join Date: May 2009
Device: none
In trying to think for some use for such mad-sized PDFs, I've been thinking of a KJV bible with appropriately linked bible commentary and strong's words.

The internal linking would have to be pretty extensive and (while simple to use) pretty complicated...

I think on each page, the verse numbers would have to be links to where the verse appears by itself, with commentary below and on subsequent pages, and each word that has a corresponding STRONG's entry also a link to said entry. Also probably a variety of navigation links (back to main portion, previous verse, next verse, et cetera).

Do you think something like that would be doable, Jellby?

- Ahi
ahi is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Adobe Error (-432) Lord Hutton Sony Reader 13 04-08-2010 08:51 PM
merge pdf page by page jakeluck Sony Reader 4 12-19-2006 02:27 PM
PDF page size DuckDodgers iRex 2 08-09-2006 02:17 PM
Alex speaking from Flight LH 432 Alexander Turcic Lounge 25 03-02-2005 09:41 PM
Alex speaking from Flight LH 432 follow-up Alexander Turcic Lounge 0 02-27-2005 01:15 PM


All times are GMT -4. The time now is 12:05 AM.


MobileRead.com is a privately owned, operated and funded community.