Register Guidelines E-Books Search Today's Posts Mark Forums Read

08-10-2009, 05:15 AM   #2
Jellby
frumious Bandersnatch

Posts: 6,195
Karma: 4800739
Join Date: Jan 2008
Location: Spaniard in Sweden
Device: Cybook Orizon, Kobo Aura
Quote:
 Originally Posted by ahi anyone manages to generate a similarly content-heavy PDF with over 100,000 pages, please post what you did.
You could maybe try appending the same file several times with pdftk or similar.

 Enthusiast
 08-10-2009, 07:11 AM #3 RobbieClarken Addict   Posts: 372 Karma: 1002274 Join Date: Mar 2008 Location: Australia Device: Kindle How big is the LaTeX source? If it is not too big, would you mind uploading it or emailing it to me? I wouldn't mind generating the PDF to see how it performs on the DR1000S.
08-10-2009, 07:46 AM   #4
ahi
Wizard

Posts: 1,792
Karma: 507333
Join Date: May 2009
Device: none
Quote:
 Originally Posted by Jellby You could maybe try appending the same file several times with pdftk or similar.
That's true, actually. I might do so just to so...

The reason I did not try it is because (even if different files are appended) each segment would be necessarily self-contained with no ability to cross-link into other sections.

Still... it is a reasonable way to see at what point it becomes virtually unusable.

Quote:
 Originally Posted by RobbieClarken How big is the LaTeX source? If it is not too big, would you mind uploading it or emailing it to me? I wouldn't mind generating the PDF to see how it performs on the DR1000S.
I will upload it (somewhere else) tonight. It shouldn't be too big once zipped. It will be nice to hear about how such a large file performs on other devices.

- Ahi

Last edited by ahi; 08-10-2009 at 02:04 PM.

 08-10-2009, 09:07 AM #5 ahi Wizard   Posts: 1,792 Karma: 507333 Join Date: May 2009 Device: none Actually, RobbieClarken, given that the LaTeX sources require XeLaTeX and the memory settings maxed out, you might have an easier time with the PDF file itself. (Which is only a 130 MB, as stated above.) I'll post both the source and the PDF though. - Ahi
08-10-2009, 10:04 PM   #6
RobbieClarken

Posts: 372
Karma: 1002274
Join Date: Mar 2008
Location: Australia
Device: Kindle
Quote:
 Originally Posted by ahi Actually, RobbieClarken, given that the LaTeX sources require XeLaTeX and the memory settings maxed out, you might have an easier time with the PDF file itself. (Which is only a 130 MB, as stated above.

08-11-2009, 11:46 PM   #7
ahi
Wizard

Posts: 1,792
Karma: 507333
Join Date: May 2009
Device: none
Quote:
 Originally Posted by RobbieClarken Sure thing, as long as you don't mind uploading 130 MB.
I don't, but I haven't manage to do it yet.

In the meantime, the LaTeX source (just click on "Ingyenes", then after on "Letöltés indítása"):
http://data.hu/get/1628766/summa_theologica.zip.html

I'll post again once I have the PDF up somewhere too.

- Ahi

 08-12-2009, 06:19 AM #8 Jellby frumious Bandersnatch     Posts: 6,195 Karma: 4800739 Join Date: Jan 2008 Location: Spaniard in Sweden Device: Cybook Orizon, Kobo Aura Do you actually need to explicitly duplicate the text? Can't you have a single copy of the text and \input it several times.
 08-12-2009, 06:49 AM #9 Jellby frumious Bandersnatch     Posts: 6,195 Karma: 4800739 Join Date: Jan 2008 Location: Spaniard in Sweden Device: Cybook Orizon, Kobo Aura By \input-ing the same text seven times and increasing some settings I could create this PDF (86830 pages, 170MB) with pdflatex. It could be larger, I guess. (source, 3.4MB)
 08-12-2009, 07:56 AM #10 ahi Wizard   Posts: 1,792 Karma: 507333 Join Date: May 2009 Device: none You are obviously right... I'm not sure why that didn't occur to me at the time. At what page limit does PDFLaTeX start to crap out on you? Perhaps that's what I should be working with for such large files... maybe XeLaTeX (unintuitively enough) has a lower maximum memory limit that PDFLaTeX. And how does the 86830 page monstrum work on your Cybook Gen3? - Ahi
08-12-2009, 08:44 AM   #11
Jellby
frumious Bandersnatch

Posts: 6,195
Karma: 4800739
Join Date: Jan 2008
Location: Spaniard in Sweden
Device: Cybook Orizon, Kobo Aura
Quote:
 Originally Posted by ahi At what page limit does PDFLaTeX start to crap out on you?
I don't know... and I'm not sure I want to try I had to raise max_strings, pool_size and save_size in /usr/share/texmf/web2c/texmf.cnf (I'm using tetex, it may be different in other TeX distributions), and it seems the .out file (for pdf bookmarks) was the most problematic part.

Quote:
 And how does the 86830 page monstrum work on your Cybook Gen3?
I don't know either, but I'm afraid it could crash (reboot), it does that with another largish (and not particularly complex) PDF I created. It could be different with the new firmware when it comes out

 08-12-2009, 08:59 AM #12 ahi Wizard   Posts: 1,792 Karma: 507333 Join Date: May 2009 Device: none Hmmm.... are you saying you generated the 80,000 page PDF with the default memory settings? I only got about 5,000 pages with those... can't even generate the 10,000 page (single Summa Theologica) file without raising those limits. Can you tell me more about the .out file stuff? Also, do you know how to prevent PDFLaTeX from generating any PDF bookmarks? For the Sony, with PDF links inside the content working, the PDF bookmarks are unnecessary and (I think) mainly responsible for the minutes of one-time initial load time. - Ahi
08-12-2009, 09:22 AM   #13
Jellby
frumious Bandersnatch

Posts: 6,195
Karma: 4800739
Join Date: Jan 2008
Location: Spaniard in Sweden
Device: Cybook Orizon, Kobo Aura
Quote:
 Originally Posted by ahi Hmmm.... are you saying you generated the 80,000 page PDF with the default memory settings? I only got about 5,000 pages with those... can't even generate the 10,000 page (single Summa Theologica) file without raising those limits.
Hmm... it depends what you mean with "memory settings". I had to change what I said above to generate the 80,000 page PDF, otherwise it aborts at around 60,000 pages, but nothing else, I didn't have to re-create formats or anything else. It may be that I had already modified something a couple of years ago, but I don't think so, this system's tetex should be pretty much out-of-the-box.

Quote:
 Can you tell me more about the .out file stuff? Also, do you know how to prevent PDFLaTeX from generating any PDF bookmarks?
The .out file is where hyperref stores the information for the bookmarks. I noticed that when pdflatex aborts, if I try to recompile, it then aborts again right at the beginning, when reading the .out file, that's why I think the stuff in the .out file might be one of the culprits.

To disable the bookmarks try passing bookmarks=false to the hyperref package.

08-12-2009, 09:35 AM   #14
ahi
Wizard

Posts: 1,792
Karma: 507333
Join Date: May 2009
Device: none
Quote:
 Originally Posted by Jellby Hmm... it depends what you mean with "memory settings". I had to change what I said above to generate the 80,000 page PDF, otherwise it aborts at around 60,000 pages, but nothing else, I didn't have to re-create formats or anything else. It may be that I had already modified something a couple of years ago, but I don't think so, this system's tetex should be pretty much out-of-the-box.

Quote:
 Originally Posted by Jellby The .out file is where hyperref stores the information for the bookmarks. I noticed that when pdflatex aborts, if I try to recompile, it then aborts again right at the beginning, when reading the .out file, that's why I think the stuff in the .out file might be one of the culprits. To disable the bookmarks try passing bookmarks=false to the hyperref package.
Thanks for that!

I don't suppose you want to go for the gold and try to make a 200,000 page version?

If not, I'll try my hand at it in the next day or two.

- Ahi

 08-12-2009, 09:41 AM #15 ahi Wizard   Posts: 1,792 Karma: 507333 Join Date: May 2009 Device: none In trying to think for some use for such mad-sized PDFs, I've been thinking of a KJV bible with appropriately linked bible commentary and strong's words. The internal linking would have to be pretty extensive and (while simple to use) pretty complicated... I think on each page, the verse numbers would have to be links to where the verse appears by itself, with commentary below and on subsequent pages, and each word that has a corresponding STRONG's entry also a link to said entry. Also probably a variety of navigation links (back to main portion, previous verse, next verse, et cetera). Do you think something like that would be doable, Jellby? - Ahi .mr-forums-btf-lastpost-in-sig-l-text-sm-height { width: 650px; height: 150px; margin-top: 10px; }