View Full Version : MobileRead newsletter - your input is needed!


Alexander Turcic
01-25-2007, 01:22 PM
As some of you know, we are planning to launch a newsletter for e-reader devices. In particular iLiad users might enjoy this because this newsletter should become available automatically via the iRex iDS. But of course we want to support other readers as well, such as the Sony Reader.

Yesterday I've started coding the necessary backend. It's going along fine, but now we need your help. Here are my questions and I would love to hear as many answers as possible:

1. What exactly should we include in the newsletter? Right now I've written the code to include all frontpage articles of one week, including all replies. A related question: how often should we publish? Daily? Weekly?

2. What is the best PDF format for your device? We've discussed PDF formatting ad nauseum, but I'd be happy if you could help me and just write it down here instead of me having to find out by scanning the older threads.

3. How should we make the newsletter available to disconnected device, such as the Sony Reader? Via a simple download or via e-mail subscription?

That would be all for now! ;)

Related:
Proposed MobileRead newsletter (http://www.mobileread.com/forums/showthread.php?t=8779), Whatever happened to MobileRead via iDS? (http://www.mobileread.com/forums/showthread.php?t=9445)

yvanleterrible
01-25-2007, 01:37 PM
I guess once a week.
We'd need all news, all new threads with the first entry and all threads with at least one new post that week.

If someone could write a little app. that would make it so when we are connected to the site it would prompt us to plug our reader, whatever it is, and write directly to it. Of course there are security issues. They have to be addressed. I would'nt want to have anyone write anything on my reader, and I would'nt want viruses.

It's a great idea Alex!

jęd
01-25-2007, 01:53 PM
Still not sure why the current RSS feeds are sufficient... Then anyone can use their favourite tool to convert it as it best to their device...

Hadrien
01-25-2007, 01:59 PM
I agree with jaed, RSS seems like a good choice. Anyone could still generate a PDF out of it with what we're implemeting on Feedbooks for example.

macanima
01-25-2007, 02:00 PM
The best PDF for me right now is 'not a PDF at all' - I've got an ETI-2. :/

Studio717
01-25-2007, 05:50 PM
I still only have Palms (Zire and Treo 650) or iPod (a 5G), so I'm not sure how useful my input will be. If there's a choice, I'd prefer RSS like now or via email. If some form of PDF, I'd appreciate it if it was Palm Adobe Reader-able.

Not sure what need you're trying to address that you don't already address here.

(Yes, I want an e-ink reader, but I'm waiting for 2G... and waiting... :dozey: )

Hadrien
01-25-2007, 06:41 PM
I still only have Palms (Zire and Treo 650) or iPod (a 5G), so I'm not sure how useful my input will be. If there's a choice, I'd prefer RSS like now or via email. If some form of PDF, I'd appreciate it if it was Palm Adobe Reader-able.

Not sure what need you're trying to address that you don't already address here.

(Yes, I want an e-ink reader, but I'm waiting for 2G... and waiting... :dozey: )

Well as long as the PDF is dynamically generated, it can be Palm friendly or adapted to anything else that supports PDF.

ojleblanc
01-25-2007, 08:15 PM
Hmm... I guess I have a different opinion... I'm pretty consumer tech savy, but the myriad of tools to convert docs into readable PDFs on the Sony Reader is really overwhelming. So, I would love it if MobileRead produced a newsletter that I could simply drag to my Sony Reader to view. Someone, I don't recall who, posted an issue of the Economist in PDF. It looked great on the Reader. I'll look for the thread and post here.

Also, I like the idea of an email subscription. That way, I don't have to remember to grab the newsletter.

RWood
01-25-2007, 09:52 PM
Given that you are going to produce a newsletter (if only because you can), then include all articles for the week even if they never made it to a splash presentation on the front page. Also, include all new and updated posts for the week.

If you want a daily version then include the new articles for that day along with associated threads. I would favor a version that had all new and updated threads for the day.

Now I know this could be generated on the fly as each person requests his or her own copy. My background is big iron (read mainframes) and the policy there would be to select a date change (like GMT) and preproduce the newsletter so all we have to do is to download it. This ensures that we don't miss an article if we download Wednesday's daily version 3 hours after we downloaded Tuesday's daily version or get the same article twice if we download a few hours earlier on the second day.

RSS is nice and great for those that like it. I never liked the process that much and dropped using it. For me the chance to read MR offline on the Sony Reader sounds like something I would enjoy. I could (and would) explore areas that I don't get into now.

Hadrien
01-26-2007, 02:22 AM
Hmm... I guess I have a different opinion... I'm pretty consumer tech savy, but the myriad of tools to convert docs into readable PDFs on the Sony Reader is really overwhelming. So, I would love it if MobileRead produced a newsletter that I could simply drag to my Sony Reader to view. Someone, I don't recall who, posted an issue of the Economist in PDF. It looked great on the Reader. I'll look for the thread and post here.

Also, I like the idea of an email subscription. That way, I don't have to remember to grab the newsletter.

I could do this with RSS, instead of using some tool, a simple URL would do the trick and format everything for your Sony Reader (or any other device).

Given that you are going to produce a newsletter (if only because you can), then include all articles for the week even if they never made it to a splash presentation on the front page. Also, include all new and updated posts for the week.

If you want a daily version then include the new articles for that day along with associated threads. I would favor a version that had all new and updated threads for the day.

Now I know this could be generated on the fly as each person requests his or her own copy. My background is big iron (read mainframes) and the policy there would be to select a date change (like GMT) and preproduce the newsletter so all we have to do is to download it. This ensures that we don't miss an article if we download Wednesday's daily version 3 hours after we downloaded Tuesday's daily version or get the same article twice if we download a few hours earlier on the second day.

RSS is nice and great for those that like it. I never liked the process that much and dropped using it. For me the chance to read MR offline on the Sony Reader sounds like something I would enjoy. I could (and would) explore areas that I don't get into now.

Well... I could store the last time you downloaded the PDF and then generate the new PDF only with articles published after this date. This way you generate the PDF whenever you want, and you're always up to date with all the news.
I don't think that preproducing is such a big issue, it would be easy to generate this in realtime...

Alexander Turcic
01-26-2007, 04:55 AM
A dedicated newsletter would be superior to RSS because it allows us to pull all kinds of content directly from the database. Example:

Newsletter:
MobileRead newsletter - your input is needed!
<reply 1>yvanleterrible said: ...
<reply 2>jęd said: ...
...

RSS:
MobileRead newsletter - your input is needed!
No replies.

Of course, you could try to write a program that would follow the RSS links and try to parse the replies via complicated regular expressions; but obviously this method is not as elegant as pulling the information directly from the database, plus the RSS + Regex method would cause more strain on our server.

A newsletter could also contain other information, sorted in groups. For instance, we could include a paragraph listing all new polls of the weeks. Another paragraph could mention all recently uploaded forum attachments.

And let's not forget iRex iDS. iRex offers us to host the newsletter via their iRex Delivery Service, meaning iLiad users can synchronize the newsletter directly upon connecting to iDS.

We'd try to keep the PDF as simple as possible from a formatting point of view. So no columns or other difficult structures. Even users with smaller screens, like PDAs, should be able to read them (assuming that they can read PDF). Of course, I am open to suggestion for different file formats. RTF perhaps?

So this is why I think a newsletter could be a lot of fun for most of us. Back to my original questions:

RWood mentioned two scenarios:
1. Weekly: all articles, including new and updated posts. Here, the newsletter could become quite huge which might be a problem. Plus, what do we do with an article whose posts span over several weeks? Should we just include the updated posts, meaning skip the old stuff? Take the Some suggestions for Software Updates (http://www.mobileread.com/forums/showthread.php?t=8014) article for example. Currently it has 178 replies. Assume we have two new posts by the end of this week. Should the newsletter only include the article's title plus the two newly added posts? What if these posts refer to earlier posts which were included in last week's newsletter or even before that?

2. Daily: all new and updated threads of the day. What about if you cannot download every day and hence occasionally miss a daily newsletter? Would updated threads still make sense if you missed the earlier updates and hence the posts?

I hesitate to generate the newsletter on the fly and customize it for each user. Firstly, it wouldn't work with iRex's iDS service. And secondly, it would generate considerable strain on the server.

This discussion is going very well.

yvanleterrible
01-26-2007, 09:44 AM
True about the amount of information, that could be daunting.

Since we already have an excellent email 'subscribed thread' notification, how about:
Complete weekly news, new threads of the week with the original post and a simple list of all updated thread titles. That would make up around 3 pages if you use smaller fonts and single line.

By the way I've printed some threads and there is no way to change the font size to take up less paper space.

A daily newsletter would be too much and some days there would be no news.

And maybe you could put up a 'On request' option for those who do not wish a regular feed.

jęd
01-26-2007, 10:29 AM
A dedicated newsletter would be superior to RSS because it allows us to pull all kinds of content directly from the database.

Yes... But I'm not sure what the newsletter would give that the current rss feeds do currently. It would be more elegant than html scraping, but what does it *add*...?

Alexander Turcic
01-26-2007, 10:38 AM
Yes... But I'm not sure what the newsletter would give that the current rss feeds do currently. It would be more elegant than html scraping, but what does it *add*...?
While our RSS feeds only display the first post of each article, the newsletter would display the entire thread with all replies.

jęd
01-26-2007, 10:57 AM
While our RSS feeds only display the first post of each article, the newsletter would display the entire thread with all replies.

And, in the rss feed, there's a link from the article to the discussion of that article...

Alexander Turcic
01-26-2007, 11:10 AM
And, in the rss feed, there's a link from the article to the discussion of that article...
... which isn't helpful for disconnected devices, is it? ;)

Well, I am open to all suggestions - and if the majority believes we don't need a newsletter, by all means, I am not trying to force it upon you :D

TadW
01-26-2007, 11:20 AM
RSS provides only what... the last 10 threads?

Alexander Turcic
01-31-2007, 03:56 AM
OK, attached you find a first demo of how the newsletter could look like. One is optimized for the Sony Reader, the other for the iRex iLiad. Bear in mind that I don't have an iLiad to try it out myself.

The following formats were used:

Font: Times (serif), non-embedded
Font-size: 12 (body), 10 (header)
Margins: 1.0mm to 2.0mm
Page dimensions: 3.57x4.82in (Sony Reader), 124x152mm (iRex iLiad)


This is your chance to influence how our future newsletter will look like ;) So comments are greatly appreciately. Please separate your comments on the content and the layout.

jęd
01-31-2007, 04:56 AM
From a quick flip through of the Iliad version...

* Paragraphing seems to have been lost on some stories
* Why use ^^ to denote links...? Just link the entire headline (this is what it used for the toc, why change the metaphor)?
* Some links are too wide for the page

Looks cool and compact, though...! :D

Alexander Turcic
01-31-2007, 05:04 AM
Paragraphing seems to have been lost on some stories
Can you elaborate? If you mean some specific formatting, like quotes, then yes, that is "by design" (to keep the file small and content non-redundant).

* Why use ^^ to denote links...? Just link the entire headline (this is what it used for the toc, why change the metaphor)?
Haven't thought about it - I think you're right ;)

Some links are too wide for the page
Yeah, nothing really that can be done here as htmldoc doesn't line-break long links. So they will overflow.

Looks cool and compact, though...! :D
Glad that you like it ;) More content will follow. Like, latest from the Wiki, latest polls, stats, you name it.

TadW
01-31-2007, 05:08 AM
Content: Cool! What exactly is included in the sample files? Only new threads of a week (including posts), or also updated threads that go back more than one week?

Layout: Looks fine on the Reader. Suggestion... how about inserting a page break after the end of each thread? This way every thread will start on top of a new page.

jęd
01-31-2007, 05:41 AM
Yeah, nothing really that can be done here as htmldoc doesn't line-break long links. So they will overflow.

Perhaps time to drop using htmldoc and use Latex as a constuction method...?

Alexander Turcic
01-31-2007, 05:48 AM
Perhaps time to drop using htmldoc and use Latex as a constuction method...?
I'd love to - but I need help. I have never done Latex :(

Hadrien
01-31-2007, 06:12 AM
I'd love to - but I need help. I have never done Latex :(

That's what we use on Feedbooks, I guess we could help you or host it, if you're doing LaTeX.

Alexander Turcic
01-31-2007, 07:39 AM
That's what we use on Feedbooks, I guess we could help you or host it, if you're doing LaTeX.
Hosting is not a problem. It's the markup - I've never bothered to look at LaTeX closer. Would it be a lot of pain to do it, and would it be worth it rather than using something restrictive like htmldoc?

Hadrien
01-31-2007, 08:07 AM
Hosting is not a problem. It's the markup - I've never bothered to look at LaTeX closer. Would it be a lot of pain to do it, and would it be worth it rather than using something restrictive like htmldoc?

Using LaTeX is really easy, it shouldn't be much of a problem. You could do a php file that would output the .tex file and then use pdflatex to transform it into a PDF.
So no, it shouldn't be a lot of pain in your case (mostly because you're dealing with a single output, the newsletter will always look the same), and it is much more powerfull than htmldoc.

Alexander Turcic
01-31-2007, 08:42 AM
Now I am intrigued ;) Hadrien, do you have some concrete hints how I should start with LaTeX->PDF? Like, what tools should I download, how should I start preparing the .tex file?

I really appreciate your help!

Hadrien
01-31-2007, 11:21 AM
Well, any LaTeX distribution should do the trick for you... You just need a bunch of basic packages and the pdflatex binary.
As for the beginning of your .tex file (Sony Reader here):
\documentclass[english,oneside]{article}
\usepackage[english]{babel} \usepackage[colorlinks=true,linkcolor=black,bookmarks=true,boo kmarksnumbered=true,hypertexnames=false,plainpages =false,pdfpagelabels]{hyperref}
\geometry{paperwidth=90mm,paperheight=120mm}
\geometry{left=1mm, right=1mm, bottom=1mm, top=1mm}
\begin{document}
...
\end{document}

Alexander Turcic
02-01-2007, 05:21 AM
Thanks for the pointer, Hadrien. I read a LaTeX primer last night in bed, and I am impressed about the quality.

While I think I will continue reading about LaTeX, I am not so sure if it's my best choice for the newsletter. The information I pull from our databases has markup (bbcode); I would have to write a bcode->latex conversion script first in order to use LaTeX. Will see if it justifies the work, that is, if you guys don't like how the current newsletter looks like.

Hadrien
02-01-2007, 08:34 AM
Thanks for the pointer, Hadrien. I read a LaTeX primer last night in bed, and I am impressed about the quality.

While I think I will continue reading about LaTeX, I am not so sure if it's my best choice for the newsletter. The information I pull from our databases has markup (bbcode); I would have to write a bcode->latex conversion script first in order to use LaTeX. Will see if it justifies the work, that is, if you guys don't like how the current newsletter looks like.

That's part of what i've been doing lately. We're adding features to Feedbooks using bbcode like markups, this way people can add quotes, verses, bold, italic, horizontal rule etc...

llasram
02-01-2007, 09:10 AM
While I think I will continue reading about LaTeX, I am not so sure if it's my best choice for the newsletter. The information I pull from our databases has markup (bbcode); I would have to write a bcode->latex conversion script first in order to use LaTeX. Will see if it justifies the work, that is, if you guys don't like how the current newsletter looks like.

I agree that LaTeX is the way to go if you have the time to put into writing the conversion scripts then tweaking the typesetting of each edition. If not, you might find that html2ps+ps2pdf gives better results than htmldoc. You provide html2ps a stlysheet -- it understands most of CSS2 -- so you can control the formatting more precisely, plus it can do justified paragraphs w/ hyphenation.

I did a quick-and-dirty run on the HTML in the main RSS feed so you can see if it maybe looks better, keeping in mind that many details of the look can be easily improved by modifying the applied stylesheet.

Alexander Turcic
02-01-2007, 09:23 AM
Looks very nice too, Ilasram!

html2ps+ps2pdf is this one, right? http://www.tufat.com/script19.htm

llasram
02-01-2007, 01:50 PM
Looks very nice too, Ilasram!

html2ps+ps2pdf is this one, right? http://www.tufat.com/script19.htmOh, interesting -- two people named nomething html2ps. :-) Looks like a PHP port? of the original Perl version. Haven't tried that one, but the web page makes it look like it doesn't support arbitray page sizes (!).

This is the html2ps I meant -- http://user.it.uu.se/~jan/html2ps.html . The ps2pdf script is a part of ghostscript (which I think you may need for html2ps anyway).

RWood
02-01-2007, 08:48 PM
Back to content and format of the newsletter for a moment. (I only reviewed the Sony version on the Reader.)

While the titles at the top are helpful, please brand each page (perhaps centered at the bottom) with the name "MobileRead Networks" and the date of publication.

The size of the base (text) font is fine even at the "S" setting. Maybe provide the headings in san serif and the body (text) in serif for more contrast between them.

I agree with the comment to remove the "^^" as it is not needed.

This is far more readable and useful than the trial version posted under the Sony Guidelines thread.

Alexander Turcic
02-02-2007, 03:46 AM
Back to content and format of the newsletter for a moment. (I only reviewed the Sony version on the Reader.)

While the titles at the top are helpful, please brand each page (perhaps centered at the bottom) with the name "MobileRead Networks" and the date of publication.

The size of the base (text) font is fine even at the "S" setting. Maybe provide the headings in san serif and the body (text) in serif for more contrast between them.

I agree with the comment to remove the "^^" as it is not needed.

This is far more readable and useful than the trial version posted under the Sony Guidelines thread.
Excellent suggestions, all of them. I'll play with html2ps this weekend and see what I then come up with.

Hadrien
02-02-2007, 05:17 AM
Using both html2ps and ps2pdf should be a quick way to generate something yes... I'm working on something where I'm totally in control of the .tex file though, I'd rather have limited support first for HTML/CSS but have much more complex control over how everything looks like.

Alexander Turcic
02-05-2007, 08:35 AM
llasram, I used html2ps over the weekend and I think it's the best solution for the newsletter. Compared to LaTeX, it's much easier to use with the information we pull from the database. Compared to htmldoc, it has more options (with limited CSS) to tweak the output. Alas, it's also considerably slower since it's in Perl and not in C code.

Anyways, I have a problem with the page size. No matter what I enter in my config -- either under @html2ps in the paper block, or under @page -- the page size of the output is 215,9x279.4mm. Could you show me the settings you used for rss-feed.pdf?

Alexander Turcic
02-05-2007, 09:57 AM
OK, I think I got it... I have to pass the page size also to ghostscript (ps2pdf). Still, if you could share your config file, I'd be very happy ;)

yvanleterrible
02-05-2007, 10:10 AM
I've just tried the first pdf, it's great! I won't have to spend my time on line to study everything!

Alexander Turcic
02-05-2007, 11:34 AM
Thanks yvan. It's a starter, but there is still room for improvement, and I received a lot of useful commentary already how to make it better.

Alexander Turcic
02-06-2007, 06:03 AM
Hm, unfortunately html2ps has a serious bug that doesn't allow me to use headers/footers. It seems you need to add a relatively big margin, like 2cm, for headers and footers to appear (no matter the font size) - which is obviously too much screen asset to give away for our purposes.

yvanleterrible
02-06-2007, 08:59 AM
Hey Alex! I spent a good half hour reading that newsletter yesterday evening. You gave me a bypass around the 'No computer after 1900h' rule we have. :happy2:

Funny thing I should say this with the old eyes I have, but you should reduce the font size a bit and put less empty lines.

Having the site as a newsletter like this gave me the time and patience to thoroughly read and understand the techno stuff I usually skip.

Alexander Turcic
02-06-2007, 09:01 AM
yvan, I am glad you like it! We will definitely have our newsletter soon - I am just evaluating the various suitable options to convert to PDF. Seems like there is no perfect solution ;(

Alexander Turcic
02-07-2007, 08:53 AM
Hm, unfortunately html2ps has a serious bug that doesn't allow me to use headers/footers. It seems you need to add a relatively big margin, like 2cm, for headers and footers to appear (no matter the font size) - which is obviously too much screen asset to give away for our purposes.
Fixed it. html2ps had a 40pt margin hard-coded for the header + footer, that's ON TOP of the actual margin settings you define. Will tweak it some more, than post the results here.

Alexander Turcic
02-09-2007, 03:13 PM
OK, I am back to htmldoc (and possibly LaTeX in the future - when I have more free time to invest). With html2ps I found the following limitations:


hyperlinks converted with html2ps and gs (or ps2pdf) are not working in the PDF viewer of the Sony Reader.
very hard to read code, makes it difficult to customize it

On the other hand, htmldoc suffers from very limited styling capabilities and from the fact, that long links or words get truncated at the end of a line. Also, html2ps supports auto-hypenation, which is quite nice, but not supported by htmldoc. Still, the fact that internal hyperlinks with html2ps didn't seem work for the Reader was a killer for me.

Here I have a new version of the newsletter for you to try. This time I added more text formatting and also a few small tweaks as suggested by others in this thread. Inline images *are* included, and I am not sure if we should do this in the final version of the newsletter, as PDF files with images tend to become quite big. Here are the download links:

Sony Reader version: http://www.mobileread.com/newsletter-sony.pdf
iLiad version: http://www.mobileread.com/newsletter-iliad.pdf

Comments are appreciated!

llasram
02-09-2007, 06:27 PM
Still, the fact that internal hyperlinks with html2ps didn't seem work for the Reader was a killer for me.That's rather odd... I was just able to use html2ps + ps2pdf to generate a PDF where internal hyperlinks do work. Could you post / send the HTML used to to generate the non-working PDF?

llasram
02-09-2007, 06:33 PM
OK, I think I got it... I have to pass the page size also to ghostscript (ps2pdf). Still, if you could share your config file, I'd be very happy ;)Perhaps a bit late, but attached. And yes... I forgot to mention the size issue -- I have just gotten used to adding -g2500x3227 to most Ghostscript commands :-).