My Manga Program

[jeffcobb]

Originally Posted by lilman

Thanks for answering that question. Here is a list of unrar executables that Canti looks for on a Mac:

/usr/bin/rar
/usr/bin/unrar
/usr/bin/unrar-free
/usr/local/bin/rar/rar

I don't have a Mac so I've never been able to test this myself. Please let me know if there is another unrar executable for Mac and I will add the info to Canti.


I've received requests from others for a GUI as well. I will try to get, at the very least, an extremely basic GUI out. At the moment I am doing research for another project and also working on speeding up Canti (been working on the speed up for a while... very tricky), so it may be some time before a GUI comes out.

For now, to run Canti in Windows you need to:
1) Download the latest version of Canti
2) Unzip it (will give you Canti.jar and readme.txt)
3) Follow the instructions in the readme.txt file. It will explain how to make a default settings file for your ereader device and then how to run Canti.

I am kinda in that same spot on my FOSS app; it works great at the cmd line (which is great for running multiple instances on headless boxes and a GUI would just kind of mess up), have had a few GUI requests but I put that task down below getting it working as best it can before worrying about a GUI. Does that limit my "market" or target audience? Probably but instead of writing it for other people, I wrote it because I needed something and just made the source available to others if they want and have the resources to use it...

[lilman]

Originally Posted by jeffcobb

I just found this on a search here; I was all ready to write essentially the same thing for Android. Curious: how did your PDF conversions work out? I have a buttload of Marvel stuff in that format from when they were selling those DVDs with like 600 issues of Spiderman, 500 Ironman, etc. I also have a lot of Battle Angel Alita and Rurouni Kenshin stuff like that I want to convert for Nook.

Pdfs work just fine. Pdf processing, both as input (pdf -> image folder) and output (image folder -> pdf), is handled by the oh-so-awesome ImageMagick. Pdf as input was a bit harder than as output. I believe pdf is an open format so I could have written my handler from a tech doc (which is what I did for epub), but ImageMagick does a good job so I never bothered.

Originally Posted by jeffcobb

I don't do Windows but can fire up a VM if need be...

No Windows required, Canti was written in Java so it runs on all major os's. ImageMagick also has install binaries for most os's.

Originally Posted by jeffcobb

I am kinda in that same spot on my FOSS app; it works great at the cmd line (which is great for running multiple instances on headless boxes and a GUI would just kind of mess up), have had a few GUI requests but I put that task down below getting it working as best it can before worrying about a GUI. Does that limit my "market" or target audience? Probably but instead of writing it for other people, I wrote it because I needed something and just made the source available to others if they want and have the resources to use it...

I have a similar background. I wrote Canti for my own use, and when it was done I thought "hey, might as well post it online so that others who have a similar need but aren't programmers can enjoy it." For personal use I don't want a GUI, it will just slow me down both in development time and run time. If I make a gui, it will be to gain the experience. I don't have a target audience... my view is use Canti if you like it or don't.

[Manuel7]

Hi,
I have a kindle 3 and I have a problem with the pdf file. The correct size page its only working in the first and last page. In the middle, the pages are like the double correct size.

I'm using width 800 and height to -1 for landscape view. In the kindle i have to set the zoom to "actual size" so i can do a correct vertical pan (scroll down) of the image. These work correct for the first and last page, but for the middle pages are like 200% zoom or more.

I look the images that your program generate and are in the correct size (800x12xx).

I'm using windows 7 64bits in Spanish (metric system).

Here is the pdf file: http://www.2shared.com/file/eQnnjRtY...77_kindle.html
and here is my properties file: http://www.2shared.com/document/3RGP5ZA-/Canti.html
(click in the small text on the bottom: "Save file to your PC: click here", dont click those big ugly spam buttons)

thanks for your help.

[jeffcobb]

Amen. Sorry if I did not read far enough into the docs, deathmarching ATM. Query; I see the formats it looks like you convert *from* but what is the best default viewing format for such material? I have found that stuff that starts with color and stays color seems fine but if you force some of it to gray-scale it can be painful on the eyes. I may still rev an Android version of this or something like it. As you say "for the experience". I have done GUIs before but don't expect this to need much of a GUI. As for my FOSS app being command-line, it works for me b/c my app is (essentially) a smart-video ripper so with a basic home network, a pile of xterms I can rip and label a whole season of a series in an hour. GUI would make this not-as-easy.

[lilman]

Originally Posted by jeffcobb

Query; I see the formats it looks like you convert *from* but what is the best default viewing format for such material? I have found that stuff that starts with color and stays color seems fine but if you force some of it to gray-scale it can be painful on the eyes.

Sorry, I don't understand your question. Could you rephrase it?

Originally Posted by jeffcobb

I may still rev an Android version of this or something like it. As you say "for the experience". I have done GUIs before but don't expect this to need much of a GUI.

You want a comic processor running on an android device? Some image processing ops are rather heavy, not to mention you would probably want to sync back processed images in case of data loss (I've had to resync my processed comic collection to my iPad a bunch of times since sometimes my comic viewer required a reinstall which would wipe out all the comics). I would love to write my own viewer app though.

Originally Posted by jeffcobb

As for my FOSS app being command-line, it works for me b/c my app is (essentially) a smart-video ripper so with a basic home network, a pile of xterms I can rip and label a whole season of a series in an hour. GUI would make this not-as-easy.

I view GUI the same way. I think if I make one it will be separate from the main app and will simply act like it is making a command line call. You could still handle multiple calls using a queue that could be saved (similar to a shell/batch script).

[lilman]

Here's Canti v1.90

Note: Rewriting Canti's brain was a massive undertaking. It took me over a month just to figure out the logic behind it, let alone having to actually program it. Since so much code as changed there may be some new bugs, so please let me know if you encounter any problems.
Note 2: You may have noticed the previous version of Canti was v1.87, and here we have v1.90. I didn't release the in-between versions since they were still works-in-progress towards Canti's new brain.

New to version 1.90:
-New brain
I rewrote most of the brain code for Canti (brain code is the main logic of the program... it makes the decisions on what work to do).
The purpose of the new brain is to do the bare minimum of work, causing the program to run faster.
The biggest speedup is that image folder restoration (both in -parent_folder and -output_folder) is only performed as needed, as opposed to the previous versions of Canti which would restore everything in -parent_folder and -output_folder.
-Include unsplit lanscape scan along with two page splits
If you enabled @auto_split_landscape_scans, then @auto_split_keep_unsplit_scans determines if Canti should include the full landscape scan even when it is split.
This is useful if you want to be able to see the two-page view to get a sense of the full image and still get the benefit of reading individual split pages.
If enabled, the page ordering of landscape split scans will be: unsplit landscape scan, first split page, second split page
This is set up in the .properties file.
-General bug fixes

Enjoy and let me know if it works for you

[squall_dc]

Hi lilman,

Just got a kindle 3 and was trying out different manga conversion programs (mangle and calibre) and was about to write my own since they didn't do everything I want, when I happened to find Canti which nearly does everything.

Great program! Really love the auto splitting and the new option to include the unsplit-landscape image. The auto compressing to cbz is really awesome too.

I just have a few issues:

- Is there an option to add white borders to an image to fill it to the target size (like on Mangle, so the image is always eg 600x800)?

- For the landscape splitting, would it be easy to add an option to extend the width by a user specified amount (ie. a safe zone). So eg if the landscape image is 1000 x 800, and the auto split detection finds a split at 400, could the resultant images be (0, 0) -> (400 + safezone, 800) (1st image) and (400-safezone, 0) -> (1000, 800) (2nd image).
ie. there will be some overlap in both images

- For the suffix, I get a crash if I turn on "use_original_filenames". If I don't turn it on, I don't get a crash, but the folder names are broken.

eg. C:\My Manga Vol 1\Chapter 01
C:\My Manga Vol 1\Chapter 02
C:\My Manga Vol 1\Chapter 03a
C:\My Manga Vol 1\Chapter 03b

Ends up like:

My_Manga_-_Ch
My_Manga_-_Ch.1
My_Manga_-_Ch.2

Crash mentioned above:
Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String ind
ex out of range: -1
at java.lang.String.substring(Unknown Source)
at java.lang.String.substring(Unknown Source)
at manga.CMFileObject.<init>(CMFileObject.java:49)
at manga.ConvertManga.main(ConvertManga.java:1397)

- also I can't find the "bind.txt" file mentioned in the readme.txt.

- Not sure if it's Canti or not, but while converting to png, Image Magik sometimes randomly crashes (always on a landscape image), if I delete the temp files and run the conversion again, sometimes it works, but mostly still crashes.

Sorry for the long post, and Thanks for the wonderful program. Currently I'm using Canti initially and then Mangle to do the final conversion (which add's borders). But would love to use Canti alone.

[jozicka]

please help me with command line (I'm lost a little bit)

I have on d:\temp2\asterix.cbr
I have installed d:\ImageMagick
I have on d:\Canti.properties for K3

I would like to create d:\temp3\asterix.cbr (or directly pdf) , which will be auto splitted and in correct size 800*600.

Can please somebody write the correct command line for this task?

[craigg]

So...this is unix only?
As mangle seems a bit naff for kindle 3 and an alternative is needed.
IThe instructions for this seem to be unix based though.

[badbob001]

My K3 hasn't arrived yet but I wanted to try out the latest Canti to provide feedback. I'm using this on Windows XP 32bit.

The component that deals with zip/cbz seems to be somehow faulty if the file contains folders within. resulting with:
java.io.FileNotFoundException.
(The system cannot find the path specified)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(Unknown Source)
at java.io.FileOutputStream.<init>(Unknown Source)
at common.utils.FileUtils.unzipFile(FileUtils.java:51 9)
at manga.ConvertManga.main(ConvertManga.java:1314)

If I recompress the pages into a rar or a zip with no subfolders, then it works fine. It looks like when canti is processing a file, it extracts the content to a temp folder in the same place as the original file. When it processes a rar file with a subfolder, I see the subfolder within the temp folder. But when it processes a zip file with a subfolder, the temp folder contains a file with the same name as the subfolder, making me think the program is incorrectly trying to extract the subfolder as a file instead of a folder. Perhaps a quick workaround is to force the use of unrar to proccess zips.

While testing, I took the same set of pages and created a zip and a rar with different file names. After I ran canti against the parent folder, I was expecting it to result with two separate PDFs with the same content. Instead, I only got one PDF. Could it be that pages with the same filename will overwrite each other? I think it may be caused by the app putting every page under the same 'title'/folder. Perhaps once you get the 'use_original_filenames' option fixed, this conflict will be avoided.

What is the purpose of the max_dpi setting and what factors does it affect? We already know the input and output resolution, so dpi seems unnecessary. Is it perhaps relevant for PDF generation? With my output set to be 800x-1, displaying the PDF 100% shows the graphics expanded well over its 1:1 pixel resolution, which is wrong to me. Would changing the DPI setting affect the sharpness of PDF pages?

When processing a zip with about 250 images, my computer slowed to a crawl as convert.exe used more than 2GB of ram, most of which involved the page file since I only have 2GB of real ram. Is Image Magick trying to process all the files within memory at the same time? It may be less resource heavy to have Image Magick process pages in batches.

Can you limit the cropper to not crop the image to less than the specified output resolution? See attached 2-page spread as what can cause problems for Canti. I'm using 800x-1 so canti will only know the resolution for one side. Perhaps a hack like: 800x-600, which means that negative dimensions are ignored unless in special cases.

There is also an issue with pages that are smaller than the target resolution. One manga had an opening credit page that is originally 700x500. Canti rotated and cropped it and the resulting result was 306x445 (rotated). The sharp size reduction is due to the page being mostly black around the borders. The resulting image file looks fine but is a blurry mess when viewed from within the PDF, since the PDF viewer will automatically expand the page to fill the screen. For these small pages, perhaps you can use my previous idea to limit the crop and then use squall_dc's idea of filling in the borders to make the image meet the target resolution.

The readme file says in step 3: java -jar Canti.jar -create_device [device_name].
It should be: java -jar Canti.jar -create_properties [device_name].

Thanks!

[lilman]

Note: There were a few people asking questions above this post, so I will answer each person's questions in a separate post.

Originally Posted by squall_dc

I just have a few issues:

- Is there an option to add white borders to an image to fill it to the target size (like on Mangle, so the image is always eg 600x800)?

That option currently does not exist, but luckily I wrote the code to add borders to an image months ago (I just never used it). I will add this option in the next release.

Originally Posted by squall_dc

- For the landscape splitting, would it be easy to add an option to extend the width by a user specified amount (ie. a safe zone). So eg if the landscape image is 1000 x 800, and the auto split detection finds a split at 400, could the resultant images be (0, 0) -> (400 + safezone, 800) (1st image) and (400-safezone, 0) -> (1000, 800) (2nd image).
ie. there will be some overlap in both images

Yes, adding in that option would be easy. I will include it in the next update.

Originally Posted by squall_dc

- For the suffix, I get a crash if I turn on "use_original_filenames". If I don't turn it on, I don't get a crash, but the folder names are broken.

eg. C:\My Manga Vol 1\Chapter 01
C:\My Manga Vol 1\Chapter 02
C:\My Manga Vol 1\Chapter 03a
C:\My Manga Vol 1\Chapter 03b

Ends up like:

My_Manga_-_Ch
My_Manga_-_Ch.1
My_Manga_-_Ch.2

Crash mentioned above:
Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: -1
at java.lang.String.substring(Unknown Source)
at java.lang.String.substring(Unknown Source)
at manga.CMFileObject.<init>(CMFileObject.java:49)
at manga.ConvertManga.main(ConvertManga.java:1397)

I will look into use_original_filenames crashing. Should be a simple bug to fix.

As for the auto sense suffix not naming things right, that's because it is only designed to handle numbers at the end of a filename. That's why it could correctly handle "01"->"1" and "02"->"2", but when it saw something like "03b" it stopped trying to figure out the suffix as soon as it saw the "b". In your case you will want to use use_original_filenames, so I'll make sure to get it working properly.

Originally Posted by squall_dc

- also I can't find the "bind.txt" file mentioned in the readme.txt.

The bind.txt file is something you make yourself and place into parent_folder. The instructions explaining how to make one is included in the readme.

Originally Posted by squall_dc

- Not sure if it's Canti or not, but while converting to png, Image Magik sometimes randomly crashes (always on a landscape image), if I delete the temp files and run the conversion again, sometimes it works, but mostly still crashes.

Actual image processing is handled by ImageMagick, so the problem shouldn't be Canti. I don't know what to tell you... maybe convert to another image format?

Originally Posted by squall_dc

Sorry for the long post, and Thanks for the wonderful program. Currently I'm using Canti initially and then Mangle to do the final conversion (which add's borders). But would love to use Canti alone.

You're welcome. With the next update you should be able to use Canti alone

[lilman]

Originally Posted by jozicka

please help me with command line (I'm lost a little bit)

I have on d:\temp2\asterix.cbr
I have installed d:\ImageMagick
I have on d:\Canti.properties for K3

I would like to create d:\temp3\asterix.cbr (or directly pdf) , which will be auto splitted and in correct size 800*600.

Can please somebody write the correct command line for this task?

So you have Canti.jar and Canti.properties in d:\. First read through Canti.properties and make any changes that you want (this is where you decide the image width and height, what formats to output to, etc.). Then open a command line to d:\ and run:

java -jar Canti.jar -parent_folder D:\\temp2\\asterisx.cbr\\ -title "Whatever the Title Is" -output_folder D:\\temp3\\

Oh, and make sure you have a rar handler installed on your system (see the readme for more information).

[lilman]

Originally Posted by craigg

So...this is unix only?
As mangle seems a bit naff for kindle 3 and an alternative is needed.
IThe instructions for this seem to be unix based though.

Canti is written in Java so it runs on any platform (Windows, Mac, Linux).

[lilman]

Originally Posted by badbob001

My K3 hasn't arrived yet but I wanted to try out the latest Canti to provide feedback. I'm using this on Windows XP 32bit.

The component that deals with zip/cbz seems to be somehow faulty if the file contains folders within. resulting with:
java.io.FileNotFoundException.
(The system cannot find the path specified)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(Unknown Source)
at java.io.FileOutputStream.<init>(Unknown Source)
at common.utils.FileUtils.unzipFile(FileUtils.java:51 9)
at manga.ConvertManga.main(ConvertManga.java:1314)

If I recompress the pages into a rar or a zip with no subfolders, then it works fine. It looks like when canti is processing a file, it extracts the content to a temp folder in the same place as the original file. When it processes a rar file with a subfolder, I see the subfolder within the temp folder. But when it processes a zip file with a subfolder, the temp folder contains a file with the same name as the subfolder, making me think the program is incorrectly trying to extract the subfolder as a file instead of a folder. Perhaps a quick workaround is to force the use of unrar to proccess zips.

I will look into unzip not working properly with subfolders.

Originally Posted by badbob001

While testing, I took the same set of pages and created a zip and a rar with different file names. After I ran canti against the parent folder, I was expecting it to result with two separate PDFs with the same content. Instead, I only got one PDF. Could it be that pages with the same filename will overwrite each other? I think it may be caused by the app putting every page under the same 'title'/folder. Perhaps once you get the 'use_original_filenames' option fixed, this conflict will be avoided.

Can you post the names of the zip and rar files you used that resulted in a single pdf. I'm guessing they had the same auto sensed suffix.

Originally Posted by badbob001

What is the purpose of the max_dpi setting and what factors does it affect? We already know the input and output resolution, so dpi seems unnecessary. Is it perhaps relevant for PDF generation? With my output set to be 800x-1, displaying the PDF 100% shows the graphics expanded well over its 1:1 pixel resolution, which is wrong to me. Would changing the DPI setting affect the sharpness of PDF pages?

max_dpi is the maximum allowable dpi for output images. If an image has a dpi greater than max_dpi, it is processed to max_dpi; if an image has a dpi lower than max_dpi, no dpi processing is performed (since increasing an image's dpi uses pixel estimation, quality would be no better). max_dpi reflects the maximum dpi of the particular ereader you are processing manga for. Placing an image on an ereader with a dpi greater than the ereader screen can support does not make the image look any better, and at the same time dpi directly relates to file size so you are wasting file space without any gain in quality.

Originally Posted by badbob001

When processing a zip with about 250 images, my computer slowed to a crawl as convert.exe used more than 2GB of ram, most of which involved the page file since I only have 2GB of real ram. Is Image Magick trying to process all the files within memory at the same time? It may be less resource heavy to have Image Magick process pages in batches.

ImageMagick processes each image individually, except for pdf creation. With pdf creation, ImageMagick has to deal with all of the images that will become part of the pdf at the same time. For larger pdfs (greater than 150 images) this can be a real resource hog, and can possibly even fail. I'm guessing that is what is slowing down your system... I can't think of any other Canti operation that involves dealing with more than one image at a time. I purposely did not program multi-threads in Canti for fear of insufficient resources.

Originally Posted by badbob001

Can you limit the cropper to not crop the image to less than the specified output resolution? See attached 2-page spread as what can cause problems for Canti. I'm using 800x-1 so canti will only know the resolution for one side. Perhaps a hack like: 800x-600, which means that negative dimensions are ignored unless in special cases.

Um, I'll have to think about that. Somebody else requested an option for adding borders to an image that is smaller than the specified size, I will probably go with that.

Originally Posted by badbob001

There is also an issue with pages that are smaller than the target resolution. One manga had an opening credit page that is originally 700x500. Canti rotated and cropped it and the resulting result was 306x445 (rotated). The sharp size reduction is due to the page being mostly black around the borders. The resulting image file looks fine but is a blurry mess when viewed from within the PDF, since the PDF viewer will automatically expand the page to fill the screen. For these small pages, perhaps you can use my previous idea to limit the crop and then use squall_dc's idea of filling in the borders to make the image meet the target resolution.

I will program the add border option... that should fix this problem.

Originally Posted by badbob001

The readme file says in step 3: java -jar Canti.jar -create_device [device_name].
It should be: java -jar Canti.jar -create_properties [device_name].

Nice catch, I'll fix that.

[jozicka]

Originally Posted by lilman

First read through Canti.properties and make any changes that you want (this is where you decide the image width and height, what formats to output to, etc.). .

Thank you very much for help, so the auto-split option to single frames from one page is setup in canti.properties?