Selective binarization and re-encoding program for use with KoReader

[embryo]

The app themes are great!!
Great improvement.
Really appreciate it..🙏

Originally Posted by LegeApps

The adaptive binarization default value change was really the big fix. I use fixed threshold a lot because I know the good values to pick (180 for dark color pages, 128 for light color pages) but I dont expect users to know that, and adaptive binarization is a more complex process that has its place.

For the Fixed Threshold and also for the Custom Adaptive values, you're right. The user can't know what is what (Custom adaptive doesn't even have a max/min value at the tooltip)..
So, the solution I propose is a drop-down menu (or some buttons if the GUI lib does not provide one), with 3-4 presets like "Cleaner text - Balanced - Cleaner background, or whatever the outcome is likely to be.
You know better than us what the values do, so you can name them in a helpful for the user way..

Not sure what you mean. either you saw it using it in the CLI or it doesnt. The GPU is doing all it can do, I can guarantee that at this point. You should look at the code or ask a model to explain whats happening.

rendering=cpu via pdfium
resize=gpu where it helps
inference=gpu
binarization=gpu
encoding=cpu

This has all been nearly completely optimized.

I use GPU-Z to monitor the GPU's processing, memory, power draw, etc., but you can see it if you look at the Windows task manager (Performance>Cuda).
95% of the processing is for Preparing Pipeline / Initializing.
At this stage only 1 CPU thread is working (I have 8 cores - 16 threads). No GPU work at all.
After that its the Render | Infer | Encode stage that takes almost no time at all.
At this stage I see some spikes of GPU processing (only for some ms) and then it ends.
This is what I mean by "overall the processing was done at the CPU".
A great improvement would be if this was utilizing all the available threads, but maybe the library doesn't support this.

That's not how the adaptive option works, it's a completely different binarizatio method than fixed threshold; it uses a fusion of Otsu and Sauvola which are two well known page binarization techniques. The "heavy model" is also Sauvola, but more intensive. I do not think implementing what you suggest would be coherent in code. What is the effect you are seeing in outputs that makes you want a feature like that?

As I said, I had to use the Fixed threshold because of the false letters that the adaptive was detecting.
I do thing that an adaptive mode would work better in a book with text and images of different brightness, since it will treat every page differently, but it might need some fine tuning like the book I mentioned.
If there is no such option, let's forget it..

I did it in 1.4.61, I added dark modes based on color palettes I always wanted to add, from a Sanzo Wada color book. check them out in the new release. I think you'll like it.

No nitpicking is what im here for, keep it up, as long as it's coherent nitpicking that I can act on. Thank you for your nitpicking.

OK then, more problems.

- Cropping is not working again.
It crops wherever it likes.
Turning the Layout detection Off, improves things sometimes, but not always.
I do recall that you said that you're gone set Free Aspect as default.
Well, I don't thing it is.
Even in the cases that cropping is almost working, all pages have almost the same size, even the ones that don't need to (a lot of blank space leftover)

- This is the taskbar icon of the app

and this is the app's window icon at dark themed Windows

It is almost impossible to be seen.
What I usually do for that, is to add a 1-pixel white (or light gray) runaround that is invisible at a light background, but shows the image if the background is dark..

[LegeApps]

https://github.com/LegeApp/Lege/rele...Lege-win64.zip

Originally Posted by embryo

The app themes are great!!
Great improvement.
Really appreciate it..��

Great..

For the Fixed Threshold and also for the Custom Adaptive values, you're right. The user can't know what is what (Custom adaptive doesn't even have a max/min value at the tooltip)..
So, the solution I propose is a drop-down menu (or some buttons if the GUI lib does not provide one), with 3-4 presets like "Cleaner text - Balanced - Cleaner background, or whatever the outcome is likely to be.
You know better than us what the values do, so you can name them in a helpful for the user way..

In the documentation towards the end i explain what the options that I think are best for any given job..documentation can be seen from about page. it's been there since nearly the beginning of the program. I don't want to clutter the GUI with that stuff.

I use GPU-Z to monitor the GPU's processing, memory, power draw, etc., but you can see it if you look at the Windows task manager (Performance>Cuda).
95% of the processing is for Preparing Pipeline / Initializing.
At this stage only 1 CPU thread is working (I have 8 cores - 16 threads). No GPU work at all.
After that its the Render | Infer | Encode stage that takes almost no time at all.
At this stage I see some spikes of GPU processing (only for some ms) and then it ends.
This is what I mean by "overall the processing was done at the CPU".
A great improvement would be if this was utilizing all the available threads, but maybe the library doesn't support this.

My custom GPU library is very high performance and doesn't need to use much GPU. I can guarantee that it is using your GPU if you have one. Just stick to reporting what you see versus what you would like to see, rather than what you think the code does, in the future, with any feedback reporting..i've spent so much time on multithreading. I'll indulge you for a second..:

pdfium - not threadsafe. behind a mutex. cannot be parallelized.
my custom onnx runtime can but if pdfium can't, it doesn't matter, everything downstream depends on pdfium. in PNG folder mode, the multithreading works. binarizing and encoding always fan out to threads.

As I said, I had to use the Fixed threshold because of the false letters that the adaptive was detecting.
I do thing that an adaptive mode would work better in a book with text and images of different brightness, since it will treat every page differently, but it might need some fine tuning like the book I mentioned.
If there is no such option, let's forget it..

It already works per page, it never gets an entire book average. Why would a book have different brightness per page anyway? This just seems so complex for a user to use in the GUI that I have. Might as well use Scantailor if you need to do that.

OK then, more problems.
- Cropping is not working again.
It crops wherever it likes.
Turning the Layout detection Off, improves things sometimes, but not always.
I do recall that you said that you're gone set Free Aspect as default.
Well, I don't thing it is.
Even in the cases that cropping is almost working, all pages have almost the same size, even the ones that don't need to (a lot of blank space leftover)

Fixed in 1.4.63. Thank you for the bug report. It was working on my test files. It works on even more now.

- This is the taskbar icon of the app

and this is the app's window icon at dark themed Windows

It is almost impossible to be seen.
What I usually do for that, is to add a 1-pixel white (or light gray) runaround that is invisible at a light background, but shows the image if the background is dark..

I added a change for this, but if it doesn't show up, whatever.

But the big change this patch besides the backend stuff is grayscale mode, which should further address the "bad photocopy" complaint (i don't see it that way but thats ok). now there is an MRC based grayscale mode. It cleans the page via a different method than binarizing, then overlays a JBIG2 mask on top of the grayscale, so that the file size is only 20% larger than binarized output, and you get aliased text that looks better. It's not perfect but it's pretty good. Dare anyone to find a better one in a free program. I hope this will be the last version for awhile but something always comes up.

[embryo]

Just downloaded the latest GitHub release, but it's not 1.4.63. The About box says 1.4.62.
- One of my sample file doesn't get converted at all (used to get converted just fine)
- Cropping is still non-usable
.. didn't check more ..
I'll wait for an updated version..

[LegeApps]

Originally Posted by embryo

Just downloaded the latest GitHub release, but it's not 1.4.63. The About box says 1.4.62.
- One of my sample file doesn't get converted at all (used to get converted just fine)
- Cropping is still non-usable
.. didn't check more ..
I'll wait for an updated version..

Hi thanks for this update, although it's vague and more like a complaint than a bug report. I already finished an update to cropping with agent help, this will be in the next release, which will be 1.4.7, since there is going to be a major improvement to the pipeline in it.

It turns out that existing PDF renderers, including pdfium which Lege uses, are garbage [july 22 edit- it's not garbage, just began from obsolete architecture and never changed]. I believed they were single threaded by nature for some fundamental reason but they aren't. with fable and codex help I have created a stateless PDF renderer, first of its kind, that is currently rendering up to 10x faster than pdfium (this will not hold true in production due to thread contention). It was 2x faster without any optimization at all. This also allows a much better pdf viewer to be created, since every single PDF viewer besides Adobe Acrobat uses pdfium or poppler under the hood. So I am doing that in conjunction. Never been done before. Real things happening. Not just slop over here folks. I've got a 15,000 strong PDF library for regression tests and it isn't failing on a single one now.

Also the PDf writer has been replaced with custom code, so after the renderer is in, Lege will be custom end to end. Thanks to this year's models. May have been possible last year to some extent and I just wasn't asking the right questions.

Sorry for the more glaring bugs but they should be largely fixed now. I am doing my best to work it all out in the service of something even better and more advanced.

[LegeApps]

Originally Posted by embryo

Just downloaded the latest GitHub release, but it's not 1.4.63. The About box says 1.4.62.
- One of my sample file doesn't get converted at all (used to get converted just fine)
- Cropping is still non-usable
.. didn't check more ..
I'll wait for an updated version..

Hey, so 1.4.7 is finished, and it's so fast that I don't think I even want to release it for free, or at least not the performance version. But since you helped me bug fix it, I want to send you the fast version compiled for windows. Smaller lighter and faster with working cropping compared to the version you have. send an email to read@legeapp.com with your contact info. thanks.

Will make a decision otherwise about the rest later. Program no longer opensource since I now have the fastest PDF renderer in existence (8x+ faster than MUPDF with more coverage). - 7/28 edit - fastest open source renderer, tested and proven claim. not sure about closed source commercial editors. i have seen a few that might be competitive

[LegeApps]

Hi all, the newest releases, with the new renderer are now available on the Releases page. Version 1.4.73. It is the fastest version, and thanks to embryo for his bug testing while I figured my stuff out.

https://github.com/LegeApp/Lege/releases

[shuvashish76]

The project is now proprietary?

[LegeApps]

For the 8 months that it was opensource, I got 0 contributions, 0 pull requests, people just reported bugs for me to fix, so yes, it is closed source now. There is no benefit to me or to the project for it to be opensource now. edit - jp2lam, jbig2enc-rust, djvulibrust which are some of the major decoder/encoders in the program are opensource and will remain so, seen on my github page. there's no outside contributions or pull requests for those either.

For a relevant laugh check out the Rust PDF renderer Hayro at - https://github.com/laurenzv/hayro - It has 20+ contributors and 15+ pull requests for a project that is much slower and with less pdf coverage than my renderer that I made with LLMs (I heavily researched and planned everything though). Thats what opensourcing gets you, crowdsourced schlock. I used their pdf.js adversarial library in testing but none of their code.

[pazos]

8 months for a niche application would, at best, get you some users. No matter if it is free software or not.

The only benefit I can see by making it a blob is to build your portfolio. Otherwise better with a permissive license. Specially when you said you used LLMs to develop it (you need to feed the cow that feeds you )

Your users are not entitled to any kind of support in any case.

[JSWolf]

Originally Posted by shuvashish76

The project is now proprietary?

The project is to take your eBook and make it look like a mimeographed copy.

[LegeApps]

1.4.74 out soon, will have automatic Table of Contents and title/author metadata, speed/memory improvements otherwise. I am aware that this is an extremely niche program. It is meant for raster scans, not digitally created documents.

Originally Posted by JSWolf

The project is to take your eBook and make it look like a mimeographed copy.

Have any better ideas for what to do with 500mb raster scans of old books or are you just complaining due to lack of understanding of both the problem and the solution?

The only benefit I can see by making it a blob is to build your portfolio. Otherwise better with a permissive license. Specially when you said you used LLMs to develop it (you need to feed the cow that feeds you )

Yea, even contributions done via LLM I would have accepted. But there was a lot of testing and tradeoffs that only I know about and that isn't documented, so I get that no one contributed.

[rantanplan]

Originally Posted by LegeApps

For a relevant laugh check out the Rust PDF renderer Hayro at - https://github.com/laurenzv/hayro - It has 20+ contributors and 15+ pull requests for a project that is much slower and with less pdf coverage than my renderer that I made with LLMs (I heavily researched and planned everything though). Thats what opensourcing gets you, crowdsourced schlock. I used their pdf.js adversarial library in testing but none of their code.

What is your expectation with contributors? You are making a lot more progress with LLMs than a person ever could. So another contributor can't easily participate unless they also use LLMs. But then why should anyone contribute if you already make so much progress?

[Frenzie]

Maybe don't tell people expressing an interest to shut it if you're actually interested in contributions.

[LegeApps]

Originally Posted by rantanplan

What is your expectation with contributors? You are making a lot more progress with LLMs than a person ever could. So another contributor can't easily participate unless they also use LLMs. But then why should anyone contribute if you already make so much progress?

I don't have an expectation and I was only trying to explain the closed source decision rather than complaining about lack of contribution. If people didn't want to contribute when it was bad, but they want to contribute when it's good, that means i'm the only true believer. but the program still isn't saleable since the kind of people who want to process book scans are generally not the people who want to pay for software. i'm one of them.

Maybe don't tell people expressing an interest to shut it if you're actually interested in contributions.

I didn't mean for it to come off that way, but the ideal situation for me is a true believer comes along who knows more about this stuff than I do, and I pass the project off in its entirety to them so I don't have to think about it anymore. This person would have more knowledge of ONNX based OCR and layout detection models, the training of them, better knowledge of what makes a good algorithmic binarization method, and the compute to train and test it all. Anything less doesn't help much.

Also I was fishing for input and contribution in many places before I came to this forum, but I did get the most help of all of them here. So, thanks.

Also that guy above who said the mimeograph comment is from the same neighborhood as me, so I figured I could speak frankly to him and I know we will probably never agree because there's curmudgeons on every corner there. He couldn't have known that of course.

[pazos]

Maybe this thread can be moved outside KOReader subforum, into "creating and editing".

The software has nothing to do with us and I don't feel we would want to promote random blobs on the internet even if they turn to be useful for some people.

A "background cleanup" was added to the program lately: https://github.com/koreader/koreader/pull/15401