Automation of scanning on phone or tablet

[Pajamaman]

I want to scan an old hardback using a phone or tablet, both android. I could use an ipad if absolutely necessary, but I prefer android.

Key point. I want an app that will automatically do a scan every x seconds without user intervention. As such, I will only need to turn the pages, and the app will automatically photo each page at regular intervals.

Does such an app exist in android? If not, does it exist on an ipad?

If you have any other tips, I'm all ears. I am thinking of building some kind of frame to cradle the book, and to place the tablet over the book.

[Turtle91]

https://www.diybookscanner.org/

They have all the answers to your questions: scanners, software, cameras, etc.

[Pajamaman]

Many thanks.

[Tex2002ans]

anonlivros wrote a great "Photographs->Ebook" tutorial earlier this year:

Tutorial-from Paper Book to Ebook PDF - 400 pages in 4 hours

His method used a bendable "gooseneck mount" + cellphone.

On the phone, you can set it to take photos based on a noise. You can then easily:

- turn page
- make noise
- turn page
- [...]

until your photographs are complete.

From there, you can do the usual Raw-Images -> Cropping/Cleanup -> OCR.

Within that MR thread, I also described in detail how to use ScanTailor Advanced for that middle stage.

[Pajamaman]

Thanks. Unfortunately the author's portugese tutorial is a pdf stored as a file on github, and as such, google translate cannot translate it. Perhaps I am missing something. I've downloaded it and uploaded it to google docs, but still no translation option. I'll try and work it out.

From gleaning, the resulting posts, it seems the OP uses voice activated software to scan, which is a great idea, and probably superior to automating the scans at regular intervals. I will look through the Portugese source to work out what app he is using (he's using a phone, so presumably he's using an app).

[Pajamaman]

Thank you again for the link. It contains more information than I knew I needed.

The guide includes information on how to trigger by sound and by timer. Doubtless there is more essential information in there that I will parse through.

For reference to others, here are the translated sections:

Tripping by beep

This approach is the most efficient. But it can only be applied in environments that are not too noisy.
Install EasySelfie android app https://play.google.com/store/apps/d...oft.easyselfie
The app informed allows you to take pictures whenever your cell phone 'hears' noise above a predetermined volume. In this way, all you have to do is pronounce a syllable aloud, such as 'vai' or 'pa', and the cell phone will trigger the photograph.
Then, turn the book page and beep again for the cell phone to take another photo.
Important: If there is someone in the environment, let them know about the process so they don't think you've lost your mind in saying 'pa-pa-pa-pa' every 5 seconds. :for
The great advantage of this method is that the cell phone respects speed variations in its ability to turn pages. So there will be a strong synchrony between the frequency of shots taken and their manual movements. No shooting at the wrong time and no time wasted waiting.

Self-timer shooting

This approach is less productive than with Beep Trigger.
Install android app Camera Photo Sequence https://play.google.com/store/apps/details? id=com.augusto.cameratimelapse&hl=pt_BR
The informed app makes it possible to take photos automatically every ‘x’ seconds, so that the time of capture intervals matches your personal manual speed of switching pages safely, minimizing errors in the process.

Setting the timer

- Test the sequential photography timer set for 10 seconds and reduce it according to your skill in turning pages safely and quickly. - It is important to note that if the timer is set to a time below your manual ability to switch pages, more manual errors will result. In other words, errors such as taking a photo when handling pages. These errors demand greater rework to delete disposable photos and increase the time invested in the capture process. Thus, it is important to set a sequential capture time that suits your particular page-turning ability.

[Tex2002ans]

Originally Posted by Pajamaman

Thanks. Unfortunately the author's portugese tutorial is a pdf stored as a file on github, and as such, google translate cannot translate it. Perhaps I am missing something. I've downloaded it and uploaded it to google docs, but still no translation option.

???

It's all pure text. Right here:

https://github.com/anonlivros/anonlivros

You can toss that into Google Translate (or DeepL, or Bing Translate, or whatever). It should translate the original Portuguese -> English for you.

Originally Posted by Pajamaman

Thank you again for the link. It contains more information than I knew I needed.

The guide includes information on how to trigger by sound and by timer. Doubtless there is more essential information in there that I will parse through.

You'll also want to peruse the diybookscanner forums, just like Turtle91 recommended. That site has lots of different designs+builds+tips as well.

Like one enhancement you may want is a V-shaped mount to hold the book + plexiglass in order to press pages flat. This will help keep pages perfectly straight, so you don't have to deal with wavy/bent text. (This can be somewhat corrected in software, but it's better to not introduce those errors in the first place.)

[retiredbiker]

Here is my DIY scanner effort from about 8 years ago. Made from scrap wood. I've used many different lights and cameras over the years. This current arrangement shows an LED floodlight and an ancient video camera (materials on hand...) and it does a great job. Obviously I use the video camera in single frame mode! The center "V" tray has two pieces of glass (from dollar store picture frames) to keep the pages flat, and that is really important for OCR.

The reason for the floodlight: with cheap old cameras I found lots of light (~high f-stop) improves depth of field and focus.

[anonlivros]

Hi! I am the creator of the tutorial mentioned.
I'm glad the tutorial continues to help more people. Thanks for share, Tex!

Tips:
1) About the app used to take photos, over time I found a better one.
The app is called 'Open Camera'
https://play.google.com/store/apps/d...hl=pt_BR&gl=US
He is better because:
- Allows you to generate images with minimal compression, which increases quality (more definition in the outline of the letters). Provides higher quality images taken with other apps I've tested.
- Allows you to lock the camera's Focus, which eliminates the risk of blurry photos. This risk exists when the 'auto focus adjustment' option is enabled, as during the act of turning the page, the camera may try to readjust the focus and the photograph may be taken during this adjustment, resulting in blurry photography.


2) For sound trigger, I suggest whistling low, like the chirping of a chick. It's more practical than speaking a syllable.

3) I also strongly recommend using Scantailor for dewarp and crop, generating B&W images. This step must be prior to abbyy finereader.

4) I also recommend equalizing the size of page content. When taking left/right photographs, rather than both pages simultaneously, depending on book thickness, there will be a page size distortion of up to 10% between consecutive pages. The distortion is most severe at the beginning and end of the book, being milder as it gets closer to the middle pages. This natural distortion is caused by the difference in distance between the page and the camera.
Some ebook producers don't mind this size distortion, as their ultimate goal is to produce an Epub. (Like Tex hehe)
But if you care to eliminate visual distortion, producing a PDF with better aesthetics, I've produced a little python script that fixes them from just 4 pieces of information.
Check the post:
Page Size Equalizer-To single-page capture camera scan


I know, I know... The tutorial needs to be updated.
I've learned some interesting things over the last few months, especially from sharing experience with Tex. (Which I reserve a gratitude that I couldn't properly express in this post)

I hope your digitization projects help a lot of people.
Keep sharing.

[Quoth]

I use Open Camera. Quality ultimately depends on the camera.

BUT my 2002 vintage flatbed scanner with optional ADF is close to 30 M pixels at 600 dpi for a larger book. The advantage of a camera is a V shaped holder to avoid damaging the spine. Pirates and professional scanning of cheap common books cut off the spine and use a duplex ADF.

Obviously if the camera has a good lens, the page size is A5 or smaller or a V support is needed then the camera wins on DPI if it's a decent resolution.

Very even lighting is needed. Skinny florescent tubes and a diffuser as on older light boxes are good. LED sources may need to be further away. Glass or plastic to hold pages flat is needed if not using a scanner.

All the problems solved nearly 50 years ago!

In 1974, Ray Kurzweil founded Kurzweil Computer Products, Inc. and led development of the first omni-font optical character recognition system, a computer program capable of recognizing text written in any normal font. Before that time, scanners had only been able to read text written in a few fonts. He decided that the best application of this technology would be to create a reading machine, which would allow blind people to understand written text by having a computer read it to them aloud. However, this device required the invention of two enabling technologies—the CCD flatbed scanner and the text-to-speech synthesizer. Development of these technologies was completed at other institutions such as Bell Labs, and on January 13, 1976

Google uses a camera and the V shaped book holder. However much of their output is only good enough for "Search" and thus poor quality. A lot is on the Internet Archive and all of it as search results in Google Books. They don't bother with proofing. They also ought to have lost the court case because they are scanning and storing complete copyright works without the copyright holder's permission.

In 2000 a really good copy typist rather than Scan + OCR + human proof read/edit was still cheaper a for a quality transcription. Such people are now rare.

I stopped printing to paper or proofing on PC screen nearly seven years ago. I make an ebook and proof correct on that (was Kindle now Kobo) and read back the annotations.

[Tex2002ans]

Originally Posted by anonlivros

I'm glad the tutorial continues to help more people. Thanks for share, Tex!

Originally Posted by anonlivros

3) I also strongly recommend using Scantailor for dewarp and crop, generating B&W images. This step must be prior to abbyy finereader.

ScanTailor Advanced.

And the steps would go:

• Images/Scans
  
  • Follow the anonlivros tutorial (and/or DIY Book Scanner forums)
• ScanTailor Advanced
  
  • Crop + Dewarp + Grayscale/B&W
• ABBYY Finereader (or Tesseract)
  
  • Optical Character Recognition (OCR)

This would get you digitized PDFs with text backend, which would allow you to search/copy/paste, etc.

From there, you can do the usual PDF->EPUB steps. (Which are their own in-depth workflows.)

Originally Posted by anonlivros

I know, I know... The tutorial needs to be updated.

I've learned some interesting things over the last few months, especially from sharing experience with Tex. (Which I reserve a gratitude that I couldn't properly express in this post)

I didn't check on the Github to see if it was expanded since.

And we definitely have to meetup and talk again before the end of the year!

Originally Posted by Quoth

BUT my 2002 vintage flatbed scanner with optional ADF is close to 30 M pixels at 600 dpi for a larger book. The advantage of a camera is a V shaped holder to avoid damaging the spine. Pirates and professional scanning of cheap common books cut off the spine and use a duplex ADF.

Any of these methods work, but I'd rank everything as a balance of these 4 categories:

Non-Destructive->Destructive

• Camera
• Scanner*
• Feed Scanner

High->Low Quality

• Scanner
• Feed Scanner
• Camera*

Fast->Slow Speed

• Camera
• Feed Scanner
• Scanner

High->Low Labor

• Scanner
• Camera
• Feed Scanner

Non-Destructive vs. Destructive

If the book must stay in-tact, then Scanner or Camera.

If you don't mind destroying the book, then cutting the spine off + feeding it into the scanner as a stack of paper saves tons of time+labor.

* Note: If you have very fragile/large books, the Scanner may be too rough on the spine, so your only non-destructive choice is Camera + V-shaped holder.

Quality

The most important, because every later stage depends on this.

Remember, digitizing books is an entire process, and getting pictures of pages is just Stage 1.

High quality input:

• High DPI (300/600+) + great/even lighting + non-warped pages
• = very accurate OCR + little image cleanup

Low quality images:

• Low DPI + bad lighting (haloing) + warped pages
• = OCR has more errors + more typos/formatting issues + fixing images takes longer

You may even have to redo a lot of your work when you stumble across a non-recoverable error later on. (Like a photograph/chart/graph being distorted beyond fixing... or horrible speckling that only appears when you try to B&W your image.)

* Note: 20+ years ago, most cameras were still too low DPI. The images may have been readable to a human, but feeding that lower-quality image through OCR, you'd get a much higher error rate.

Within the past 10 years though, the quality of your typical cellphone camera has dramatically improved. Now, everyone carries something in their pockets that may work "well enough".

This is where anonlivros's gooseneck+cellphone method comes into play. It's an extremely cheap (<$50) way of reusing materials you most likely already have (cellphone + lamp)... getting you 80%+ of the way there.

* * *

Recommendation: If doing conversion professionally though, I'd recommend a superb quality scanning company like Golden Images. He was recommended by Hitch+me quite a few times over the years.

The immaculate quality scans will save you all that time+labor in the long-run.

Higher quality input = better+faster conversion with less errors.

* * *

Speed

Cameras, when you get the workflow down, can take a few seconds per page.

Scanners, at higher DPI, are very slow.

Labor

Scanners+Cameras require you to turn pages, make sure everything is lined up, hold books down, no fingers in the way, etc.

Feed scanners, you stick in a stack of paper and can go off doing something else.

Originally Posted by Quoth

Google uses a camera and the V shaped book holder. However much of their output is only good enough for "Search" and thus poor quality.

It's quite good. And all the raw images are still there as JPEG2000. They are much higher quality than the auto-generated B&W/Color PDFs.

A lot of this was discussed a few months ago in:

2021: "Archive.org ePub"

I even showed the difference between the auto-generated Archive.org "EPUB"s vs. EPUBs generated out right out of Finereader with minimal intervention.

Note: Although again, you can rapidly get 80%+ of the way there in your PDF->EPUB, but that final quality push is what takes up the majority of time:

• Headings + TOCs
• Splitting chapters
• Formatting
  
  • bold/italics + super-/sub-script
    
  • blockquotes/poetry/indexes/scenebreaks
    
  • Alignment (left/center/right)
    
• Fixing tables/figures/images
• Clickable footnotes
• Typo corrections
• [...]

99.99% text accuracy also seems fantastic... until you actually read a book with .01% errors in it. (At least a few errors on every page.)

Originally Posted by Quoth

A lot is on the Internet Archive and all of it as search results in Google Books. They don't bother with proofing. They also ought to have lost the court case because they are scanning and storing complete copyright works without the copyright holder's permission.

Fair Use. It's a library. And you don't "need permission".

There was even an article on that court case a few days ago:

Techdirt.com: "Internet Archive Would Like To Know What The Association Of American Publishers Is Hiding"

and a few days before that, on the ridiculousness of "library ebook licenses":

Techdirt.com: "Publishers Want To Make Ebooks More Expensive And Harder To Lend For Libraries; Ron Wyden And Anna Eshoo Have Questions"

A UK campaign to fight that development in the world of academic publishing, called #ebookSOS, spells out the problems. Ebooks are frequently unavailable to institutions to license as ebooks. When they are on offer, they can be ten or more times the cost of the same paper book. [...] One title cost £29.99 as a physical book, and £1,306.32 for a single-user ebook license. As if those prices weren't high enough, it's common for publishers to raise the cost with no warning, and to withdraw ebook licenses already purchased. [...]

On Controlled Digital Lending (CDL), I'd also recommend checking out these fantastic webinars given by Internet Archive a few months ago:

• "Implementation & Integration: CDL For All Libraries"
• "Empowering Libraries Through Controlled Digital Lending" by Chris Freeland

Libraries already use these methods + Interlibrary Loans to loan scans of their own works (especially fragile/rare/non-public-facing material).

This allows you to consolidate it, instead of every individual library having to (poorly) re-scan + re-digitize.

Similarly with the Blind/Low-Vision/Disabilities associations + universities within each country all independently digitizing. Why? Why waste all that time and effort when you can do it once, in high-quality PDFs/EPUBs, then lend it from there?

On universities digitizing, see the great webinar:

• DAISY Consortium: "The Accessible EPUB Ecosystem in Action: Following the Journey from Publisher to Student"
• DAISY Consortium: "Describing Images in Publications – Guidance, Best Practices and the Promise of Technology"
  
  • University explaining best practices for describing images (alt text) to students who request more accessible formats/materials.
    
  • They also had an absolutely fantastic 3-part series on "The Art and Science of Describing Images"

On Copyright/Libraries + "Permission Culture" see:

• "Libraries Do Not Need Permission To Lend Books: Fair Use, First Sale, and the Fallacy of Licensing Culture"
• "Libraries Have Never Needed Permission To Lend Books, And The Move To Change That Is A Big Problem"
• "Copyright Is Broken: COVID-19 Pandemic Revealing Just How Messed Up Our Permission-Based Culture Is"
• "Permission Culture And The Automated Diminishment Of Fair Use"
• "Dear Permission Culture: This Is Why No One Wants To Ask For Your OK"

Side Note: Luckily, Internet Archive has been archiving scholarly articles as well:

• "Search Scholarly Materials Preserved in the Internet Archive"

And speaking of digitizing + Link Rot... ~18% websites cited within scholarly articles are already dead:

• "Internet Archive Releases Refcat, the IA Scholar Index of over 1.3 Billion Scholarly Citations"

This knowledge needs to be preserved + easily searchable/accessible.

[Quoth]

Library schemes, fees per book loaned and permissions vary by country and by media type. Not everyone is in the USA. USA laws do not apply outside the USA. Sadly eBooks are rarely by the same rules as paper.
Also in most countries the Copyright Holder has the right to decide how or if copies are done. Not USA Courts or Google or the Internet Archive, no matter how virtuous they are at preserving information.
From the point of view of Irish and UK law both Google and the Internet Archive have been breaking copyright (civil issue, not a criminal) for over a decade by scanning British and Irish works still in copyright.
Also Google invented their own definition of "Orphan Works".

The big USA Corporations, OTOH, are responsible for the disgraceful extensions of Life + term and Corporate Copyrights, DRM and DMCA. All immoral corporate "theft" that does nothing for creators and immediate children.

[Tex2002ans]

Originally Posted by Quoth

The big USA Corporations, OTOH, are responsible for the disgraceful extensions of Life + term and Corporate Copyrights, DRM and DMCA.

No disagreement there.

You may also be interested in an upcoming debate (November 15, 2021) at the Soho Forum:

https://www.thesohoforum.org/

Resolution: All patent and copyright law should be abolished.

For the affirmative:

N. Stephan Kinsella (LL.M., King’s College London-University of London; JD, Paul M. Hebert Law Center, LSU; BSEE and MSEE, LSU) is a libertarian writer and patent attorney in Houston. He was previously General Counsel for Applied Optoelectronics, Inc., a partner with Duane Morris (Philadelphia and Houston), and adjunct law professor at South Texas College of Law. His publications include Against Intellectual Property (Mises Institute, 2008), Trademark Practice and Forms (West/Thomson Reuters 2001–2013), International Investment, Political Risk, and Dispute Resolution: A Practitioner’s Guide (Oxford, 2020), and the forthcoming Law in a Libertarian World: Legal Foundations of a Free Society.

For the negative:

Richard A. Epstein is the Peter and Kirsten Bedford Senior Fellow at the Hoover Institution, the Laurence A. Tisch Professor of Law, New York University Law School, and a senior lecturer at the University of Chicago. The author of many books and articles, Richard researches and writes in a broad range of constitutional, economic, historical, and philosophical subjects. His most recent book, The Dubious Morality of Modern Administrative Law, was released in March 2020. Richard appears on “The Libertarian,” a weekly podcast sponsored by the Hoover Institution that features a conversation with him on issues of the day. This will mark his fourth appearance as a debater at the Soho Forum.

A few days later, it will be uploaded in video+audio at the "Past Events" section.

(I'm for digitizing and freeing up any and all works.)

* * *

Last night, I was also digging through the ol' podcasts:

• Techdirt Podcast: "197: The Grand Re-Opening of the Public Domain"

which also discussed the Public Domain 2019 celebration:

• Internet Archive Blog: "Join us for A Grand Re-Opening of the Public Domain"

Full video of the conference here:

• "A Grand Re-opening of the Public Domain"

Keynote given by Creative Commons founder (Lawrence Lessig).

And in the Techdirt podcast, they mentioned a fantastic talk by a blind person who explains the entire process from her point of view. (Lightning Talks began in part #2 of the conference at 3:06:00. Her talk started a few minutes later.)

[Quoth]

I don't agree with abolishing patents, design patents (UK Registered Designs) or copyright. That's anarchy. But the USPTO is broken since the Victorian era and instead of reform it's worse. They don't do due diligence. They make more money from approval than rejection. The theory is that the courts should decide validity. That's weighted toward US companies and also those with deep pockets.

Creative Commons is not a solution. Anyone can have whatever relaxation of copyright they want, CC is just one of many templates often incorrectly applied.