02-09-2023, 04:51 PM | #31 |
the rook, bossing Never.
Posts: 11,552
Karma: 87456643
Join Date: Jun 2017
Location: Ireland
Device: All 4 Kinds: epub eink, Kindle, android eink, NxtPaper11
|
Close to 70,000 today.
|
02-09-2023, 04:57 PM | #32 | |
the rook, bossing Never.
Posts: 11,552
Karma: 87456643
Join Date: Jun 2017
Location: Ireland
Device: All 4 Kinds: epub eink, Kindle, android eink, NxtPaper11
|
Quote:
And having a UPS And hitting Save regularly I found on LO Writer that autosave still runs usually if you somehow managed to freeze the GUI. Just wait the autosave interval you set. |
|
Advert | |
|
02-10-2023, 06:57 AM | #33 |
Connoisseur
Posts: 66
Karma: 14170
Join Date: Oct 2011
Device: kindle 1
|
damn! that would make me want to just sit and cry. 8 gigs is not enough memory in that situation for large operations in calibre much less the rest of the system needing memory also and to have that on an old school hard drive is just brutal. I would bet a author or title search takes several minutes if not more.
|
02-10-2023, 07:05 AM | #34 | |
Connoisseur
Posts: 66
Karma: 14170
Join Date: Oct 2011
Device: kindle 1
|
Quote:
Going from old system in last few months with 1000 mbs nvme and 300,000+ books to new faster cpu and ram and 3500mbs nvme took searches from 4 seconds average to 2 seconds. last comparison is not apple to apple so it is harder to quantify reasons for performance increase than first one that the only difference was going from spin disk to standard ssd. Going from standard 550 mbs ssd to 1000mbs nvme really didn't seem to boost performance a lot when I did that upgrade on my old system. again maybe that had more to do with underlying motherboard as it was 8 years old or so even at that time. I think its about 12 years old now. Actually its the computer im on now. for most desktop tasks it is still super fast. Just limited in IO by older tech. Last edited by audeojude; 02-10-2023 at 07:41 AM. |
|
02-10-2023, 07:09 AM | #35 | |
Connoisseur
Posts: 66
Karma: 14170
Join Date: Oct 2011
Device: kindle 1
|
Quote:
In general I only have about 7 or 8 columns displayed. |
|
Advert | |
|
02-10-2023, 07:34 AM | #36 |
Connoisseur
Posts: 66
Karma: 14170
Join Date: Oct 2011
Device: kindle 1
|
it has been commented about ssd's failing without warning. I have seen that but it is really rare. SSD's do use S.M.A.R.T and it should let you know if you are monitoring it about issues before complete failure.
That being said. If your not backing up data in at least two places other than where your using it your asking to lose it. I use spinney drives to backup my library. One that I keep in a fire proof safe and bring up to date every month or so and one that I do more often. I have a script created just for my library using rsync that I run when I want it to back up. So it is a manual solution and if I get lazy or forgetful I put myself at risk. I have the hardware to build another little server to just use for backups and automate everything but I haven't done it as I don't have enough drive capacity to backup everything that needs it. about 60TB.. for my library yes but for all my other personal and business stuff No. Instead I have a huge stack of labeled hard drives in the 500GB to 4tb range that I break all the data I have up across and I will just drop a drive or drives in an external usb docking station and initiate the rsync job for that drive/datasets. Before anyone points it out I totally agree that that there are much better ways of doing this. However the cost of doing it right due to the size of stuff to be backed up is prohibitive for me. All my data sits on Big raid 10 arrays on a server, made up of 8tb and 10 tb drives other than the library that needs it's own nvme for performance. So for most of it a bare minimum of redundancy before the single backup to external drives. Library gets a little bit more in the way of backup. When I win the lottery I will have multiple backups with my own personal off site cloud backup also For most of you with small library's of under 50,000 books, (well any size iibrary really) you should get an external usb hard drive and use the calibre export all books feature as your backup. Then you should get a large flash drive from a reliable manufacture and do it to that also. Use the hard drive for daily or weekly backups and take the flash drive and put it in a fire proof safe or even better off site at a relative or friends house in a small fireproof safe there. Swap it out monthly. The reason I recommend using the export function is it backups up not just the books but also the preferences, plugins and everything else you have customized in your library. Just copying the library directory and database file with rsync like I do for the majority of my backups will not save a lot of that stuff for you. now that you have a off site safe. Take and put copies of all your important documents such as Id, birth certificates, titles etc in it also Last edited by audeojude; 02-10-2023 at 07:38 AM. |
02-10-2023, 08:17 AM | #37 |
the rook, bossing Never.
Posts: 11,552
Karma: 87456643
Join Date: Jun 2017
Location: Ireland
Device: All 4 Kinds: epub eink, Kindle, android eink, NxtPaper11
|
Maybe a SATA SSD uses SMART, but there is no SMART on my NVMe PCIe SSD. The HDD on same PC does have SMART.
|
02-10-2023, 08:47 AM | #38 | |
Connoisseur
Posts: 66
Karma: 14170
Join Date: Oct 2011
Device: kindle 1
|
Quote:
NVME standard in general does not use the SMART system for health reporting but it does have health reporting. My understanding is that the SMART standard is old and really complex (to much so) with a lot of crazy stuff added in over the years hard drive development. The people who developed the NVME standard decided they didn't want to even try and use it because of problems in that it would make them build in stuff they didn't want into the NVME drives. (scratch head, my understanding on this and why is minimal) However they did build in all the health reporting stuff that you would need. I'm assuming based on below that some of the smart software is being modified to read some NVME information now. example https://nvmexpress.org/resources/nvm...architectures/ https://www.pcworld.com/article/5414...free-tool.html https://www.techtarget.com/searchsto...h-of-NVMe-SSDs While smart and sata drives have been around for 3+ decades and is a mature tech, NVME storage is relativly new and doesn't use the sata specification so it's not as cohesive at a user level yet but it is there. |
|
02-10-2023, 09:01 AM | #39 |
the rook, bossing Never.
Posts: 11,552
Karma: 87456643
Join Date: Jun 2017
Location: Ireland
Device: All 4 Kinds: epub eink, Kindle, android eink, NxtPaper11
|
Much in HDD SMART reporting is irrelevant to SSD.
command line smartctl for Linux only reports on SATA SSD. There is an NVMe command line tool (not installed) but it doesn't work for some or many NVMe SSDs. So I'm mostly right. Anyway, whatever you are using backups are important. There is always user error, so even if you are living in 1990s and using a magneto-optic disc (which I did use and seem to be near infinite liife and survive a washing machine), there are users. Arghh! That directory has the same name as in another path! RAID certainly doesn't affect needing backups. We sold it in late 1990s as high availability, never as a backup solution. But you know that. Now RAID rebuild times are silly, so now you need two and faster to make a fresh copy than rebuild? Back then we were using 2GB Ultra wide and fast 10,000rpm SCSI. Very noisy! Last edited by Quoth; 02-10-2023 at 09:05 AM. |
02-10-2023, 12:03 PM | #40 |
Well trained by Cats
Posts: 29,944
Karma: 55705602
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
|
RAID does not protect against real 'melt downs' that can happen. Fire, Water, wild power...
Off site, a goodly distance away and not in a similar location. (I have photos from this month of the families , off grid retreat where the creek that has been dry for the last 3+years, has made the road near impassible to common vehicles (A SxS can make it with detours) for almost 6+miles . ) Plan where carefully. |
02-12-2023, 03:13 PM | #41 |
Connoisseur
Posts: 66
Karma: 14170
Join Date: Oct 2011
Device: kindle 1
|
Sorry if I implied that raid was a backup, I did not intend it to sound like that. Raid is a redundancy from single disk failure I was just specifying where most of my data resides.
Sadly I was able to afford a asustor 6000 series NAS With 10 drive bays that I eventually populated with 6 8tb and 4 10tb drives. years later I still haven't been able to really afford a backup solution other than a huge stack of 500GB to 4tb used sata drives that I can piecemeal put data on for backups. I love my Asustor NAS and their support when in upgrading the size of the raid array it ate the array was stellar. they were able to fix it with no loss of data.. took 4 days of re-initializing the array but they fixed it. Best I could tell there was a bug in the firmware that affected the exact method I tried to use in expanding the size of the array. Other than that one issue it has run like a rock for years now. the sadly comes in that I want to purchase a second one and migrate the data from this one to the new one and use the old one for a backup, but 1200 for the bare NAS and another min 2500 for new larger drives. Ideal would be 3400 dollars for 20tb drives and then have the same issue going forward of no backups after I exceed the old units storage capacity unless i spend another 1500 dollars for 4 20 tb drives to use as external backup. Having lots of data cost lots of money to maintain it correctly Other than the drives I could cheap out and not buy the NAS and just get a used server off of ebay for a roll my own NAS for 3 or 4 hundred dollars that would be way more powerful. The downside of that is your power bill to run the server is high and your power bill to cool the room the server is in is high. Over time you end up spending way more money running it than the cost of a dedicated NAS. My NAS that I am using is specd to at MAX draw 65 watts. So about 1/4 the power usage of most of the 10+ bay servers I could afford on ebay. When I got this NAS and paid the then 900 dollars for it, I was going to do the server route and save 500 to 600 dollars on the cost of it. Before I bought the server I found a server power/cost calculator that you could put how much it cost you per kilowatt and what server you had and get an estimated cost for power to run and cool it. I just about had a heart attack at the estimated annual costs. I paid for the difference in purchase price in the first year of usage on the Asustor unit. About 2500 dollars or more in savings since 2018 when I got it. It's all a bunch of trade off's between what you can afford vs what you want vs what you need vs initial cost vs cost of ownership. |
02-12-2023, 05:52 PM | #42 | |
the rook, bossing Never.
Posts: 11,552
Karma: 87456643
Join Date: Jun 2017
Location: Ireland
Device: All 4 Kinds: epub eink, Kindle, android eink, NxtPaper11
|
Quote:
I've built decent "servers" using cheap (or thrown out) tower PCs. You only need a "real" server box if you want hot swap HDD trays and Dual hot swap PSUs (swap failed one while running), which needs two separate UPSes! Oh, and serving 100s to 1000s of users rather than a family. Started with NT Server 3.5, then by 2002 switched to Cent OS (basically Red hat), then Debian and now Mint (sort of Debian/Ubuntu). Last Windows server (in parallel with a Linux one for a few years) was Win2000 Server, Server 2003 was too bloated for the old HW. Last edited by Quoth; 02-12-2023 at 05:57 PM. |
|
02-12-2023, 07:23 PM | #43 |
null operator (he/him)
Posts: 20,652
Karma: 26966376
Join Date: Mar 2012
Location: Sydney Australia
Device: none
|
Moderator Notice
This thread has drifted so far off topic I'm thinking I might call Justin & Joe and get them to shoot it down BR |
03-28-2023, 08:59 AM | #44 |
Connoisseur
Posts: 66
Karma: 14170
Join Date: Oct 2011
Device: kindle 1
|
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Limit on Number of books? | kblechman | Library Management | 1 | 04-02-2011 10:05 AM |
NookLibrary : Limit for number of books? | Kolfe | Barnes & Noble NOOK | 6 | 12-02-2010 06:27 PM |
Limit to number of books in Calibre? | ficbot | Calibre | 1 | 02-26-2010 08:46 AM |
Can calibre handle different books' editions? | italianReader | Calibre | 1 | 01-03-2010 11:13 AM |
Limit to the number of MP3 files? | dhbailey | Sony Reader | 19 | 04-15-2007 04:44 PM |