Sloooow update

roger64 · 10-23-2020, 09:02 PM

Hi

I use monthly OTA updates. I did it some days ago. The process was reasonably quick for the first two sliders, but the processing of the third and last slider was really awfully slow (between to 4.5kb down to 1.5kb). I had to resume updating five times before incrementally completing the update (good we do not have to resume from zero..).

My current location is in China. Is there any reason for the huge difference in speed between the first two sliders and the third one?

NiLuJe · 10-23-2020, 09:49 PM

It's a (series of) range requests, so, it's (somewhat) different each time.

Given that the master is located in the US (+ CloudFlare), and the mirrors in Canada (-na), Western (-fr) & Central (-pl) Europe, I'd indeed expect slow to downright terrible results in Asia, doubly so in China, because, err, The Wall.

I'd try the -na and -pl mirrors, just in case, but I wouldn't necessarily hold my breath

.

With sane peerings, the limiting factor is usually the device's I/O, and/or the Wi-Fi signal itself, on devices with not-downright-terrible I/O.

As a random example, I'm seeing roughly the same 1.4MB/s that I see with LAN<->LAN Wi-Fi transfers to the same device in the same location on the same network:

Code:

#################--- 88.1%
#################--- 88.1% 155.6 kBps 26:52 ETA 
#################--- 88.1% 495.2 kBps 8:55 ETA  
##################-- 90.3% 1006.9 kBps 0:13 ETA
###################- 95.2% 1207.8 kBps 0:02 ETA
###################- 98.2% 1449.5 kBps 0:00 ETA
###################- 99.2% 1495.2 kBps 0:00 ETA
###################- 99.3% 1547.1 kBps 0:00 ETA
#################### 100.0% 1444.0 kBps DONE

roger64 · 10-23-2020, 11:23 PM

Thanks for your answer.

The download speed for local Arch repository is between 2 to 4 MB/sec but slower for some Aur updates.

I'll try to use other repositories for OTA updates. How could i change it?

pazos · 10-24-2020, 08:54 AM

Quote:

Originally Posted by roger64

I'll try to use other repositories for OTA updates. How could i change it?

Burguer menu -> update -> settings -> server

roger64 · 10-24-2020, 08:52 PM

Quote:

Originally Posted by pazos

Burguer menu -> update -> settings -> server

Thank you.

My previous selected mirror was http://koreader.ak-team.com/

I'll try different other mirrors and report the result for the next update.

I know that the Tsinghua university in Beijing provides a fast mirror for Arch repositories (not only that distro). Maybe they would accept to host a Koreader mirror for Chinese users ?

NiLuJe · 10-24-2020, 09:45 PM

Oh, well, that probably didn't help

.

That one's got barely any bandwidth, and is hosted in the north of France

.

It's at the end of the list for a reason, it's really only there as a last resort in case your ISP has a severe allergy to CloudFlare CDNs and OpenStack object storage based mirrors

.

roger64 · 10-25-2020, 01:58 AM

I never paid any attention to mirror selection when in Europe nor willlingly selected this particular mirror (as it happened, it just worked reasonably before).

Good to know they are ranked by speed (sort of). But, the download speed obtained varies according to the server, to the location (or IP) of the user and may change over time. So this ranking is at best an advice.

I use a provided bash script to select mirrors by speed with Arch. Maybe this kind of tool could be adapted to Koreader?

https://wiki.archlinux.org/index.php...#List_by_speed
https://www.archlinux.org/mirrorlist/

The other idea would be to find a Chinese host since Koreader is now a world class application.

Frenzie · 11-04-2020, 04:53 PM

Quote:

I use a provided bash script to select mirrors by speed with Arch. Maybe this kind of tool could be adapted to Koreader?

That's Cloudflare's job really. You can pretty much regard all the others as backup unless you're in China.

roger64 · 11-07-2020, 05:50 AM

Quote:

Originally Posted by Frenzie

... unless you're in China.

I am

Frenzie · 11-07-2020, 08:04 AM

In that case it's conceivable that one of the other mirrors would do better. Not because they're geographically closer or have more bandwidth or anything like that, but just because Cloudflare is a well-known entity and @NiLuJe's server probably isn't. Then again, you said the French one was pretty slow and it gives me 80+ Mbit…

(Well, only 10 Mbit or so on my H2O of course but that's simply its speed limit.)

NiLuJe · 11-07-2020, 09:41 AM

The last entry in the list *is* technically slower, because it's not actually a fancy object storage frontend, it's served by a plain old nginx web server on real hardware, and it has teeny tiny throughput because...

Code:

[   15.660441] e1000e 0000:00:19.0 eno1: NIC Link is Up 100 Mbps Full Duplex, Flow Control: None

Which makes it ten times slower than my home network ^^

.

The others are actually OpenStack frontends, and should offer better throughput.

(e.g., you can check in a browser, they all have file listing enabled: if it's pretty and naturally sorted, it's nginx, if it's dog-ugly, it's OpenStack

).

(There's a bit of a tradeoff: CloudFlare/OpenStack is *terrible* at range requests, but offers us essentially unlimited bandwidth and fairly decent PoP coverage. nginx is *awesome* at range requests, but my own server has very limited bandwidth).

Granted, since the move to zsync2, that's less of an issue than before. That was in fact one of the main goal of the whole thing, being able to get decent speeds out of the "proper" master/mirrors

.

JSWolf · 11-07-2020, 10:07 AM

Quote:

Originally Posted by roger64

Thanks for your answer.

The download speed for local Arch repository is between 2 to 4 MB/sec but slower for some Aur updates.

I'll try to use other repositories for OTA updates. How could i change it?

Why would you use the Arch suppository when it doesn't work well?

DiapDealer · 11-07-2020, 01:36 PM

Quote:

Originally Posted by JSWolf

Why would you use the Arch suppository when it doesn't work well?

Moderator Notice
Knock it off.

Frenzie · 11-09-2020, 05:38 AM

Quote:

Originally Posted by NiLuJe

Code:

[   15.660441] e1000e 0000:00:19.0 eno1: NIC Link is Up 100 Mbps Full Duplex, Flow Control: None

Still plenty to offer a good experience for a pretty decent volume of users. Most downloads will be over in seconds, even without zsync. Your upload volume limit is much more likely to be a relevant restraint imho.

(Unless you have some Cacti graphs or something proving me wrong. :-D )

NiLuJe · 11-09-2020, 12:24 PM

@Frenzie: Oh, yeah, it's mostly an issue for non-embedded platforms (and/or manual D/Ls), where a single client is likely more able to saturate the BW than a poor Kobo

.

10-23-2020, 09:02 PM	#1
roger64 Wizard Posts: 2,625 Karma: 3120635 Join Date: Jan 2009 Device: Kindle PW3 (wifi)	Sloooow update Hi I use monthly OTA updates. I did it some days ago. The process was reasonably quick for the first two sliders, but the processing of the third and last slider was really awfully slow (between to 4.5kb down to 1.5kb). I had to resume updating five times before incrementally completing the update (good we do not have to resume from zero..). My current location is in China. Is there any reason for the huge difference in speed between the first two sliders and the third one?

10-23-2020, 09:49 PM	#2
NiLuJe BLAM! Posts: 13,506 Karma: 26047202 Join Date: Jun 2010 Location: Paris, France Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E	It's a (series of) range requests, so, it's (somewhat) different each time. Given that the master is located in the US (+ CloudFlare), and the mirrors in Canada (-na), Western (-fr) & Central (-pl) Europe, I'd indeed expect slow to downright terrible results in Asia, doubly so in China, because, err, The Wall. I'd try the -na and -pl mirrors, just in case, but I wouldn't necessarily hold my breath . With sane peerings, the limiting factor is usually the device's I/O, and/or the Wi-Fi signal itself, on devices with not-downright-terrible I/O. As a random example, I'm seeing roughly the same 1.4MB/s that I see with LAN<->LAN Wi-Fi transfers to the same device in the same location on the same network: Code: #################--- 88.1% #################--- 88.1% 155.6 kBps 26:52 ETA #################--- 88.1% 495.2 kBps 8:55 ETA ##################-- 90.3% 1006.9 kBps 0:13 ETA ###################- 95.2% 1207.8 kBps 0:02 ETA ###################- 98.2% 1449.5 kBps 0:00 ETA ###################- 99.2% 1495.2 kBps 0:00 ETA ###################- 99.3% 1547.1 kBps 0:00 ETA #################### 100.0% 1444.0 kBps DONE Last edited by NiLuJe; 10-24-2020 at 09:34 AM.

10-24-2020, 09:45 PM	#6
NiLuJe BLAM! Posts: 13,506 Karma: 26047202 Join Date: Jun 2010 Location: Paris, France Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E	Oh, well, that probably didn't help . That one's got barely any bandwidth, and is hosted in the north of France . It's at the end of the list for a reason, it's really only there as a last resort in case your ISP has a severe allergy to CloudFlare CDNs and OpenStack object storage based mirrors . Last edited by NiLuJe; 10-24-2020 at 09:47 PM.

10-25-2020, 01:58 AM	#7
roger64 Wizard Posts: 2,625 Karma: 3120635 Join Date: Jan 2009 Device: Kindle PW3 (wifi)	I never paid any attention to mirror selection when in Europe nor willlingly selected this particular mirror (as it happened, it just worked reasonably before). Good to know they are ranked by speed (sort of). But, the download speed obtained varies according to the server, to the location (or IP) of the user and may change over time. So this ranking is at best an advice. I use a provided bash script to select mirrors by speed with Arch. Maybe this kind of tool could be adapted to Koreader? https://wiki.archlinux.org/index.php...#List_by_speed https://www.archlinux.org/mirrorlist/ The other idea would be to find a Chinese host since Koreader is now a world class application. Last edited by roger64; 10-25-2020 at 05:54 AM. Reason: speed

11-07-2020, 09:41 AM	#11
NiLuJe BLAM! Posts: 13,506 Karma: 26047202 Join Date: Jun 2010 Location: Paris, France Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E	The last entry in the list is technically slower, because it's not actually a fancy object storage frontend, it's served by a plain old nginx web server on real hardware, and it has teeny tiny throughput because... Code: [ 15.660441] e1000e 0000:00:19.0 eno1: NIC Link is Up 100 Mbps Full Duplex, Flow Control: None Which makes it ten times slower than my home network ^^ . The others are actually OpenStack frontends, and should offer better throughput. (e.g., you can check in a browser, they all have file listing enabled: if it's pretty and naturally sorted, it's nginx, if it's dog-ugly, it's OpenStack ). (There's a bit of a tradeoff: CloudFlare/OpenStack is terrible at range requests, but offers us essentially unlimited bandwidth and fairly decent PoP coverage. nginx is awesome at range requests, but my own server has very limited bandwidth). Granted, since the move to zsync2, that's less of an issue than before. That was in fact one of the main goal of the whole thing, being able to get decent speeds out of the "proper" master/mirrors . Last edited by NiLuJe; 11-07-2020 at 09:46 AM.

10-23-2020, 11:23 PM	#3
roger64 Wizard Posts: 2,625 Karma: 3120635 Join Date: Jan 2009 Device: Kindle PW3 (wifi)	Thanks for your answer. The download speed for local Arch repository is between 2 to 4 MB/sec but slower for some Aur updates. I'll try to use other repositories for OTA updates. How could i change it?

11-07-2020, 08:04 AM	#10
Frenzie Wizard Posts: 1,786 Karma: 731691 Join Date: Oct 2014 Location: Antwerp Device: Kobo Aura H2O	In that case it's conceivable that one of the other mirrors would do better. Not because they're geographically closer or have more bandwidth or anything like that, but just because Cloudflare is a well-known entity and @NiLuJe's server probably isn't. Then again, you said the French one was pretty slow and it gives me 80+ Mbit… (Well, only 10 Mbit or so on my H2O of course but that's simply its speed limit.)

11-09-2020, 12:24 PM	#15
NiLuJe BLAM! Posts: 13,506 Karma: 26047202 Join Date: Jun 2010 Location: Paris, France Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E	@Frenzie: Oh, yeah, it's mostly an issue for non-embedded platforms (and/or manual D/Ls), where a single client is likely more able to saturate the BW than a poor Kobo .

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Moving or copying books between libraries is sloooow	Enesco	Calibre	15	01-23-2014 12:35 PM
won't update...after completing the update cycle edge returns to the first screen	WeAreBorrg	enTourage Archive	2	03-18-2011 07:39 PM
How can I update creation time for a title when I update the book info or convert it	setherd	Calibre	3	10-30-2010 02:51 PM
Firmware Update Can I reinstall the official amazon update after the update file from this site ?	yuenslhk	Amazon Kindle	2	06-20-2010 12:24 PM
Update on Google eBook Sales Plan Update	poohbear_nc	News	6	05-05-2010 10:00 AM