View Single Post
Old 10-27-2015, 05:18 PM   #3
knc1
Going Viral
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 17,212
Karma: 18210809
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
Transfer priorities

The transfer managing daemon handles prioritized transfers.

The lower number is serviced first:
Spoiler:

Code:
+151027:144934 tmd[2525]: I start_transfer:starting:id=218,dir=up,pri=1600,qpos=0 of 4,net=wan,on=wifi,was=DEFERRED,defby=NETWORK_DOWN,try=1,ceh=0x43a70ce0:
+151027:144940 tmd[2525]: I start_transfer:starting:id=219,dir=up,pri=1600,qpos=0 of 3,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=1,ceh=0x43a70ce0:
+151027:144941 tmd[2525]: I start_transfer:starting:id=220,dir=down,pri=600,qpos=0 of 4,net=wan,on=wifi,was=WAITING,try=1,ceh=0x43a70ce0:
+151027:144941 tmd[2525]: I start_transfer:starting:id=223,dir=down,pri=600,qpos=1 of 7,net=wan,on=wifi,was=WAITING,try=1,ceh=0x43ab70a0:
+151027:144943 tmd[2525]: I start_transfer:starting:id=221,dir=down,pri=600,qpos=0 of 6,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=1,ceh=0x43a70ce0:
+151027:144943 tmd[2525]: I start_transfer:starting:id=224,dir=down,pri=600,qpos=1 of 6,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=1,ceh=0x43ab70a0:
+151027:144945 tmd[2525]: I start_transfer:starting:id=222,dir=down,pri=1200,qpos=0 of 4,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=1,ceh=0x43a70ce0:
+151027:144946 tmd[2525]: I start_transfer:starting:id=225,dir=down,pri=200,qpos=0 of 5,net=wan,on=wifi,was=WAITING,try=1,ceh=0x43a70ce0:
+151027:144947 tmd[2525]: I start_transfer:starting:id=231,dir=down,pri=1000,qpos=0 of 10,net=wan,on=wifi,was=WAITING,try=1,ceh=0x43a70ce0:
+151027:144948 tmd[2525]: I start_transfer:starting:id=232,dir=down,pri=200,qpos=0 of 11,net=wan,on=wifi,was=WAITING,try=1,ceh=0x43a70ce0:
+151027:144949 tmd[2525]: I start_transfer:starting:id=231,dir=down,pri=1000,qpos=0 of 10,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=2,ceh=0x43a70ce0:
+151027:144950 tmd[2525]: I start_transfer:starting:id=226,dir=down,pri=1050,qpos=0 of 14,net=wifi,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=1,ceh=0x43a70ce0:
+151027:144950 tmd[2525]: I start_transfer:starting:id=238,dir=down,pri=1000,qpos=0 of 15,net=wan,on=wifi,was=WAITING,try=1,ceh=0x43a70ce0:
+151027:144950 tmd[2525]: I start_transfer:starting:id=239,dir=down,pri=1000,qpos=0 of 15,net=wan,on=wifi,was=WAITING,try=1,ceh=0x43a70ce0:
+151027:144951 tmd[2525]: I start_transfer:starting:id=226,dir=down,pri=1050,qpos=0 of 14,net=wifi,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=2,ceh=0x43a70ce0:
+151027:144951 tmd[2525]: I start_transfer:starting:id=233,dir=down,pri=1050,qpos=0 of 13,net=wifi,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=1,ceh=0x43a70ce0:
+151027:144953 tmd[2525]: I start_transfer:starting:id=227,dir=down,pri=1100,qpos=0 of 12,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=1,ceh=0x43a70ce0:
+151027:144953 tmd[2525]: I start_transfer:starting:id=228,dir=down,pri=1100,qpos=0 of 11,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=1,ceh=0x43a70ce0:
+151027:144954 tmd[2525]: I start_transfer:starting:id=229,dir=down,pri=1100,qpos=0 of 10,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=1,ceh=0x43a70ce0:
+151027:144954 tmd[2525]: I start_transfer:starting:id=240,dir=down,pri=100,qpos=0 of 11,net=wan,on=wifi,was=WAITING,try=1,ceh=0x43a70ce0:
+151027:144955 tmd[2525]: I start_transfer:starting:id=229,dir=down,pri=1100,qpos=0 of 10,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=2,ceh=0x43a70ce0:
+151027:144955 tmd[2525]: I start_transfer:starting:id=230,dir=down,pri=1100,qpos=0 of 9,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=1,ceh=0x43a70ce0:
+151027:144955 tmd[2525]: I start_transfer:starting:id=234,dir=down,pri=1100,qpos=0 of 8,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=1,ceh=0x43a70ce0:
+151027:144956 tmd[2525]: I start_transfer:starting:id=235,dir=down,pri=1100,qpos=0 of 7,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=1,ceh=0x43a70ce0:
+151027:144956 tmd[2525]: I start_transfer:starting:id=236,dir=down,pri=1100,qpos=0 of 6,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=1,ceh=0x43a70ce0:
+151027:144957 tmd[2525]: I start_transfer:starting:id=237,dir=down,pri=1100,qpos=0 of 5,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=1,ceh=0x43a70ce0:
+151027:144957 tmd[2525]: I start_transfer:starting:id=222,dir=down,pri=1200,qpos=0 of 4,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=2,ceh=0x43a70ce0:
+151027:144958 tmd[2525]: I start_transfer:starting:id=219,dir=up,pri=1600,qpos=0 of 3,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=2,ceh=0x43a70ce0:
+151027:144958 tmd[2525]: I start_transfer:starting:id=241,dir=down,pri=1200,qpos=0 of 4,net=wan,on=wifi,was=WAITING,try=1,ceh=0x43a70ce0:
+151027:144958 tmd[2525]: I start_transfer:starting:id=219,dir=up,pri=1600,qpos=0 of 3,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=3,ceh=0x43a70ce0:
+151027:144959 tmd[2525]: I start_transfer:starting:id=216,dir=up,pri=1700,qpos=0 of 2,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=1,ceh=0x43a70ce0:
+151027:144959 tmd[2525]: I start_transfer:starting:id=242,dir=up,pri=1600,qpos=0 of 3,net=wan,on=wifi,was=WAITING,try=1,ceh=0x43a70ce0:
+151027:144959 tmd[2525]: I start_transfer:starting:id=216,dir=up,pri=1700,qpos=0 of 2,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=2,ceh=0x43a70ce0:
+151027:145000 tmd[2525]: I start_transfer:starting:id=217,dir=down,pri=1800,qpos=0 of 1,net=wan,on=wifi,was=DEFERRED,defby=HIGHER_PRIORITY_REQUEST,try=1,ceh=0x43a70ce0:
+151027:145000 tmd[2525]: I start_transfer:starting:id=243,dir=down,pri=1800,qpos=0 of 1,net=wifi,on=wifi,was=WAITING,try=1,ceh=0x43a70ce0:


Books on-order are transfered with PRI=600
Spoiler:

Code:
+151027:144941 tmd[2525]: I request_download:received:id=223,pri=600,pc=ContentTransfer:B00GCIN0OM_EBOK,net=wan,at=194941.83:
+151027:144941 tmd[2525]: I start_transfer:starting:id=223,dir=down,pri=600,qpos=1 of 7,net=wan,on=wifi,was=WAITING,try=1,ceh=0x43ab70a0:
+151027:144942 tmd[2525]: I header:list:id=223,headers={:RESPONSE::HTTP/1.1 200 OK},{Content-Length:8579},{Content-Type:application/json},{Date:Tue, 27 Oct 2015 19:49:42 GMT}:
+151027:144942 tmd[2525]: I end_transfer:COMPLETED:id=223,dir=down,pri=600,net=wan,on=wifi,curl=0,srv=200,try=1,resumes=0,declines=0,mm=receiving_body,len=8579,saved=0:
+151027:144942 tmd[2525]: I rate_stats1:COMPLETED:id=223,t1_tmdreq=@194941.83,t2_enq=28ms,t3_actv=4ms,t4_hprt1=0ms,t5_hprt=@194941.87,t6_hpet1=19ms,t7_hpet=@194941.88,t8_start1=@194941.88,t9_start=@194941.88,t10_start=0ms,t11_work=254ms,t12_dns=60ms,t13_conn=90ms,t14_ssl=200ms,t15_reqsent=1ms,t16_respst=374ms,t17_eoh=0ms,t18_dstart=12ms,t19_dend=14ms,t20_end=@194942.89,try=1:
+151027:144942 tmd[2525]: I rate_stats2:COMPLETED:id=223,session_time=1008ms,chunk_time=14ms,chunk_len=8579 of 8579,adj_len=2579,session_rate=2558 B/sec,chunk_rate=184214 B/sec,overall=1s,active=1s,lost=0s,type=(unknown),start_wss=-1,end_wss=-1,c1_redir=0ms,c2_dns=36ms,c3_conn=90ms,c4_ssl=200ms,c5_pret=1ms,c6_dstart=374ms,c7_dend=27ms,c8_total=729ms,try=1,mcc=,mnc=:


Updates are transfered with PRI=1800:
Spoiler:

Code:
+151027:145000 tmd[2525]: I request_download:received:id=243,pri=1800,pc=,net=wifi,at=195000.04:
+151027:145000 tmd[2525]: W qmgr:fopen_to_write:id=243,path=---,err=Is a directory(21):
+151027:145000 tmd[2525]: I start_transfer:starting:id=243,dir=down,pri=1800,qpos=0 of 1,net=wifi,on=wifi,was=WAITING,try=1,ceh=0x43a70ce0:
+151027:145000 tmd[2525]: I end_transfer:FAILED:id=243,dir=down,pri=1800,net=wifi,on=wifi,curl=0,err=8,srv=200,try=1,resumes=0,declines=0,flags=515,mm=resolving,did=-1 (0+-1) of -1:
+151027:145000 tmd[2525]: I rate_stats1:FAILED:id=243,t1_tmdreq=@195000.04,t2_enq=7ms,t3_actv=7ms,t4_hprt1=395ms,t5_hprt=@195000.45,t6_hpet1=17ms,t7_hpet=@195000.47,t8_start1=@195000.48,t9_start=@195000.48,t10_start=6ms,t11_work=0ms,t12_dns=0ms,t13_conn=0ms,t14_ssl=0ms,t15_reqsent=0ms,t16_respst=0ms,t17_eoh=0ms,t18_dstart=0ms,t19_dend=8ms,t20_end=@195000.49,try=1:
+151027:145000 tmd[2525]: I rate_stats2:FAILED:id=243,session_time=8ms,chunk_time=8ms,chunk_len=-1 of -1,adj_len=-1,session_rate=0 B/sec,chunk_rate=0 B/sec,overall=0s,active=0s,lost=0s,type=(unknown),start_wss=-1,end_wss=-1,c1_redir=0ms,c2_dns=0ms,c3_conn=0ms,c4_ssl=0ms,c5_pret=0ms,c6_dstart=0ms,c7_dend=0ms,c8_total=0ms,try=1,mcc=,mnc=:
+151027:145000 tmd[2525]: I moreinfo:FAILED:id=243,curl=0 "No error",err=8 (The local file path is on a currently inaccessible filesystem):
+151027:145000 tmd[2525]: I obliterate:request_received:id=243:
+151027:145000 tmd[2525]: E dequeue:notfound:id = 243:
+151027:145009 tmd[2525]: I def:malloc_trim::start
+151027:145009 tmd[2525]: I def:malloc_trim:ret=1:end
+151027:145009 tmd[2525]: I tq_mh:memory_reclaimed:before=7124, after=6736, reclaimed=388:
## next id ##
+151027:145034 tmd[2525]: I request_upload:received:id=244,pri=1700,pc=,net=wan,at=195034.19:


There are a surprising number of things being transfered, and many of them use different priorities than these two shown.

The transfer manager is running multiple, concurrent, transfers (in both directions at times).
Its a pretty sophisticated daemon.

= = = =

Almost bottom line -
The system will **usually** download your books before downloading a pending update.

**usually**

If there is an available thread and no higher priority transfers **to start** (even though the higher priorities have not yet completed), it will start downloading the pending update.

Bottom line -
Even War and Peace (I have a Kindle copy) is smaller than an update.
Your book downloads should beat the update download, even if they run concurrently (which they can).

But that is not a guarantee.
If your book is on a very slow server in the Amazon cluster -
and
Your update is on a very fast server in the Amazon cluster -

Your screwed.

So side-load your books.

Last edited by knc1; 10-27-2015 at 05:23 PM.
knc1 is offline   Reply With Quote