I've been hitting the refresh manually from day one. It never worked until yesterday. I always do a manual sync. I'm not alone with this problem. Lots of ranting at BN.com from others. One person mentioned their consistent success.
Also, I've never seen it mentioned that highlights and notes will be synced. I recall folks being concerned they would lose their notes and highlights should they have to do a factory reset.