View Single Post
Old 05-13-2019, 12:40 AM   #3
ilovejedd
hopeless n00b
ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.
 
ilovejedd's Avatar
 
Posts: 5,110
Karma: 19597086
Join Date: Jan 2009
Location: in the middle of nowhere
Device: PW4, PW3, Libra H2O, iPad 10.5, iPad 11, iPad 12.9
Quote:
Originally Posted by NiLuJe View Post
There's possibly a bit of hashing involved, but take that with a grain of salt, as I'm extremely unfamiliar with that bit of code.

Are the actual files identical on your end? How do you transfer them? (Might be a case of Calibre using a different output template?).
Files are bit for bit identical. I was able to trace the digest function used.

Spoiler:
Code:
-- calculate partial digest of the document and store in its docsettings to avoid document saving
-- feature to change its checksum.
--
-- To the calculating mechanism itself.
-- since only PDF documents could be modified by KOReader by appending data
-- at the end of the files when highlighting, we use a non-even sampling
-- algorithm which samples with larger weight at file head and much smaller
-- weight at file tail, thus reduces the probability that appended data may change
-- the digest value.
-- Note that if PDF file size is around 1024, 4096, 16384, 65536, 262144
-- 1048576, 4194304, 16777216, 67108864, 268435456 or 1073741824, appending data
-- by highlighting in KOReader may change the digest value.
function Document:fastDigest(docsettings)
    if not self.file then return end
    local file = io.open(self.file, 'rb')
    if file then
        local tmp_docsettings = false
        if not docsettings then -- if not provided, open/create it
            docsettings = require("docsettings"):open(self.file)
            tmp_docsettings = true
        end
        local result = docsettings:readSetting("partial_md5_checksum")
        if not result then
            logger.dbg("computing and storing partial_md5_checksum")
            local bit = require("bit")
            local md5 = require("ffi/MD5")
            local lshift = bit.lshift
            local step, size = 1024, 1024
            local m = md5.new()
            for i = -1, 10 do
                file:seek("set", lshift(step, 2*i))
                local sample = file:read(size)
                if sample then
                    m:update(sample)
                else
                    break
                end
            end
            result = m:sum()
            docsettings:saveSetting("partial_md5_checksum", result)
        end
        if tmp_docsettings then
            docsettings:close()
        end
        file:close()
        return result
    end
end


Looks like KOSync uses the partial_md5_checksum as identifier for syncing. My problem was I just overwrote fanfics with updated versions leading to wrong partial_md5_checksum in metadata.epub.lua.

I deleted the sidecar folder forcing KOReader to recalculate the md5 sum and sync worked.
ilovejedd is offline   Reply With Quote