Finally I figured it out. It is not a calibre or kobo bug. After reading through many posts on this forum, I know that calibre doesn't read database created by kobo (nickel), but creates its own database on kobo storage. So books are not sideloaded by calibre but by other ways (copy paste or via
kobo cloud services in my case) will not be recognized by calibre. Thank you for your patient help @davidfor.