Sigil has trashed more files
 

Several weeks ago I reported a bug on the Sigil website, that the program had truncated one of the files in the book. I was able to recover that file. After that I saved before I did anything like split and merge and checked that all was well before I saved again. I finally got through the first pass and had gone back to proofread, when I discovered two more files that had been truncated. I was able to recover those from a backup I had made at the time of the first problem. Now, this morning I added a cover page and cover and wanted to see what it looked like in Kindle Previewer. Previewer couldn't compile the book. Why? Too many unresolvable links in the toc. Three more truncated files. One, the only thing that is left is the heading.

The only place those edited files exist is in the last mobi file created by Previewer, and I don't know if it's even possible to get at those.

So, all that's left I guess is to go back to the raw files and rebuild the truncated files from scratch. A LOT of work. And, of course, I'll have to find some other program to do this with, lest Sigil trash more files in the process. And I'll have to keep Sigil running while I do this and pray Windows doesn't do and update before I get it done. No telling what Sigil would do with all those unresolvable links, probably throw them away.

Version 0.7.1, but I've downloaded at least two new versions while working on this book. (Oh, and you moved the Windows versions from Program Files to Program Files (x86), so my registry's screwed up somewhere and I can no longer use Open with for Sigil. Thanks.)

There's no real question here, I'm just venting. Of course, if anyone has any suggestions, I'd be happy to hear them. Have a nice day. :)

What's left of the above mentioned file:

You've not got HTML Tidy turned on to auto, have you?

It's not really moved to Program Files (x86), just that there isn't a 64-bit version anymore. Perhaps you should clean your computer of all things Sigl and just reinstall the latest version.

I don't let Tidy have a go at trying to fix things and I've never had any files truncated.

I've had Tidy turned off from the beginning.

I don't know how to help with the data loss issue, but there is an easy solution to solve the Open with problem:
Just reinstall Sigil (again...I know) and then tick the "Associate ebook files with Sigil" option in the window that appears at the beginning of the installation. That's all.

If you ever get an error about a tag not closed, etc., do you ever tell Sigil to fix it automatically? I never do and I've never experienced what you're getting.

When I'm doing major changes to a book, I often make a copy of the epub so if I screw something up badly, I still have the full version of the untouched book to recover. Until you sort out why this is happening, I'd suggest always making a copy before you start editing again.

No, I never have Sigil fix anything.

Have you been doing search and replace? This function can cause things to go sideways fast. It may be possible depending on your source that undisplayed characters might cause a problem.

Is the problem isolated to just this one file? Is there another you could try to see if this particular file is troubled? If it is a cheesed copy of a copyrighted book who knows what is in it.

You can probably get back something of a copy by running the .mobi through calibre. At least that way there should not be anything wrong with file except too much calibre stuff in it...it is ugly, but functional.

Yes, I did a few global search and replace and I suspect that may have been at least part of the problem (two of the files were truncated right where there would have been a replace.)

There have been six files total...so far. Previewer said there were unresolvable links to these latest three, and I'm just hoping the rest are intact. And, no, I'm not ripping off someone else's work.

I was mistaken about the mobi files. The files were already gone in the earliest mobi copy I made. There was an update to Previewer and maybe that's why it didn't complain before.

JimmyG
If your S&R goes sideways and leaves a malformed section, you could end up with anything if you let Sigil fix the trouble before exiting..
:bulb2: always save before doing a global S&R. Discard (and reload the document) is your friend if you then find yourself over the cliff ;)

Not just save, but make an actual copy of the book for backup. The problem is there's no way to know if part of a file has gone south, short of checking each file. Reports tells me that file above is 68K...

I've got two of the files rebuilt and working on the third (outside of Sigil.) When I get them done and put back into Sigil, I'm going to have to go through and check each file (54, it's a BIG book) and see if there are any others missing.

I've got a list of things I have to check, repeated errors I've noticed, to see if I've caught them all. Do you know if finding and replacing each instance in All html files is as hinky as Replace all?

Not sure what "hinky" means? S & R in itself probably isn't the problem. But it tends to do EXACTLY what you tell it to! Never forget it acts on the HTML, not just the text. I would always do a few individual S & R operations before committing to a Replace All. Even then, Save before and check immediately afterwards.

I have frequently seen S&R skip (on the find) some matches. If you do a second pass, it now finds them.

It could be something hinky with my pretty-basic pattern , but I have also seen this happen with a long standing, saved search.

It has been a Long-ago time (~V4) since I have seen code just disappear (no S&R). Then the code usually had WEB page (java) artifacts or major code problems.
:bulb2: You might W3C validate BEFORE you let Sigil touch unproven code.

Well, I got the three files rebuilt and put back into Sigil (one missing right angle bracket,) and checked all the other files and found them intact. Now, I can go ask the question I was going to ask before the shtf.

No idea about the Sigil issue but you can recover your HTML files from your Mobi file using Calibre with the MobiUnpack plugin.

I was wrong the files were already gone when I made the mobi files, but that's good to know for future reference. Thanks.

I had a similar situation yesterday. Did a global replace and then opened some other html files. Some were fine, some were truncated, showing only the first paragraph or so. Went back to an earlier version and tried again as I had already saved before noticing the issue. Same replace but this time different truncated files. Finally had one of the truncated files also displayed a message at the top that there was an error in the html and that it was rendering everything up to the error, was looking at them in bookview. But not all displayed that message. Checked in codeview, saw the error and fixed it and things were fine then. I'm sure it was related to some auto clean up of the html that went bad.

That is why you should never trust auto cleanup and always take care of your S&R. It can go horribly wrong.

Actually...

This has happened to me. And damned if I know why. I don't believe I was S&R'ing, but...I was working on a file, very short, simple. After I had finished it, I split it into its designated "chapters." For reasons I can't quite fathom, (and, yes: Tidy is never turned on), when I tried to go to BookView, I got an "unresolved error" warning, and then...kablammo, the file was truncated. Content simply disappeared, ala Ye Olden PSO'D (Ye Olden Pink Screen O'Death, for you noobs). I was pretty gobsmacked. And, yes, I then had unresolved links in the ncx, because of course, content had gone MIA.

My comprehension of what happened was that somehow, after doing a simple s&R (not regex), an element did not have a closing tag. When I attempted to go to bookview, I got the error, and the inquiry "clean it or do it manually?" I said manually, but when I got the file "back," half of it was simply GONE, with the big honking pink error message.

I replaced the original (not altered) html content from the source file (pre-sigil) and repaired it the long way round, but honestly--still not quite sure what made it go "hinky." (Wombat: bad, smelly, something rotten. Twisted. ="Hinky.") I stared at it for a while, but never did figure out what made it go bats**t. This happened about 3 times in a row, and I don't know what triggered it, other than an unclosed element that I could not find, but, more importantly, that I was not able to fix manually. When I chose "fix manually," the content had been "cleaned." (And had disappeared). But Tidy is OFF. :chinscratch:

So, yes: I've had the PSO'D again, now in the latest version of Sigil. Win7, running the 32-bit version, as there isn't one for those of us with 64.

Hitch

I've had this on occasion as well. I don't have HTML Tidy on, but Pretty Print is.

Because of this (as well as other reasons), I never work on the original file, and before making a significant change have a habit of using Save As so that I have a few copied at different stages, so if it cocks up I can revert back. Once I know the significant change has not had a detrimental effect on the book, I'll save it and delete the extra copies as I now know the file is ok.

Ditto:

If this helps in the troubleshooting, I have Pretty Print on as well.

H

Hmm, it really sounds if there is something peculiar in the source HTML file which makes Sigil to act funny. Strange that you can't find it. Have you tried looking at it via a hex-editor?

+1 to having Pretty Print on.
It is RARE that I have seen this.
Currently, I get random (QT) windows crashes more often. And NO, I have not filed a bug report as I have not identified any sequence.

I have had the sense, but never been able to prove it, that manual is not really manual anymore, when you get that error box.

If this a QT5 bug, then I will take the old horrible one in its place. At least it was more predictable in its badness.

Wow-- scary stuff here. Glad I'm in the habit of starting each day with a Save As to make a backup.

I've been working on this book for two months now. I am constantly using the Validate EPUB option to check for errors. And I've had Tidy on on the whole time.

I work predominantly in the Book View; going into code view, only to clear up <div> tags or any Validate errors that crop up. AND I have used search and replace many times, often with the Replace All option.

So far, everything works like a charm. But, hearing all these horror stories, I'm going to be a lot more careful, from now on.

Hi, Tox:

No, I have not. Good idea, though. I don't know if I could replicate it. I think that very simply, there was, somehow, via regex, an unclosed tag. I now forget what I was doing (changing spans to...nope. Not that. I was changing a paragraph class to a header class, I think). Yes, I think I likely had an unclosed header tag, i.e. an opening class of h1 and a closer of </p>. Either that, or, I've found that Sigil can get very witchy about improperly called style classes, when I have a brain-fart.

But I think I recall that I was changing paragraph classes to header classes, and then I was changing a named para class to another named class--just a name swap. Then, kablammo! PSO'D. I was seriously bummed. I was extremely glad that I'd had my html editor (NoteTab Pro) open with the originally-cleaned file in it...so I swapped out the ending html. It occurred a second time, so...obviously, I'd screwed the pooch somewhere, but the issue is, I never got the chance to "manually clean" the errata; what happened is when I endeavored to pop over to BookView (not yet in habit of using "preview" mode, TBH), we had error message--answer "manual"--and PSO'D, no "manual cleaning" option. Somehow, the combination of an error, and what seems to be an "auto-clean" when you slide over to BV, causes this to occur. That's my best remembrance right now.

Hitch

That's basically what happended to me.

I was cleaning out messy Calibre code and replacing <p> tags with proper heading tags when I must have missed something and then it all disappears with an auto-clean.

I've gotten religion on the backup front myself.

I'm just the opposite, I always work in code view. The only time I look at book view is to make sure something looks the way I want it to. The other day I was doing just that when I noticed a misplaced apostrophe and deleted it. When I went back to the code view, it yammered at me for a nbsp entity. What? The only entity I use is amp. Where was that nbsp? Right where I had deleted that apostrophe. Sigil put it there and then complained to me about it! LOL.

I've heard that many times. I don't know how you guys do it. At best, I can do maybe 30 words a minute. I don't stop for spelling, or even to correct errors in tense or punctuation. When the ideas are flowing, I just want to get them out. Then later, during the rewrite, I correct all that stuff.

If I were in Code View, adding tags along the way, that would really put a wrench in my creative process.

When it comes to the whole data loss problem, the Book View seems to be is a LOT safer. I've never lost anything! As a word processor, Sigil's Book View is pretty robust. And--that Table of Contents window is great as an outline processor.

Very simply, because most of us just don't use Sigil as a word-processor. I wouldn't, and I live with the thing daily.

If you have/use Word, I have to say that its outlining mechanisms are fabulous. I use Outline View, but I also love the Document Nav view, which rocks. It's one of Word's little secrets (View-->Navigation Pane). Use headers for structure, not styling, and the Nav Pane is an outline. :2thumbsup Significantly less data loss, as well.

Hitch

I agree with Hitch. Use heading styles (and other/own styles) in Word for structure. That will help tremendously. Styling I do in Sigil and my structure is already there in that case.

Clicking on the p taskbar icon in Sigil when you want a new paragraph is comparable to pressing Enter in Word. The same for clicking h1 for a header. Shouldn't hold up your creative flow too much!

The answer to that is, of course, don't enable auto-clean and restrict yourself to Pretty Print for manual tidying. And press Ctrl-S first!

I have to say, it is not intuitive to me that Pretty Print would "clean" upon a switch from CV to BV. That's not something that I comprehended. After all, to my mind, that's not "open" or "save;" it's just a view switch. (Hmmm...I ponder, would it happen if you used the Preview from Code View? The PSO'D?).

I certainly have historically not set any type of auto-clean, but I see that in my latest upgrade to 0.7.1, it seems to be reset to auto-clean on open/save with PP. This certainly partly explains the issue, BUT....there's still something glitchy, because as I described, I never did have the opportunity to "manually clean" the missing tag. It was as I detailed, to the best of my recollection, and as it happened twice or so, I think I recall it reasonably well, given how many books have come since. ;-)

Hitch

This just happened to me TWICE while I was working on a book yesterday. It occurred after I renamed some images. All of the code occurring after the renamed image -> end of file went poof. I didn't catch this until long after the image rename on a quality pass (luckily I just copied/pasted the missing code from a previous version).

I will have to keep a real eye out for this next time, to try to catch exactly when this "bug" is occurring, so I can supply exact test cases.

After upgrading, it's best to check the cleaning options. With 0.7.2 if you have Open checked, then when you switch from Book View to Code View your code will be cleaned. This is because you are opening the code Book View created (Book View editing is provided by Qt, and Qt will not keep the same code layout you have in Code View).

If you switch from Code View to Book View, then if the code is invalid, it should stop you from switching. Forcing the code to be valid before switching prevents losing text/tags.

The same should happen when saving an EPUB or opening an EPUB - you should be prompted that the file is invalid and that cleaning could cause data loss, although it will allow you to open/save an invalid file.

If, as has been suggested, you are not being prompted about invalid code when switching to Book View, then that's an issue. I haven't been able to reproduce it, but will try to look into it more to see how that might be possible.

I wonder if the prompt to manually or automatically clean should be removed anyway - so that if the code is invalid it will just refuse to switch to Book View. Though if the bug is that the prompt isn't triggering on an invalid file, it won't help much.

I have 0.7.1, and I thought I'd set it. Apparently not. Still, not intuitive to me. (Also, am I the only one who finds the Sigil Preferences on the edit menu weird? I mean, this isn't insurmountable, obviously...but I would never think to look for a program's defaults or preferences on the edit menu. Just me being whiny.)


See....here's where I think we are having an issue. Sure, I agree; the program should not allow you to save bad code, or switch to BV with bad code. My problem is that I never got the chance to clean the bad code. Despite selecting "manual," all that happened was the PSO'D, and half my html file vaporized.

No, no: that's not what I suggested (that I was not being prompted about invalid code when switching to BookView). What I said was, I was prompted, and when I selected "manual," the dreaded giant pink error message showed up and half my file disappeared--everything from the "error" to the EOF. Yes, I got the prompt, as far as I can recall; it's what happened after that that was problematic. I think it was as though it ignored the "manual" selection, and then rendered BV anyway, and in so doing, showed the (now) corrupted file. Does that make sense to you?

As far as recreating the issue...try having a paragraph with an opening header class and a closing p class, see if that does it, or, try a named paragraph class with some typos, like...forget the italics around a class name, or something like that. I know I had at least one of those errors. I think the "PSO'D" error was the opening a line with a header class and closing it with a paragraph class (or a closing span tag, instead), but...if I can recreate it, I'll send you the file.

Hitch

I think this is a current Open Software standard.. It makes some sense, you are editing something :D

My e-mail (really old, Eudora 7) and News (NNTP) programs moved the options to the Tools menu :rolleyes: (Are Account Settings, Tools? :chinscratch:)

At least it make more sense to me than 'Files'