Query about GitHub and User Guide correction

[philja]

I forked and then cloned the User Guide repo to my desktop. I opened Sigil with its default of a new blank epub3 (without any user templates).

Then, as per the README.md file, I used the latest version of the FolderIn plugin to bring in the 'src' directory. I used the toc in Preview to locate the two files I wanted to edit. I did the edits and then changed one image. I never opened any other file in Code View. Next, still following the README, I did the 'Mend and Prettify' and ran epubcheck. Then I used FolderOut to save the mods to my local cloned repo.

All seemed correct, and using the cli I committed and pushed the changes to my GitHub account.

(I did save the modified epub to a temp file to look at my changes - they seemed ok.)

When I examine the commits in my GitHub, I was surprised and somewhat shocked to find 48 files changed with +2,662 -2,593 changes. Closer examination showed they include the changes to the image and the two xhtml files I edited. Other changes seem of little consequence - a space eliminated, line breaks added.

I suppose these changes were all the work of Mend and Prettify? Before offering my edits as a PR, I'd like your comments as to whether what I see is normal or if I have messed up somewhere. I'm certainly not an expert on GitHub.

[DiapDealer]

I'm pretty sure there have been some slight changes to Sigil's prettify (and some whitespace preservation changes in mend) since the last push to the User Guide repo. That might explain some of the changes. Or maybe we've not followed our own instructions for running mend & prettify for quite a while when pushing changes. I'll let Kevin speak to what the Folder In/Folder Out process may have contributed.

For small focused changes, I just tend to make direct edits to files in the git repo rather than using Sigil. That way I can use git --difftool --dir-diff to see what changed before committing them. But that involves installing/configuring a difftool for git. I use the cross-platform Meld program on Linux and Windows.

Can you share your github repo so we can see the changes before an Pull Request gets made? That way we might be able to determine whether it's a lot of changes due to pretty-print/mend changes? If so, they will probably make it into the official repo at some point anyway.

TL;DR: I don't think you did anything wrong.

[KevinH]

But I would only use Mend not Mend and Prettify across the entire epub. I would only uuse Prettify in the 2 files you changed if needed.

FolderIn and FolderOut should make no changes.

[philja]

Originally Posted by DiapDealer

Can you share your github repo so we can see the changes before an Pull Request gets made? That way we might be able to determine whether it's a lot of changes due to pretty-print/mend changes? If so, they will probably make it into the official repo at some point anyway.

TL;DR: I don't think you did anything wrong.

Relieved that you don't think I cocked it up. How do I share the repo? Just give you the url of the commit page?

[DiapDealer]

Just your GitHub account name should be enough for me to find your public repo(s):

https://github.com/<Name>

But you may want to try what Kevin suggested. You can delete your forks (both on github and locally) and start over with a new fork/clone edit. Might be a small pita but we'd know for sure.

[philja]

Originally Posted by DiapDealer

Just your GitHub account name should be enough for me to find your public repo(s):

https://github.com/<Name>

But you may want to try what Kevin suggested. You can delete your forks (both on github and locally) and start over with a new fork/clone edit. Might be a small pita but we'd know for sure.

I think I'll go for the remake. It shouldn't take long - the edits were not substantial and it will reinforce my learning on GitHub.

[DiapDealer]

Originally Posted by philja

I think I'll go for the remake. It shouldn't take long - the edits were not substantial and it will reinforce my learning on GitHub.

Fair enough.

Do yourself a favor and do a "git diff" locally before committing and pushing any changes. That way if anything goes wonky you can simply delete any uncommitted changes with a git command (git reset --hard HEAD). Then you're already to try again without needing to refork and reclone.

[philja]

I started afresh. Deleted the fork in my GitHub and the local clone and branch. And then reforked, cloned, and created a new branch.
Opened Sigil with a default epub3 - no user templates. Used FolderIn to load the 'src'. Opened one xhtml file in Code View - made the edits, deleted one image and added a replacement. Used right-click menu in Code View to Mend/Prettify in just that one file.

Opened the second xhtml, made the edit, same single file Mend/Prettify.

Validated with epubcheck and used FolderOut to make the export.

Next, in the cli, I ran 'git status' and found that a further 12 files had been modified in addition to the image I changed and the two xhtml files I edited. Of course, when src was loaded in, cover.xhtml was opened but I didn't touch it.

The two xhtml files I edited were tutorial_*.xhtml

See the attached screenshot.

Any ideas what caused the changes in the other files?

I haven't done a commit - leaving it at that for tonight.

[philja]

Seeing DiapDealer's suggestion of a git diff prompted me to dump the diffs into a text file.

I have attached this file. It shows remarks about each of the 'changed' files.

[DiapDealer]

A few things jump out:

1) There are many files that have nothing changed but:

-</html>
+</html>

Not sure what's going on there if you didn't make any changes to those files. Unless it's just a line ending change, But I don't know what would prompt that--especially only one at the end of the file. But that's not a huge deal by itself. It would be nice to find out what's going on though.

2) Non-breaking space entities are being converted to their unicode character equivalents in some of the files you didn't edit. I'm guessing you don't have the &#160; entity in your EPUB3 Preserve Entities list in Sigil's Preferences. But I don't know what's prompting those edits unless the Automatic Mend on Save thinks something is malformed. I would prefer that those particular entities remain entities if possible.

3) The content.opf file is being modified because you're deleting the original image and adding a new one with the same same. And because Sigil's version meta has changed, along with modified meta.

4) The really troubling/puzzling one is that the toc.ncx is being wildly overhauled. And I have no idea why. I don't think it's overall functionality has been altered though--which only makes it weirder.

I may have to do some of my own testing with Sigil and FolderIn/FolderOut to see if I can figure out what might be going on. Hold tight.

[philja]

Originally Posted by DiapDealer

A few things jump out:

1) There are many files that have nothing changed but:

-</html>
+</html>

Not sure what's going on there if you didn't make any changes to those files. Unless it's just a line ending change, But I don't know what would prompt that--especially only one at the end of the file. But that's not a huge deal by itself. It would be nice to find out what's going on though.

2) Non-breaking space entities are being converted to their unicode character equivalents in some of the files you didn't edit. I'm guessing you don't have the &#160; entity in your EPUB3 Preserve Entities list in Sigil's Preferences. But I don't know what's prompting those edits unless the Automatic Mend on Save thinks something is malformed. I would prefer that those particular entities remain entities if possible.

3) The content.opf file is being modified because you're deleting the original image and adding a new one with the same same. And because Sigil's version meta has changed, along with modified meta.

4) The really troubling/puzzling one is that the toc.ncx is being wildly overhauled. And I have no idea why. I don't think it's overall functionality has been altered though--which only makes it weirder.

I may have to do some of my own testing with Sigil and FolderIn/FolderOut to see if I can figure out what might be going on. Hold tight.

1. As regards the files I didn't edit, cover file opens automatically but for the rest, I didn't touch them in any way - not even to open them in code view.

2. You're correct about the preserve entities. I've now added &#160; to the preferences file.

3. sounds logical

Standing by -

[BeckyEbook]

Ad 1. Specifically, this involves removing the 0A character after the closing HTML tag.

Ad 2. I think it would be a good idea to add this note about the need to enable &#160; in the "Preserve Entities" window to the "Preferred Workflow" section of the readme file.
BTW There is one entity-free character in the advanced_topics.xhtml file

Ad 3. Sure.

Ad 4. My guess is: Since the Guide is an EPUB3 file, to get a fresh toc.ncx file, you have to do it manually by running "Generate NCX and OPF Guide from EPUB3 Nav," and this hasn't been done in a long time, which is why there are so many changes.

[DiapDealer]

Originally Posted by BeckyEbook

Ad 1. Specifically, this involves removing the 0A character after the closing HTML tag.

But what is triggering this removal in many xhtml files if only the two xhtml files are being edited?

Originally Posted by BeckyEbook

Ad 4. My guess is: Since the Guide is an EPUB3 file, to get a fresh toc.ncx file, you have to do it manually by running "Generate NCX and OPF Guide from EPUB3 Nav," and this hasn't been done in a long time, which is why there are so many changes.

But OP has specifically said only three files were edited by him: 2 xhtml and 1 image file. So what triggered the toc.ncx rebuild if he didn't? His specific edits certainly wouldn't have warranted a manual ncx rebuild.

[DiapDealer]

So far, I have not been able to recreate what @philja is experiencing. I've tried to duplicate as close as possible the steps they took and the changes they made. I'm using the latest version of Sigil (on Linux) and the latest versions of the FolderIn/FolderOut plugins

• I have an up to date clone of the sigil-user-guide repository
• I use the FolderIn plugin to load the guide's source into a new epub in Sigil
• I make the same modification @philja did to the tutorial_find_replace.xhtml file (using Sigil's Code View)
• I run Mend and Prettify on that one file
• I delete the saved-searches-find-replace-context-menu.png file and add a different image with the same name (using Sigil's Delete and Add Existing tools from BookBrowser)
• I make the same two modifications @philja did to the tutorial_regex_reference.xhtml file (using Sigil's Code View)
• I run Mend and Prettify on that one file
• I use the FolderOut plugin to write the the epub's src back to the sigil-user-guide repository

Using the git command line in the sigil-user-guide repository, git status shows me 4 files have been modified:

• the saved-searches-find-replace-context-menu.png file (binary diffs not shown)
• the content.opf file
• the tutorial_find_replace.xhtml file
• the tutorial_regex_reference.xhtml file

git diff shows me the very minimal changes to those two xhtml files and the content.opf file that I would expect.

My first question would be which platform (Windows, Linux, macOS) are you using to do this?

My second would be how exactly are you making theses edits? Are you manually editing in Sigil's Code View, or are you using external editing tools? Are you making the simple edits your diffs show, or are you copy/pasting larger swathes of code (or possibly whole file contents)? Are you copy/pasting anything into the sigil-user-guide repository at a file system level, or is everything being done from within Sigil?

P.S. I did a Mend & Prettify on the entire epub just as a test and the results were no different.

[philja]

Originally Posted by DiapDealer

So far, I have not been able to recreate what @philja is experiencing. I've tried to duplicate as close as possible the steps they took and the changes they made. I'm using the latest version of Sigil (on Linux) and the latest versions of the FolderIn/FolderOut plugins

Using the git command line in the sigil-user-guide repository, git status shows me 4 files have been modified:

• the saved-searches-find-replace-context-menu.png file (binary diffs not shown)
• the content.opf file
• the tutorial_find_replace.xhtml file
• the tutorial_regex_reference.xhtml file

git diff shows me the very minimal changes to those two xhtml files and the content.opf file that I would expect.

My first question would be which platform (Windows, Linux, macOS) are you using to do this?

My second would be how exactly are you making theses edits? Are you manually editing in Sigil's Code View, or are you using external editing tools? Are you making the simple edits your diffs show, or are you copy/pasting larger swathes of code (or possibly whole file contents)? Are you copy/pasting anything into the sigil-user-guide repository at a file system level, or is everything being done from within Sigil?

P.S. I did a Mend & Prettify on the entire epub just as a test and the results were no different.

Sounds like you got my procedure correct. Before starting, I upgraded the FolderIn/FolderOut plugins to the latest available.

For your first question: I did this editing using linux UbuntuStudio 26.04 LTS with Sigil 2.8.1 (Sigil-2.8.1-x86_64.AppImage).

Second q: mostly manual editing, typing by hand into Code View just the minor amount of edits you saw, with two exceptions.

First, in the file 'tutorial_advanced_find.xhtml', I copied the link for the Regular Expression Reference from the para at line 21, pasted it into a new para at new line 110 and then edited the para text by hand in Code View.

The second exception was in the file 'tutorial_regex_reference.xhtml' where I cut an item from the list of Multiple Characters and pasted it into the list higher up of Single Characters.

I never copy/pasted anything from outside the document. Nor did I use any external editing tools. All was done inside Sigil. And I never copy/pasted anything into the repository at file system level.