Issue with Multiple File Reference Errors in Spine

[KevinH]

@DNSB and @BeckyEbook,

Another idea to explore is to handle this is to add two special cases in the ReadingOrder (EditRO) dialog that only allows non-contiguous selection for block move (not for drag and drop) to move a selection of files to either the top of the list (keeping selection relative order) or to the bottom if the list (again keeping selection relative order).

That would allow you to separate things into two groups one following the other but keep initial relative order within the selection.

Wouldn't that be useful in the case that Becky indicated?

Update:

Since BookBrowser already allows non-contiguous multiple selection, we could add 2 pop-up menu items:

Collect at Top of Reading Order
Collect at Bottom of Reading Order

Where:

At Top moves all selected files (one by one in reverse order) (ie from the bottom of the selection) to the top of the reading order

At Bottom moves all selected files (one by one in forward order) (ie, from the start of the selection) to the bottom of the reading order.

That should be pretty easy to implement since it would not use Qt's buggy drag and drop.

I could easily add that feature to BookBrowser.

Just let me know if any of these options would help.

And fwiw, by the time you non-contiguously multiselect files, you could have used the keyboard shortcuts in the EditRO to move them one by one to the Top or Bottom faster.

That is why ToTop and ToBottom exist in that Reading Order Editor.

[BeckyEbook]

That's an interesting idea.
I can select a few non-contiguous elements and move them either to the very top or the very bottom, but that would make it easier to set the order for larger projects.

EDIT:
`Collect at Top of Reading Order`
`Collect at Bottom of Reading Order`

I'm all for it.

[byword]

The ability to select multiple consecutive files in BookBrowser and change their reading order is an extremely useful feature that I use frequently when editing books.

After exporting an InDesign (.indd) file to HTML and editing it in Sigil, the reading order is not always sequential because of the structure of the exported .indd file. As a result, files that should be adjacent are often scattered throughout the BookBrowser. The ability to move multiple consecutive files together is very useful for restoring the correct reading order.

This issue did not occur in versions prior to Sigil 2.7.x. In those versions, selecting and moving consecutive files worked correctly. It appears to be a regression introduced in the 2.8.x series.

Would it be possible to compare the code between versions 2.7.x and 2.8.x to identify what changed and locate the cause of this regression?

[KevinH]

You misunderstand consecutive drag and drop moves always have worked and still work properly. I tested it.

But your test moved 2, 4, and 6 which are non-consecutive. It is only non-consecutive (ie. the set of resources to move are not contiguous) drag and drops that does not work. And it has always been broken.

Your example only fails because you tried to move non-consecutive resources by drag and drop.

Sigil's code is not actually involved in BookBrowser drag and drop, just Qt internal code in their QStandardItemModel. So there is no Sigil code involved and therefore no change.

My change only prevents drag and drop when you have selected multiple but *non-consecutive* files. Selecting consecutive files and using drag and drop still works correctly even after the change I just pushed.

If you read through the thread we are now discussing alternative ways for users to quickly group non-consecutive files at the top or bottom of the reading order via a new pop-up menu.

When we come to a general agreement, I will also implement that approach.

So unless you can show a testcase where normal drag and drop of consecutive files is failing in some repeatable manner (just like your non-consecutive testcase showed) I think we have the correct solution in master to prevent that bug from hitting anyone in the future, and will add a tool to handle selections that are non-consecutive.

I used the term "non-contiguous" here and earlier to be a synonym for a selection that has non-consecutive members. I used the term "contiguous" to mean all selected items are side by side in one block (ie, consecutive). Please re-read this thread with that in mind.

So I hope this explains things a bit more.

If you do have a repeatable testcase where contiguous (consecutive) selections fail under drag and drop in Bookbrowser please do post it here and I will look into it as a separate issue.

[DiapDealer]

Originally Posted by byword

The ability to select multiple consecutive files in BookBrowser and change their reading order is an extremely useful feature that I use frequently when editing books.

Originally Posted by byword

As a result, files that should be adjacent are often scattered throughout the BookBrowser. The ability to move multiple consecutive files together is very useful for restoring the correct reading order.

Originally Posted by byword

This issue did not occur in versions prior to Sigil 2.7.x. In those versions, selecting and moving consecutive files worked correctly. It appears to be a regression introduced in the 2.8.x series.

I'm confused. You described a bug that was triggered by selecting then dragging and dropping NON-consecutive files, but now you seem to be suggesting there's a problem dragging/dropping multiple consecutively selected files.

We're not interested in preventing the ability to select multiple consecutive OR non-consecutive files in Book Browser. As mentioned previously: selecting multiple files (contiguous or non-) is important for telling plugins which files to operate on. We're not even interested in preventing the dragging and dropping of consecutively selected multiple files. We're mainly interested in preventing the moving of non-consecutive multiple selections (which your bug report describes).

So which behavior is different from Sigil 2.7.x and Sigil 2.8.x: the ability to move multiple non-consecutively selected files? Or the ability to move multiple consecutively selected files?

It is the former that has been prevented by Qt itself for a long, long time. The latter should still be possible.

EDIT: what Kevin just said.

[byword]

Originally Posted by KevinH

You misunderstand consecutive drag and drop moves always have worked and still work properly. I tested it.

But your test moved 2, 4, and 6 which are non-consecutive. It is only non-consecutive (ie. the set of resources to move are not contiguous) drag and drops that does not work. And it has always been broken.

Your example only fails because you tried to move non-consecutive resources by drag and drop.

Sigil's code is not actually involved in BookBrowser drag and drop, just Qt internal code in their QStandardItemModel. So there is no Sigil code involved and therefore no change.

My change only prevents drag and drop when you have selected multiple but *non-consecutive* files. Selecting consecutive files and using drag and drop still works correctly even after the change I just pushed.

If you read through the thread we are now discussing alternative ways for users to quickly group non-consecutive files at the top or bottom of the reading order via a new pop-up menu.

When we come to a general agreement, I will also implement that approach.

So unless you can show a testcase where normal drag and drop of consecutive files is failing in some repeatable manner (just like your non-consecutive testcase showed) I think we have the correct solution in master to prevent that bug from hitting anyone in the future, and will add a tool to handle selections that are non-consecutive.

I used the term "non-contiguous" here and earlier to be a synonym for a selection that has non-consecutive members. I used the term "contiguous" to mean all selected items are side by side in one block (ie, consecutive). Please re-read this thread with that in mind.

So I hope this explains things a bit more.

If you do have a repeatable testcase where contiguous (consecutive) selections fail under drag and drop in Bookbrowser please do post it here and I will look into it as a separate issue.

I apologize. I made a mistake. I completely understand what you meant.

You are right; they are non-contiguous files. What I wanted to point out is that I had been continuously moving these non-contiguous files in versions prior to Sigil 2.7.x without encountering any such errors. I first noticed this error in version 2.8.x. This was the point I wanted to make. Since you mentioned that this error existed previously as well, I will double-check that part too.

[byword]

Originally Posted by DiapDealer

I'm confused. You described a bug that was triggered by selecting then dragging and dropping NON-consecutive files, but now you seem to be suggesting there's a problem dragging/dropping multiple consecutively selected files.

We're not interested in preventing the ability to select multiple consecutive OR non-consecutive files in Book Browser. As mentioned previously: selecting multiple files (contiguous or non-) is important for telling plugins which files to operate on. We're not even interested in preventing the dragging and dropping of consecutively selected multiple files. We're mainly interested in preventing the moving of non-consecutive multiple selections (which your bug report describes).

So which behavior is different from Sigil 2.7.x and Sigil 2.8.x: the ability to move multiple non-consecutively selected files? Or the ability to move multiple consecutively selected files?

It is the former that has been prevented by Qt itself for a long, long time. The latter should still be possible.

EDIT: what Kevin just said.

I made a mistake in my writing. I apologize for the confusion. You are correct that the error only occurs with non-contiguous files.

[KevinH]

If you reread this thread you will see that making any other drag and drop change that is consecutive seems to fix this problem. Note: this is a well known Qt limitation in their QStandardItemModel and has been even back to Qt5.

You were probably just lucky to always have your last drag and drop be consecutive which force removes any duplication but the duplication was still there prior to that. Again, all of BookBrowser drag and drop code is internal code in Qt's QStandardItemModel.

[byword]

Originally Posted by KevinH

But your test moved 2, 4, and 6 which are non-consecutive. It is only non-consecutive (ie. the set of resources to move are not contiguous) drag and drops that does not work. And it has always been broken.

I just installed Sigil version 2.7.6 on Windows 11 and tested moving non-contiguous files. In version 2.7.6, the spine organizes properly without any issues. I edit dozens of books every month using Sigil, and I first noticed this error in version 2.8.x. (Perhaps the error occurred intermittently in the past, but since it wasn't common, I might have just manually fixed the spine and moved on.)

I hope this information is helpful in reviewing potential improvements.
Thank you as always for your prompt responses and continuous updates.

[DiapDealer]

2.7.6 uses the same version of Qt that 2.8.1 uses. So nothing on that particular front has changed.

[BeckyEbook]

But maybe "something" was done in version 2.8.0 – after all, a new feature has been added:

- create a separate Change Reading Order dialog for those who can not use drag and drop

I'll run some tests on 2.7.6 when I have a free moment.

[elibrarian]

Originally Posted by BeckyEbook

But maybe "something" was done in version 2.8.0.

I'll second that. I don't know if there's any connection, but:

I usually add an .xhtml file as the first file after the title page. Up until and including 2.7.6, the imported file was placed after the file in the Bookbrowser that I right-clicked to get the menu with the Add Existing Files.

In 2.8.0 the imported file was invariably placed as the last file in the Bookbrowser, and I had to manually place it where it belonged.

In 2.8.1 the functionality is restored to the 2.7.6-behaviour.

So "something" must have happened in 2.8.0.

Regards

Kim

[KevinH]

Again, to be clear, the drag and drop code is question is internal to Qt and has not changed! A web search has made it clear that Qt code has never properly handled drag and drop for non-consecutive/non-contiguous selections. That is the issue we are tring to address here.

The code in AddExisting is different and a number of bugs associated with bulk addition have been fixed in 2.8.1 including when to make "bulk" additions vs slower normal additions that allowed us to bring back targeting where to exactly to add a file. Bulk additions in AddExisting had broken that.

This is one of the primary bug fix reasons for putting out 2.8.1.

The changes to add a Edit Reading Order, are independent and have users re-order a simple list of file paths (not actually reorder BookBrowser itself to be safe). Once that list is happy, the code sets the reading order in the OPF and all we do is rebuild BookBrowser from the updated OPF. So Edit Reading Order can but need not use Qt's QStandardItemModel at all to do any reordering. Drag and Drop in Edit Reading Order is limited to a single item move so non-consecutive moves are not possible and therefore not part of this bug.

So people are conflating 3 different changes that have nothing to do with each other or this issue.

Many Sigil 2.8.0 bugs have been reported in the last couple months and all are fixed in Sigil 2.8.1 which is why we called it a bug fix release.

As far as I know, this bug is the only new bug reported for Sigil 2.8.1 and it is only triggered by unusual circumstances that we have now prevented from happening.

[KevinH]

All of that said, I just tried the original test case 3 times with Sigil 2.7.6 and not once did I see the duplication.

I checked and both Sigil 2.8.1 and Sigil 2.7.6 are running the exact same Qt version and even Qt build (at least on MacOS).

So I can not explain why this happens at all.

It is inconsistent with all the "this is broken in Qt" webite searches (and I did a lot) and it is really true that the actual drag and drop code in BookBrowser actually does *not* use any Sigil code at all.

So maybe there was a change associated with how we update the opf *after* the drag and drop but that would be inconsistent with what the Qt QStandardItemModel actually states in its own source code.

Maybe it is simply a timing change?

[DiapDealer]

Originally Posted by KevinH

I checked and both Sigil 2.8.1 and Sigil 2.7.6 are running the exact same Qt version and even Qt build (at least on MacOS).

The Windows Qt build is the same for those two versions as well.

Originally Posted by KevinH

Maybe it is simply a timing change?

That would be my guess.