Tolerance and parsing mistakes

[KevinH]

And since clipboards are system level, I have never seen this on Mac OSX. Does it occur on Windows?

I will start to look at the Sigil startup code to see if anything related to things on the clipboard is done. My guess is on linux, the x11 clipboard has its own event thread and is not playing well with Qt for some reason ... but that is just a wag.

Well at least we have something to work with? Does anyone have Kate (a QT based text editor) on their machine. Does the precence of items on the clipboard upon starting it up impact Kate as well? Or any other Qt app text app on Linux? Or perhaps some other KDE text editor? Does it impact calibre's editor at all?

KevinH

[eschwartz]

It doesn't affect calibre's editor.

I don't use Kate, I use (g)Vim.
I do have ReText, which is totally unaffected.

[KevinH]

Okay,
In the clipboard history code of Sigil, anytime the clipboard changes, a changed() signal is sent. So as you highlight code in X11 it is constantly sending this signal where the current contents of the clipboard are compared against the highlighted text so far and added to the clipboard if and only if there is something in the system clipboard.

There is a screen to filter these signals out by mode (data or selection) but it appears to fail for x11. This only happen for X11 because it has a selection mode. On other systems you must hit copy not simply highlight something to activate the clipboard. I will see if the datachanged signal with proper filter would work better on Linux. The docs on changed(mode) signal are obviously incorrect for Linux or Qt has a bug since the mode check on the changed() signal in Sigil should be preventing this but obviously is not.

I will post something to try and fix this tomorrow for linux users to try. I'm off to sleep now.

KevinH

[DiapDealer]

Originally Posted by KevinH

Okay,
In the clipboard history code of Sigil, anytime the clipboard changes, a changed() signal is sent. So as you highlight code in X11 it is constantly sending this signal where the current contents of the clipboard are compared against the highlighted text so far and added to the clipboard if and only if there is something in the system clipboard.

There is a screen to filter these signals out by mode (data or selection) but it appears to fail for x11. This only happen for X11 because it has a selection mode. On other systems you must hit copy not simply highlight something to activate the clipboard. I will see if the datachanged signal with proper filter would work better on Linux. The docs on changed(mode) signal are obviously incorrect for Linux or Qt has a bug since the mode check on the changed() signal in Sigil should be preventing this but obviously is not.

I will post something to try and fix this tomorrow for linux users to try. I'm off to sleep now.

KevinH

I think the code is successfully filtering out the x11 selection signal (at least qDebug printouts indicate no x11 selection change events are getting past the

if (mode != QClipboard::Clipboard) {
        return;
    }

test in ClipboardHistorySelection.cpp), but I think that filtering those selection events is actually what's causing the performance bottleneck. I could be all wet, though.

[KevinH]

Hi DiapDealer,

Yes, I agree with your assessment. So I have just now pushed the code to use QClipboard's dataChanged() signal so the total number of signals being delivered just to update the damn clipboard history feauture should be greatly reduced.

So please try a pull from just now from Master and let me know if this has any impact at all. If not, then we are back to square one on this bug. There just are not many places the QClipboard is used in the Sigil codebase. FWIW, Mac OS X works well with dataChanged() and I am assuming Windows will as well.

Take care,

KevinH

[DiapDealer]

No discernable performance diifference on my end, I'm afraid, Kevin. When Sigil is launched with data in the clipboard, text-selection in Code View is still extremely sluggish. With no data in the clipboard upon Sigil launch, text-selection works as expected (for the entire session).

You say you're not experiencing this on your Linux Mint 17.2? Can I ask what desktop you're using? I'm using 17.2 with Cinnamon.

Is the whole QClipboard rigamarole to support Sigil's Paste from Clipboard feature (to make sure the most recent copy shows first in the dialog)? Or is there more to it? I'm afraid I'm just not up to snuff on why fhe clipboard contents get written/read to the preferance store/ini, or why the clipboard events/signals are being intercepted and manipulated in the first place.

[KevinH]

Wow, not good. I had such high hopes. This whole nonsense is just to keep a local copy of the last 20 things you have copied tot he clipboard. You only see this dialog if you go to the edit menu and select paste from clipboard history.

Would you please try one thing for me to see if it has any impact at all.

In Sigil/src/Dialogs/ClipboardHistorySelector.cpp please comment out the following lines:

void ClipboardHistorySelector::ApplicationActivated()
{
    // Turned on when Sigil is activated, put the latest text if any at the top of the history  

    // DiapDealer - comment out these next two lines                                
    // ClipboardChanged();
    // connect(QApplication::clipboard(), SIGNAL(dataChanged()), this, SLOT(ClipboardChanged()));

    // If we are currently showing this dialog, refresh the display                                                             
    if (isVisible()) {
        SetupClipboardHistoryTable();
    }
}

and similarly in this routine

void ClipboardHistorySelector::RestoreClipboardState()
{
    if (m_ClipboardHistoryItems->count() > 0) {
        QApplication::clipboard()->setText(m_ClipboardHistoryItems->at(0));
    }

    // DiapDealer please comment out the next line as well
    // connect(QApplication::clipboard(), SIGNAL(dataChanged()), this, SLOT(ClipboardChanged()));
}

Then try rebuilding. With those changes, we will have disabled the whole remember the clipboard nonsense.

And then try rebuilding and see if it has any impact. If not, then we need to start looking for things inside Qt as we really do not play much with the QClipboard.

Please let me know what you see.

KevinH

[KevinH]

Hi,

I have Linux Mint Cinamon 17.1 in a VM I believe. And I simply do not see any change no matter how full my clipboard is upon entry (or perhaps I do not know how to clear the external clipboard properly?). Is there a way to clear the system level clipboard in Linux Mint?

KevinH

[DiapDealer]

Originally Posted by KevinH

Hi,

I have Linux Mint Cinamon 17.1 in a VM I believe. And I simply do not see any change no matter how full my clipboard is upon entry (or perhaps I do not know how to clear the external clipboard properly?). Is there a way to clear the system level clipboard in Linux Mint?

KevinH

There's probably a lot of ways to clear the clipboard, but I just use xsel to do it.

sudo apt-get install xsel

and then xsel -bc to clear.

I'll do your test when I get home a little later.

[eschwartz]

+1 xsel

As for the changes, it is still slow, until you add something to the clipboard while in Sigil.
Then it finally speeds back up.

Progress, I suppose?

[KevinH]

Hi DiapDealer,

Ah, that helped. I can now see the difference. I tried disabling the ClipBoard History code as above and it has no impact. There really is not a reason for what we are seeing as far as I can tell.

Something inside Qt must be slowing down the javascript used to highlight text only when something exists on the clipboard upon entry to Sigil.

This is so strange. The funny thing is that even disabling everything to do with clipboard and signals does nothing for this issue.

KevinH

[DiapDealer]

Perhaps it has something to do with QPlainTextEdit's selectionChanged signal? Or ExtraSelections? Just WAGgin' it, here, away from my computer. Perhaps we're back to line numbers and syntax-highlighting.

[KevinH]

After doing a bug search at Qt, I am wondering if this in related to the INCR protocal used to talk to X11 based Windowing systems. It has a timeout. After looking at the code in qxcbclipboard.cpp in Qt source, I notice they had problems copy data out of the app into other X11 apps, and I am wondering if they are having the same trouble copying things into the Sigil app.

Interesting, they disable this INCR socket protocol for Motif Window managers.

So strange. I will keep looking.

KevinH

[DiapDealer]

Since it doesn't appear to be related to Sigil's Clipboard diddling, I have to take it as a good sign that calibre's editor can select text in a QPlainTextEdit widget (with line numbers and syntax highlighting) with no noticeable lag on the same machines that lag using Sigil (with comparable recent versions of Qt5). It's the optimist in me.

Should we revert the change to dataChanged() back to changed(QClipboard::Mode) in master?

[KevinH]

Hi DiapDealer,

The dataChanged() is fine as is. It will help reduce the number of times signals are flying around.

I have played with things a bit more and the amount of text in the clipboard when Sigil is started has no impact. It is just that there is something in clipboard at all no matter how small (such as the single letter "I"). That means that none of our syntax highlighting or anything else is related to the bug as these stayed the same across the two different cases (starting Sigil with something on the clipboard vs starting Sigil without something on the clipboard).

My guess the problem is something to do with the protocol of how Qt communicates with the X11 xcb clipboard. If there is something in the clipboard, some communication with the X11 windowing system seems to be timing out all of the time slowing all text selection down immensely. If there is nothing in the clipboard, then this problem never happens for the entire session.

Then there is the fact this is entirely a Linux issue and can not be seen on Windows or Mac points toward a Qt bug as well.

Is there an easy way on Linux mint to fire up a Motif based Window Manager and then fire up Sigil to see if the problem goes away or not. If it stays, that would probably rule out my INCR theory. Calibre via PYQT could be using a native python interface to the clipboard for all I know. This one really has me stumped. Adding a single letter "I" to the system clipboard before starting up Sigil has no effect on any other platform but Linux.

This is a real puzzler?

KevinH