Suggestions for Sigil Improvements or New Features

[KevinH]

Some replies ...

- No BOM is needed or used for utf-8 encoded files. A BOM is only needed for utf-16 and utf-32 given either could be in little endian or big endian byte order (thus Byte Order Mark) There is no little or big endian for 8 bit encoding. No file saved as utf-8 in Sigil needs or uses a BOM. There are many applications that can read the saved CSV files with no issues even on Windows. Using a BOM on an 8-bit encoded text file is just wrong. And for Windows in this day and age to default to assuming an ansi/ascii file when it detects an 8 bit encoded file is just stupidity on their part. In all other platforms utf-8 (with no BOM) is the default.

According to google: To open a utf-8 encoded csv file in Excel, you should not double-click it. Instead, use the Data > Get Data > From File > From Text/CSV menu option in Excel, which allows you to specify the encoding as 65001: Unicode (UTF-8).

- selection by copy to a clipboard is restricted to a single element, I will look into changing that if easily doable.

- the status line's length will always change. It did for changing line number and column numbers before this change. As for the status line order, that question was already asked in an earlier post in this thread. I argued against it. But if most testers would prefer the order to be:

CODEPOINT NAME (U+0000) - Line: 1, Col 1 .

I would consider it. You are the third to ask for that change. We need to hear from other testers to get a consensus.

- What reason do you have from disabling the character name? What benefit would it give to most users? IMHO, Sigil already has too many settings right now, so adding one to disable a new feature is probably not going to happen unless there is a negative impact on editing speed, but none has been detected or reported. So there are no downsides to adding that info to the status line.

Yes, suggestions are still open. But please read the first post about what kinds of suggestions are acceptable. Include arguments about how and why this would help the majority of users.

[KevinH]

Related: would it be possible to add the option to copy the selected lines? (with or without a menu = currently, CTRL+C only copies the first one).

Okay, I was able to add a copy selected to clipboard menu item for SpellcheckEditor that uses tab separator chars. This should allow pasting into any app. It is can also be launched via the normal copy shortcut key.

This change has already been added to master.

[Doitsu]

Originally Posted by jugaor

In Windows, CSV files are usually associated with MS Excel... which doesn't know how to decode UTF files properly if they don't have a BOM (even the latest 365 version: see before/after screenshots). Can the file be saved with the BOM added automatically?

This is a known Windows MS Excel bug. By default it considers all UTF-8 files without a BOM, CP1252 files.
You can easily add a BOM with Notepad++. Simply open the .csv file with it, select Encoding > UTF-8-BOM and save it.

[KevinH]

FWIW, I thought Windows 11 fixed this by moving its default encoding to be utf-8 (Unicode 65001) for files and console use but maybe it was just Python on Windows that made that change.

Windows is alone in using an unnecessary BOM for utf-8 files as it breaks many many things on unix/linux/macos systems (ie try concatenating two text files and ending up with a BOM in the middle of the resulting file and breaking string compare, etc.) and it is against Unicode recommendations.

FWIW, the entire networked world has moved to default utf-8 without an extraneous BOM. IMHO, it is time Windows joined the networked world instead of adding tracking and advertising to its operating system. But that is just my opinion. Sigil will not support it except to strip it out.

[BetterRed]

This Windows user associates CSV files with Nirsoft's CSVFileView. Spreadsheet programs like Excel, Calc, 123 and Quattro are massive overkill for something like a list of misspelt words in a novel.

In defence of Microsoft: Excel is used by millions. It is used by large corporates (banks, pension fund managers etc) in their Line of Business processes, and in Mission Critical (life and death) applications - e.g. Fire & Rescue resource scheduling. Removing a defect in Excel may have material consequences.

It's even [mis]used by storied Harvard economists ==>> https://en.wikipedia.org/wiki/Growth_in_a_Time_of_Debt

BR

[KevinH]

So that means you never fix bugs? No, it means you fix the bug and test the hell out of it before releasing the fix. Microsoft has done neither.

[icallaci]

Originally Posted by KevinH

the status line's length will always change. It did for changing line number and column numbers before this change. As for the status line order, that question was already asked in an earlier post in this thread. I argued against it. But if most testers would prefer the order to be:

CODEPOINT NAME (U+0000) - Line: 1, Col 1 .

I would consider it. You are the third to ask for that change. We need to hear from other testers to get a consensus.

No preference here. Either way is ok by me.

[BetterRed]

Originally Posted by KevinH

So that means you never fix bugs? No, it means you fix the bug and test the hell out of it before releasing the fix. Microsoft has done neither.

The issue in question is not a 'bug', it's a legacy feature in a 40 year old application that is the most widely used in its class… by a country mile.

As for testing the 'hell out of it" - I couldn't agree more, especially with large corporate and public service clients who use it in their LOB and Mission Critical business processes.

There used to be an adage: "Software should only be changed when the cost of not changing it exceeds the cost of doing the change - including loss of market share." Is it still around. For some reason i associate that with Tom Peters.

I suspect MS doesn't suffer from FONKU

[DiapDealer]

I've never understood why people use Excel to manipulate csv files.

[KevinH]

Originally Posted by BetterRed

The issue in question is not a 'bug', it's a legacy feature in a 40 year old application that is the most widely used in its class… by a country mile.

Actually utf-8 was only generally adopted in the 2008-2012 timeline and by Microsoft much later than that. So this "bug" was introduced in the last 15 years or so, so that Microsoft could keep using local 8 bit encodings like cp-1251 or cp-1252 when the world moved to unicode. Excel right now can read these files properly by importing them after setting the encoding.
Windows 11 should have fixed all this nonsense.

[jugaor]

@KevinH: Thank you so much for your quick replies!

Originally Posted by KevinH

- No BOM is needed or used for utf-8 encoded files…

I understand your well-founded reasons. The BOM suggestion wasn't specifically for me, but for the millions of Win-Excel users who just double-click and that's it. It was just a suggestion to save you potential "It's not working properly" reports
No problem for me.

Originally Posted by KevinH

- selection by copy to a clipboard is restricted to a single element, I will look into changing that if easily doable.

THANK YOU!!! That was on my list of suggestions. Check. Much more convenient this way.

Originally Posted by KevinH

- the status line's length will always change. It did for changing line number and column numbers before this change.

"Ideally", the width of both values ​​could be fixed (five digits?), so that the Line: Col: headers wouldn't jump when moving the cursor (but that might be asking too much, hehe).

Originally Posted by KevinH

As for the status line order, that question was already asked in an earlier post in this thread. I argued against it. But if most testers would prefer the order to be:

CODEPOINT NAME (U+0000) - Line: 1, Col 1 .

I would consider it. You are the third to ask for that change. We need to hear from other testers to get a consensus.

Thanks for considering it.

Originally Posted by KevinH

- What reason do you have from disabling the character name? What benefit would it give to most users? IMHO, Sigil already has too many settings right now, so adding one to disable a new feature is probably not going to happen unless there is a negative impact on editing speed, but none has been detected or reported. So there are no downsides to adding that info to the status line.

Again, the request wasn't specifically for me. But whenever an application adds a feature, someone might prefer to disable it: because they don't find it useful, because of limited screen space, etc. Providing an option to disable it, if it's not difficult to implement, is always welcome.
That's all, no other interest on my part.


Thanks again for your time and consideration!

[jugaor]

Originally Posted by Doitsu

You can easily add a BOM with Notepad++. Simply open the .csv file with it, select Encoding > UTF-8-BOM and save it.

Thank you very much for your kind reply. I use another application for that (EmEditor), but the process is the same.

Originally Posted by DiapDealer

I've never understood why people use Excel to manipulate csv files.

Because the MS installer always hijacks the default CSV association? Microslop

[KevinH]

Originally Posted by jugaor

Again, the request wasn't specifically for me. But whenever an application adds a feature, someone might prefer to disable it: because they don't find it useful, because of limited screen space, etc. Providing an option to disable it, if it's not difficult to implement, is always welcome.
That's all, no other interest on my part.

I am against the idea of user overrides on everything. It is one of my prime complaints about calibre which is a wonderful book library management system, but has so many confusing and conflicting settings it drives me crazy at times. And as I said, Sigil already has too many settings. So unless a feature has negative downsides, I will not be adding setting to enable or disable it. The existing status bar is primarily unused so no impact on smaller screens.

[KevinH]

Originally Posted by jugaor

I understand your well-founded reasons. The BOM suggestion wasn't specifically for me, but for the millions of Win-Excel users who just double-click and that's it. It was just a suggestion to save you potential "It's not working properly" reports

Luckily, Sigil does not have millions of Win-Excel users . And promoting a broken idea just for one set of users is not a good idea for any cross-platform app in my humble opinion.

FWIW, a websearch shows Windows 11 does finally have a new feature to make the default encoding utf-8 for all apps, as it is in Notepad now, even for older apps.

To make UTF-8 the default system encoding in Windows, enable the "Beta: Use Unicode UTF-8 for worldwide language support" option in the Region settings. This improves compatibility with web apps and Linux, though it may cause issues with older, non-Unicode applications. A system restart is required.

But off course it defaults to off, is hidden under Region setting even though utf-8 is regionless, etc. But at least it is there. Hopefully Windows 11 users will find this and enable it so that their system can be just like Web, MacOS, and Linux systems which have had this capability built in for years, so that they can make/edit text that can contain any word, no matter the language. The days of old 8-bit encoding codepages has long passed.

For existing 8-bit code page text docs, many apps have the capability to bulk transcode 8-bit encoding to utf-8 to future (present)-proof them, especially for archiving. Sigil does this for all xhtml files on initial load (detects encoding either by charset settings or detection algorithms and converts it to utf-8).

[Turtle91]

Here's a suggestion:

Can you make the "Edit/Paste from Clipboard History (Ctrl-Alt-V)" a dockable window...or at least non-modal. I find myself constantly pasting from recent history (not necessarily the same text which would better serve as a clip) and having it auto-close after every paste is an annoyance.