"Delete unused stylesheet classes" problems with overloaded styles

[BeckyEbook]

Makefile.win32 from DiapDealer is better because it creates an exe file not only smaller but also without additional DLL dependencies.

[DiapDealer]

You still need the vs c++ runtimes, but if you have sigil, you should already have them anyway.

[wrCisco]

I compiled and tried the new cli cssparser, it went almost all well and good with various pseudo-classes, pseudo-elements, at-rules, vendor-prefixed properties (these with warnings, since they are not part of the standard) and vendor-prefixed at-rules (@-o-keyframes and the likes... not likely to be used in an epub, I just had an old css for a web page with animations...).

I encountered two problems:
- it didn't recognized the "@import" rule at the beginning of the stylesheet, and it deleted the space between "import" and the subsequent "url(...);" when printing;
- (minor) it lost one level of indentation in the serialization of nested at rules, so this code

@media amzn-mobi {
  @media only screen and (max-width: 650px) {
    p:nth-of-type(2n+3)::first-letter {
      color: red;
    }
  }
  .aclass:not(.anotherclass), h1 + p > span:last-child {
    color: blue;
  }
}

became

@media amzn-mobi {
    @media only screen and (max-width: 650px) {
    p:nth-of-type(2n+3)::first-letter {
        color:red;
    }
}

.aclass:not(.anotherclass),h1 + p > span:last-child {
    color:blue;
}    
}

(it's just an example: I know this code doesn't work on mobi, and probably this feature of nested conditional group rules is not even part of the epub standard, since it's still a candidate recommendation for the w3c, but it has been supported by all major browser engines for many years now).

A heads up on the more recent version of the Sigil fork for the gumbo query integration (https://github.com/kevinhendricks/Sigil): I had to replace "#include <exception>" with "#include <stdexcept>" in Query/CParser.cpp in order to compile it on Linux.

As for the Sigil's python interface, I couldn't got it to work with a list of integers as argument for the "runInPython" method: I had to convert all the numbers to strings. Then, with a QStringList, it was easy enough... Is it expected? (Maybe I just got it all wrong with the QVariant conversions).

[KevinH]

@wrCisco

In reverse order ...


QVariant toList() to get a list of QVariants and each of those QVariants toInt() should be how it works. If not it is a bug and I will track it down.

I will try adding stdexcept to the query code ifdef'd to linux in my own github site and exception for other platforms.

@import are not tokenized. I will add a space. So they should appear in the import std::vector. I will check.

[wrCisco]

Originally Posted by KevinH

QVariant toList() to get a list of QVariants and each of those QVariants toInt() should be how it works. If not it is a bug and I will track it down.

Since my program crashed while calling PyObject_CallObject (inside runInPython), it was probably a problem with the conversion the other way around, from QList<int> to QVariant.

[KevinH]

Regarding @import issues with CSSParser:

I tried the following variations at the top of a test css file:

@import url(../hello/junk.css);
@import "../hello/junk1.css";
@import url("../hello/junk2.css");

Other than the missing space after @import the css code seemed to be parsed properly.


So I am now confused as to what you meant by @import "not being recognized".

Do you mean the @import command was not parsed into appropriate pieces or do you mean that no import / inclusion of the specified spreadshhet was done. If the latter, that is expected behaviour of the parser. It is not the parser that should handle any physical inclusion, just parsing the request. Upstream code from the parser would be responsible for locating the file and actually including it and that typically is done only when used for rendering.

So when you get a free moment would you explain what you meant.

Thanks.

[KevinH]

@wrCisco,

I eyeballed the code in EmbeddedPython.cpp and nothing is jumping out to me at all.

So when you get a free moment would you please supply to me the smallest/simplest python code target and a snippet of the cpp/qt code you use to invoke it and I will add some debug to EmBeddedPython code in Misc and test the passing of a List of Ints as one argument to your python routine and get this tracked down and fixed.

We have plenty of examples of passing string lists and that part of the bridge seems to work well and the general List code mirrors it closely so it should work.

So the problem could be specific to QList<Ints> or perhaps impact all other QList types such as float, unsigned int, etc.

Thanks!

Originally Posted by wrCisco

Since my program crashed while calling PyObject_CallObject (inside runInPython), it was probably a problem with the conversion the other way around, from QList<int> to QVariant.

[KevinH]

Originally Posted by wrCisco

A heads up on the more recent version of the Sigil fork for the gumbo query integration (https://github.com/kevinhendricks/Sigil): I had to replace "#include <exception>" with "#include <stdexcept>" in Query/CParser.cpp in order to compile it on Linux.

I looked into this and #include <exception> should work on Linux as it is used in the sigil_exceptions.cpp code that works across all platforms. So I googled and it seems some platforms that use std::runtime exceptions need to include <stdexcept> as well.

And on all platforms that I looked at <stdexcept> also included <exception> to get its base class std::exception.

So I have just now pushed code to do the following for all platforms:

#include <exception>
#include <std::except>

and this seems to work fine on macOS and should hopefully work fine on Linux as well.

Thanks!

[KevinH]

Originally Posted by wrCisco

Since my program crashed while calling PyObject_CallObject (inside runInPython), it was probably a problem with the conversion the other way around, from QList<int> to QVariant.

Okay I ran the following testcase and got this to work. I think a QVariant of a QList should work directly but I could not get it to work.

In PythonRoutines.cpp and .h
-------------------------------------

int PythonRoutines::SumIntList(const QList<int>& ilist);

int PythonRoutines::SumIntList(const QList<int>& ilist)
{
    int results;
    int rv = -1;
    QString error_traceback;
    QList<QVariant> args;
    QVariantList qvl;
    foreach(int v, ilist) {
       qvl.append(QVariant(v));
    }
    args.append(QVariant(qvl));

    EmbeddedPython * epython  = EmbeddedPython::instance();

    QVariant res = epython->runInPython( QString("testcase"),
                                         QString("sum_int_list"),
                                         args,
                                         &rv,
                                         error_traceback);
    if (rv == 0) {
        results = res.toInt();
    }
    return results;
}

And in the python3lib I used the following testcase.py

import sys
import os

def sum_int_list(alist):
    sum = 0
    for v in alist:
        sum = sum + v
    return sum

And here is how I invoked it:

   // test pass ing list of ints to python routine                                                                             
    QList<int> alist;
    alist << 1 << 3 << 6 << 9;
    PythonRoutines pr;
    int sum = pr.SumIntList(alist);
    qDebug() << "Sum from SumIntList is: " << sum;

I will work on lists.

But this should get you going.

[wrCisco]

For the @import issue, I just saw two anomalies in the program output: the absence of TYPE and Data, as in

Information: 0
import: url(a_css.css)
Type: SEL_START  Data: @font-face
Type: PROPERTY  Data: font-family

and the missing space. I thought that it was not parsed properly, but I did not dig deeper than that, sorry if it wasn't clear.

As for the QList<int> to QVariant, what I tried was to convert directly the QList<int> to a QVariant with QVariant::fromValue and that led me to a segfault during the execution of runInPython. I can confirm that with your code it all works smoothly, thanks!
I don't have the old code I wrote anymore, but if you replace the lines regarding the QVariantList in your snippet with

args.append(QVariant::fromValue(ilist));

it should be close enough to what I did.

[KevinH]

Yes, QMetaType of Qt already defines a QVariant for QStringList, QVariantList and QList of Variants but does not define other basic QList or QVector types such as QList<int> and QList<double>.

In order to properly store and retrieve these from QVariant, their custom QMetaTypes must be created and registered so proper QVariant typing for introspection can be used by EmbeddedPython.

So when I wrote Sigil's EmbeddedPython I only supported the predefined QMetaTypes in mapping them to their equivalent python types. Since QVariantList can contain both lists of any type and therefore can hold and pass any structure data members even of mixed types, I did not add support for other non-predefined Qt types.

I will add and register a QMetaType for QList<int> and expand EmbeddedPython code to support it so your one line conversion of a QList of int to a single QVariant will work.

I will push those changes to Sigil master today.

Until them building a QVariant from a QVariantList is the way to go and should work even on older versions of Sigil.

[KevinH]

Originally Posted by wrCisco

For the @import issue, I just saw two anomalies in the program output: the absence of TYPE and Data, as in

Information: 0
import: url(a_css.css)
Type: SEL_START  Data: @font-face
Type: PROPERTY  Data: font-family

and the missing space. I thought that it was not parsed properly, but I did not dig deeper than that, sorry if it wasn't clear.

Yes csstidy did not create tokens (or even token types) for @charset or @import and @namespace and so handle them directly in their own variables outside of the token list.

I think they did this as those can only come right at the top of the css file and so were special cased, but I do not really like that approach.

I have changed and modified CSSParser.cpp so much away from csstidy, that I will probably define and add in token types for those and put them into the css token list and remove the special cases.

I will work on the indentation issue but nested media queries are not css3 spec so the code will have to be modified to support them.

I am also going to add file offset starting positions for each token so that its output can be used to more easily modify the existing css file.

Thanks for all your testing and feedback!

[KevinH]

@wrCisco,

I have pushed the code changes for EmbeddedPython to support passing a QList<int>.

To use it in your local tree you will want to pull in the changes to EmbeddedPython.cpp, EmbeddedPython.h, and PythonRoutines.h from Sigil-Ebook Sigil master.


I also somehow inadvertently pushed all of my own local tree changes into Sigil master! So all of the Gumbo Query and CSSParser changes are there now but not actually being used (I disabled running the Query tests in main.cpp).

But instead of reverting a long list of changes inSigil master that we may want to add in the end, DiapDelaer and I have decided to leave them there and remove anything we decide we do not need/want later.

[KevinH]

@wrCisco, and anyone else following along,

I have modified CSSParser to extend the set of tokens to include CHARSET, IMPORT, NAMESP and these now are tokenized properly instead of special casing.

I have tracked file offset positions for specific internal parser state transitions and added a pos field to the token structure returned so that file offsets can be used to edit css outside the CSSParser itself.

I have enabled proper indent level tracking to enable nesting of selectors, and nesting @media rules.

I have added a method for a csstoken list (after outside processing or editing), be set back inside of CSSParser so that the serialize_css function can serialize it back to a css file/string.

See attched cssparser_v2.0.zip for people to evaluate.

Thank you for your testing and feedback.

PS: Have completed the cleanup and now have posted cssparser_v2.0.zip

[KevinH]

Again, for those following along, given the CSSParser appears to work on a wide range of input, I plan to convert CSSParser.cpp from std::string, std::map, and std::vector to use QString, QMap, and QVector, so that position information for each token matches the QString location used inside of Sigil. Current position information returned represents the position in a utf-8 string so it can not be used easily inside Sigil without a mapping of utf-8 offsets to QChar (utf-16) offsets in a QString.

I will keep the std::string versions in its own repo someplace for those interested.