Sad Sigil news?

[Katsunami]

Originally Posted by crutledge

I have been browsing through the sigil code and see the major problem with sigil naintenance. NO PROPER DOCEMENTATION!!!

Unless the programmer thinks, eats, breathes, and dreams in C++, understanding what is going on is very difficult and not being totaly knowledgeable in the general functioning (epub operations) maintaining the code will be a huge learning curve.

If there is indeed no documentation, then it would be almost impossible to pick up this project except for a programmer who has, as you say, a lot of experience in writing C++ (I have that), a lot of knowledge of the libraries Sigil uses (I have, with regard to BOOST, but none with regard to QT or other frameworks or components used), and good knowledge of the EPUB spec (I have none).

Each modile should be accompanied by a pseudo code file defining and demonstrating the desired inputs, outputs, and data manipulation to be performed.

Maybe the pseudocode is a bit much, but a description of the main functions and objects would indeed be helpful... however, it's hard to fault user_none for not making that documentation, as it costs a lot of time. Even (very) big companies often omit vital documentation (because it costs time that can't be spent on the product itself), not realising that replacing a programmer/designer that leaves the project becomes virtually impossible later on.

I've been in such a situation a few times; required to replace someone within a few weeks, coding in a language and using frameworks I've never seen before, with no documentation with regard to project, program and database structure (if databases are used). I could never do it. It's impossible to AND learn the language AND it's default library, AND the third party frameworks AND find out how the entire project works, in just a few weeks if there is no documentation whatsoever.

As I said; I can't fault user_none for not making any documentation (as he doesn't need it, being Sigil's only maintainer), but if there is indeed none, then I don't expect Sigil to be picked up by anyone else... ever. Sadly enough.

[rkomar]

@crutledge: You're absolutely right; this is the major problem of all large projects in all languages. It is especially problematic with projects with volunteer developers since coding is fun and documenting usually isn't. The problem here is that to properly document at a high level, you need full understanding of what the code is doing, and if you have that, then you're probably working on bug fixes or new features rather than documenting (the squeaky wheel gets the grease, and it's more fun). So, if it's already hard to get volunteers to work on the code, getting volunteers to only do documentation is even harder. The only hope is to use some sort of automated documentation generators on the current source base, and to get some of those experienced developers to take some time to write out some very high level outlines of what's going on. The latter shouldn't take a lot of time, so it has a chance of getting done, and even better, being kept up to date. Documentation that doesn't match the code is worse than useless, it's actually harmful. So, in a project like this, it's probably best to keep the stuff that needs to be manually updated to a minimum.

This is really a problem in organization, which is why it's so good that John is offering to stick around to help manage the project.

[user_none]

Originally Posted by crutledge

Each modile should be accompanied by a pseudo code file defining and demonstrating the desired inputs, outputs, and data manipulation to be performed.

So you're saying write the code twice. Instead of reading pseudo code just read the code. All you're doing by writing pseudo code (other than writing the same thing twice) is writing code for someone who doesn't understand C++. If someone isn't versed in C++ it's still going to be difficult for them to contribute even if they know what the code is supposed to do. You still have to know C++ regardless.

Also, the user of pseudo code assumes you can represent the concept in the pseudo code. All you can do is show how something should work but not how it actually does work. For someone who knows C++ this is precisely what the class definition(s) in .h files do.

That said I will concede that Sigil could have more documentation. Especially when it comes to class dependance and how they work together.

Originally Posted by crutledge

The US military tried to overcome these problems with ADA

And look how that turned out.

[DrChiper]

Originally Posted by user_none

So you're saying write the code twice. Instead of reading pseudo code just read the code. All you're doing by writing pseudo code (other than writing the same thing twice) is writing code for someone who doesn't understand C++.

You really respond as a programmer (no pun intended): I've seen these kinds of reactions many times. The thing is: pseudo code should be context / implementation free and just grasp the intention of program, in other words: what must it do. It should be left to the developer how to practically realize that and also select a certain language which is best suited for that job.

Many applications (and believe me also many commercial apps you do not want to know) just "grow" organically without ever have established some kind of "vision" document to explain why it is as it is and how things are originally where supposed to work together.

Regarding ADA: the language was not bad, but most important DoD also gave us an accompanying documentation standard (Mil-2167A, now Mil-498) to be completed and approved mandatory before being allowed to start to program. There is/was a reason for this. Think many employees, many subco's, many third parties, many requirements changes, maintainability over many years, and you get the picture.

Further developers where and are just afraid for documenting: hé, I think with my keyboard, my programs are self-descripting, yeah, right, keep on dreaming.

What can we learn from this? It always pays to document at least the main features. Documents do not have to be that big: 1 sheet often fills the bill as long as it is to the point and up-dated to reflect the current status. Skipping docs is penny wise/pound foolish as many managers will find over and over again, but then: that problem is for their successors. Time to move on.

Just my 2cts.

[tompe]

Originally Posted by DrChiper

You really respond as a programmer (no pun intended): I've seen these kinds of reactions many times. The thing is: pseudo code should be context / implementation free and just grasp the intention of program, in other words: what must it do. It should be left to the developer how to practically realize that and also select a certain language which is best suited for that job.

But just describing the intention is not pseudo code. Pseudo code as I understand it is for describing what the actual implementation do and often describe before you implement.

The problem with this kind of pseudo code is that you have duplicated the information. If there is a mismatch you do not know if the pseudo code or the code is correct if both are working according to the interface specification.

[DrChiper]

Originally Posted by tompe

But just describing the intention is not pseudo code. Pseudo code as I understand it is for describing what the actual implementation do and often describe before you implement.

The problem with this kind of pseudo code is that you have duplicated the information. If there is a mismatch you do not know if the pseudo code or the code is correct if both are working according to the interface specification.

Your statement is indeed what confuses most developers in their interpretation of what pseudo code is: it does not describe the actual implementation. The latter is up to the developer to decide: its his job, his area of expertise. Pseudo code is just an (high level) algorithm with the basic sequence of steps taken to solve a problem.

True, most "pseudo code" is (wrongly) written to even at bit-level, using real named parameters and hence, an 1-to-1 translation (cut/paste) can easily be made. In fact most "pseudo code" is just actual code because they get the abstraction wrong.

Spoiler:

Pseudo code example

WHILE calibre files are present DO:

Input calibre file

IF calibre file genuine and valid DO:

clean calibre file TOC
store result
update bookkeeping
update database
update GUI where necessary

ELSE

inform user about failure
log failure

END

END

Print totals in GUI

But this is basic programming skill and actually IMHO

[latepaul]

I remember learning Pascal at university in the 80s. For our projects we had to submit a program (as a printed listing naturally) together with the pseudo-code we used when we designed it.

I wrote my program as I always did, got it working and then wrote a script that turned the source code into pseudo-code and used that. Got full marks.

Currently I work as a C programmer (with a smattering of other languages but nothing so modern as python) on a very large codebase (10s of millions of lines of code) with very little external documentation. The thing that helps me most day to day is well-commented code and coding standards. When I joined the team I work in now I had no knowledge of the source at all. I asked for documentation and was given something a decade and a half out of date. However I did have mentors who knew the code. But I asked an "overview" sort of question I often found that they would do the same as me - go looking in the code itself.

Funnily enough I've been looking for a project to teach myself C++/QT for a while. I got so far but I needed something to build rather than examples from tutorials. However I think Sigil would be jumping in at the very deep end. The technical stuff I'm sure I could master it's the commitment needed (sometimes I want to do non-programming things) and the already-mentioned demanding users.

Anyway the way this thread is going you'll have it folded into Calibre before I've got past "Hello World" stage.

[crutledge]

Originally Posted by DrChiper

Your statement is indeed what confuses most developers in their interpretation of what pseudo code is: it does not describe the actual implementation. The latter is up to the developer to decide: its his job, his area of expertise. Pseudo code is just an (high level) algorithm with the basic sequence of steps taken to solve a problem.

True, most "pseudo code" is (wrongly) written to even at bit-level, using real named parameters and hence, an 1-to-1 translation (cut/paste) can easily be made. In fact most "pseudo code" is just actual code because they get the abstraction wrong.

Spoiler:

Pseudo code example

WHILE calibre files are present DO:

Input calibre file

IF calibre file genuine and valid DO:

clean calibre file TOC
store result
update bookkeeping
update database
update GUI where necessary

ELSE

inform user about failure
log failure

END

END

Print totals in GUI

But this is basic programming skill and actually IMHO

Excellent, well put.
Software design is a skill for which I paid top dollar. There are tools just for design which can carry the system down to the lowest levels. The designer begins with the specification and expands from there. The manager can then assign related components to programmers for implementation in code. The language is immaterial.

DOXYGEN is said to be a useful reverse engineering tool. I have never used it.
It is free under the GNU General Public License. It supports both C++ and Python.

[QUOTE]http://www.stack.nl/~dimitri/doxygen/index.html[/QU.
OTE]

There are others both free and commercisl.

[Katsunami]

Originally Posted by latepaul

Currently I work as a C programmer (with a smattering of other languages but nothing so modern as python) on a very large codebase (10s of millions of lines of code) with very little external documentation. The thing that helps me most day to day is well-commented code and coding standards. When I joined the team I work in now I had no knowledge of the source at all. I asked for documentation and was given something a decade and a half out of date. However I did have mentors who knew the code. But I asked an "overview" sort of question I often found that they would do the same as me - go looking in the code itself.

Now imagine that situation without having any mentors. Like this: "Our two main programmers decided to start their own company and left / died in a car crash / where fired because they mishandled the secretary". You'll be completely lost without any current documentation, even if the code is commented. It will take you weeks just to get up and running and write your first line of code AND know what you're doing.

I've been in such a situation twice, and wasn't hired. "What? You CAN'T sit down at an application having a million lines of code, 7 external libraries you never heard of before, and just start writing code... like RIGHT NOW? You actually need documents to EXPLAIN stuff? Then you're not a real programmer. Go away."

Before that happened, I often went away myself already, mostly at the point where I figured out that if I had a question, nobody from the past who could have answered it was left at the company. I don't even want to think how these programs look nowadays.

Originally Posted by crutledge

Excellent, well put.
Software design is a skill for which I paid top dollar. There are tools just for design which can carry the system down to the lowest levels. The designer begins with the specification and expands from there. The manager can then assign related components to programmers for implementation in code. The language is immaterial.

That is how it should work. Nowadays however, the project is often already sold to a client, deadline and all, including a first demonstration next week, before the first design decision has been made. So, a demo is hacked up to show the customer "how far we're into the project already", and because the next demo (and the next, and next) are also already planned, there actually isn't any time to properly design the application/website/whatever. It's just a bunch of (carefully scripted) demo's of different half-working functions, that are cobbled together into one application at one point.

I've seen this a few times at different companies.

Oh, and you can FORGET about any documentation.

If we built houses the way a lot of software is built, they would come crashing down our ears as soon as the builders left the premesis. Often I'm thinking I should have studied architecture or something.

[DaveinJapan]

Yikes. I used Sigil exclusively to put together my first book — I don't know what I'll do without it! Hopefully it won't come to that, but it doesn't sound too promising at the moment...

[st_albert]

Originally Posted by DaveinJapan

Yikes. I used Sigil exclusively to put together my first book — I don't know what I'll do without it! Hopefully it won't come to that, but it doesn't sound too promising at the moment...

Well, don't lose too much sleep just yet. The present Sigil should continue to serve you well at least until (or if) epub 3 becomes de rigueur. And by then... well, as the Fremen say, "there's always a way to water."

But in the mean time, better hurry up with your next book!

Albert

[mrmikel]

User_none should not only have given his free time away, but doubled the time by documenting it???? Especially when some of the code was from the previous maintainer?

[kiwidude]

Just to offer some info for anyone wondering. I'm one of the guys who stepped in to help give Sigil a kickstart after John took over. I worked pretty intensively (but enjoyably) seven days a week for around 4 months with Dave and John in getting up to speed with the codebase and then helping add a large number of new features that were of interest. As well as stomping on many of the most painful bugs or quirks that had irritated me ever since I first started using it.

IMO the reality for anyone working on an open source project (where they are not getting paid for it) is that there comes a time where you have to step away as you just cannot spend every minute of your life writing code on top of a day job without at some point getting burnt out, losing interest or having time/financial pressures. In my case, I had nothing more I needed to put in features wise and we had got Sigil to the most stable it had ever been in its history, plus I need to start my day job contracting again to pay the bills!

It is a shame things have tapered off since. That was an intense period for all of us so it was inevitable there would be a lull afterwards. It does what I need it to for my very simple requirements, and right now I work 100 hour weeks in my day job and when not doing that I have other open source development etc to work on. It was never going to be a long term thing for me.

Right now I don't really see the sky falling in quite just yet. I'm not using the very latest Sigil but AFAIK it is stable and working. That it is not actively being hammered with new features is not necessarily a bad thing - it is a product which for the limited needs of the majority of users has reached a point of maturity. Yes there will always be commercial users pushing the boundaries who would love "more" from it and I have no facts at all to support this but surely they are the minority. I don't mean that in a derogatory sense at all - clearly with the ongoing forum support and perhaps even financial from the comments I have seen they have been a very important part of Sigil's life. However unless a particular feature/requirement happens to tickle the personal fancy of a developer it just won't happen unless someone pays a developer to do it. As Sigil is such a small niche in scope and userbase combined with technology choices that it was always going to struggle to be self sustaining as a non-commercial product.

Admittedly I have stepped away big time since my involvement, I haven't even visited these forums until today. So maybe there is some new EPUB specification or whatever on the horizon which means Sigil will stop working at some point. If that happens... well, we shall see. As long as there is no alternative product, there is always the possibility someone will step up to help John out again to "save the day", then while they are there knock off some of the backlog.

To anyone contemplating getting involved - if you want a C++ project to dabble with and a top bloke in John to be in touch with then I say go for it. I greatly enjoyed the debates with Dave and John we had during the period I worked on it. Clearly there is an "opening" there now for someone to step up, even for a little while to keep things chipping along. The codebase may sound a bit scary but in reality it is not that bad, even for C++ noobs like me. Where things do come a bit more unstuck is the need for cross platform support (I was the Windows guy, John was our only Mac guru and Dave with John covered Linux) so ensuring something works "for everyone" is a challenge. However Sigil clearly continues to have some enthusiastic supporters in the forums and with John's experience I am sure that is nothing that cannot be overcome one way or another.

As for John - kudos to you mate for staying at the coalface for as long as you did (and to Dave as well until relatively recently). I wish you all the best in helping keep Sigil roll forward in the understandly more limited involvement you can put in at this point.

[st_albert]

Many thanks for your efforts as well, Kiwidude! You and Dave and John can be rightfully proud of the advances you made in Sigil. I for one never expected all the new features that y'all added -- not to mention the stability.



Albert

[DaveinJapan]

Originally Posted by st_albert

But in the mean time, better hurry up with your next book! Albert