The Technology Vent and Rant Thread

[Greg Anos]

Picking up rant from Vent and rant thread about becoming a software engineer - per moderator's request.


Katsunami
"Just read on one of the biggest job vacancy sites in the Netherlands that it is...

... A great idea to put in some job application to become a programmer or a software engineer. There's lots of work in that sector, a shortage of people, and you don't need a diploma; most companies are happy to teach you the skills on the job, and if all else fails, there's always Google."

So that is the status of the job of software engineer nowadays, junk drain for anyone who doesn't have a proper job? You don't need an education, because you can just Google everything?

Now, that's all I needed to know to make me regret even more that I got into "the job of the future" 15 years ago.

And then, people keep asking why there are always so many problems with IT-related stuff in companies. If the above is actually TRUE and companies DO hire random people without a proper computer science education just because they're cheap, I can tell you why. If you hire junk employees who Google all of their stuff, you get junk software and/or infrastructure!

Yes, you can google an answer to some questions or look up documentation on a programming language or framework or whatever (I do too, because you can't know -everything- by heart), but you can't Google on the fly how to design a huge piece of software while you're doing it. You have to KNOW how to handle that. It's never going to work.

There's a huge discrepancy on the Dutch job market with regard to computer science.

Companies: Ask 75 skills, be acquainted with everything from systems programming and GUI design on the Web and everything in between, expecting people to know everything at every point in time, all the time...

The public: You don't need to know anything, you can learn everything on the job in a few weeks and Google the rest...

Realistic software engineers (like me): Both aren't true. Software engineers are people who know their stuff but don't know -everything- (you can't; you maybe could, in the 60's and 70's). We're trying to convince companies we know enough to do the job, and to convince the public that it's utterly stupid and impossible to try and start writing big software (enterprise-grade programs or bullet-proof firmware for use in factory automation for example) when you've never seen a line of code in your life, let alone have no education on how to design it.

I couldn't be happier if the government would put software engineer among the protected professions, just like architect. Nobody in their right mind thinks that anyone without an education in the field will be able to design a fracking building that has no structural problems. Then why does everybody think that those same people will be able to properly design a piece of software and expect it to have no problems and be maintainable?!

Making software engineer a protected profession would potentially solve a lot of problems in IT. Then, educate people to use stable, well known frameworks, libraries, and languages for their stuff, instead of trying to use the in-flux latest and greatest, or even trying to re-invent the wheel over and over again... (The profession is called "Software Engineer" and not "Software Experimenter" for a reason.)"


This hasn't changed in 50 years. Non technical people are always looking for a "magic bullet" to get rid of IT. To management, IT was a monetary sink hole, staffed by a bunch of uppity nerds, who had no respect for the corporate ladder. There are always salespeople willing to promise the moon "my stuff will cut your IT cost in half", cash the check, and run for the hills!

You may not realize it, but COBOL was the first of these "magic bullets" in 1958. I could fill half a page with the history of "magic bullets".

Some hardware, some software, some "wetware" (What was the great offshoring trend of the 2000's, but a way to get IT "at half the price"?)

There was a saying in the 1970's. . . ."If builders built buildings, the way programmers wrote programs, the first woodpecker that came along would destroy civilization."

[Katsunami]

Originally Posted by Ralph Sir Edward

There was a saying in the 1970's. . . ."If builders built buildings, the way programmers wrote programs, the first woodpecker that came along would destroy civilization."

That was said by Edsger Dijkstra I think, one of the pioneers of software engineering and designing software on sound principles, as opposed to just 'programming' a computer.

And he has a lot of quotes.

About knowing enough to name software engineering a respectable discipline, he said the following:

After having programmed for some three years, I had a discussion with A. van Wijngaarden, who was then my boss at the Mathematical Center in Amsterdam, a discussion for which I shall remain grateful to him as long as I live. The point was that I was supposed to study theoretical physics at the University of Leiden simultaneously, and as I found the two activities harder and harder to combine, I had to make up my mind, either to stop programming and become a real, respectable theoretical physicist, or to carry my study of physics to a formal completion only, with a minimum of effort, and to become....., yes what? A programmer? But was that a respectable profession? For after all, what was programming? Where was the sound body of knowledge that could support it as an intellectually respectable discipline? I remember quite vividly how I envied my hardware colleagues, who, when asked about their professional competence, could at least point out that they knew everything about vacuum tubes, amplifiers and the rest, whereas I felt that, when faced with that question, I would stand empty-handed. Full of misgivings I knocked on van Wijngaarden’s office door, asking him whether I could “speak to him for a moment”; when I left his office a number of hours later, I was another person. For after having listened to my problems patiently, he agreed that up till that moment there was not much of a programming discipline, but then he went on to explain quietly that automatic computers were here to stay, that we were just at the beginning and could not I be one of the persons called to make programming a respectable discipline in the years to come? This was a turning point in my life and I completed my study of physics formally as quickly as I could. One moral of the above story is, of course, that we must be very careful when we give advice to younger people; sometimes they follow it!

And that was in 1972, where one of the greatest minds in computer science ever already said: "We as software engineers know nothing." (So how can companies expect you to know everything?)

And, about writing software without understanding the founding principles of software engineering (and as such, desiging good, maintainable software as much as is possible at the time when you're desiging it), he said:

The required techniques of effective reasoning are pretty formal, but as long as programming is done by people that don't master them, the software crisis will remain with us and will be considered an incurable disease. And you know what incurable diseases do: they invite the quacks and charlatans in, who in this case take the form of Software Engineering gurus.

In short, there are a lot of people active in IT that shouldn't even be let near a computer.

Being an auto-didact can only get you so far: I learned that when I was 15. I had been programming for 5 years, and I wrote my own chess engine... until, at some point, I just couldn't add any functionality anymore because the code was a mess, and I didn't understand it clearly any longer. The chess engine had grown, like a weed, without any design, until it could play legal moves and think 1-2 moves ahead, but that was it...

I basically was stuck at that point until I went to uni and studied computer science. It didn't make me a better programmer; I already could (and can) write concise, powerful and efficient code very well, but it _did_ make me a much better software engineer/designer.

[DMcCunney]

Originally Posted by Katsunami

In short, there are a lot of people active in IT that shouldn't even be let near a computer.

Yes.

Being an auto-didact can only get you so far: I learned that when I was 15. I had been programming for 5 years, and I wrote my own chess engine... until, at some point, I just couldn't add any functionality anymore because the code was a mess, and I didn't understand it clearly any longer. The chess engine had grown, like a weed, without any design, until it could play legal moves and think 1-2 moves ahead, but that was it...

The problem with being an auto-didact is one I've seen elsewhere, and not just in computing.

I know some people who are very bright, and auto-didacts with magpie minds that soak up facts. To some extent, that describes me. But the problem is knowing what you don't know. You can soak up all the facts you want, but without a conceptual framework to fit them into, there can be enormous gaps in your knowledge you simply aren't aware of, and it can come back to bite hard.

An old friend and former housemate is in that situation. He came to a parting of the ways with his business partner, and declared bankruptcy with an impressive listing of debts. I don't know details, and don't want to - I know his former business partner is suing him for an enormous amount, but since they'd worked together for 15 years, I don't know what prompted the abrupt about face. I do know the man I knew back when wouldn't have done what he's being accused of. I suppose the man he is now might have, but I find it unlikely.

My best guess is that he simply didn't know as much about what he was doing as he thought, and dug himself into a deep hole. He seemed to simply assume his plans would work out as intended, and had no fallback position if they didn't.

I basically was stuck at that point until I went to uni and studied computer science. It didn't make me a better programmer; I already could (and can) write concise, powerful and efficient code very well, but it _did_ make me a much better software engineer/designer.

It's the designer part that's critical. I've seen too many people jumping in and writing code with only a vague notion of what the code is supposed to do, and no knowledge at all of the current practices and workflows of the folks who are the intended users. That works about as you would expect, which is not at all.

First, define the problem you are trying to solve, and hint! It may not be what you think it is. Be prepared to pivot when you discover that.

I've done my best to acquire context, and have a framework into which I can place facts. Many of those facts have pointed out areas where I need more framework. There are enormous amounts I don't know, but I have at least some conception of what the gaps are.
______
Dennis

[Terisa de morgan]

Originally Posted by DMcCunney

It's the designer part that's critical. I've seen too many people jumping in and writing code with only a vague notion of what the code is supposed to do, and no knowledge at all of the current practices and workflows of the folks who are the intended users.

What do you say when, yes, you don't know about it... but not because of your lack of interest but the structure of the company itself, which does not allow that knowledge to reach you so you can do your work? And this is a very real situation in a lot of companies.

[DMcCunney]

Originally Posted by Terisa de morgan

What do you say when, yes, you don't know about it... but not because of your lack of interest but the structure of the company itself, which does not allow that knowledge to reach you so you can do your work? And this is a very real situation in a lot of companies.

And those companies shoot themselves in both feet, which is something else I believe Kats has complained about.

If you can, you find a more clueful employer. If you can't, you keep your head down, do what is asked of you, try to minimize potential damage, and keep records that might at some point prove the failure wasn't your doing, but occurred at a higher level.
______
Dennis

[Katsunami]

Originally Posted by DMcCunney

The problem with being an auto-didact is one I've seen elsewhere, and not just in computing.

You can learn a lot of stuff on your own, but at some point you get stuck, inevitably, because there are things you can't come up with yourself, simply because some of them require YEARS of study to discover.

As I said, I hit a wall at around 15-16, when I just couldn't improve the chess engine anymore. Refactoring wasn't even an option, because everything was tangled with everything (I now know).

Other walls I've hit are playing Chess and Go.

It's well known that, for most people with some keen interest, an ELO-rating of 1800-1850 in Chess, and a strength of 5 kyu in Go are attainable just by playing a lot to gather experience, but at that point, there's a wall you can't climb by just 'playing even more.'

I've never actively studied chess, or Go, and my strengths are... well... ELO 1835 in Chess, and 6 - 4 Kyu in Go, exactly what can be expected. to advance, I'd need a teacher in both games to show me the stuff I don't know I don't know.

It's the designer part that's critical. I've seen too many people jumping in and writing code with only a vague notion of what the code is supposed to do, and no knowledge at all of the current practices and workflows of the folks who are the intended users. That works about as you would expect, which is not at all.

I follow a YouTube-channel, called The Coding Train, by Daniel Shiffman. Sometimes, it's very cheesy, but he explains mathematical concepts very well, and visualizes them using the Processing an p5.js languages/frameworks.

He is an MSc. in math, and it shows. Whenever he 'just' needs to put a formula into a function and write some stuff to visualize it, everything goes very well. A month ago, he tried to write an implementation of the game 2048.

As usual, he jumped in without any design or plan, and then he got stuck. After hours of jacking around with the code, he had to abandon his attempt (in Processeing), only to restart a month later in p5.js.

He did manage to get the game mechanics coded and had a lot of fun doing it, but from a software engineer's standpoint, it wasn't pretty.

He has stated numerous times that he teaches "programming": I *HOPE* he teaches how to visualize mathematical concepts and data, because he does that very well. If he teaches software engineering, well... let's say there will be some VERY poor software engineers coming from New York University.

This guy should teach Maths, not programming, and certainly *NOT* software engineering.

(Note that I'm not trying to bash Michael Shiffman. He is a great teacher, and his channel is fun to watch. He is good at clearly explaining difficult subjects in a fun way, and good at visualizing data. He is, however, very poor at writing maintainable code... but I guess that's not the point of his channel.)

[Katsunami]

Originally Posted by Terisa de morgan

What do you say when, yes, you don't know about it... but not because of your lack of interest but the structure of the company itself, which does not allow that knowledge to reach you so you can do your work? And this is a very real situation in a lot of companies.

Originally Posted by DMcCunney

And those companies shoot themselves in both feet, which is something else I believe Kats has complained about.

If you can, you find a more clueful employer. If you can't, you keep your head down, do what is asked of you, try to minimize potential damage, and keep records that might at some point prove the failure wasn't your doing, but occurred at a higher level.

Indeed.

I know, for most non-technical people, designing software is not a thing. They see it as a waste of time and money to have designs made and documentation kept up to date.

They understand you can't just start putting up walls to build a house and expect it to stay up without it being designed properly first, but they *fully* expect people to just sit down and start writing code, off the bat, that does work and is maintainable.

Heck, even for many software engineers/programmers designing is not a thing. They hate to do it. They'd rather write code. I was like that, at some point, when I wrote software that was easy enough to keep in my head. In university, I *often* thought "why should I *design that* ?! I can just write it out, right now."

What I didn't know back then is that I *did* design the solution first, in my head, and it was simple enough to just remember and visualize the diagrams without actually writing anything down.

When you start to work on bigger and bigger software, this doesn't work anymore. If you start to work on software on which two, three, four or even more people are at work, it *REALLY* doesn't work anymore, because you: 1... can't even remember your own designs and decisions and 2.... you don't even *KNOW* what other people on the team are doing, let alone remember it and keep everything compatible.

The bigger the software becomes, the more you get to love the design and documentation aspect, because without it, writing the code is *not* fun.... and maybe not even possible.

It's the reason why I tell everyone I'm a Software Engineer, and NOT A PROGRAMMER.

In the Netherlands, there are actually three levels of computer science:

1. MBO (Vocational school): Basically, teaches people how to program. These people would end up implementing functions and such, as designed by the Nr. 2 in this list:

2. HBO (Uni of applied science, Bachelor or Master level): Applied Computer Science or Software Engineering, depending on the uni. These people learn how to *design* software. They write code, and basically also learn everything that is part of 1. above, but in the end, their main job would be designing software, and then implementing it, assisted by people from 1.

3. University (Research Uni, BSc./MSc.) These people study computer science in a theoretical way. Most of the time they don't design software, and don't write it. They do things like coming up with a sorting algorithm that saves two seconds when sorting 10 billion pieces of data.

(Of course there is some overlap here and there.)

The problem in the Netherlands is that 1. and 2. are often merged, with a disastrous results. You get Nr. 1 people who 'grow into' software design somehow, but clearly lack the knowledge to do it right, and thus the software will be poor, or... you get Nr. 2 people who only get to bang out code and are unhappy because "that was not what they studied for" (I've been in this position).

[Greg Anos]

I think part of it is why you are developing automation.

Is it to support physical reality? (things like hardware, or something that is clearly definable)

Is it to support soft concepts like ownership. or projecting the future?

They are <very> different.

I've almost always designed and code in the latter world. But then again, my academic background was micro/molecular biology, where a variance of 10 to the 3rd power was normal (among friends), and every rule was only a guideline, with lots of exceptions.

Not exceptions in realization, but exceptions in the underlying concepts you work with. Where you don't know all the inputs, all the outputs, or even all the rules. And never will.

Where whimsy and irrationality rule, with the money to back up those whimsies. Lots of money. . . Lots and lots and lots of money - with the "beeper" as your friend 24x6 or (7).

These environments will never achieve mathematical perfection, or follow mathematical rules.

But the money's good . . .

[wodin]

Ralph that sounds like me, as an IT Project Manager, I don't profess to be a Subject Matter Expert in anything. I need to have enough knowledge to understand roughly what you're talking about, and who the SME for that particular subject is.

And yes, the money's good!

[John F]

Originally Posted by Ralph Sir Edward

...

There was a saying in the 1970's. . . ."If builders built buildings, the way programmers wrote programs, the first woodpecker that came along would destroy civilization."

Originally Posted by Katsunami

That was said by Edsger Dijkstra I think, one of the pioneers of software engineering and designing software on sound principles, as opposed to just 'programming' a computer.

...

A Google says it was Weinberg.

[Greg Anos]

The problem is not that the coder has a vague idea of what they are coding, but the requester only has a vague idea of what they want.

No amount of engineering is going to fix that. And the requester has the money that pays you. Nor do they want to be educated, either. (How <dare> you tell a senior manager that they don't understand what they are asking! Ahem. . . not habit forming. . .)

Let me give you a real life example. I was working for one of the top three US banks. (For legal reasons, don't ask who. At the time, they were the largest handler of mortgage security payments in the world. Yes, <that> big.)

The US law prevents banks for paying interest on cash balances for non-profit organizations. (At least at that time. Who know what the law is today.) But top management wanted to pay interest to get more business. The question was, how can we cheat and do this anyways? (This goes on all the time with the "Big Boys" in the financial industry.)

The lawyers said, 'You can pay rebates for fees legally. Everybody charges fees out the wazoo, so you can reduce their fees up to zero amount, and get away with it. It's not interest <wink>!"

So my phone rings. Can I code non-profits to get interest without calling it interest? Of course, they want it flexible, based on a widely available index, like Fed Funds rate, or LIBOR. Example, non-profit X would get rebates at Fed Funds rate - X percent. And could they have it yesterday?

Yes, I can do it, but it will take a couple of weeks to code and test, and another month to get it through the bureaucracy.

I will now ask, where is the major flaw in the request. Not from a legal standpoint, not from a moral standpoint, but from a sheer design/implementation standpoint. The flaw that will cause an emergency call a couple years later?

The hand grenade was - "rate - X percent". What to do if the base rate fell below minus percent? The calculated rate would become negative, which would cause fees to jump even above what was legally allowed. Not good for the business - potentially. . .

After an hour or so of analysis (I'm a full service programmer), I point this out. I was firmly told that base rates would never get that low, don't code a negative branch. (Remember, these are the people who sign the checks. . . ) I went back to my desk, shook my head, and murmurer "morons", and coded a negative branch to load zeros to the rebate amount anyways, so I wouldn't get that 2 AM phone call, and just didn't bother to tell the stupid user.

Sure enough, 2 years later, I'm sitting at my desk, contemplating the latest user idiocy, when the phone rings. "OMG, rates are dropping below our rebate contracts! How can we fix it before the customers get overbilled!"

I replied, "Rest easy, I put in a negate rate scenario fix when I put in the project." 'But I told you not to." "Yeah, well, you're calling me for it now, right?" "Ummm. Yes." "It's called 'being proactive'. Now I defaulted the amount to zero if it calculated out negative. Is that what you want?" "Yes." "Ok, Relax and have a cup of coffee. You can tell the brass you already had the possibility thought of and prepared for." "You'd let me take the credit?" "Sure, if you put me in for a bonus." "Fine!" (Note, I never got a bonus. I did get my just reward. I was laid off a year later. No good deed goes unpunished. That little patch, 3 lines of COBOL, saved the bank more money than than my entire salary for my entire career there. So they had to save more . . . )

My point on this yarn is that you can't have engineering discipline when you don't have fixed definitions. In an environment where laws and rules are always changing, there can be no "by the book". There's no "book" to be by.

That's why I'm a programmer, not a Software Engineer.

[Bilbo1967]

[Terisa de morgan]

Originally Posted by Ralph Sir Edward

My point on this yarn is that you can't have engineering discipline when you don't have fixed definitions. In an environment where laws and rules are always changing, there can be no "by the book". There's no "book" to be by.

Corollary: Walking over water and meet software requirements are an easy task if both, water and requirements, are frozen.

[Greg Anos]

Originally Posted by Terisa de morgan

Corollary: Walking over water and meet software requirements are an easy task if both, water and requirements, are frozen.

Which is easy in SIberia, not so easy in Hawaii. . . .

[Katsunami]

Somehow, I've trashed my main backup drive.

I've got five copies of my data:

- On the desktop
- On the laptop
- Main backup drive (MBD)
- Second backup drive
- One off-site backup drive
- (And a sixth of my music on the NUC dedicated as a music player.)

Normally I work on my desktop, and I keep my MBD and laptop in sync with it. Lately though, I've been mainly working on my laptop, and a lot of data has changed (didn't touch the desktop for close to six weeks), so I synced the MBD with the laptop, and the intention to then sync the desktop and at the same time, the second backup drive.

The laptop has a TPM, and its internal drive has been using Bitlocker since it was installed. Since a month or two ago, I've also Bitlocked the backup drives. No problems there. Everything has been working fine.

The desktop doesn't (yet) have a TPM.

So, when I connected the MBD to the desktop, I already suspected some sort of error. Windows 10 asked for a password, and I supplied it. The drive was unlocked (the lock disappeared), but when I wanted to open it, I got an error: drive structure unreadable, drive corrupted.

Well... I had expected something like "Can't be unlocked. TPM missing." To be certain, I connected the drive to the laptop again and unlocked it using its password... and got the same error there. The drive is trashed, or at least, the file system is.

Now I beg the question:

- Did the desktop trash the drive due to trying to unlock it without a TPM? (I tested this drive on my previous, almost 10 year old old laptop, which does have a TPM, and it unlocked and worked as expected there.)
- Did something else, somewhere, go wrong?

So now I've removed the partition, Bitlocked the drive again on the laptop, and I'm recreating the entire backup. When done, I'll try it on the desktop again and I'll see what happens.

(I don't actually know if a Bitlocker ToGo drive needs a TPM to be unlocked, or if the password is sufficient. I can imagine the TPM only being necessary to store the decription key, for internal drives set to auto-unlock. Nevertheless, I do intend to install the Asus TPM onto my desktop's mainboard, as it only costs €9.95.)