The Technology Vent and Rant Thread

[jbjb]

Originally Posted by ratinox

Once they identify how the salts are generated they can use this to generate custom tables, which is orders of magnitude faster than brute force and doesn't require infinite storage.

It's still the same effort to build the custom table as it is to brute-force a single password - you still have to compute the hashes of all the possible passwords if you want the table to be complete.

Even for a short-ish (10 characters or so) password that takes a lot of compute, particularly if a deliberately compute-intensive hash algorithm (nested hashes, bcrypt etc.) is used.

The point of rainbow tables is that you only have to do that once to attack many passwords with the same hash - you still need to generate a separate table for each salt.

[jbjb]

Originally Posted by ratinox

Once they identify how the salts are generated they can use this to generate custom tables, which is orders of magnitude faster than brute force and doesn't require infinite storage.

I've just reread this, and don't get the point you're trying to make. If you've got hold of the passwd file, or whatever, with the salted hashes, you already know all the salts - they're sitting right there in plain text. What would knowing the salt generation algorithm gain you? (Genuine question!)

[ratinox]

Originally Posted by jbjb

In any sane environment the salts are generated properly randomly, using a high quality source of entropy. Not something that can be reverse engineered, unless you can spy on the source of entropy.

This is past to the edges of my expertise and experience so I'm not going to say yay or nay to the point (I have implemented hash-based auth but that was a very simplistic system for a POP3 client). But given that hardly a day goes by without another major breach being announced, it appears that doing things correctly is the exception, not the rule.

It's always just a matter of time, but if that time is many lifetimes of the universe, we're probably OK.

"If". Attacks always get better, and there are no takesies-backsies.

[ratinox]

Originally Posted by jbjb

I've just reread this, and don't get the point you're trying to make. If you've got hold of the passwd file, or whatever, with the salted hashes, you already know all the salts - they're sitting right there in plain text. What would knowing the salt generation algorithm gain you? (Genuine question!)

Unix password files are well-documented, but any given site like mobileread can do things differently, so an attacker would need to identify that in order to generate usable hash tables. Or to identify a more efficient attack if one exists.

[Renate]

MD5 itself is not the most brilliant algorithm and it's probably possible to find clashes pretty easily. But the Linux password algorithm MD5Crypt uses 1000 rounds of it, both to hide weaknesses and to make it 1000 times harder to calculate a "rainbow table". Of course time has caught up with this and a million would be a better number.

[jbjb]

Originally Posted by ratinox

This is past to the edges of my expertise and experience so I'm not going to say yay or nay to the point (I have implemented hash-based auth but that was a very simplistic system for a POP3 client).

That's more expertise than most! Thanks for the interesting discussion. I hope I haven't come across as too pedantic - I've got a few decades of experience in related fields, and admit I have a tendency to get too picky (as my wife will confirm).

But given that hardly a day goes by without another major breach being announced, it appears that doing things correctly is the exception, not the rule.

Nail meet head . By far the most significant issue is weakness in implementation.

"If". Attacks always get better, and there are no takesies-backsies.

Agreed.

[jbjb]

Originally Posted by ratinox

Unix password files are well-documented, but any given site like mobileread can do things differently, so an attacker would need to identify that in order to generate usable hash tables. Or to identify a more efficient attack if one exists.

That's true, but in most real-world implementations I'm aware of (admittedly, most of which are at the more secure end in terms of requirements) the hashes are not generated solely by an algorithm, but use local (unreproducible) entropy.

[jbjb]

Originally Posted by Renate

MD5 itself is not the most brilliant algorithm and it's probably possible to find clashes pretty easily. But the Linux password algorithm MD5Crypt uses 1000 rounds of it, both to hide weaknesses and to make it 1000 times harder to calculate a "rainbow table". Of course time has caught up with this and a million would be a better number.

Bcrypt is your friend.

[ratinox]

Originally Posted by jbjb

That's more expertise than most! Thanks for the interesting discussion. I hope I haven't come across as too pedantic - I've got a few decades of experience in related fields, and admit I have a tendency to get too picky (as my wife will confirm).

Oh, no. Not at all. Security can be messy and complicated and weirdly counter-intuitive.

[jbjb]

Originally Posted by ratinox

Security can be messy and complicated and weirdly counter-intuitive.

That is most certainly true!

[Renate]

Originally Posted by ratinox

Security can be messy and complicated and weirdly counter-intuitive.

The experts have finally decided that forcing people to change their passwords periodically is counterproductive.

[ratinox]

Originally Posted by Renate

MD5 itself is not the most brilliant algorithm and it's probably possible to find clashes pretty easily. But the Linux password algorithm MD5Crypt uses 1000 rounds of it, both to hide weaknesses and to make it 1000 times harder to calculate a "rainbow table". Of course time has caught up with this and a million would be a better number.

Hash collisions are a problem for general use, but much less so for authentication systems. For example, hardly anyone is going to care if a 10-character password collides with a 100K word novel in their auth system (except for the admin who has to clean up after someone pastes an entire novel into the password field and their system chokes).

And certainly, MD5Crypt has its weaknesses but so did 3DES which it replaced. And so, as expected, MD5Crypt is supplemented or entirely supplanted by stronger algorithms including Blowfish, SHA-256 and SHA-512. These in turn will be supplemented or supplanted by newer algorithms as time and tech progress.

And then the rise in use of multi-step verification (those authenticator apps, often mistakenly called multi-factor authentication). These don't fix the problems with passwords but they do make it more difficult for attackers to compromise. Though this in turn has lead to a rise in MFA/MSV fatigue attacks where an attacker spams the system until the victim hits "yes" or whatever on their app to get it to stop.

TOTP/HOTP apps are not multi-factor. A factor is a unique thing, commonly "something you know" such as your ATM PIN and "something you have" such as your ATM debit card. You cannot access your account without having both factors in your possession. These apps seem like something you have: your phone. But really, they're a variant of something you know since the seeds for these are entirely independent of the device -- they're information rather than the device itself. While multi-step is marginally weaker than multi-factor, it's still a big step up from plain login/password auth.

[jbjb]

Originally Posted by ratinox

Unix password files are well-documented, but any given site like mobileread can do things differently, so an attacker would need to identify that in order to generate usable hash tables. Or to identify a more efficient attack if one exists.

Sorry for reopening this, but it's clear I haven't made my point well.

The point is that:

• generating the rainbow table for a particular salt is about the same effort as brute-forcing one password
• hence, the point of rainbow tables is that the table can be used against multiple passwords if they use the same salt (i.e., in the real world, if they're unsalted)
• whether you know the salt through reverse-engineering the salt-generating algorithm or from reading it from a hacked file, you still need a table specific to that salt
• based on all the above, for a properly salted system you're basically back to brute-forcing each password

[ratinox]

Originally Posted by Renate

The experts have finally decided that forcing people to change their passwords periodically is counterproductive.

To be fair, security experts have know and advocated this for decades. It's just taken that long for the NIST to catch up.

[ratinox]

Originally Posted by jbjb

Sorry for reopening this, but it's clear I haven't made my point well.

The point is that:[LIST][*]generating the rainbow table for a particular salt is about the same effort as brute-forcing one password[*]hence, the point of rainbow tables is that the table can be used against multiple passwords if they use the same salt (i.e., in the real world, if they're unsalted)

No, I get it. If salts are random/unique for each hash then there's no practical difference between direct brute force and generating a unique rainbow table for that hash, except that brute force should be faster. Rainbow tables are useful against unsalted hashes or when salts are reused. Then you get two or more for the price of one.