Gregg's Nearly Interminable Adventures in Linux Land

[eschwartz]

Actually, the different DDR generations have different shapes (they really don't want you to mess up and use the wrong one).

But it is pretty obvious which of the four generations you have, if you look at the current stick(s).

...

RAM also has a clock frequency, which is slightly harder to verify.
It is also fully compatible, i.e. PC3-10600 is slower than PC3-12800 but even if your motherboard only supports PC3-10600 a PC3-12800 stick will work, but simply run at PC3-10600 speeds.

You don't need Crucial's program to tell you what shape your RAM is, but it might be helpful to see what maximum speed you want.
There are programs (CPU-Z, Speccy) to identify your hardware on Windows, but on linux you can probably just examine the output of dmidecode.

[Gregg Bell]

Hey, shifting gears a bit. I' used to be able to install OSs from a bootable USB flash drive. Now when I do I get one of two errors. One is this "SYSLINUX 6.03 EDD 20150813 Copyright (C) 1994-2012 H. Peter Anvin et al
Boot Error" and the other is a blue Unetbootin Default screen that gets stuck in an endless loop. (The first error comes from flash drives made in Startup Disk Creator, the second kind from those made in Unetbootin.) If you happen to know what might cause either of those errors please let me know.

That said, I can install OSs from a bootable CD. But the CD is limited to 700 MB and many distros are bigger than that. So I happened to be looking at Xubuntu's download page and they referred to this minimal CD thing. At first I thought it was just a really small version of Xubuntu but I went to the link and saw it was for putting bigger distros onto CDs.

https://help.ubuntu.com/community/In...tion/MinimalCD

If I can't figure out how to get the installations working with the USB flash drives again, would this minimal CD route be a good way to go? Thanks.

[Gregg Bell]

Originally Posted by eschwartz

Actually, the different DDR generations have different shapes (they really don't want you to mess up and use the wrong one).

But it is pretty obvious which of the four generations you have, if you look at the current stick(s).

...

RAM also has a clock frequency, which is slightly harder to verify.
It is also fully compatible, i.e. PC3-10600 is slower than PC3-12800 but even if your motherboard only supports PC3-10600 a PC3-12800 stick will work, but simply run at PC3-10600 speeds.

You don't need Crucial's program to tell you what shape your RAM is, but it might be helpful to see what maximum speed you want.
There are programs (CPU-Z, Speccy) to identify your hardware on Windows, but on linux you can probably just examine the output of dmidecode.

So you're stuck with whatever memory type you have then because of the shape, right? So DDR2 has a different shape than DDR3?

There's no sense of "I'm going to speed this computer using DDR2 by getting rid of the DDR2 and using DDR3"?

I ran

dmidecode

and got "permission denied." It didn't ask for sudo or anything.

[eschwartz]

The minimal CD can be useful -- IIRC, it requires the command-line to install, though, since it saves space by not including a desktop or GUI.

But if you avoid UNetbootin and simply write the Xubuntu ISO download directly to the flashdrive, it should work fine.
UNetbootin is known to cause weird problems.

[eschwartz]

Originally Posted by Gregg Bell

So you're stuck with whatever memory type you have then because of the shape, right? So DDR2 has a different shape than DDR3?

There's no sense of "I'm going to speed this computer using DDR2 by getting rid of the DDR2 and using DDR3"?

Well, you could only it would involve upgrading your motherboard as well.

I ran

dmidecode

and got "permission denied." It didn't ask for sudo or anything.

dmidecode requires sudo.
sudo never asks to be run -- if you need it but don't use it, you get a permission denied error.
If you use sudo, it might ask for your password.

[kacir]

Originally Posted by DMcCunney

And I'd look seriously at installing an SSD. The usual concern is that SSDs have finite write operations. A cell on an SSD can be written to about 10,000 times. Beyond that, it becomes inaccessible. Current SSDs use firmware that attempts to spread writes evenly over all cells, and the firware also attempts to transparently migrate data on failing cells to good spares and mark the failing cell as bad. In practice, you are likely to upgrade to a new maching lone before you even notice SSD wear.

Also, please note that there are three kinds of SSD disks:

SLC - Single Level Cell - one memory cell holds one bit - those are SSD disks for fancy servers that you see selling for thousands of dollars (for an SSD not for the entire server) - those have memory cells that can be re-written millions of times

MLC - Multi Level Cell - one memory cell holds two bits so it has to have four voltage levels representing 00, 01, 10, 11 values per cell. Those have hard physical limit of about 7000 writes. You have to use higher voltages than on SLC to be able to be able to distinguish between 4 voltage levels and this physically wears out memory cells. ALL SSD disks sold today have very fancy wear leveling mechanisms that moves data around behind your back, so that all cells are worn out to a similar level, so that one region holding swap file or a temp directory or something doesn't wear out much faster than parts that are only read after OS installation.

TLC - Triple Level Cell - one memory cells holds three bits, EIGHT voltage levels. This has significantly lower number of writes per memory cell. At the time MLC were named nobody expected that one day they would use 8 voltage levels per cell. Otherwise MLCs would have been named DLC.
Many modern "budget" SSDs have TLC, but manufacturers do not like to advertise the fact that the disk uses TLC instead of MLC and has much lower wear reserve. You usually have to google extensively to identify whether the SSD has MLC or TLC.
One of most popular models from Samsung - 840 Evo had a big scandal some time ago - this was one of first popular drives to use TLC and it turned out that when the data sits on the disk for half a year untouched the read speed is extremely slow - due to voltage drop. They still haven't solved that problem. There *is* a firmware update available for those disks but I suspect that they just re-write the data periodically, further increasing the wear of the disk.

The situation is further complicated by the fact that the increased density of modern silicon makes the memory cells less resistant to wear.

Recently there were vertical NANDs introduced by Samsung and they have insanely high warranties and guaranteed writes. Again, those come in MLC and TLC variety, with MLC being cheaper and less wear resistant.

I have purchased a huge 1TB SSD for this Linux machine.
In the last 516 days I have written 4559GB to the disk - overwriting it 4.5 times (out of several several thousand that are physically possible and several hundred that manufacturer guarantees).
I use the disk quite extensively, with a large Calibre Library, some development ...
The only thing I have done in this Linux is
- installed plenty of RAM and set swappiness in Kernel parameters to a low value, so the kernel does not start to use swap preemptively, before it really needs it
- set noatime parameter in fstab, so that the filesystem doesn't set time when you read the file - this would needlessly increase writes and for normal home desktop use the access time - atime parameter is never used anyway.
- made sure fstrim is working automatically. You have to TRIM the SSD periodically to free up cells after delete of data. SSD has to erase the whole block of data at once when you change something, and reclaiming this space as you go (when you run out of available cells) is very time consuming. Please note that modern distros should have this working out-of-box

I was very reluctant to purchase the SSD, because I need to work with a very large number of small files (in Calibre, development and other scenarios) and I worried about how long the SSD would last. BLOODY expensive SSD, mind you ;-). Yet, this is scenario where SSD really shines. If you imagine that each memory cell has a very limited lifetime of several thousand writes you *have* to be anxious, especially if you know how many files are written when you do a bulk rename in Calibre or dabble in software development with some projects that have thousands of files that need to be updated frequently. Even Calibre itself is some 200MB and gets updated once a week. Virtual machines can also cause lots of writes to the image file.

After the SSD purchase I wrote a script that queries SMART data on disk and prints how much data I write daily and over the lifetime of the disk. I see that even with intensive use the disk will be obsoleted *long* before it is written to death, even if it was a dreaded TLC ;-)

Originally Posted by DMcCunney

Last but not least, pay attention to what video is offered on the machine you get. Will it be adequate for your needs, or will you want to add a video card? Will the video card you use be supported by Linux? Is there a manufacture's driver for Linux, or are you limited to existing Linux support for the card? You didn't mention details on what sort of video editing you wanted to do, but you need decent video on the machine to support it.

I personally prefer the built-in Intel graphics. It is adequate for my needs and this way I have least amount of hassle with graphics drivers and tinkering with graphics system

[DMcCunney]

Originally Posted by Gregg Bell

Eight slots, wow. Probably one of those 64GB gaming machines.

Not just gaming. Eric S. Raymond recently built a machine he calls the Great Beast of Malvern. It has 64GB RAM among other things. But he's not a gamer. The Great Beast is designed to convert software repositories from one version control system to another. There are a number of version control systems out there, including Bazaar, CVS, git, Mercurial, RCS, sccs and SVN. (And that's just some of the open source products. There are commercial products as well.) Eric wrote tools to automate most of the conversion from one to another. A lot of products first written decades ago were still under CVS, one of the earliest VCSes. They really needed to be migrated to something more modern, but when the repository is many gigabytes in size with decades worth of commit histories, it's a daunting task. To grease the wheels, Eric built a machine that can run his software to do the conversion and accomplish it within hours instead of days or weeks. His goal is to eliminate CVS in his lifetime. He has good reason for wanting to make it go away.

I like Crucial. That's what I got when I added a stick to my computer. But that was when the computer was Windows before I converted it to Xubuntu. Crucial does this "guarantee" that the memory will work. You download their little program and it runs and tells you just what you need. Nice. Only thing is they don't do it for Linux computers. So how do I know that the memory I buy for my Linux computer will work? (Or do I ever know?)

<sigh> RAM is hardware. RAM needs to be the correct speed and form factor to be installed in and run correctly in the machine. If it will work in the hardware, it doesn't matter what the OS is. On an x86 box, it could be Free/Net/OpenBSD, <whatever> Linux, MSDOS, Netware, OS/2, OS/X, Solaris, Unix, Win3.x/9.x/NT/2K/XP/7/8.1/10... The OS is built to run on the hardware, and all will access it the same way.

Whatever makes you think the fact that you're running Linux would matter here?

I like Crucial too, and have also used MemoryX. Crucial is a unit of long time DRAM manufacturer Micron Technology, and a Name Brand. Their tool to examine your system and tell you what kind of RAM you need is a boon. You don't have to pop the hood and peer to see what's installed.

Crucial is also in the SSD business these days (since NAND Flash is a form of non-volatile memory), and I have a Crucual MX-100 SSD in my desktop.
______
Dennis

[DMcCunney]

Originally Posted by eschwartz

If RAM is that much of a desperate concern that you need to jump desktops, why not settle for some lightweight window manager, say, dwm. It runs in ~1 MB of memory.

<grin> I've looked at stuff like that. It's a bit too minimal.
______
Dennis

[DMcCunney]

Originally Posted by arjaybe

As I understand it, some userland programs for their Linux developers working on their cloud services. Running on Win10.

At present, yes, though I believe more extensive support is planned down the road.

The key here is Linux support libraries that will be part of a Win10 upgrade.

I had a Unix machine at home before I got a PC running MSDOS, and spent time and effort trying to make the PC look and act as much like the Unix machine as possible. The eventual solution was a commercial package called the MKS Toolkit, that offered all of the Unix commands that made sense in a single-user, single tasking OS like DOS. The selling point here was a complete implementation of the Korn shell. Running under the MKS Korn shell, you had to dig a bit to discover it wasn't a Unix machine.

I kept up the effort when I moved to Windows, and at various times ran Microsoft's Service for Unix (based on Interix, which had roots in the MKS product), AT&T's UWin, and Cygwin, which ported the Gnu Linux toolchain including GCC to Windows. UWin and Cygwin both implemented POSIX compatibility layers encapsulated as DLLs. A lot of *nix code builds "out of the box" under Cygwin because it links against the DLL and sees the *nix system calls it expects.

But that sort of arrangement carries a penalty in speed and complexity of environment. These days, I run "native" Win32 ports of Linux commands built with MinGW, a version of GCC set up to build using native Windows runtimes. (Git for Windows includes a pretty complete set, including the bash shell.)

With underlying support for *nix code built into Win10, an assortment of things become possible, including full Ubuntu running alongside Win10 without using a VM. Down the road, I expect to see actual container support in Windows, but that will take a bit, and isn't the direction Canonical took for this effort.
______
Dennis

[DMcCunney]

Originally Posted by Gregg Bell

Ha ha, Dennis, thanks for the vote of confidence!

You're welcome, and it is a vote of confidence. If I didn't think you could handle it, I wouldn't suggest doing it.
______
Dennis

[Dngrsone]

I haven't gone the SSD route myself yet... I moved over to laptops almost ten years ago and have stopped building my own computers.

Yes that eight-slot machine I referred to was a gaming machine, though I bought it obsolete and set it up as a firewall (it had two built-in NICs, which is what I wanted).

Damn Small Linux was a favorite minimal OS that I used to use, Nowadays, I carry Puppy around on my thumb-drive, though I haven't booted it in a while...

[DMcCunney]

Originally Posted by Gregg Bell

That said, I can install OSs from a bootable CD. But the CD is limited to 700 MB and many distros are bigger than that. So I happened to be looking at Xubuntu's download page and they referred to this minimal CD thing. At first I thought it was just a really small version of Xubuntu but I went to the link and saw it was for putting bigger distros onto CDs.

https://help.ubuntu.com/community/In...tion/MinimalCD

If I can't figure out how to get the installations working with the USB flash drives again, would this minimal CD route be a good way to go? Thanks.

I used the Minimal CD to install Ubuntu on my old Fujitsu notebook. It works fine, and is certainly small enough.

But what you get is a bare ones command line installation. It does not install a GUI. That's not a huge issue, as "working installation" means "working network support", so you can use apt-get from the command line to install a GUI and whatever else you want. Select the GUI you want to use, and Xorg and the necessary support to run a GUI will come along as dependencies. You can then reboot into the GUI and proceed as you normally might.

I'd be more interested in why your installing from USB stopped working. When was the last time it did work? What changed since then?
______
Dennis

[DMcCunney]

Originally Posted by Gregg Bell

So you're stuck with whatever memory type you have then because of the shape, right? So DDR2 has a different shape than DDR3?

Yes.

There's no sense of "I'm going to speed this computer using DDR2 by getting rid of the DDR2 and using DDR3"?

Correct. The machine can't actually use DDR3. And while there are different speeds of RAM within DD2, DDR3, etc, there's no point in buying faster RAM than the machine can use. The RAM just needs to be fast enough.

The first way to boost performance on an existing machine is to add more RAM. As mentioned, machines are I/O bound, and spend most time in a wait state waiting for I/O to occur. Anything that reduces the need to access disk cuts I/O time, and more RAM allows better caching and less disk access.

The second way to boost performance is to move to a solid state drive. Those are an order of magnitude faster than standard hard drives, so stuff that does access the disk happens a lot quicker. (Note that this speeds up booting and program loading. It makes no difference in performance once things are loaded and running.)

The third way is to get a new machine with a faster processor, but most cases can be handled by steps one and two.

I ran

dmidecode

and got "permission denied." It didn't ask for sudo or anything.

It won't. Try running it from a terminal as "sudo dmidecode".
______
Dennis

[DMcCunney]

Originally Posted by Dngrsone

I haven't gone the SSD route myself yet... I moved over to laptops almost ten years ago and have stopped building my own computers.

You really ought to look at it.

My current desktop is is a refurb Dell product. My prior one was built from components. These days, refurb packaged systems are good enough to eliminate a lot of the need to roll your own, unless you're a hard core gamer.

Damn Small Linux was a favorite minimal OS that I used to use, Nowadays, I carry Puppy around on my thumb-drive, though I haven't booted it in a while...

DSL ran into a wall a while back. A design spec was an ISO that would not get larger than 50MB. It finally reached a point where there was nothing more that could be crammed into an ISO that size, even with content choices based on small size.

I have Puppy on an older machine. It works, but it's quirky. I began on *nix with AT&T Unix System V a couple of decades ago, and I've been an admin on multi-user Unix and Linux machines. As someone who has spent the odd hour locking down machines so users can't get root, Puppy's "You always run as root" approach gives me hives.

For something I'd boot off a flash drive these days, I'd probably look at Tiny Core Linux.
______
Dennis

[DMcCunney]

Originally Posted by kacir

Also, please note that there are three kinds of SSD disks:

<...>

Good summary, thanks.

Given wear leveling on current SSDs, having one wear out is not a concern. (Though on the desktop, stuff that uses its own cache, like the browser, gets told to put in on a ramdrive.)

I have purchased a huge 1TB SSD for this Linux machine.
In the last 516 days I have written 4559GB to the disk - overwriting it 4.5 times (out of several several thousand that are physically possible and several hundred that manufacturer guarantees).

Which one did you get?

I have a 240GB Crucial MX-100 here. I have no need for a TB model.

I use the disk quite extensively, with a large Calibre Library, some development ...

I have a large Calibre library too, though it happens to live on the HD, not the SSD, and gets mirrored to a thumbdrive for redundancy and portability.

I'm not really doing development on this machine.

The only thing I have done in this Linux is
- installed plenty of RAM and set swappiness in Kernel parameters to a low value, so the kernel does not start to use swap preemptively, before it really needs it
- set noatime parameter in fstab, so that the filesystem doesn't set time when you read the file - this would needlessly increase writes and for normal home desktop use the access time - atime parameter is never used anyway.
- made sure fstrim is working automatically. You have to TRIM the SSD periodically to free up cells after delete of data. SSD has to erase the whole block of data at once when you change something, and reclaiming this space as you go (when you run out of available cells) is very time consuming. Please note that modern distros should have this working out-of-box

I did similar things here, since I dual boot.

I was very reluctant to purchase the SSD, because I need to work with a very large number of small files (in Calibre, development and other scenarios) and I worried about how long the SSD would last. BLOODY expensive SSD, mind you ;-). Yet, this is scenario where SSD really shines. If you imagine that each memory cell has a very limited lifetime of several thousand writes you *have* to be anxious, especially if you know how many files are written when you do a bulk rename in Calibre or dabble in software development with some projects that have thousands of files that need to be updated frequently. Even Calibre itself is some 200MB and gets updated once a week. Virtual machines can also cause lots of writes to the image file.

Wwith current wear leveling that tries to spread the load evenly over all cells, the question becomes "How long will it take for any particular cell to actually reach that limit?" The answer will normally be "a long time". SSD wear was not a huge concern here. I did things like you did to limit it, but that was reflex.

After the SSD purchase I wrote a script that queries SMART data on disk and prints how much data I write daily and over the lifetime of the disk. I see that even with intensive use the disk will be obsoleted *long* before it is written to death, even if it was a dreaded TLC ;-)

Exactly.

I personally prefer the built-in Intel graphics. It is adequate for my needs and this way I have least amount of hassle with graphics drivers and tinkering with graphics system

If what you do is basically 2D graphics, Intel is fine. It falls down on 3D. I tossed in a $50 ATI card here because I do a few things where 3D is used and wanted a bit better performance. It's not the latest and greatest whiz bang GFX card, and is decently supported under Linux as well as Windows.
______
Dennis