Comment How strapped for hardware are you? (Score 1) 359
Since you, the OP, have not said much about the computer he plans to use, I'm going to make a few assumptions on your words and assume that he only has the mentioned computer (an old desktop minit-ower, maybe?) and give a plan of action based on my own experience with my own handmade NAS out of old parts.
** INTERLUDE/RANT BEGINS **
Oh, and the answer is, for a consumer, pure software RAID always: Hardware RAID was conceived and it's intended use case were ageing servers that needed higher, cheaper storage capabilities while also still working as the servers that were designed to be. So hard disk matrixes to emulate bigger disks was a logical idea. The problem was that the moment that you added redundancy, the CPU costs for them shoot sky high (for that age). Having a four threads *200mhz server (as in four single thread processors) wasting half it's processing power to manage it's disk storage was not efficient. It clipped them as servers.
The solution? Plug a cheaper risk processor (a 400mhz single core PowerPC, like LSI and rebrands use), between the disk controller and the CPU. Does 2*200mhz equals 1*400mhz? It loads a proprietary binary blob (a software RAID, but running in the PowerPC processor) that emulates a fast single disk toward the CPU, and manages internally the RAID operations and redundancy. Also, since it's proprietary, you are now captive of whatever implementation the card maker has chosen to use, and since it is a binary blob loaded/updated from the room or a driver, the card maker sent you a limited version, and unlocking capabilities implies paying extra for a driver/blob update. "Wow! DLCs before internet gaming! Who knew that they existed? Datacenter guys!" ^_^
Of course that was then, and this is now, where people throws away computers for being too old that have 2 or 4 cores at 1600~2600mhz. The RAID managing costs in CPU are now incredibly cheap. So, HW RAID implementations do not make sense today unless you plan to use huge amounts of disks (>8), or complex catching systems, and need high 24/7 availability in a data center. For a home user? Heck, no! And for a mere NAS, even less!
** INTERLUDE/RANT ENDS **
So, the plan, in simple steps:
1. Examine the motherboard and it's specs:
1.1 Check that you have at least 4 SATA ports. Having read that said motherboard has a FakeRAID, probably it means that you are probably a lucky ba*$t4rd and have an old southbridge chipset with 6 SATA ports. Usually ports 0~3 would work either as AHCI or a 0/1/0+1 FakeRAID, and ports 4~5 either as AHCI, or emulated PATA IDE, for old OS. If you have only 4, it is still ok. Only two, change the motherboard for another used one. Check Ebay, etc.. It will be cheaper that getting a decent 4-port SATA controller card.
1.2 Check that you have at least a 1GB ethernet port. If it's only 10/100 and you can, upgrade it.
1.3 Ensure you have at least 2GB of RAM. It will be probably what it does carry, for an old Windows Vista/7 PC. Lucky if you have more.
2. If the mobo passes the test, remove all other extra cards. Enter BIOS and set all SATA ports as AHCI, and enable Wake-On-Lan if you can.
3. Crawl under that bed and other places, find, buy or steal old/cheap pendrives, of 16GB or more of storage. Ideally get at least 3 USB3 pendrives, even if you are going to use them as USB2 only. Mainly because sometimes those have faster write speed than older USB2 ones. Look, ma! A solid state disk just as slow as an HDD, instead of even slower. ^_^U
4 Get a Debian net installer in another pendrive. Plug monitor and keyboard to the NAS machine, and remove all other hardware/cards except network and video if it does not have an integrated video card (that means no HDDs at all).
4.1 Plug in both pendrives, the installation and the fast one, in USB2 ports. Boot up from the net install. pendrive, and use the installer assistant to make a minimal installation on the other pendrive.The Debian installer may fail and do the install but failing at setting up grub, so it won't boot. Update manually the grub of the installation pendrive instead of the target pendrive. You may have to go to console before rebooting, and force it to update the right grub manually.
4.2 During the installation, choose a single partition for the root volume, and no swap. It will end creating two partitions, a Minix one for the grub, of a few megabytes, and an ext3 for the rest of the disk. Like Windows, it will align partitions to the megabyte, and start it at 0 and 10MB.
I read about some of the cheaper memories had incredibly wide memory cells (8, 16 or 32mb), like SSDs, but without dedicated RAM for cache, it made them incredibly slow for writing, and the missalignmen made them even slower, the wider they were, the worse. Using 32MB ensures always alignment, even if you don't know or can't figure the width of your memory cells (it's next to impossible w/o opening the casing). In my case the improvement was small but existed.
Make the partition smaller that your smaller 16GB pendrive. They are not really 16GB, they are 15.78~15.82 GBs, some space reserved for management and spare blocks. Pick less than that. I choose 15.5GB. Those grant 32MB alignment both to beginning and end.
Using USB2 ports, it's because few BIOSes and kernels support fully the USB3 controllers when booting from them, even if they will happily chug data from them down once the system is up.
5 Ready your fists to punch yourself in the face repeatedly until you manage to install it without trouble, make it boot automatically (choose the OS pendrive as default / highest priority boot device in the BIOS), and keeps working once you remove the installation pendrive. If it fails, try again, choosing another model of pendrive for the OS, and GOTO 4. If it works, CONTINUE.
6 Log in at the system (it will have to be as the standard user you created on installation). Logging in as root will be disabled and "sudo" will not yet be installed. You will have to use "su -" to switch from user to root. That normal user will be the default user, like "pi" for raspbian, or whatever for ubuntu. So plug in the ethernet and set up the DCHP client on your ethernet device. Reboot and check (ping other IPS in your network, do "apt update" to update the system, etc.) If ethernet fails, try again (GOTO 6). If it works, CONTINUE.
7 Do "apt update" and "apt upgrade" to update the system. Check a tutorial and switch to Debian 11 (bullseye), because you want Linux kernels v5.0 and other modern goodies. (swap repositories, apt update or apt distro-update, or whatever it is required now). When the system it's updated, reboot again for testing. Then enable/install ssh and test it to connect from the network. If it works, ok.
7.1 You can now shut down the proto-NAS and make a backup of your OS. Simply clone the working pendrive onto another empty one. Clone only the partitioned space, so no matter if the pendrives have the same, nearly the same, or way more space that the original, only the first 15.5GB are used. BACKUP DONE! NEXT!
8 Shut down the system, install the HDDs you plan to use, and then turn it on. Fiddle with the cables and the BIOS until all HDDs are recognised by the BIOS, all are in AHCI mode, and it stills boot fast and cleanly from the OS pendrive. Check that all HDDs are detected by Linux. Use "lsblk" or "blkid". Check how each HDD can have an unique UUID. if they had partitions, these will be unique too.
9 Install mdadm and it's tools, and look for a tutorial about how to zero the beginning/end of the HDDs. This is a single command, and it erases standard/GPT partition tables, RAID configuration blocks on the disks, and their backup an the end of the disk. No need to zero it fully.
10 If you were semi lucky an have 4 sata ports, use the mdadm command to create a RAID5 with those 4 devices. Use if you can when you create them with the UUIDs instead of the device name, and give it a name. Let default values for the rest for now. Once done, if you are using "lsblk" will show that now the disks have kind of raid pseudopartitions, all of them with the same UUID. That UUID is the UUID of the RAID itself.
11 If you want to use other filesystem instead of ext3 or ext4 install them and related tools now, like zfs, or in my case XFS. Format the device with the proper mkfs.xfs /dev/md0 or whatever. If it fails, retry.
12 So create a mounting point "mkdir /mnt/raid", and add the raid to /etc/fstab, using the uuid. That is because sometimes the raid will be detected by the kernel as mdadm device /dev/md0, and others as /dev/md127, but the UUID will never change. Check that it's detected automatically, check it's status with "cat /procc/mdstat", and reboot then to check that it boots w/o trouble and mounts automatically.
Remember too to make sure the mount config int /etc/fstab its specifying that it must run with the uid and gid of the standard user (1000:1000), so the standard user can access and modify the RAID data fully w/o swapping to root. If it does not work, fiddle and try again until it works. BACKUP! NEXT!
12 Now install samba. Stop the samba service/daemon with "service smbd stop". Edit the config file and add /etc/raid to the file shares. Sync the Linux users with the samba users, so the standard user can access to the shares through samba. Test from a windows machine. If the distro it's updated, mdadm, xfs and raid are updated enough that Windows 10 will recognize it w/o doing more. You will need to specify the IP to access it, since samba does not use the modern net broadcast protocol to announce itself, and samba v1 and it's broadcast protocol is deprecated and deactivated by default. Once it works, BACKUP! NEXT!
13 Google for a package and a repo for the wsdd daemon. It is the broadcast protocol. It can exist in python script or in c executable form. The repo does carry the c executable as a daemon, so install it with apt and enable it with "service wsdd start" From then on, it will show on Windows and Mac machines. Once it works, BACKUP! NEXT!
14 Shut it down. close the case, remove keyboard and monitor. Park it out of the way. Start it up again. Check if it still works. Connect to it with ssh to manage it.
The end, you are DONE.
Cost? An old core or core2 mobo/cpu with barely any useful amount of ram, and at least 3 HDDs of a decent size. 3 hdd SATA 3 or 4 hdd SATA 2+ disks, even if slow (5400 rpm) will hopefully fill your net output (1Gbit/s = 125GBytes/s).
Next step is upgrading your network to 1GB/s. If feasible, 2.5GB/s are becoming affordable and use standard ethernet cabling, but only NICs (network cards), for direct NAS to PC connection. Switches are still pretty expensive. If you are lucky and find one cheap secondhand, though? ^_-
Also, if you have at least 2 2TB hdds, don't fill the rest with 1TB. Try to fill at least 4 HDDs: buy secondhand or new cheap external 3 1/2 ones and chuck them out of their cases, or user bigger disks. 3+1 it's the best data / redundance ratio for 4 disks. If you were lucky and had 6 SATA with AHCI, you can opt for 5+1 in RAID 5 or 4+2 in RAID 6, depending if you want space or security, which it is important.
Taking into account that, for example my own system it is using:
* 1 WD disk (Caviar? before they started using the color brands, 10y/o?)
* 1 WD green disk (external, 7 y/o)
* 1 Seagate (old, external 5?y/o)
* 1 Toshiba (external, ?)
* 1 Seagate barracuda (internal, 6?y/o)
Some of the disk are nearly 10 years old, and the system it's moving them happily.
** INTERLUDE/RANT BEGINS **
If you buy new cheap, just don't buy Westend Digital, ever. Years ago they were awesome, but then they made so their external cases only worked with their disks. Disk were fine, though (WD green, WD red). Now they made so their disks (WD white) only work with their cases (they don't work inside computers without a simple, but bothersome hardware workaround). You don't want that failure point in your NAS.
** INTERLUDE/RANT ENDS **
Can't go cheaper than that.
If you don't want to mess up with Linux, use FreeNas, TrueNas or whatever. There are preconfigured distros that will run from a CD or a pendrive and will give you way more, but will require ant least 4GB of ram, ideally 8GB. That may mean replacing the ram for higher sticks, assuming you can install that many. It will give you a web interface and allow you to install applications and services, but those consume RAM and CPU. Some, like video transcoding for streaming, require a compatible videocard. They are easier and give you more, but they will start costing you money in new pieces instead of reusing old ones. Some can be installed in pendrives too, to use all the HDDS as storage space.
Prebuilt NAS are the same, except slimmer, and running in an underpowered ARM processor to barely consume any electricity, in exchange of switching to a locked system you can't mess with.
** INTERLUDE/RANT BEGINS **
Oh, and the answer is, for a consumer, pure software RAID always: Hardware RAID was conceived and it's intended use case were ageing servers that needed higher, cheaper storage capabilities while also still working as the servers that were designed to be. So hard disk matrixes to emulate bigger disks was a logical idea. The problem was that the moment that you added redundancy, the CPU costs for them shoot sky high (for that age). Having a four threads *200mhz server (as in four single thread processors) wasting half it's processing power to manage it's disk storage was not efficient. It clipped them as servers.
The solution? Plug a cheaper risk processor (a 400mhz single core PowerPC, like LSI and rebrands use), between the disk controller and the CPU. Does 2*200mhz equals 1*400mhz? It loads a proprietary binary blob (a software RAID, but running in the PowerPC processor) that emulates a fast single disk toward the CPU, and manages internally the RAID operations and redundancy. Also, since it's proprietary, you are now captive of whatever implementation the card maker has chosen to use, and since it is a binary blob loaded/updated from the room or a driver, the card maker sent you a limited version, and unlocking capabilities implies paying extra for a driver/blob update. "Wow! DLCs before internet gaming! Who knew that they existed? Datacenter guys!" ^_^
Of course that was then, and this is now, where people throws away computers for being too old that have 2 or 4 cores at 1600~2600mhz. The RAID managing costs in CPU are now incredibly cheap. So, HW RAID implementations do not make sense today unless you plan to use huge amounts of disks (>8), or complex catching systems, and need high 24/7 availability in a data center. For a home user? Heck, no! And for a mere NAS, even less!
** INTERLUDE/RANT ENDS **
So, the plan, in simple steps:
1. Examine the motherboard and it's specs:
1.1 Check that you have at least 4 SATA ports. Having read that said motherboard has a FakeRAID, probably it means that you are probably a lucky ba*$t4rd and have an old southbridge chipset with 6 SATA ports. Usually ports 0~3 would work either as AHCI or a 0/1/0+1 FakeRAID, and ports 4~5 either as AHCI, or emulated PATA IDE, for old OS. If you have only 4, it is still ok. Only two, change the motherboard for another used one. Check Ebay, etc.. It will be cheaper that getting a decent 4-port SATA controller card.
1.2 Check that you have at least a 1GB ethernet port. If it's only 10/100 and you can, upgrade it.
1.3 Ensure you have at least 2GB of RAM. It will be probably what it does carry, for an old Windows Vista/7 PC. Lucky if you have more.
2. If the mobo passes the test, remove all other extra cards. Enter BIOS and set all SATA ports as AHCI, and enable Wake-On-Lan if you can.
3. Crawl under that bed and other places, find, buy or steal old/cheap pendrives, of 16GB or more of storage. Ideally get at least 3 USB3 pendrives, even if you are going to use them as USB2 only. Mainly because sometimes those have faster write speed than older USB2 ones. Look, ma! A solid state disk just as slow as an HDD, instead of even slower. ^_^U
4 Get a Debian net installer in another pendrive. Plug monitor and keyboard to the NAS machine, and remove all other hardware/cards except network and video if it does not have an integrated video card (that means no HDDs at all).
4.1 Plug in both pendrives, the installation and the fast one, in USB2 ports. Boot up from the net install. pendrive, and use the installer assistant to make a minimal installation on the other pendrive.The Debian installer may fail and do the install but failing at setting up grub, so it won't boot. Update manually the grub of the installation pendrive instead of the target pendrive. You may have to go to console before rebooting, and force it to update the right grub manually.
4.2 During the installation, choose a single partition for the root volume, and no swap. It will end creating two partitions, a Minix one for the grub, of a few megabytes, and an ext3 for the rest of the disk. Like Windows, it will align partitions to the megabyte, and start it at 0 and 10MB.
I read about some of the cheaper memories had incredibly wide memory cells (8, 16 or 32mb), like SSDs, but without dedicated RAM for cache, it made them incredibly slow for writing, and the missalignmen made them even slower, the wider they were, the worse. Using 32MB ensures always alignment, even if you don't know or can't figure the width of your memory cells (it's next to impossible w/o opening the casing). In my case the improvement was small but existed.
Make the partition smaller that your smaller 16GB pendrive. They are not really 16GB, they are 15.78~15.82 GBs, some space reserved for management and spare blocks. Pick less than that. I choose 15.5GB. Those grant 32MB alignment both to beginning and end.
Using USB2 ports, it's because few BIOSes and kernels support fully the USB3 controllers when booting from them, even if they will happily chug data from them down once the system is up.
5 Ready your fists to punch yourself in the face repeatedly until you manage to install it without trouble, make it boot automatically (choose the OS pendrive as default / highest priority boot device in the BIOS), and keeps working once you remove the installation pendrive. If it fails, try again, choosing another model of pendrive for the OS, and GOTO 4. If it works, CONTINUE.
6 Log in at the system (it will have to be as the standard user you created on installation). Logging in as root will be disabled and "sudo" will not yet be installed. You will have to use "su -" to switch from user to root. That normal user will be the default user, like "pi" for raspbian, or whatever for ubuntu. So plug in the ethernet and set up the DCHP client on your ethernet device. Reboot and check (ping other IPS in your network, do "apt update" to update the system, etc.) If ethernet fails, try again (GOTO 6). If it works, CONTINUE.
7 Do "apt update" and "apt upgrade" to update the system. Check a tutorial and switch to Debian 11 (bullseye), because you want Linux kernels v5.0 and other modern goodies. (swap repositories, apt update or apt distro-update, or whatever it is required now). When the system it's updated, reboot again for testing. Then enable/install ssh and test it to connect from the network. If it works, ok.
7.1 You can now shut down the proto-NAS and make a backup of your OS. Simply clone the working pendrive onto another empty one. Clone only the partitioned space, so no matter if the pendrives have the same, nearly the same, or way more space that the original, only the first 15.5GB are used. BACKUP DONE! NEXT!
8 Shut down the system, install the HDDs you plan to use, and then turn it on. Fiddle with the cables and the BIOS until all HDDs are recognised by the BIOS, all are in AHCI mode, and it stills boot fast and cleanly from the OS pendrive. Check that all HDDs are detected by Linux. Use "lsblk" or "blkid". Check how each HDD can have an unique UUID. if they had partitions, these will be unique too.
9 Install mdadm and it's tools, and look for a tutorial about how to zero the beginning/end of the HDDs. This is a single command, and it erases standard/GPT partition tables, RAID configuration blocks on the disks, and their backup an the end of the disk. No need to zero it fully.
10 If you were semi lucky an have 4 sata ports, use the mdadm command to create a RAID5 with those 4 devices. Use if you can when you create them with the UUIDs instead of the device name, and give it a name. Let default values for the rest for now. Once done, if you are using "lsblk" will show that now the disks have kind of raid pseudopartitions, all of them with the same UUID. That UUID is the UUID of the RAID itself.
11 If you want to use other filesystem instead of ext3 or ext4 install them and related tools now, like zfs, or in my case XFS. Format the device with the proper mkfs.xfs
12 So create a mounting point "mkdir
Remember too to make sure the mount config int
12 Now install samba. Stop the samba service/daemon with "service smbd stop". Edit the config file and add
13 Google for a package and a repo for the wsdd daemon. It is the broadcast protocol. It can exist in python script or in c executable form. The repo does carry the c executable as a daemon, so install it with apt and enable it with "service wsdd start" From then on, it will show on Windows and Mac machines. Once it works, BACKUP! NEXT!
14 Shut it down. close the case, remove keyboard and monitor. Park it out of the way. Start it up again. Check if it still works. Connect to it with ssh to manage it.
The end, you are DONE.
Cost? An old core or core2 mobo/cpu with barely any useful amount of ram, and at least 3 HDDs of a decent size. 3 hdd SATA 3 or 4 hdd SATA 2+ disks, even if slow (5400 rpm) will hopefully fill your net output (1Gbit/s = 125GBytes/s).
Next step is upgrading your network to 1GB/s. If feasible, 2.5GB/s are becoming affordable and use standard ethernet cabling, but only NICs (network cards), for direct NAS to PC connection. Switches are still pretty expensive. If you are lucky and find one cheap secondhand, though? ^_-
Also, if you have at least 2 2TB hdds, don't fill the rest with 1TB. Try to fill at least 4 HDDs: buy secondhand or new cheap external 3 1/2 ones and chuck them out of their cases, or user bigger disks. 3+1 it's the best data / redundance ratio for 4 disks. If you were lucky and had 6 SATA with AHCI, you can opt for 5+1 in RAID 5 or 4+2 in RAID 6, depending if you want space or security, which it is important.
Taking into account that, for example my own system it is using:
* 1 WD disk (Caviar? before they started using the color brands, 10y/o?)
* 1 WD green disk (external, 7 y/o)
* 1 Seagate (old, external 5?y/o)
* 1 Toshiba (external, ?)
* 1 Seagate barracuda (internal, 6?y/o)
Some of the disk are nearly 10 years old, and the system it's moving them happily.
** INTERLUDE/RANT BEGINS **
If you buy new cheap, just don't buy Westend Digital, ever. Years ago they were awesome, but then they made so their external cases only worked with their disks. Disk were fine, though (WD green, WD red). Now they made so their disks (WD white) only work with their cases (they don't work inside computers without a simple, but bothersome hardware workaround). You don't want that failure point in your NAS.
** INTERLUDE/RANT ENDS **
Can't go cheaper than that.
If you don't want to mess up with Linux, use FreeNas, TrueNas or whatever. There are preconfigured distros that will run from a CD or a pendrive and will give you way more, but will require ant least 4GB of ram, ideally 8GB. That may mean replacing the ram for higher sticks, assuming you can install that many. It will give you a web interface and allow you to install applications and services, but those consume RAM and CPU. Some, like video transcoding for streaming, require a compatible videocard. They are easier and give you more, but they will start costing you money in new pieces instead of reusing old ones. Some can be installed in pendrives too, to use all the HDDS as storage space.
Prebuilt NAS are the same, except slimmer, and running in an underpowered ARM processor to barely consume any electricity, in exchange of switching to a locked system you can't mess with.
