Raspberry Pi Christmas Project

Like a lot of people, I am spending this Christmas/New Year holiday by myself because of COVID-19. It really hasn’t been a bad thing but I did want to plan something different for me to do on Christmas Day. I decided to try to build a network accessible storage (NAS) server to use primarily for my photos. I wanted to create storage that would be accessible from any of my devices while I’m at home. I have a Dropbox subscription that works really well for this for most of my files but I have chosen not to store my photos there, primarily because there are so many of them. I could have bought an NAS server like this one but I thought it would be fun to build one myself. And it was fun!

I decided that I would base my NAS on the Raspberry Pi, mainly because I thought it would be fun to try out a couple of different projects. For those who don’t know, a Raspberry Pi is a credit card sized computer that can be customized for a variety of tasks. It provides a way to learn about and practice a lot of computing skills.

For my playing, I chose a starter kit from Vilros. As you can see in the image above, the kit comes with a Raspberry Pi board which includes a number of ways to connect to other devices (USB, Ethernet, wireless LAN, HDMI), a case to enclose the board, a couple of heat sinks that need to be installed, a power supply, a slot for a microSD card, and the NOOBS (New Out of the Box Software) installer on the card to install the Raspberry Pi operating system which is a version of Unix designed specifically for the Pi. I followed the instructions in the Quick Start Guide to insert the heat sinks. I then followed the instructions to insert the 32GB microSD card.

In order to work on the Raspberry PI, you need to attach a monitor and keyboard. I used the HDMI cable to connect the Pi to my TV for use as a monitor. I already owned an Eagletec wireless keyboard and mouse so I inserted the wireless dongle into one of the Pi’s USB ports. I then plugged the Pi into the power supply and followed the instructions to install and configure the operating system which is called Raspbian.

Raspbian provides a graphical desktop environment that looks a lot like Windows or macOS. But most of the work that I did in this project is easier to do using text-based commands. Raspbian provides a tool called Terminal to allow you enter these commands. It looks like this:

In doing this project, it was definitely helpful that I already know how to use the command line. It was also helpful that I have used Unix systems a lot in the past and know about most of the commands I needed to use. This helped with debugging when things looked different than how the tutorials I was using suggested they would look. But I think with persistence anyone can do this project even without previous Unix experience. If nothing else, this project is a great way to learn about Unix.

One of the first things I did was to rename my Raspberry Pi to something other than the default name. I called it “cathienas”. To rename the device, I typed the following command (without the quotes) at the command prompt in Terminal: “hostname cathienas“.

So the basic idea for this project is that I wanted to have storage attached to the Pi that I can both read from and write to wirelessly from my other devices (laptops, phone, iPad). In doing research before starting the project, I decided that I wanted to set the NAS to be set up as a RAID (redundant array of inexpensive disks). This means that when I write a file to the NAS, it is copied to 2 storage devices automatically. This is helpful because if one of the devices fails for some reason, you still have a copy of all of your files. Plus, I just thought it would be fun and challenging to set up a RAID since I have never done it before.

The next decision that I had to make was what devices to use for the storage. My first thought was that I would use a couple of portable external hard drives (something like this). This kind of drive is powered through a USB connection but the Pi isn’t powerful enough to power 2 external hard drives. So to use portable hard drives, I would also need to use a powered USB hub (something like this). A second option is to use 2 desktop hard drives that each have their own independent power supply (something like this). Note that my links to these products does not represent a recommendation for those particular options. I didn’t choose either of these alternatives for storage so I can’t vouch for any of those products. Instead, I chose to use 2 thumb drives for my RAID because I can see myself reusing these in other projects should I decide to abandon my NAS. The thumb drives have no moving parts which means they are less likely to wear out and also that they don’t need a power supply. So I purchased 2 SanDisk 512GB thumb drives. Note that the Raspberry Pi that I purchased only supports USB 2.0 and these thumb drives are USB 3.0. Luckily, the 3.0 standard is backward compatible to 2.0 so I was confident these drives would work.

Once I plugged the thumb drives into the Pi, I was ready to set up the RAID. I relied on this tutorial although I did have to change a couple of things. All of the following commands (which are in blue) should be typed at the command prompt in Terminal (see above).

First, check to make sure the drives are available to the system. Type “lsblk” (without the quotes) at the command prompt. This command means list all block devices that are part of the system. There should be 3 (unless you have added something other than what I’ve talked about in this post). The first one is “mmcblk0” which is the microSD card that contains the Raspbian operating system. The other two should be “sda” and “sdb” (storage device a and storage device b).

Second, format sda and sdb so they can be used as part of the RAID. Type “sudo fdisk /dev/sda” at the command prompt. “sudo” means “superuser do” and will run the fdisk command with the privileges of the superuser. “fdisk” mean “format disk” and “/dev/sda” is the directory path for storage device a. Within the fdisk command, I first had to delete the current partition on the drive by typing “d” and then I could create a new partition by typing “n“. I answered all of the questions that came up by just pressing Enter (which means to use the default answer). Once all of the questions have been answered, type “w” to write these changes to the disk (that is, save the changes). This will cause fdisk to exit back to the command prompt.

Next, repeat the steps above for sdb. That is, type “sudo fdisk /dev/sdb“, delete its current partition, create a new partition with default settings, and write the changes to the disk.

At this point, I had to make sure my operating system was up-to-date. To do this, type “sudo apt update“. Recall that “sudo” means “superuser do”. This time, we are running the “apt” command which stands for “advanced package tool” and we’re telling it to “update”. Then type “sudo apt -dist upgrade” which asks the advanced package tool to upgrade the distribution of the operating system. Don’t forget the single minus sign in front of dist in the command.

Now we are ready to create the RAID. To do this, we need to make the Pi see sda and sdb as one storage location that it will write every file to twice (once on sda and once on sdb). We need to install the RAID manager which is called mdadm (which stands for Multiple Disk And Device Management). Type “sudo apt install mdadm” which I think is self-explanatory.

Next, we need to tell mdadm to create the RAID. Type “sudo mdadm –create –verbose /dev/md0 –level=mirror –raid-devices=2 /dev/sda1 /dev/sdb1” all in one long command. Note the double minus signs in front of create, verbose, level, and raid-devices. The command runs the RAID manager and asks it to create a new device with a root directory of /dev/md0. This device will be a mirrored device (–level=mirror) which means that it will make multiple copies of anything added to it and it will have two RAID devices. The first will be the partition we created on sda (which is called sda1 and don’t forget the 1 or you will get an error) and the second will be the partition we created on sdb (and again don’t forget the 1). –verbose asks the mdadm command to give long feedback about what it is encountering. Now Pi will see the 2 thumb drives as 1 device.

In Unix-based systems, we need to “mount” storage devices in order for the system to see them. So in order for our Raspbian operating system to see our new RAID, we need to mount /dev/md0. This will take several commands. First, type “sudo mkdir -p /mnt/raid1“. “mkdir” means “make directory”. A directory in Unix is the same as a folder in Windows. So this command says make a directory called “raid1” inside the directory called “mnt”. The -p (that’s with a single minus sign) says to make the parent directories if they don’t already exist. In other words, if the directory called “mnt” doesn’t already exist, create it. Next, type “sudo mkfs.ext4 /dev/md0” which means make a new file system (of type extension 4 which is too complicated to explain here) for our RAID (which is called /dev/md0). Next, we will actually mount the RAID by typing “sudo mount /dev/md0 /mnt/raid1/“. (Note the / after raid1.) So our mounted RAID will be located in the directory /mnt/raid1/. To list the contents of our mounted RAID, type “ls -l /mnt/raid1/“. “ls” means “list” and “-l” (with a single minus sign) means in long format. You see only the item “lost+found” in the list and that’s how you know the RAID is operational.

We don’t want to have to manually mount the RAID every time we reboot the Raspberry Pi so we will need to add a command in the start up instructions for the file system. These start up instructions are located in a file called fstab which stands for file system table. To do this, we will use a text editor called nano. Type “sudo nano /etc/fstab” which will open the fstab file in nano. Type the following line somewhere within the file: “/dev/md0 /mnt/raid1/ ext4 defaults,noatime 0 1“. This adds the RAID to the file system table. To exit nano, type CTRL-x and answer the question about overwriting the file with “y“.

Now to make sure the RAID mounts correctly on start up, type “sudo mdadm –detail –scan | sudo tee -a /etc/mdadm/mdadm.conf“. This command uses the pipe operator | to feed the results of “sudo mdadm –detail –scan” (with double minus signs in front of detail and scan) into “sudo tee -a /etc/mdadm/mdadm.config” (with single minus sign in front of a). So this command puts the details of the current mdadm set up into the configuration file for mdadm. When the Raspberry Pi reboots, that configuration file will set up mdadm as it is set up now.

Reboot the Raspberry Pi and the RAID is ready to use.

But how do we access files on the RAID from other devices on the network? We will use the SMB protocol but since that protocol is not installed by default on the Pi, we will need to install it. The protocol is known as Samba on the Pi. So to install the protocol, type “sudo install samba samba-common-bin“. Answer any questions with the default answer.

We have to make sure that we have a shared space on the RAID that is accessible to people on our network. We will make a directory to be shared by typing “sudo mkdir /mnt/raid1/shared“. Note that the directory name “shared” can be any name that you choose. There is nothing magic about the word “shared”. Now we need to make that directory available for everyone to read, write, and execute (or run programs). Type “sudo chmod 777 /mnt/raid1/shared“. “chmod” is the command to change file permissions in Unix. One of my favorite parts of the Unix OS is how it handles file permissions. There are three groups of people that you can give permissions to for each file: the person who owns the file, people in the file’s group, and everyone else. For each of these 3 groups, you can give read (can open and view the file), write (can edit, modify, and delete the file), and execute (can run the file if it is a script or program of some sort) access. Typically, the permissions for the owner of the file are set to rwe which means that person can read, write, and execute the file. This would be written as 111. If for some reason, the owner did not have access to execute the file, their permission would be rw- or 110. In binary, 111 is 7 and 110 is 6. So a 7 in the first position of the number after “chmod” means that the owner has read, write, and execute access to this directory and the files within the directory. The second 7 means anyone in the group also has read, write, and execute access. The third 7 means everyone else has read, write, and execute access as well. Knowing binary helps here: 7=111=rwe, 6=110=read and write, 5=101=read and execute, 4=100=read only, 3=011=write and execute, 2=010=write only, and 1=001=execute only. Isn’t that cool?

Next, we need to tell Samba to make the RAID shared directory available on the network. To do that, we will change the Samba configuration file using nano. Type “sudo nano /etc/samba/smb.conf“. Scroll to the bottom of the file using the arrow keys and type each of the following on a separate line: “[shared]“, “path = /mnt/raid1/shared“, “writeable=Yes“, “mask=0777“, “directory mask=0777“, “public=no“. Exit the file by typing CTRL-x and answer “y” to the question about overwriting the file. Now restart Samba by typing “sudo systemctl restart smbd“. This command uses the systemctl tool to restart the SMB daemon which is the software that runs the SMB protocol.

The default user on the Raspberry Pi is called pi. We have to give that user a password to access the Samba share from other devices. Type “sudo smbpassword -a pi“. “-a” (with a single minus sign) means add the user “pi” to the list of smbpasswords. You will be prompted for a password which you will need to enter twice.

How do we now access this shared space on the RAID from other devices? If you’re using a Windows computer, you will map a network drive to the RAID. In Windows 10, you do this by opening File Explorer and navigating to “This PC”. At the top of the window, you will see “Computer” as shown below:

 

Click on “Computer” and then click on “Map Network Drive”. You will see the following dialog box:

Change the drive to whichever letter you want and then type “\\cathienas\shared” in the Folder box but replace “cathienas” with whatever you named your Pi and “shared” with whatever you called your shared folder. Be sure to check the box that says “Connect using different credentials”. You will then be prompted to enter the username (probably “pi”) and password that you created for Samba. A new window will open that contains the contents of your shared folder on your Raspberry Pi RAID and you can treat it like any other folder on your Windows 10 computer.

To access the shared folder on my Android phone, I downloaded a new File Manager app that has the SMB protocol built into it. I chose File Manager Plus from the Google Play store. I installed it and navigated to the Home page which looks like:

Click on the Remote icon at the bottom of the screen. Choose Add a remote location and you will be prompted for the type of remote location you want to add. Choose SMB and enter “//cathienas/shared” again replacing “cathienas” and “shared” as needed. Enter your username and password using the ones you set up on your Raspberry Pi. You will now have access to the shared folder on the Pi RAID on your phone.

As I said, this was a really fun project. And I have a networked location to store the photos that I take. I’m still figuring out the work flow but it’s certain to be more convenient than the non-networked work flow I have been using up until now.