Turn an old Computer into a Local Fileserver (Mostly Graphical Setup)

We’re running out of space. With photos, astrophotography files, and music recordings and work and just life stuff; we are running low on hard drive space. Over the years we’ve bought a few external USB hard drives but even those aren’t manages well and being portable get moved around a lot. That’s not the safest way to handle hard drives. Unless they are SSDs, they can be damaged by moving them around and such. We needed a better solution. I prefer using GUIs to set things up but will for through the terminal if I have to… I just haven’t used my unix fileserver skills since I worked in IT and my linux is a bit rusty as well.

Googling around you’ll find a ton about setting up a NAS or RAID5 or somesuch, but that’s overkill. I’m not looking to spend any money with this, otherwise I’d just have bought yet another portable drive. I already have a bunch of hard drives full of stuff (and honestly, there are a LOT of duplicates of each file taking up tons of space from the crappy way I store my backups). I want to use those drives and be able to clean out the duplicates to gain space. The first step is to build a basic file server.

I had a linux machine (my workhorse during grad school) that wasn’t being used, and  bunch of old hard drives that I wanted to use/access  It was a variety of drives from old PCs (back to win XP actually, and even older). I had a few external drives that broke so I just plopped them into this machine as well. I didn’t want to format them for a RAID or anything as they were all different sizes and I had data like pics and projects and work stuff from ages ago on them I wanted to keep. The best solution for me was the JBOD (just a bunch of drives) style server. All the NAS examples looked annoying to setup and I’d easily forget what I had done to set them up so JBOD it was.  I couldn’t find any single document explaining how to do all the steps I wanted so I spent a couple of weeks failing in my spare time… breaking and unbreaking my linux fileserver and documenting the working parts here for when I need to work on it again. I prefer doing things with GUI if possible, but I’ll do some things using the terminal because you just have to but I tried to keep it to a minimum.

Installing Linux:

If you don’t already have linux running on this computer, I suggest you install it. It sounds hard, but it is super easy to do actually. Just download any flavor of ubuntu you like (lubuntu, kubuntu, linux mint, cinnamon, etc.) I recommend the versions that say “LTS” as those are supported for 2 years or so. There’s a lot of options here.

Install linux without formatting hard drive or losing data:

  1. Download Wubi. This will let you install linux without killing your windows files or messing anything up. you can always uninstall linux later if you want using this method.

Start clean on either blank hard drive or format an old drive:

  1. Download ubuntu flavor you prefer
  2. Format a USB flash drive that’s at least 2GB (lubutnu and xubtunu) or 4GB (all other flavors).
  3. Use Balena Etcher to burn this ISO file to the USB drive.
  4. Once finished, you can reboot your computer.
  5. Enter the boot menu when you see your computer’s startup logo (before windows logo) by pressing either F10, F11, F12, or the delete key. Different manufacturers have different keys they choose for this.
  6. Select to boot off the USB drive.
  7. Doubleclick the icon on the desktop to install *buntu. Follow the menus or youtube videos on how to install it.
  8. Remove the USB drive and reboot when done.

I was already running ubuntu so I just needed to put all the drives in the computer case. I could have 2 drives that were IDE on the same ribbon cable, and 4 SATA drives on the motherboard of my old linux box. Sadly, that’s all this mobo could accommodate. I see spots for 4 more SATA drive on the PCB but they aren’t populated. I filled all 6 spots with drives, making sure that all the drives were set up as slaves except the bootable one (see another post about how I set that up).  I also had a TON of USB ports on this motherboard as well (literally 6 on the back, 2 on the front and I added 2 more to a breakout cable inside for a total of 10 USB ports). These can add more portable drives and other drives with some of  these awesome USB to IDE/SATA drive cables I have to the server. As I loaded these drives in the bays I made a note as to their serial numbers and capacities. I wrote it on paper, but I also used a sharpie and wrote it on the chassis so I could keep track of what is what and where.

When I turned the machine on, I went into the BIOS and set up the boot order for the hard drives so that my /filesystem always boots first in order of hard drives.  When booting the linux machine, hold the Delete key to enter BIOS. Then go to Advanced BIOS Features.

bios1

Next select “Hard Disk Priority”. Here’s what that looks like:

bios2

Finally, use the arrow keys to select a particular hard drive (the one you want to boot from) and then use the + key to increase its priority until it is number 1:

bios3

Once this is done, Hit F10 to save and exit BIOS. It will reboot the machine into linux as it usually did.

Accessing all the Hard Drives:
Open your ubuntu menu and search for the “disks”  program. This should be part of gnome-system-tools, which is a standard program in ubuntu. If you don’t have it then open your package manager and search for it. Mark it for installation and click the “Apply” button. or open a terminal and enter the following:

sudo apt-get install gnome-system-tools

You can copy that line of text paste it into the terminal using ctrl+shift+v to paste in a terminal.

All your hard drives should be visible in the Disks app. It lays out all the gory details. Serial numbers, model numbers, etc. Click each drive in the left and note what the “mount point” is for them. I have my Filesystem (this is where I installed the linux operating system)  and /home mount points on separate drives. /home is like the user folder in Windows with a folders for Desktop folder, Documents folder, etc. Separating this from the operating system installation file tree allows me to reinstall linux without breaking my user’s documents and settings and Desktop items, etc. I have another post on that and it has served me very well over the years.

For my other drives, they have mount points that start with /mnt  and end in a weird code.  That code is the UUID (hardware ID) for that hard drive and it isn’t very descriptive for humans.  Let’s change that to something helpful. For instance I changed my 1TB drive’s mount point to /mnt/_old_1TB_MyBook

image

Once all the drives have normal names click the link to one of them and it’ll take you to the folder in your file browser. Go up one directory to see all the hard drives. You’ll note the old folders with the UUIDs in there as well as any old names you had mounted in the past, but they are empty. Only the mount points you named in Disks are connected to your hard drives. You can just delete the other empty old mount points. You have to be root to do so. I used the terminal and the command sudo rmdir <folder name> to delete the ones I was CERTAIN were empty old mount points.

All we’ve done here is essentially modify the /etc/fstab file which contains drive mount info. Lots of other tutorials out there can show you how to do this in the terminal if you want to know how.

Now we need to install all the other software we’ll need for the other steps of setting up the server. in your start menu, search for your package manager software. Mine is “Synaptic package manager” and install the following (make sure you check the box next to these entries, select “Mark for Installation” and when all are selected, click the “Apply” button.) Don’t be surprised if some of these are already selected, just ignore them then:

gnome-system-tools

samba

samba-common

python-glade2

system-config-samba

Or you could open a terminal and paste the following which will install them all for you automatically

sudo apt-get install gnome-system-tools samba samba-common python-glade2  system-config-samba

Once these all install you will create your linux users who can access this machine. You need linux users to be able to create Samba users. Samba is the file server software. From your start menu button, search for the app “Users and Groups.” Add whoever needs users for the fileserver.

linuxUsers

Now you can start fiddling with Samba, which is the actual fileserver software. Since there’s no icon to select to load this app, you need to open a terminal (hit ctrl+alt+t) type or paste:

sudo -H system-config-samba

You can copy that line of text paste it into the terminal using ctrl+shift+v to paste in a terminal.

You have to enter your password, but then a window should pop up. The very first time I did this I got the following error:

SystemError: could not open configuration file `/etc/libuser.conf': No such file or directory

ok, so the file doesn’t exist… I’ll simply make a blank file there and try again.

sudo touch /etc/libuser.conf #creates a blank file named libuser.conf in /etc folder
sudo -H system-config-samba

Success! It opened!

samba server config

Now we need to figure out which folders we want to share and how they are set up in linux.

As we saw before, the internal hard drives all have mount points in the /mnt folder. So let’s create a share for that. Click the big plus sign button and make yours match mine below. Be sure to click the “Access” tab and make it open for everyone.

image

Do the same kind of thing for the /media folder. This is where any USB storage devices will mount to by default.

And finally, I have a TON of files on my old desktop and user folder, so I want to make the entire /home folder accessible as well. You have to be careful with this though because if you allow all users (and even guests) edit access to your home folder, they can delete stuff, and snoop to their hearts’ content. Any sensitive data should be locked so that users are required to log in and can only access their files. If for example you wanted each linux user we created earlier to access only their home folders, you’d set that up as /home/adam then you could set the access rights to only allow user “adam” to view it.

Once you’ve set this up, now you must create the Samba users. While in the same application, select Preferences –> Samba users. Here you will add a new user with the same name as each linux user. You will select each user from the “Unix user” dropdown box (this is why we had to create all the users before). You can have the same or different passwords than your linux login. Once you added everyone, close this app. The steps we just took replace the manual process of editing a configuration file for Samba. This is a plain text file stores in /etc/samba called smb.conf and this is usually what most tutorials will do, show you how to edit this file manually. I didn’t have luck with that. Also, you might see tutorials using a tool called gadmin-samba however when I used that it completely hosed up my system hardcore. I had to purge all samba stuff, delete all the files and start from scratch several times to fix the issues.

File/Folder Access Rights:

The final step here that is required is to set the access rights of each folder you want to share. This is important because unless we change these permissions, we will end up only being able to VIEW files and folders on the server but we can’t create new files, paste, move, or delete anything yet. To fix this we need to change the permissions on the folders we set up to share.

We need to open the folders as root or administrator. Depending on the file manager software you might have different ways to do this, but in mine you simple right click ANY folder and choose “open as Administrator” or “Open as root” then I can simply navigate to the filepath “/” If yours doesn’t offer this, then open a terminal and type in the name of your file manager but call it from sudo like one of the following lines:

sudo nautilus /
#or
sudo nemo /
#or
sudo caja /

Right click on the folder of interest, for example “/mnt” and view the Properties. Click the Permissions” tab and make sure that they all show the ability to “Create and delete files.”

But if you’re using the terminal anyway, you can simply use the chmod commands. Here’s the best explanation of how to use it I’ve ever seen. You can make /mnt and /media fully writable like this:

sudo chmod 777 –R /mnt
sudo chmod 777 –R /media

Or you can give others the ability to look at files in your home user folder, but not create or delete like this:

sudo chmod 755 –R /home/adam

And you can block others from your folders and have them only viewable and editable by you like so:

sudo chmod 700 –R /home/adam

On the Windows machine:

Now your fileserver is set up you need to access it. Go on your windows machine and open a file browser window. Right-click on “This PC” and select “Add a Network Location”  Click Next until it asks for the location. Click “Browse” and wait a minute for it to search your network. You should see your fileServer’s name pop up in the list. Click literally any folder under the fileServer and select Next.

fileserver List

Shorten the terribly long name it gives you (I just delete the highlighted part) and you’re done.

name1

It’ll pull in the other accessible folders on that drive as well. Now you should be able to access and edit anything on those folders on the fileserver. To get to this again simply choose the fileserver option in the left-pane of the windows file explorer. If this doesn’t appear, then make sure your server is turned on and is on the same network as your windows machine and repeat the instruction in this section of the post.

remote

The first time you visit something in here it might ask you for a username and password. Just enter one of the Samba username and password combos we created earlier. You can make it save these credentials and never ask you again if you’d like.

Cherry on top – Remote Shutdown of server:

If you don’t want to connect a keyboard, mouse and monitor to the screen just to shutdown the fileserver, you can do so remotely by installing and using SSH. SSH is a way to securely tunnel into the fileserver’s terminal remotely. That way you can send it the standard linux shutdown message from your windows PC when you are done with it for the day.

Open synaptic and you should select the following:

or run the following command:

sudo systemctl status ssh

This will install the needed files for SSH. Next, we need to let the firewall built into linux know you want to allow ssh. In a terminal window run the following:

sudo ufw allow ssh

Now you can go to the windows computer and use putty.

Putty1

Enter your data here (keep the port as 22) and when you click “open” a black command window will pop up asking you who you want to login as. Type your linux username and hit enter and it’ll ask for a password. Now you’re commanding the terminal of the linux machine from windows. You can do literally anything the terminal can handle. You can’t do anything that requires GUIs though this way. That’s fine though because this is enough to send the reboot or shutdown signal. The shutdown takes about a minute before it actually shuts down, so don’ t think it isn’t doing anything, just give it some time.

sudo shutdown
#or
sudo reboot

Remote Desktop:

If you want a graphical environment for controlling your server, there’s 2 options. One is to simply remote Desktop into it.  There are many ways to do this, but the easiest is to simply install and enable the xrdp app on the linux machine and use the built-in RDP app in windows to connect.

Open synaptic package manager and search for xrdp and xorgxrdp.  Mark it for installation and click Apply.

Once this is done, go to the in the App search, look for “System Settings

Go to Sharing and make sure Remote Login is ON.

image

Next, in system settings, click “Network” in the left column and then click the gear button next to your wired (or wireless) connection. This will give you your IP address. It should look similar to your router’s address. Mine’s 192.168.x.x. You’ll need this to connect in windows.

To do all this in the terminal on the linux machine (or through SSH) run the following:

sudo apt install xrdp
sudo systemctl enable xrdp
ifconfig

This last command, ifconfig, will give you a screen showing you the local IP address of the machine. It should match closely to your router’s address. Mine is 192.168.x.x so I look for something that matches that. You’ll use this on the windows machine.

Now on the windows machine, click the start button and type “RDP” and hit enter. Type in the IP address of the linux machine.

I found an easier way. Simply right click on the fileserver in your explorer folder and select “Remote into this machine”

image

When you click connect, you’ll be able to login. I had a little trouble with this connecting. I tried all the selections in the dropdown menu a couple of times and on the 2nd or 3rd try it worked. Once I was logged in I got an error asking me to login to create a color device over and over. After entering my password 3 times I just canceled it and the window stopped popping up. I haven’t done this in like 15 years, but I don’t remember the experience being so crappy. I can access the desktop, but it is displaying a GNOME environment instead of the Mate environment I have installed on the linux machine. Nonetheless I can get do things graphically without issue.

Other Thoughts:

The way I’ve done it is quite insecure. I’m allowing Guests on the network access to create and destroy all the shared files which is kind of dumb… I’ll live with it though because I will only turn this fileserver on if I am immediately accessing the files on it, then immediately turning it off again.  In that short time I hope I don’t get hacked…

Storing all your files in one location is not the best for preservation. You can make a personal cloud like Google Drive, Dropbox, Box, OneDrive, Apple Cloud, etc. by using the Unison program.  It essentially synchronized two folders. These can be on the same computer, one on a portable USB drive, or even across the network. Anything added, modified, or deleted in one folder is automatically synched to the other in both directions.

Windows Networks Issues:

If the network disappeared from the “Networks” thing in Windows 10 but still somehow appears in the sidebar and is still accessible, no prob. Follow these instructions: https://www.wintips.org/fix-windows-10-network-computers-not-showing/#method-5

File Manager Share plugins:

Previously, I had some luck with using my file-brower’s “share” plugin. Nautilus, Nemo, and Caja all have their respective “share” app you can install in apt-get or synaptic eg “caja-share“.  This will allow you to simply right-click on a folder and select “share settings” where you can set this up yourself. The problem with this is that this info is NOT stored in the basic /etc/samba/smb.conf configuration file. In fact, it is nowhere to be found anywhere in that folder.  This is because this plugin stores this info in the following path:

  /var/lib/samba/usershares

To create the Samba users and set up their SAMBA passwords (these can and maybe *should* be different than their linux user passwords)

sudo smbpasswd –a user1
sudo smbpasswd –a user2

Now enable the users

sudo smbpasswd –e user1
sudo smbpasswd –e user2

Check the status of the samba server. You should see “active” somewhere. If not, you hosed it somehow. Tips on how to fix this at the end…

sudo systemctl status smbd

Restarting samba (after every big change to users, folders, etc you should do this. Or you can reboot the machine, which is foolproof method of restarting the server the right way)

sudo systemctl restart smbd
sudo systemctl restart nmbd

or do it directly for smbd and nbmd

sudo smbd stop
sudo nmbd stop
sudo smbd start
sudo nmbd start

My Introduction to Astrophotography

I’ve been interested in astonomy my whole life, and a few years back, I got a second-hand Meade LX-10 telescope. This is a 8″ diameter scope… definitely not a toy. It is great for planetary viewing and can even track the planets as the Earth turns. I have used it from time to time to try to take some pictures of celestial objects, but not very good ones. Astrophotography is a growing interest of mine now and I found the learning curve quite steep. I’m throwing together everything I’ve learned over the past couple of years including links to software and such into a few posts. There are multiple steps involved in this project and it took me a long time, working here and there and amongst about 1000 other projects, to finally get everything together for this.

To take amazing amateur astrophotography images you need to decide whether you want to look at deep space objects (DSO) like galaxies or nebulae, or if you want to focus on planetary viewing (within our own solar system). You don’t really want to zoom very much if you want DSO images because zooming narrows your field of view and most DSOs you want to get are very dim so you don’t want to zoom in to make their light spread out across your eye or camera sensor too much, you want bright pictures. You can take some amazingly cool pictures with just a DSLR and a “barn door” tracker. (This is just a device that rotates counter to the Earth’s rotation so your camera will continue pointing at the same object for minutes or hours at a time, meaning it moves a full 360 degrees in 24 hours).

In my case, I want to do a little of both planetary and DSO. So I have my LX-10 telescope, which has an 8″ diameter mirror, meaning it can collect a lot of light, and it has a tracker so I can point a camera at one point in the sky as long as I want. However, unless you perfectly align the scope with true celestial north, the scope will still drift a bit over the night. This is because it is an open control system. You just point it at a star, turn it on and hope you aligned it well when you started. The best images are taken with closed-loop control systems. That is, they continuously look to see if it is drifting off target, then takes actions to move back on target when needed. This guy has my telescope and has taken some amazing pictures of planets, sunspots, and DSOs. Some are mindlbowingly good!  To be fair, he’s using some reducer lenses like this one to change his F-stop to make it much quicker (for example F/3.75 and F/4.6 in some images), oh and he’s using a 4x Powermate lens  a lot which costs as much as I spent on my entire telescope… But what is great about his site is that he tells all the settings he used for the images. This is similar to Reddit’s astrophotography subreddit.

My scope is what they call “slow,” “long,” or “dark.” This refers to the F-stop number. This is also called the focal ratio or relative aperture. It is a ratio of the len’s focal length and the diameter of the scope. THe focal length of my telescope is 2000mm. And the aperture is 8″ or 203mm. The F-stop of this would be about 2000mm / 200mm = 10. So this means my scope is a F/10 lens. The higher the f-stop number is, the darker the image will be (all other things being equal) when compared to a lens with a smaller f-stop number. For example an F/4 is considered a pretty “fast” telescope. My F/10 number basically means that to take a nice bright picture of Jupiter for instance, I need to play with the “sensitivity” or ISO number of the camera sensor (used to be film) as well as the duration I leave the shutter open on the camera.

You might say, “Well I can just crank up the sensitivity then. Higher sensitivity will make the image come out brighter, right?” The answer is yes and no. You can raise the sensitivity, but then a lot of other things (such as heat) can trigger a pixel to register a value in a digital camera. This will increase the noise in the image, making it staticy.

You might then think “Ok, the other option is to just open the shutter for a long time, exposing the camera sensor to more light over a longer time period.” yeah… not really. This can work as well, however you will increase the overall light pollution in your images and worse, you risk blurring the image. Since the Earth keeps on spinning, objects in a fixed telescope’s field of view move. Without tracking perfectly, long exposures will be blurred.

Even if you can fix all that, there’s blurring you simply cannot fix. This is due to heat inversion in the air column between you and the object you are viewing. The Hubble space telescope was designed to fix this problem… by simply being above all the air on earth orbiting in a pretty high orbital plane (at the farthest reach that the space shuttles could fly). Air at different temperatures has different densities. This acts like a prism to bend the shape of a beam of light. Look at a straw or pencil in a glass of water. Notice how it looks like it’s broken in half at the interface of the water and the air? That’s an example of the different densities acting like a prism. Since air is, well, air… it is gaseous and mixes and moves around a lot. It is in constant motion (unlike the water in your glass compared to the air sitting on top of it.) Hot air rises and cool air falls, making all sorts of weird prism effects in our viewfinder. In the column of air between you and the top of the atmosphere where Hubble is, there’s also a lot of dust. The dust as well as the temperature inversions is what makes starts look like they are twinkling.

Side note: Planets don’t twinkle in the sky when viewed by eye. This is because a star is so far away, you are only seeing it as a point light. This makes it easy for dust or temperature inversion to affect your view of it. But planets are much closer, and their light is spread slightly wider across the retina in your eye. This means there much less of a chance that a mote of dust will block it, or temperature inversion will guide the light too far from your retina.

So how do you fix the problem of having a dark image from the telescope without getting errors from too high of a sensitivity setting or blurring from too long an exposure and temperature inversions? You can thank Woz for the home computer! Using a laptop, I’m going to connect a closed-loop control system to my computer to track the objects I want to image very closely, reducing blur of long exposure images. I’m also going to take multiple images of the same object, then do a process called “stacking” where a computer algorithm will take the sharpest views of different parts of the planet, for instance, and stitch them together into a composite image that is overall much sharper. Then I’ll be able to do some image processing on the composite image to get some great results. Stay tuned for more posts on this theme!

Guiding Telescope with a Webcam Setup

I’ve finally gotten jealous enough for the astrophotography subreddit to get back to work on this project. Jess bought me a Meade LX10 8″ diameter telescope several years ago for my birthday. I’ve used it quite a bit to view planets and try to take deep sky astrophotography pictures. This telescope isn’t one of those fancy ones you can type in whatever cool thing you want to see and it’ll drive itself to point right to it, that’s called a “GO TO”. Rather it has a simple “barn door” tracker motor. Basically, if you align to perfect true north, and set the wedge (the thing that mounts the telescope to the tripod) to your latitude, whatever you point the scope it will stay in view for hours in the eyepiece. If I know where to look, I can attach a camera to the scope and leave the shutter open and get some amazing pictures of nebulae and galaxies.

Being that I’m no good at polar alignment, I decided a few years ago to build an arduino interface that will connect my scope to my computer.  The way this works is that I attach a webcam to the spotter scope (the small telescope that helps you find stuff) which looks at a particular star.  The webcam pipe data into a program that sends signals out to the arduino to move the scope to keep the star in the same part of the webcam’s view.  This way, I don’t have to be perfectly polar aligned, the software will help adjust the position of the scope for me.

I went on the hunt for a webcam that would work well with Windows and linux.  This is because a lot of people are buying Raspberry Pi boards,connecting a webcam to them and attaching the whole setup to the telescope. Right now I’m testing on a windows machine so I need a webcam that’ll play well with both.  I looked up the Linux Universal video Class (UVC) drive list to find a good modern camera. This list shows a good number of webcam models and brands that are known to work natively in recent linux distros.

The camera I landed on is the Logitec HD Webcam C270.  It is a very cheap 720p 3 megapixel webcam. That’s overkill for the telescope, but it’s a good general use webcam and we can use it for video chats and such as well.  This means my solution to attaching the camera to the scope can’t be permanent.

I keep a bunch of 3/4″ PVC pipes and connectors in the garage for prototyping, so I grabbed a 3/4-inch T connector.  This connector can easily accommodate my 1″ outer diameter sighting scope.

webcamTele

The scope doesn’t fit perfectly, so I added some 2mm sticky-backed craft foam for a snug pressfit. (On a side note, I can’t tell you how useful it is having this kind of foam in the toolbox for all sorts of random purposes. I use it all the time)  To accommodate the webcam, I used a hacksaw to cut a portion of the PCV connector off as shown.  Then I wrapped a 3/8″ piece of foam on each of the cut edges of the PVC where it will touch the camera. This will help the camera seat well and stay in place when I attach it to the scope.

3webcamTele  4webcamTele

Finally, I used a smooth “ouchless” hair tie to hold the camera to the PVC tightly and aligned the camera with the hole in the PVC T-joint. Again, believe it or not, these hair ties are pretty useful for random jobs.  In fact, I use a 8-inch smooth headband made of the same material to hold on my cheapo dew shield (more on this in another post.)

5webcamTele  6webcamTele

7webcamTele

 

The final product is easy to use and quite robust. I think it’ll work quite well with my the rest of my setup.  Since I’m still working that all out, I’ll post more as I learn more.8webcamTele