Introduction

In Part 1, we covered home server concepts and hardware. Now in Part 2, we will get hands-on with installing the operating system (OS). If hardware is the body, the OS is the soul. What OS you choose greatly affects what you can do and how you manage it.

In this article, we will compare the most popular operating systems for home servers and provide a detailed walkthrough of installing Ubuntu Server, the most versatile option.

1. Types of Home Server Operating Systems

Home server operating systems can be broadly divided into two categories: general-purpose Linux distributions and special-purpose operating systems.

1.1 Ubuntu Server

This is probably the most commonly recommended OS. I also started with Ubuntu Server when I first began.

Pros:

  • Extensive documentation - most problems have solutions online
  • LTS (Long Term Support) versions provide 5 years of security updates
  • Best compatibility with Docker and various server software
  • Relatively accessible for beginners

Cons:

  • Snap package system is controversial
  • May be affected by Canonical company policy changes

Recommended for: Home server beginners, Docker-based service operators

1.2 Debian

This is the distribution that Ubuntu is based on. It is more stable and conservative than Ubuntu.

Pros:

  • Extremely stable - only verified packages included
  • Lightweight with no bloat
  • Completely open source, no corporate influence
  • Most Ubuntu documentation is compatible

Cons:

  • Package versions are somewhat outdated
  • Installation process slightly more complex than Ubuntu

Recommended for: Users who prioritize stability, Linux-experienced users

1.3 Proxmox VE

This is a virtualization-specialized OS. Choose this when you want to run multiple virtual machines (VMs) or containers (LXC) on a single server.

Pros:

  • Web UI for managing virtual machines
  • Supports both KVM virtualization and LXC containers
  • Free to use (subscription is optional)
  • Built-in snapshot, backup, and clustering features

Cons:

  • Performance loss due to virtualization overhead
  • Has a learning curve
  • Inefficient on low-spec systems

Recommended for: Users who want to test multiple OSes simultaneously, homelab builders

1.4 TrueNAS (Core/Scale)

This is a storage-specialized OS. It is optimized for file server and NAS use cases.

TrueNAS Core (FreeBSD-based):

  • Data integrity guaranteed with ZFS filesystem
  • Mature and stable
  • Service expansion through plugins (Jails)

TrueNAS Scale (Linux-based):

  • Native Docker/Kubernetes support
  • More apps available due to Linux compatibility
  • Relatively new and still maturing

Recommended for: Users whose primary focus is large-capacity storage management

1.5 Unraid

A paid OS, but extremely popular in the home server community.

Pros:

  • Can pool HDDs of different sizes into one
  • Intuitive and beautiful web UI
  • Easy Docker and VM management
  • Active community app store

Cons:

  • Paid (Basic $59, Plus $89, Pro $129)
  • Parity-based so performance is somewhat lower

Recommended for: Users who want to utilize HDDs of various sizes, GUI preference users

1.6 OS Comparison Summary Table

OS Difficulty Primary Use Cost Recommended RAM
Ubuntu Server Easy General Purpose Free 2GB+
Debian Medium General Purpose Free 1GB+
Proxmox VE Medium Virtualization Free 8GB+
TrueNAS Medium Storage Free 8GB+ (ECC recommended)
Unraid Easy General/Storage Paid 4GB+

2. Preparing for Ubuntu Server Installation

This guide is based on Ubuntu Server 24.04 LTS. It is the most versatile option with abundant documentation, making it recommended for beginners.

2.1 What You Need

  • USB drive (8GB or more)
  • Ubuntu Server ISO file
  • Bootable USB creation program (Rufus or balenaEtcher)
  • Keyboard and monitor (only needed during installation)

2.2 Downloading the ISO

Download the ISO from the official Ubuntu website:

  1. Go to ubuntu.com/download/server
  2. Select "Ubuntu Server 24.04 LTS"
  3. Download (approximately 2GB)

2.3 Creating a Bootable USB

Using Rufus on Windows:

  1. Download and run Rufus (rufus.ie)
  2. Device: Select your USB drive
  3. Boot selection: Select the downloaded ISO file
  4. Partition scheme: Select GPT (for UEFI boot)
  5. Click 'Start'

Using balenaEtcher on Mac/Linux:

  1. Download and run balenaEtcher (etcher.balena.io)
  2. 'Flash from file' - Select ISO
  3. 'Select target' - Select USB
  4. Click 'Flash!'

3. Ubuntu Server Installation Process

3.1 Changing Boot Order

  1. Insert USB into server and power on
  2. Enter BIOS/UEFI (usually F2, F12, or Del key)
  3. Set USB as first in boot order
  4. Save and restart

3.2 Running Through the Installation Wizard

1. Language Selection

English is recommended. Selecting other languages may cause display issues in the terminal.

2. Keyboard Layout

Select your appropriate keyboard layout (e.g., English (US))

3. Installation Type

Select "Ubuntu Server" (not the minimized version)

4. Network Configuration

If connected via ethernet, it will automatically receive a DHCP IP. Proceed with defaults for now; we will change to static IP later.

5. Proxy Configuration

Leave blank unless you have a specific need

6. Mirror Server

Proceed with the default. It will select a mirror appropriate for your region.

7. Disk Configuration

This is the most important step:

  • Use an entire disk: Uses the entire disk for Ubuntu (recommended)
  • Keep LVM settings as default
  • Check "Encrypt the LVM group" if encryption is needed

8. User Configuration

  • Your name: Display name
  • Your server's name: Hostname (e.g., homeserver)
  • Username: Login ID (lowercase, letters only)
  • Password: Set a strong password

9. Ubuntu Pro

Select "Skip for now" (free registration is available for personal use, but can be done later)

10. SSH Setup

Make sure to check "Install OpenSSH server"! This is required for remote access.

11. Featured Server Snaps

Do not select anything now. It is better to install Docker etc. manually later.

12. Installation Complete

When installation completes, select "Reboot Now". Remove the USB when prompted, then press Enter.

4. Initial Configuration

4.1 First Login

After reboot, log in with the account created during installation.

homeserver login: username
Password: (enter password, nothing will display)

4.2 System Update

First, update the system to the latest state:

sudo apt update && sudo apt upgrade -y

This may take some time. Go grab a cup of coffee.

4.3 Timezone Configuration

Set the server timezone to your local time:

# Check current timezone
timedatectl

# Change to your timezone (example: New York)
sudo timedatectl set-timezone America/New_York

# Verify the change
date

4.4 Changing Hostname (If Needed)

If you want to change the hostname set during installation:

# Check current hostname
hostname

# Change hostname
sudo hostnamectl set-hostname newname

# Also modify /etc/hosts file
sudo nano /etc/hosts

In the hosts file, also change the hostname on the 127.0.1.1 line.

4.5 Static IP Configuration

A server must have a fixed IP address. A DHCP-assigned IP can change after rebooting or over time, which would break port forwarding and other configurations.

Method 1: DHCP Reservation on Router

The simplest method. In your router's admin page, reserve a specific IP for your server's MAC address.

Method 2: Set Static IP Directly on Server

Since Ubuntu 18.04, Netplan is used:

# Check network interface name
ip a

You will typically see names like enp0s3, eth0, or eno1.

# Edit Netplan configuration file
sudo nano /etc/netplan/00-installer-config.yaml

Modify as follows (example):

network:
  ethernets:
    enp0s3:  # Modify to match your interface name
      dhcp4: no
      addresses:
        - 192.168.0.100/24  # Your desired IP
      routes:
        - to: default
          via: 192.168.0.1  # Router IP (gateway)
      nameservers:
        addresses:
          - 8.8.8.8
          - 8.8.4.4
  version: 2

Apply the configuration:

sudo netplan apply

Warning: Incorrect IP settings can disconnect your network connection. Make sure to work with a monitor and keyboard connected.

5. SSH Server Configuration

SSH is an essential tool for remotely accessing your server. If you installed OpenSSH during installation, it should already be running.

5.1 Checking SSH Status

# Check SSH service status
sudo systemctl status ssh

# If not enabled
sudo systemctl enable ssh
sudo systemctl start ssh

5.2 Connecting via SSH from Another PC

Windows 10/11:

In Command Prompt or PowerShell:

ssh username@server_ip

Mac/Linux:

In Terminal:

ssh username@server_ip

Example: ssh admin@192.168.0.100

On first connection, when prompted about the fingerprint, type yes.

5.3 SSH Security Hardening (Optional)

Default settings are sufficient, but for enhanced security you may want to change a few settings:

sudo nano /etc/ssh/sshd_config

Recommended setting changes:

# Change default port (from 22 to another number)
Port 2222

# Disable direct root login
PermitRootLogin no

# Allow only key authentication (after key setup)
# PasswordAuthentication no

Restart SSH after changing settings:

sudo systemctl restart ssh

Note: If you change the port, you must connect using ssh -p 2222 username@server_ip.

5.4 SSH Key Authentication Setup (Recommended)

Using SSH keys instead of passwords is much more secure.

Generate keys on client (your PC):

# Windows PowerShell or Mac/Linux terminal
ssh-keygen -t ed25519 -C "your_email@example.com"

Press Enter a few times to generate the keys.

Copy public key to server:

# Automatic copy
ssh-copy-id username@server_ip

# Or manually
# Copy contents of client's ~/.ssh/id_ed25519.pub
# to server's ~/.ssh/authorized_keys file

6. Installing Basic Packages

Pre-install utilities you will use frequently:

sudo apt install -y \
    curl \
    wget \
    git \
    htop \
    neofetch \
    net-tools \
    ufw
  • curl, wget: File download
  • git: Version control (for configuration file backups, etc.)
  • htop: System monitoring (better than top)
  • neofetch: View system info (nicely formatted)
  • net-tools: Network tools like ifconfig
  • ufw: Firewall management tool

6.1 Basic Firewall Configuration

# Allow SSH (this is critical or you will be locked out!)
sudo ufw allow ssh

# Or if you changed the port
sudo ufw allow 2222/tcp

# Enable firewall
sudo ufw enable

# Check status
sudo ufw status

Conclusion

Great work! You now have Ubuntu Server installed and basic configuration complete. In summary:

  • Ubuntu Server 24.04 LTS installation complete
  • Timezone and hostname configured
  • Static IP configured
  • SSH server configured and secured
  • Basic utilities and firewall configured

You can now manage your server via SSH from another PC without a monitor and keyboard. Place your server in a convenient corner!

In Part 3, we will cover network configuration and remote access, including port forwarding, DDNS, and VPN setup for accessing your server from anywhere.

If you have any questions, please leave them in the comments!