devops-interview-handbook

Linux Interview Questions

Table of Contents


System Administration

Q1: Explain the Linux boot process.

Difficulty: Mid

Answer:

Boot Sequence:

  1. BIOS/UEFI: Initializes hardware, finds boot device
  2. Bootloader (GRUB): Loads kernel, shows boot menu
  3. Kernel: Initializes hardware, mounts root filesystem
  4. Init Process (systemd/PID 1): First process, starts all other processes
  5. Runlevels/Targets: Determines which services start
    • systemd targets: multi-user.target, graphical.target
  6. Services Start: System services and daemons start

systemd (Modern Linux):

Key Files:

Real-world Context: Server boots → GRUB loads kernel → Kernel initializes → systemd starts → Services start → System ready.

Follow-up: What is PID 1? (Init process, parent of all processes, manages system)


Q2: Explain Linux file permissions and how to change them.

Difficulty: Junior

Answer:

Permission Types:

Permission Groups:

Permission Representation:

-rwxr-xr-- 1 user group 1024 Jan 1 10:00 file.txt
│││││││││
│└┴┴└┴┴└┴┴
│ │ │ │ │ │
│ │ │ │ │ └─ Others: r--
│ │ │ │ └─── Group: r-x
│ │ │ └───── Owner: rwx
│ │ └─────── Type: - (regular file)

Numeric Notation:

Changing Permissions:

chmod 755 script.sh
chmod u+x script.sh  # Add execute for owner
chmod g-w file.txt   # Remove write for group
chmod -R 755 directory/  # Recursive

Changing Ownership:

chown user:group file.txt
chown -R user:group directory/

Real-world Context: Script needs to be executable: chmod +x script.sh. Web server needs read access: chmod 644 index.html.

Follow-up: What does chmod 777 do? (Gives full permissions to everyone - security risk, avoid in production)


Q3: Explain the Linux directory structure (/etc, /var, /usr, etc.).

Difficulty: Mid

Answer:

Key Directories:

/ (root): Root of filesystem

/bin: Essential binaries (ls, cp, mv) - system critical

/sbin: System binaries (fdisk, ifconfig) - system administration

/etc: Configuration files (system and application configs)

/var: Variable data (logs, spool, cache)

/usr: User programs and data

/home: User home directories

/root: Root user’s home directory

/tmp: Temporary files (cleared on reboot)

/opt: Optional/third-party software

/dev: Device files (represent hardware)

/proc: Process information (virtual filesystem)

/sys: System information (virtual filesystem)

Real-world Context: Configs in /etc, logs in /var/log, user data in /home, temporary files in /tmp.

Follow-up: What’s the difference between /bin and /usr/bin? (Historically: /bin on root partition, /usr/bin on separate partition. Now: /bin essential, /usr/bin user programs)


Q4: How do you manage services with systemd?

Difficulty: Mid

Answer:

systemd Commands:

Service Management:

systemctl start service-name    # Start service
systemctl stop service-name     # Stop service
systemctl restart service-name  # Restart service
systemctl reload service-name   # Reload config (if supported)
systemctl status service-name   # Check status

Enable/Disable:

systemctl enable service-name   # Start on boot
systemctl disable service-name  # Don't start on boot
systemctl is-enabled service-name  # Check if enabled

Service Status:

systemctl list-units --type=service  # List all services
systemctl list-units --type=service --state=running  # Running services
systemctl list-units --type=service --state=failed   # Failed services

Service Files:

Example Service File:

[Unit]
Description=My Service
After=network.target

[Service]
Type=simple
ExecStart=/usr/bin/my-service
Restart=always

[Install]
WantedBy=multi-user.target

Real-world Context: Start nginx: systemctl start nginx. Enable on boot: systemctl enable nginx. Check status: systemctl status nginx.

Follow-up: What’s the difference between start and enable? (Start: run now, Enable: start on boot)


Process Management

Q5: Explain process states and how to manage processes.

Difficulty: Mid

Answer:

Process States:

Process Management Commands:

ps aux              # List all processes
ps -ef              # Alternative format
top                 # Interactive process viewer
htop                # Enhanced top
pgrep process-name  # Find process by name
pkill process-name  # Kill process by name

Signals:

kill -9 PID        # SIGKILL (force kill, cannot be caught)
kill -15 PID       # SIGTERM (graceful termination, default)
kill -2 PID        # SIGINT (interrupt, Ctrl+C)
kill -1 PID        # SIGHUP (reload config)

Process Priorities:

nice -n 10 command    # Run with nice 10
renice 10 PID         # Change nice of running process

Real-world Context: Process consuming CPU: top to find PID, kill -15 PID for graceful stop, kill -9 PID if unresponsive.

Follow-up: What’s a zombie process? (Terminated process whose parent hasn’t reaped it. Parent needs to call wait() or exit)


Q6: Explain background processes, jobs, and nohup.

Difficulty: Mid

Answer:

Background Processes:

command &           # Run in background
jobs               # List background jobs
fg %1              # Bring job 1 to foreground
bg %1              # Send job 1 to background

Job Control:

nohup:

nohup command &
nohup command > output.log 2>&1 &

screen/tmux:

screen -S session-name    # Create named session
screen -r session-name    # Reattach session
screen -ls               # List sessions

tmux new -s session-name
tmux attach -t session-name

Real-world Context: Long-running script: nohup ./script.sh > output.log 2>&1 &. Disconnect SSH, script continues. Reconnect, check output.

Follow-up: What’s the difference between & and nohup? (&: background, but dies when terminal closes. nohup: survives terminal close)


File System

Q7: Explain Linux file system types and mounting.

Difficulty: Mid

Answer:

Common File Systems:

Mounting:

mount /dev/sdb1 /mnt/data    # Mount device to directory
umount /mnt/data             # Unmount
mount -a                     # Mount all in /etc/fstab

/etc/fstab:

Example:

/dev/sdb1  /mnt/data  ext4  defaults  0  2
UUID=1234  /mnt/backup  xfs  defaults,noatime  0  2

Mount Options:

Checking Disk Usage:

df -h              # Filesystem disk space
du -sh directory/  # Directory size
du -h --max-depth=1 /  # Size of top-level directories

Real-world Context: Add new disk: Partition (fdisk), format (mkfs.ext4), mount (mount), add to /etc/fstab for auto-mount.

Follow-up: What happens if you can’t unmount a filesystem? (Process using it. Use lsof or fuser to find, kill process, then unmount)


Difficulty: Mid

Answer:

Hard Links:

Symbolic Links (Symlinks):

Creating Links:

ln target.txt link.txt           # Hard link
ln -s target.txt symlink.txt     # Symbolic link

Differences:

Use Cases:

Real-world Context: Application expects config at /etc/app/config. Symlink /etc/app/config -> /opt/app/config. Move config, update symlink.

Follow-up: What happens if you delete the target of a symlink? (Symlink becomes broken, points to non-existent file)


Networking

Q9: How do you configure network interfaces in Linux?

Difficulty: Mid

Answer:

Network Interface Commands:

ip addr show              # Show IP addresses (modern)
ifconfig                  # Show interfaces (deprecated)
ip link show              # Show interfaces
ip addr add 192.168.1.10/24 dev eth0  # Add IP
ip link set eth0 up       # Bring interface up
ip route show             # Show routing table

Network Configuration Files:

systemd-networkd:

NetworkManager:

Traditional (Debian/Ubuntu):

Traditional (RHEL/CentOS):

Example Configuration:

# Static IP
ip addr add 192.168.1.10/24 dev eth0
ip route add default via 192.168.1.1
echo "nameserver 8.8.8.8" > /etc/resolv.conf

# Or use netplan (Ubuntu 18+)
# /etc/netplan/01-netcfg.yaml

DNS Configuration:

/etc/resolv.conf          # DNS servers
/etc/hosts                # Local hostname resolution

Real-world Context: Configure static IP: Edit network config file or use ip commands. Set gateway, DNS. Restart networking service.

Follow-up: What’s the difference between ip and ifconfig? (ip is modern, ifconfig is deprecated but still available)


Q10: Explain iptables and firewall management.

Difficulty: Mid

Answer:

iptables:

Basic Commands:

iptables -L              # List rules
iptables -A INPUT -p tcp --dport 22 -j ACCEPT  # Allow SSH
iptables -A INPUT -p tcp --dport 80 -j ACCEPT   # Allow HTTP
iptables -A INPUT -j DROP                       # Default deny
iptables -F              # Flush rules
iptables -S              # Show rules in command format

Common Rules:

# Allow loopback
iptables -A INPUT -i lo -j ACCEPT

# Allow established connections
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow SSH from specific IP
iptables -A INPUT -p tcp -s 192.168.1.0/24 --dport 22 -j ACCEPT

# Allow HTTP/HTTPS
iptables -A INPUT -p tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp --dport 443 -j ACCEPT

# Default deny
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT

firewalld (RHEL/CentOS):

firewall-cmd --list-all
firewall-cmd --add-service=http --permanent
firewall-cmd --reload

ufw (Ubuntu):

ufw allow 22/tcp
ufw allow 80/tcp
ufw enable

Real-world Context: Web server: Allow 22 (SSH), 80 (HTTP), 443 (HTTPS). Deny everything else. Use firewalld or ufw for simplicity.

Follow-up: What’s the difference between iptables and firewalld? (iptables: low-level, firewalld: high-level wrapper, easier to use)


Shell Scripting

Q11: Write a bash script to check disk usage and alert if over threshold.

Difficulty: Mid

Answer:

#!/bin/bash

# Configuration
THRESHOLD=80
EMAIL="admin@example.com"
PARTITION="/"

# Get disk usage percentage
USAGE=$(df -h "$PARTITION" | awk 'NR==2 {print $5}' | sed 's/%//')

# Check if over threshold
if [ "$USAGE" -gt "$THRESHOLD" ]; then
    echo "Disk usage is ${USAGE}% on $PARTITION" | \
        mail -s "Disk Usage Alert" "$EMAIL"
    echo "Alert sent: Disk usage is ${USAGE}%"
else
    echo "Disk usage is ${USAGE}% - OK"
fi

Improved Version:

#!/bin/bash

THRESHOLD=80
PARTITION="/"

check_disk() {
    local partition=$1
    local usage=$(df -h "$partition" | awk 'NR==2 {print $5}' | sed 's/%//')
    
    if [ "$usage" -gt "$THRESHOLD" ]; then
        echo "WARNING: $partition is ${usage}% full"
        return 1
    else
        echo "OK: $partition is ${usage}% full"
        return 0
    fi
}

# Check multiple partitions
check_disk "/"
check_disk "/var"
check_disk "/home"

Real-world Context: Cron job runs daily, checks disk usage, sends email if over 80%. Prevents disk full issues.

Follow-up: How would you make this script more robust? (Error handling, logging, multiple thresholds, check multiple partitions)


Q12: Explain bash scripting best practices.

Difficulty: Mid

Answer:

1. Shebang:

#!/bin/bash

2. Error Handling:

set -e          # Exit on error
set -u          # Exit on undefined variable
set -o pipefail # Exit on pipe failure

3. Variables:

# Quote variables
name="John Doe"
echo "$name"

# Use readonly for constants
readonly MAX_RETRIES=3

# Use local in functions
function myfunc() {
    local var="value"
}

4. Functions:

function usage() {
    echo "Usage: $0 [options]"
    exit 1
}

5. Input Validation:

if [ $# -lt 1 ]; then
    usage
fi

6. Logging:

LOG_FILE="/var/log/script.log"
log() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') $*" | tee -a "$LOG_FILE"
}

7. Temporary Files:

TMPFILE=$(mktemp)
trap "rm -f $TMPFILE" EXIT

8. Comments:

Real-world Context: Production script: Error handling, logging, input validation, cleanup on exit, proper error messages.

Follow-up: What does set -e do? (Exit immediately if any command exits with non-zero status)


Performance & Monitoring

Q13: How do you monitor system performance in Linux?

Difficulty: Mid

Answer:

CPU Monitoring:

top                 # Interactive process viewer
htop                # Enhanced top
vmstat 1            # System statistics
mpstat 1            # CPU statistics per core
sar -u 1            # CPU utilization (if sysstat installed)

Memory Monitoring:

free -h             # Memory usage
vmstat 1            # Memory statistics
sar -r 1            # Memory utilization
cat /proc/meminfo   # Detailed memory info

Disk I/O Monitoring:

iostat -x 1         # Disk I/O statistics
iotop               # I/O by process
df -h               # Disk space
du -sh *            # Directory sizes

Network Monitoring:

iftop               # Network usage by connection
nethogs             # Network usage by process
ss -tulpn           # Network connections (modern netstat)
netstat -tulpn      # Network connections

System Load:

uptime              # Load average
w                   # Who and load average
cat /proc/loadavg   # Load average

Real-world Context: Server slow: Check top for CPU, free for memory, iostat for disk I/O, iftop for network. Identify bottleneck.

Follow-up: What does load average mean? (1.0 = 1 CPU fully utilized. 2.0 on 4-core = 50% utilization)


Q14: Explain how to troubleshoot high CPU usage.

Difficulty: Mid

Answer:

Steps:

1. Identify High CPU Processes:

top                 # Sort by CPU (%CPU)
htop                # Better visualization
ps aux --sort=-%cpu | head -10  # Top 10 CPU processes

2. Analyze Process:

strace -p PID       # System calls (if process is stuck)
perf top            # Performance profiling
pidstat -p PID 1    # Detailed process stats

3. Check System Load:

uptime              # Load average
mpstat -P ALL 1     # Per-CPU utilization

4. Check for Zombie Processes:

ps aux | grep Z     # Zombie processes

5. Check I/O Wait:

iostat -x 1         # High %iowait = I/O bottleneck

6. Check Context Switches:

vmstat 1            # High cs = context switching overhead

Common Causes:

Real-world Context: CPU at 100%: top shows Java process. Check if infinite loop, memory issue, or I/O wait. Kill if needed, or optimize code.

Follow-up: What’s the difference between CPU usage and load average? (CPU: current utilization, Load: average over time, includes I/O wait)


Security

Q15: Explain SSH key authentication and best practices.

Difficulty: Mid

Answer:

SSH Key Authentication:

Setup:

# Generate key pair
ssh-keygen -t rsa -b 4096 -C "email@example.com"

# Copy public key to server
ssh-copy-id user@server
# Or manually:
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

SSH Config:

# ~/.ssh/config
Host myserver
    HostName 192.168.1.10
    User admin
    IdentityFile ~/.ssh/id_rsa
    Port 22

Best Practices:

Server Configuration (/etc/ssh/sshd_config):

PasswordAuthentication no
PubkeyAuthentication yes
PermitRootLogin no

Real-world Context: Generate SSH key, copy to servers. Disable password auth. Use SSH agent. More secure than passwords.

Follow-up: How do you use SSH agent? (eval $(ssh-agent), ssh-add ~/.ssh/id_rsa, enter passphrase once, use multiple times)


Q16: Explain Linux security hardening practices.

Difficulty: Senior

Answer:

1. System Updates:

apt update && apt upgrade    # Debian/Ubuntu
yum update                  # RHEL/CentOS

2. Firewall:

3. SSH Hardening:

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
Port 2222  # Change default port
MaxAuthTries 3

4. User Management:

5. File Permissions:

6. Disable Unused Services:

systemctl disable service-name
systemctl stop service-name

7. Logging and Monitoring:

8. SELinux/AppArmor:

9. Kernel Hardening:

10. Regular Audits:

Real-world Context: New server: Update, configure firewall, harden SSH, disable unused services, enable logging, set up monitoring.

Follow-up: What’s the difference between SELinux and AppArmor? (SELinux: RHEL/CentOS, more complex. AppArmor: Ubuntu/Debian, simpler)


Summary

Linux is essential for DevOps. Master system administration, process management, networking, scripting, and security. Practice troubleshooting and automation.

Next Steps: