Master essential Linux commands, file systems, processes, and system administration tasks critical for DevOps engineers. Learn to navigate, manage, and automate Linux systems with confidence.
Linux is the backbone of modern DevOps infrastructure. From cloud servers to containers, understanding Linux is essential for any DevOps engineer. This lesson covers the fundamental Linux concepts and commands you'll use daily in your DevOps career.
Why Linux Matters in DevOps:
The vast majority of servers, containers, and cloud instances run Linux. Whether you're deploying applications, managing infrastructure, or troubleshooting production issues, Linux proficiency is non-negotiable. Understanding the Linux file system, process management, and command-line tools enables you to work efficiently and solve problems quickly.
What You'll Learn:
You'll master the essential Linux commands and concepts that form the foundation of DevOps work. From navigating the file system to monitoring processes, managing permissions, and analyzing logs, you'll gain the practical skills needed for real-world DevOps scenarios.
The Linux file system follows a hierarchical tree structure starting from the root directory (/). Everything in Linux is either a file or a directory, and understanding this organization is crucial.
Key Directories:
/ - Root directory, the top of the file system hierarchy/home - User home directories/etc - System configuration files/var - Variable data (logs, caches, temporary files)/usr - User programs and utilities/bin - Essential command binaries/sbin - System administration binaries/tmp - Temporary files/opt - Optional application softwareEssential Navigation Commands:
1# Print working directory
2pwd
3
4# List files and directories
5ls # Basic listing
6ls -l # Detailed listing (long format)
7ls -la # Include hidden files
8ls -lh # Human-readable sizes
9ls -lt # Sort by modification time
10
11# Change directory
12cd /path/to/directory
13cd ~ # Go to home directory
14cd .. # Go up one level
15cd - # Go to previous directory
16
17# Create directories
18mkdir mydir
19mkdir -p path/to/nested/dir # Create parent directories
20
21# View file contents
22cat file.txt # Display entire file
23less file.txt # Paginated view (q to quit)
24head file.txt # First 10 lines
25tail file.txt # Last 10 lines
26tail -f /var/log/app.log # Follow log in real-time
27
28# File operations
29cp source.txt dest.txt # Copy file
30cp -r sourcedir destdir # Copy directory recursively
31mv oldname.txt newname.txt # Move/rename file
32rm file.txt # Remove file
33rm -rf directory # Remove directory recursively (use with caution!)
34
35# Search for files
36find /path -name "*.log" # Find files by name
37find /var/log -mtime -1 # Modified in last 24 hours
38find . -type f -size +100M # Files larger than 100MBUnderstanding Paths:
/home/user/file.txt)./file.txt or ../other/file.txt). represents current directory.. represents parent directory~ represents home directoryWorking with File Content:
1# Search within files
2grep "error" /var/log/app.log
3grep -r "TODO" /path/to/code # Recursive search
4grep -i "warning" logfile.txt # Case-insensitive
5
6# Count lines, words, characters
7wc -l file.txt # Count lines
8wc -w file.txt # Count words
9
10# Sort and unique
11sort file.txt
12sort -n numbers.txt # Numeric sort
13uniq file.txt # Remove duplicatesEvery file and directory in Linux has associated permissions and ownership. Understanding and managing these is essential for security and proper system operation.
Permission Structure:
Each file has three sets of permissions for:
Each set can have three permissions:
Reading Permissions:
1ls -l file.txt
2# -rw-r--r-- 1 user group 1234 Jan 15 10:30 file.txt
3# │││ │││ │││
4# │││ ││└─┘└─ others: r-- (read only)
5# │└┘ └───── group: r-- (read only)
6# └───────── owner: rw- (read and write)
7# First character: - (file) or d (directory) or l (link)Changing Permissions (chmod):
1# Symbolic notation
2chmod u+x script.sh # Add execute for owner
3chmod g+w file.txt # Add write for group
4chmod o-r secret.txt # Remove read for others
5chmod a+r public.txt # Add read for all (a = all)
6
7# Numeric notation
8chmod 755 script.sh # rwxr-xr-x
9chmod 644 file.txt # rw-r--r--
10chmod 600 secret.txt # rw-------
11chmod 700 private-dir # rwx------ for directory
12
13# Common permission sets:
14# 755: Standard for directories and executables
15# 644: Standard for regular files
16# 600: Private files (only owner can read/write)
17# 700: Private directoriesChanging Ownership (chown):
1# Change owner
2sudo chown newuser file.txt
3
4# Change owner and group
5sudo chown newuser:newgroup file.txt
6
7# Change recursively
8sudo chown -R user:group /path/to/directory
9
10# Change only group
11chgrp newgroup file.txtSpecial Permissions:
1# Setuid (4): Run as file owner
2chmod 4755 /usr/bin/sudo
3
4# Setgid (2): Run as group owner, or inherit directory group
5chmod 2755 /shared/directory
6
7# Sticky bit (1): Only owner can delete files
8chmod 1777 /tmpViewing Current User and Groups:
1whoami # Current username
2id # User ID and groups
3groups # Groups current user belongs to
4groups username # Groups for specific userA process is a running instance of a program. Managing processes is crucial for system administration, troubleshooting, and resource management.
Viewing Processes:
1# List all running processes
2ps aux # All processes, detailed info
3ps aux | grep nginx # Filter for specific process
4
5# Real-time process monitoring
6top # Interactive process viewer (q to quit)
7htop # Enhanced version (if installed)
8
9# Process tree
10pstree # Show process hierarchy
11ps auxf # Process tree with details**Understanding ps aux Output:**
1USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
2root 1 0.0 0.4 169564 12276 ? Ss Jan15 0:03 /sbin/initControlling Processes:
1# Start a process in background
2./long-running-script.sh &
3
4# View background jobs
5jobs
6
7# Bring background job to foreground
8fg %1
9
10# Suspend current process (Ctrl+Z)
11# Then put it in background
12bg %1
13
14# Kill a process
15kill PID # Graceful termination (SIGTERM)
16kill -9 PID # Force kill (SIGKILL)
17kill -15 PID # Same as default kill (SIGTERM)
18
19# Kill processes by name
20pkill nginx
21killall nginx
22
23# Kill all processes of a user
24pkill -u usernameManaging Process Priority:
1# Start with specific priority (nice value: -20 to 19)
2nice -n 10 ./cpu-intensive-script.sh
3
4# Change priority of running process
5renice -n 5 -p PID
6
7# Top priority (requires sudo)
8sudo renice -n -20 -p PIDSystem Services (systemd):
1# Start/stop/restart services
2sudo systemctl start nginx
3sudo systemctl stop nginx
4sudo systemctl restart nginx
5sudo systemctl reload nginx # Reload config without restart
6
7# Check service status
8sudo systemctl status nginx
9
10# Enable/disable service at boot
11sudo systemctl enable nginx
12sudo systemctl disable nginx
13
14# List all services
15systemctl list-units --type=service
16
17# View service logs
18journalctl -u nginx
19journalctl -u nginx -f # Follow logs
20journalctl -u nginx --since "1 hour ago"Effective system monitoring helps you identify bottlenecks, prevent outages, and optimize resource utilization.
CPU Monitoring:
1# View CPU information
2lscpu # CPU architecture details
3cat /proc/cpuinfo # Detailed CPU information
4
5# Real-time CPU usage
6top # Press '1' to show all CPUs
7htop # Better visualization
8
9# CPU load averages
10uptime
11# Output: load average: 1.5, 2.3, 1.8 (1min, 5min, 15min)
12# Rule of thumb: load < number of CPU cores = goodMemory Monitoring:
1# Memory usage summary
2free -h # Human-readable
3free -m # In megabytes
4
5# Output:
6# total used free shared buff/cache available
7# Mem: 15Gi 8.2Gi 1.5Gi 234Mi 5.6Gi 6.8Gi
8
9# Detailed memory information
10cat /proc/meminfo
11
12# Find processes using most memory
13ps aux --sort=-%mem | head -10Disk Monitoring:
1# Disk space usage
2df -h # All mounted filesystems
3df -h /var # Specific mount point
4
5# Directory size
6du -sh /var/log # Total size of directory
7du -h /var/log | sort -hr | head -10 # Top 10 largest
8
9# Disk I/O statistics
10iostat # Requires sysstat package
11iostat -x 1 # Extended stats, 1 second interval
12
13# Find large files
14find / -type f -size +100M -exec ls -lh {} \; 2>/dev/nullNetwork Monitoring:
1# Network interfaces
2ip addr # IP addresses
3ip link # Network interfaces status
4
5# Network statistics
6netstat -tuln # Listening ports (TCP/UDP)
7ss -tuln # Modern replacement for netstat
8
9# Active connections
10netstat -ant # All TCP connections
11ss -ant # Modern version
12
13# Network traffic
14ifconfig # Interface statistics (legacy)
15ip -s link # Interface statistics (modern)
16
17# Test connectivity
18ping google.com
19ping -c 4 8.8.8.8 # Send 4 packets only
20
21# Trace route
22traceroute google.com
23
24# DNS lookup
25nslookup google.com
26dig google.comSystem Logs:
1# System log locations
2/var/log/syslog # General system log (Debian/Ubuntu)
3/var/log/messages # General system log (RHEL/CentOS)
4/var/log/auth.log # Authentication logs
5/var/log/kern.log # Kernel logs
6/var/log/dmesg # Boot messages
7
8# View logs
9tail -f /var/log/syslog # Follow log
10journalctl -f # Follow systemd journal
11journalctl --since "1 hour ago" # Last hour
12journalctl -p err # Only errors
13journalctl -u nginx --since today # Service logs for today
14
15# Search logs
16grep -i "error" /var/log/syslog
17grep "failed" /var/log/auth.logPerformance Analysis Tools:
1# System activity
2vmstat 1 # Virtual memory stats every second
3sar -u 1 10 # CPU usage (10 samples, 1 sec interval)
4
5# I/O wait time
6iostat -x 1
7
8# Check for system bottlenecks
9top # Overall system view
10iotop # I/O by process
11nethogs # Network by processThese are the commands you'll use repeatedly in DevOps work, from managing text files to automating deployments.
Text Processing:
1# Stream editor (sed)
2sed 's/old/new/g' file.txt # Replace text
3sed -i 's/old/new/g' file.txt # Edit file in-place
4sed -n '10,20p' file.txt # Print lines 10-20
5
6# Pattern scanning (awk)
7awk '{print $1}' file.txt # Print first column
8awk -F: '{print $1}' /etc/passwd # Custom delimiter
9ps aux | awk '{sum+=$3} END {print sum}' # Sum CPU usage
10
11# Cut specific fields
12cut -d: -f1 /etc/passwd # Extract usernames
13cut -c1-10 file.txt # Extract characters 1-10
14
15# Paste files side by side
16paste file1.txt file2.txtPiping and Redirection:
1# Output redirection
2command > file.txt # Overwrite file
3command >> file.txt # Append to file
4command 2> error.txt # Redirect stderr
5command &> all-output.txt # Redirect stdout and stderr
6command > /dev/null 2>&1 # Discard all output
7
8# Input redirection
9command < input.txt
10
11# Piping
12ps aux | grep nginx | awk '{print $2}'
13cat /var/log/nginx/access.log | grep "404" | wc -lArchives and Compression:
1# tar (tape archive)
2tar -czf backup.tar.gz /path/to/dir # Create compressed archive
3tar -xzf backup.tar.gz # Extract archive
4tar -tzf backup.tar.gz # List contents
5tar -xzf backup.tar.gz -C /destination # Extract to specific location
6
7# gzip/gunzip
8gzip large-file.txt # Compress (creates .gz)
9gunzip large-file.txt.gz # Decompress
10
11# zip/unzip
12zip -r archive.zip directory/
13unzip archive.zipSystem Information:
1# System details
2uname -a # Kernel and system info
3hostnamectl # Hostname and OS info
4lsb_release -a # Distribution info
5
6# Hardware info
7lshw # Detailed hardware
8lsblk # Block devices
9lspci # PCI devices
10lsusb # USB devices
11
12# Running kernel
13uname -r
14
15# System uptime
16uptime
17
18# Who is logged in
19w
20who
21last # Login historyPackage Management:
1# Debian/Ubuntu (apt)
2sudo apt update # Update package index
3sudo apt upgrade # Upgrade packages
4sudo apt install package-name # Install package
5sudo apt remove package-name # Remove package
6apt search keyword # Search for packages
7apt show package-name # Package information
8
9# RHEL/CentOS (yum/dnf)
10sudo yum update # Update packages
11sudo yum install package-name # Install package
12sudo yum remove package-name # Remove package
13yum search keyword # Search packages
14
15# Check installed packages
16dpkg -l # Debian/Ubuntu
17rpm -qa # RHEL/CentOSEnvironment Variables:
1# View environment variables
2env # All variables
3echo $PATH # Specific variable
4echo $HOME
5echo $USER
6
7# Set temporary variable
8export MY_VAR="value"
9
10# Set permanent variable (add to ~/.bashrc or ~/.bash_profile)
11echo 'export MY_VAR="value"' >> ~/.bashrc
12source ~/.bashrc # Reload config
13
14# Add to PATH
15export PATH=$PATH:/new/pathUseful Shortcuts and Tips:
1# Command history
2history # Show command history
3!n # Run command number n
4!! # Repeat last command
5!$ # Last argument of previous command
6Ctrl+R # Search command history
7
8# Tab completion
9command <Tab> # Auto-complete
10ls /et<Tab> # Completes to /etc/
11
12# Command substitution
13echo "Today is $(date)"
14files=$(ls *.txt)
15
16# Aliases (add to ~/.bashrc)
17alias ll='ls -lah'
18alias update='sudo apt update && sudo apt upgrade'
19alias logs='tail -f /var/log/syslog'Real-world scenario: troubleshooting a slow web server using Linux commands
1#!/bin/bash
2# Scenario: Website is slow and users are complaining
3# Let's investigate systematically
4
5echo "=== SERVER TROUBLESHOOTING WORKFLOW ==="
6echo "Investigating slow website performance..."
7echo
8
9# STEP 1: Check System Load
10echo "Step 1: Checking system load..."
11uptime
12echo
13
14# STEP 2: Check Memory Usage
15echo "Step 2: Checking memory usage..."
16free -h
17echo
18
19# STEP 3: Check Disk Space
20echo "Step 3: Checking disk space..."
21df -h | grep -E '^/dev/'
22echo
23
24# STEP 4: Check Disk I/O
25echo "Step 4: Checking disk I/O (2 second sample)..."
26iostat -x 2 2 | tail -n +4
27echo
28
29# STEP 5: Find Top CPU Processes
30echo "Step 5: Top 5 CPU-consuming processes..."
31ps aux --sort=-%cpu | head -6
32echo
33
34# STEP 6: Find Top Memory Processes
35echo "Step 6: Top 5 memory-consuming processes..."
36ps aux --sort=-%mem | head -6
37echo
38
39# STEP 7: Check Web Server Status
40echo "Step 7: Checking nginx service status..."
41systemctl status nginx --no-pager | head -20
42echo
43
44# STEP 8: Check Active Connections
45echo "Step 8: Counting active connections to port 80..."
46ss -ant | grep :80 | wc -l
47echo "Active connections to port 80: $(ss -ant | grep :80 | wc -l)"
48echo
49
50# STEP 9: Check for Errors in Logs
51echo "Step 9: Recent errors in nginx error log..."
52tail -20 /var/log/nginx/error.log 2>/dev/null || echo "No error log found"
53echo
54
55# STEP 10: Analyze Access Patterns
56echo "Step 10: Top 10 requesting IPs in last 1000 requests..."
57tail -1000 /var/log/nginx/access.log 2>/dev/null | awk '{print $1}' | sort | uniq -c | sort -rn | head -10
58echo
59
60# STEP 11: Check for Long-Running Queries (if MySQL)
61echo "Step 11: Checking MySQL processlist for slow queries..."
62mysql -e "SHOW PROCESSLIST;" 2>/dev/null | grep -v Sleep | head -10 || echo "MySQL not accessible or not installed"
63echo
64
65# STEP 12: Network Check
66echo "Step 12: Checking network connectivity..."
67ping -c 3 8.8.8.8 > /dev/null 2>&1 && echo "✓ Internet connectivity OK" || echo "✗ Internet connectivity FAILED"
68echo
69
70# STEP 13: Check System Logs for Issues
71echo "Step 13: Recent critical errors in syslog..."
72grep -i "error\|critical\|failed" /var/log/syslog | tail -5
73echo
74
75# DIAGNOSIS SUMMARY
76echo "=== DIAGNOSIS SUMMARY ==="
77
78# Check if high CPU
79CPU_LOAD=$(uptime | awk -F'load average:' '{print $2}' | awk '{print $1}' | sed 's/,//')
80CPU_COUNT=$(nproc)
81echo "CPU Load: $CPU_LOAD (CPUs: $CPU_COUNT)"
82if (( $(echo "$CPU_LOAD > $CPU_COUNT" | bc -l) )); then
83 echo "⚠️ HIGH CPU LOAD detected"
84fi
85
86# Check if low memory
87MEM_AVAILABLE=$(free | awk 'NR==2 {print $7}')
88MEM_TOTAL=$(free | awk 'NR==2 {print $2}')
89MEM_PERCENT=$((MEM_AVAILABLE * 100 / MEM_TOTAL))
90echo "Available Memory: ${MEM_PERCENT}%"
91if [ $MEM_PERCENT -lt 10 ]; then
92 echo "⚠️ LOW MEMORY detected"
93fi
94
95# Check if disk space critical
96DISK_USAGE=$(df -h / | awk 'NR==2 {print $5}' | sed 's/%//')
97echo "Root Disk Usage: ${DISK_USAGE}%"
98if [ $DISK_USAGE -gt 80 ]; then
99 echo "⚠️ HIGH DISK USAGE detected"
100fi
101
102# Check nginx status
103if systemctl is-active --quiet nginx; then
104 echo "✓ Nginx is running"
105else
106 echo "✗ Nginx is NOT running!"
107fi
108
109echo
110echo "=== RECOMMENDED ACTIONS ==="
111echo "1. Review top CPU/memory processes and investigate"
112echo "2. Check nginx error logs for application errors"
113echo "3. Analyze access logs for traffic spikes or DDoS"
114echo "4. Consider adding more resources if consistently high load"
115echo "5. Optimize database queries if MySQL shows slow queries"
116echo "6. Check for full disk or high I/O wait"
117echo "7. Review recent code deployments for performance issues"This comprehensive troubleshooting script demonstrates how to systematically diagnose server performance issues using Linux commands.
The Investigation Process:
uptime to check if the system is overloadedfree -h to identify memory pressuredf -h to check for full disksiostat to identify I/O bottlenecksps aux to find resource-hungry processessystemctl to verify services are runningss to count active connectionstail and grep to find errorsawk to analyze traffic patternsKey Diagnostics:
iostatsystemctl status shows inactiveCommon Resolutions:
This workflow represents real DevOps troubleshooting and combines multiple Linux commands to solve production issues effectively.
Your progress is saved in this browser, on this device only.