Random Useful Linux Commands
How to grep for a string and truncate the results?
1 2 3 4 5 | cat ./file | grep --only-matching --line-number --extended-regexp '.{20}search string.{20}' | awk '{print $0,"\n"}' # --only-matching - Only the print the matching part. # --line-number - Prefix the results with their line number. # --extended-regexp - Following this is a regex. # awk '{print $0,"\n"}' - Print a newline after each result. |
How to find all the files of a certain extension that matches a regex pattern?
1 2 3 | grep --ignore-case --recursive -n --include \*.html -e '/http:\/\//' grep --ignore-case --recursive -n --include \*.html 'http://' # -n - Shows the relative line number of the match |
What regex can I use to find API keys in source code?
1 2 3 | # Search for patterns like "adXe5W_34dA" with a negative look-ahead to whittle down false positives =\s*"\b([a-z0-9_-]{6,})\b(?<!class|function|interface|password|disabled|gregorian|event|_blank|dateHeader|dateTitle)" # This works great in PyCharm and IntelliJ. |
What are the top ten largest folders?
1 2 3 4 5 6 7 | du -h 2>/dev/null | sort -hr | head -n 10 # Parameters: # -h - Human-readable # 2>/dev/null - Hide errors from inaccessible folders like /proc/... # -hr - Sort descending and human-readable numbers # -n 10 - Show the first 10 lines |
What are the IP address of my running Docker containers?
1 | docker inspect -f '{{.Name}} - {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $(docker ps -q) |
How many lines of code are there in a Java project?
1 2 | # Sum the number of lines from each file in each folder recursively find . -name '*.java' | xargs wc -l |
How to see all the git commits concisely?
1 | git log --oneline | nl -v0 | sed 's/^ \+/&HEAD~/' |
How long has a process been running?
1 2 3 | # Use pidof to find the PID of the running process (e.g. java) ps -eo pid,lstart,etime,cmd | grep `pidof java` # e.g. 21047 Sat Jun 15 00:51:40 2019 21-10:25:12 (21 days uptime) |
How many vCPUs do I have?
1 | grep processor /proc/cpuinfo | wc -l |
How to setup aliases in ~/.bashrc
1 2 3 4 | # Use `ls -al` as the default ls with auto color alias ll='/usr/bin/ls -l --color=auto' 2>/dev/null alias l.='/usr/bin/ls -d .* --color=auto' 2>/dev/null alias ls='/usr/bin/ls -al --color=auto' 2>/dev/null |
How to grep network traffic in realtime?
1 | ngrep -l -q -p -d eth0 -i '' 'tcp and dst host 192.168.0.106' |
REF: ngrep man pages
What iptables input rules are in effect?
1 | iptables -L INPUT -v -n | less |
Why was a process killed?
1 2 3 4 5 6 7 8 9 10 11 12 | # If it just happened, show the preceding 10 lines dmesg -T | grep -i -B10 'killed process' # Find the last time it happened and the preceding 10 lines cat /var/log/kern.log | grep -i -B10 'killed process' # Find each time it happened, each preceding 10 lines, and with human-readable timestamps journalctl -o short-precise | grep -i -B10 "killed process" # Parameters: # -T - Human-readable timestamps # -i - Case-insensitive # -B - Number of lines before the match to print # -o short-precise - Human-readable timestamps for journalctl |
REF: What killed my process and why?
REF: How to read dmesg from previous session?
What ports are open and what processes are listening?
1 2 3 4 5 6 7 8 9 10 11 | # Vanilla information - see quickly which ports are open ss -tulpn # See the full command line of the process bound to a port ss -tulpn | awk '/^./ {split($NF, pid,"/");sub(FS $NF,x);getline cmd < ("/proc/"pid[1]"/cmdline");print $0" "pid[1]"/"cmd}' # Parameters: # -t - TCP ports # -u - UDP ports # -l - Listening ports # -p - Display the process id and short program name # -n - Display numeric information (e.g. 127.0.01:25 instead of localhost:smtp ) |
REF: Netstat: See process name like in `ps aux`
How to limit the memory usage of journald?
1 2 3 4 5 6 7 8 9 10 | # Edit the journald config file nano /etc/systemd/journald.conf # Set these entries SystemMaxUse=500M RuntimeMaxUse=50M # Restart and verify the service systemctl restart systemd-journald systemctl status systemd-journald |
How to clean up journal logs?
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | # Check the current space du -hs /var/log/journal/ # Keep only n-days journalctl --vacuum-time=10d # Keep only n-MBs journalctl --vacuum-size=200M # List the number of boots journalctl --list-boots # See n lines of log journalctl -n 100 --no-pager # Delete old gzipped logs sudo find /var/log/ -type f -regex '.*\.[0-9]+\.gz$' -delete |
REF: How To Use Journalctl to View and Manipulate Systemd Logs
How many shells are connected?
1 2 3 | # List the number of shells currently connected # This excludes any screen sessions ps a | awk '{print $2}' | grep -vi "tty*" | sort | uniq | wc -l |
REF: How to count how many shell/terminals are running?
How to restart the system?
1 2 3 | shutdown -r now # Give a 1-minute warning and prevent new logins # shutdown -r +1 |
How to disable wifi and Bluetooth on Raspberry Pi 3s?
1 2 3 4 | # /boot/config.txt [all] dtoverlay=pi3-disable-wifi dtoverlay=pi3-disable-bt |
How to release and renew all the DHCP-assigned IP addresses?
1 | dhclient -r && dhclient |
What DNS server(s) am I using?
1 | nmcli dev show | grep DNS |
How many files are there in a folder and all child folders?
1 2 | cd to/the/folder find . -type f | wc -l |
How to zip an entire folder and subfolders?
1 2 3 4 5 6 | zip -r -q /path/to/output.zip /path/to/folder # Parameters: # -r - Recursive # -q - Quiet (don't show each file being added) # -0 - Store the files only, no compression |
How to gzip a huge folder tree with hundreds of thousands of files and avoid OOM?
1 2 3 4 5 6 7 8 9 10 | tar cf - /huge/folder -P \ | pv -s $(du -sb /huge/folder | awk '{print $1}') \ | gzip > /dest/folder/archive.tar.gz # Install pv with apt-get install pv # Note: du may take a while on a huge folder # du -sb - Get the total size of the folder in bytes # Warning: On small systems the OOM killer may run. # Find the process id of 'du' with ps -auxf and run # sudo echo -1000 > /proc/ID/oom_score_adj to protect your process. |
How to clear all Docker logs?
1 | sh -c "truncate -s 0 /var/lib/docker/containers/*/*-json.log" |
How to follow all the logs from Docker Compose?
1 2 3 4 5 6 | docker-compose up -d # Start all services in detached (or daemon) mode docker-compose logs -f # Parameters # -f - Follow the logs from all services as they arrive |
Disk out of space but there is plenty of space?
1 2 | # Check how many inodes are free - usage might be near 100% df -iH |
How to show all available hardware and connected devices?
1 2 | # apt-get install lshw lshw |
How to backup and restore an entire partition or disk?
1 2 3 4 5 | # Backup with compression sudo dd if=/dev/sda1 | bzip2 > ./backup-image.bz2 # Restore from a compressed image requires a boot to live environment bzcat ./backup-image.bz2 | dd of=/dev/sda1 |
How to audit all system calls?
1 2 3 4 5 6 7 8 9 10 11 | # Specify the architecture as b64 or b32 sudo auditctl -a exit,always -F arch=b64 -S execve # View the auditd rules sudo auditctl -l # Delete all the auditd rules sudo auditctl -D # Tail the audit log as invocations occur tail -f /var/log/audit/audit.log |
How to search man pages for a keyword like ‘encrypt’?
1 2 3 4 | # List available commands with a given keyword (same command as apropos) man -k encrypt # List available commands based on a regex man -k '^encrypt$' |
How to find all files modified in the last 10 minutes?
1 2 3 4 5 6 | sudo find / -type f -mmin -10 -exec ls -l {} + # sudo - Needed to search everywhere # / - Start searching at the root # -type f - Search for only files # -mmin -10 - Search for file modification less than (-) 10 minutes # -exec ls -l {} + - Exec ls on all the results at once (e.g. ls -l a.txt b.txt c.txt) |
How to rename all the files in a folder?
1 2 3 | sudo apt install rename rename -f 's/abc/xyz/' *.ext # This will rename all __abc__.ext files to __xyz__.ext |
How to Rsync the files in the current folder?
1 2 | # Note the trailing slash rsync -av . user@hostname-or-ip:/the/remote/folder/ |
How to label Screen sessions?
1 2 3 | # sudo apt install screen echo 'caption always "%{= kw}%-w%{= BW}%n %t%{-}%+w %-="' > ~/.screenrc screen |
How to share wlan0 Internet with eth0?
1 2 3 4 5 6 | sudo sysctl -w net.ipv4.ip_forward=1 sudo iptables -F sudo iptables -t nat -F sudo iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE sudo iptables -A FORWARD -i wlan0 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT sudo iptables -A FORWARD -i eth0 -o wlan0 -j ACCEPT |
How to find the largest files of a certain type recursively?
1 2 | # Find, format, sort in reverse order, limit to 10 find . -name '*some_pattern*' -printf '%s %p\n'| sort -nr | head -10 |
How to stop all running Docker containers at once?
1 | docker kill $(docker ps -q) |
How to run Linux Mint with Cinnamon in 2D mode (no hardware acceleration in a VM) and prevent the “Running software in rendering mode” message?
1 2 | # This adds a flag to enable flat or 2D UI mode echo "export CINNAMON_2D=true" >> ~/.profile |
More to come…