Skip to content

linux administration Notes

Jump to bottom
chris edited this page Sep 14, 2022 · 16 revisions

Contents for Linux & macOS Administration Notes
*nix, BSD, & macOS Universal Tooling
Linux
Useful Links
TODOs

*nix, BSD, & macOS Universal Tooling πŸ”

The concepts outlined in this section should be applicable to most if not all modern OS's derived from a UNIX platform.

Backing up with rsync πŸ”

To display a progress meter when transferring a file to a remote box, use rsync and not scp

rsync -e "ssh -p4242" --info=progress2 /path/to/mr-fancy.txt remote:/path/to/

One of the many benefits of backing up with rsync is that it performs incremental backups by default, so it should not recopy files if they have already been backed up.

Backing up a root partition using rsync πŸ”

The below write up will demonstrate how to backup an SD card that contains Raspbian on it.

To perform a full system backup of a root file system using rsync

rsync -aAXh --exclude=/path/to/rsync-exclude-file --info=progress2 /path/to/source/directory/or/file /path/to/destination/directory/or/file

To restore from an rsync backup reverse the source and destination paths.

A typical rsync excludes file, ie. rsync-excludes will contain

/.fseventsd/*
/boot/*
/dev/*
/lib/modules/*
/media/*
/mnt/*
/proc/*
/run/*
/sys/*
/tmp/*
/var/log/*
/lost+found
/etc/fstab
/etc/mtab
/etc/modules
/etc/network/interfaces

rsync > Backing up across a network, ie. LAN

rsync is useful for backing up files and folders across a network attached disk, ie. a large mass storage device attached to a Raspberry Pi on a LAN. However, there are some gotchas to be aware of.

  1. I did not get any decent results using NFS for whatever reason, I was averaging ~ 30KB transfer speeds from my MBP to ext USB device attached to a rpi.
  2. NFS is kind of a PITA to get setup and going.
  3. Installing sshfs on a local box, ie. my MBP is far less painless then setting up NFS.
  • make sure that the local $USER and GROUP exist on both the local and remote systems along with having the same UID and GID for best results.
  • even though there is a pi user on the stock Raspbian OS, I'd suggest creating a new user that has the same name and GID as on the local box (my MBP).

After all the above criteria has been met, then one can mount the mass storage device attached to the pi using the newly created credentials. If all goes well the rsync files should match the local UID and GID (my MBP) and the files contained on the USB ext drive (mass storage device) should have the same UID and GID

❗️ As far as I can tell, any non root user can mount the USB device / filesystem / partition on the pi and as long as the local user sshfs's the remote filesystem using the same remote / local credentials all should be good.

❗️ If the remote filesystem is mounted using sshfs such as,

sshfs pi@hostname.local:/remote/mnt/point /local/mnt/point

then issues could arise where when rsyncing file to the remote filesystem will the UID of the pi user, ie. NOT HOT DOG 🌭

Useful rsync flags πŸ”

  • -a, --archive archive mode; files should be archived, meaning most characteristics are preserved

the -a flag implies --recursive and also implies --links.

  • -A, --acls preserve ACLs and update the destination ACLs with the source ACLs.

  • --exclude exclude the directories contained within curly braces.

  • --info=[FLAGS] control over the output rsync displays to STDOUT.

    • --info=help to display a list of flags that can be used with --info.
      • Ex --info=progress2

  • -h, --human-readable print output in friendly human-readable format.

  • -l, --links symlinks are recreated on the destination.

  • --no-i-r, --no-inc-recurssive disable incremental recursion

  • -P, --partial --progress keeps partially transferred files

  • -r, --recursive to recurse into directories

  • -S, --sparse handle sparse files efficiently, ie. this is useful when backing up files such as Docker images or virtual hard disks for emulators such as QEMU.

  • --stats record a verbose set of statistics of the file transfer.

  • -X update the destination with extended attributes to be the same as the source.

  • -x, --one-file-system prohibits rsync from going beyond the local file system.

  • -u, --update update destination file with source files if the destination file already exists.

  • --delete-excluded removes files from destination that no longer exist on the source. This is flag is useful when running into the below error message when running rsync

cannot delete non-empty directory:

Useful rsync commands πŸ”

dat new new new rsync command

rsync \
...
--filter 'protect /path/to/dir/to/not/delete/but/add/new/files'
/path/to/src
/path/to/destination/

To backup a file or dir without preserving UID and GID

rsync -a --no-o --no-g /path/to/local/dir /path/to/backup/

To backup a directory of files from a local disk to an external volume on macOS

rsync \
--archive \
--acls \
--info=progress2 \
--human-readable \
--stats  \
--update \
-X \
--one-file-system \
--sparse \
--delete \
--delete-excluded \
--exclude-from=/path/to/rsync-excludes-file \
/path/to/src \
/Volumes/external-disk/path/to/bkp/dir/

The above command should create a 1:1 backup the source directory to destination volume, and also delete files on the destination that are no longer present on the source, and should also update file attributes and files in modifications have occurred since last backup.

A useful command that can compliment the above rsync command when backing up files is using the watch command.

There may be a yarn / NPM package that installs a watch binary that can conflict with the watch command installed by brew that is located in /usr/local/bin, so the watch package may need to be relinked, or specify the absolute path to watch

To copy files using rsync and ssh across a network

rsync -avz \
-e "ssh -o StrictHostKeyChecking=no \
-o UserKnowHostsFile=/devnull" \
--progress [user]@[hostname.local]:/path/to/remote/file_or_directory /path/to/local/file_or_directory

The above command will copy a remote file or directory using ssh authorization and using rsync for the copying / transfer process, and copy a remote file from a remote box on the local network to the computer that the rsync command was issued from.

To do a simple file / directory transfer using rsync and have it display a progress bar for each file transferred

rsync -ah --progress /path/to/src/file_or_dir[/] /path/to/dest/file_or_dir

The above command is useful for displaying a progress bar when copying large files or directories.

rsync / Copying large files over a network or between filesystems

There are times were extremely large files need to be copied across filesystems and if the rsync command does not complete or fails for whatever reason it's useful to how to resume a failed rsync operation.

To start a rsync operation that can be resumed

rsync -ah --partial --info=progress2 /path/to/lrg/src/file /path/to/lrg/dest/file ;

echo "resume cmd";
rsync -ah --append-verify --info=progress2 /path/to/lrg/src/file /path/to/lrg/dest/file ;

❗️ If a rsync job is started in the foreground and not within a terminal multiplexer the job can be suspended and moved to the background, however if the job is disowned then there is not an easy way to associate the disowned job back with the $USER who started the job and the rsync command will more than like fail for reasons I do not yet know. All that said run rsync within a terminal multiplexer to avoid this nonsense.

Useful Links rsync πŸ”

Security πŸ”

security / working with drives, disks

one way to securely wipe a hard disk, disk drive, ssd, learn more
an additional useful link for preparing a disk for resale learn more, and one more

sudo badblocks -wsv -t 0x00 /dev/sdd

To calculate the SHA1 of a file

sha1sum /path/to/file.tar.bz

Working with find πŸ”

To find / search for particular information on a topic through the system man pages, ie. editor

apropos editor

apropos searches man pages reading the description for a particular program and then printing it to STDOUT within the terminal.

To find all regular files related to a particular search query

find / -iname "*silverlight*" -type f 2>/dev/null

The above command will perform a case insensitive search of all files on the local system with the text silverlight contained anywhere within a file name, and suppress any errors to STDOUT, ie. the dreaded permission denied errors and what not.

To delete all files within a particular folder / directory and preserve the folder / directory itself.

find . -path ['*/mr_fancy_pants_dir/*'] -delete

Example

find . -path '*/node_modules/*' -delete

To find all directories on a file system with a particular name

find / -type d -name "mr-fancy-dir" -ls

To find all directories on a local file system with the word .kext in the directory name

find / -type d -name "*.kext"

To find all binary / executable files along with shell scripts within a specified directory

find /opt/code -executable -type f

To set the executable bit for all binary files within a specified directory

find /opt/code -executable -type f -exec chmod a+x {} \;

VirtualBox stores its custom kernel extensions within,

/Library/Application Support/VirtualBox/

To replace all spaces ␣ with a hyphen - for files and directories

find -name "* *" -print0 | \
sort -rz | \
while read -d $'\0' f; 
  do mv -v "$f" "$(dirname "$f")/$(basename "${f// /_}")"; 
done;

The above command requires a POSIX shell

To change permissions of files and not folders recursively

find . -name '*.jpg' -type f | xargs chmod -x

To count the total number of files and directories within a path

find /path/to/directory -type f | wc -l

The above command will output the total count of the files and directories within the path specified. An alternative, but much more verbose solution is to use the tree command with the a flag. Personally, πŸ™‹β€β™‚οΈ I would not recommend this solution.

Useful Examples using find command πŸ”

To find all .DS_Store files on a file sytem without printing errors to STDERR

find / 2>/dev/null -type f -name ".DS_Store"

The above command will redirect all error output ie. Permission denied messages to /dev/null and NOT to the screen or console, thus making STDOUT more terse.

Useful Links find πŸ”

Working with grep πŸ”

To search for expression from STDOUT, ie. searching for nerd and plex from the output of fc-list on macOS

grep -iP '^(?=.*nerd)(?=.*plex)'

To invert the matching using grep

grep -v

To search for a pattern within all text files through a directory and sub directories.

grep -r [mr-fancy-search-pattern] .

To recursively search for a pattern in all text files, ie. source code files within the current directory

grep -rnw '/path/to/mr/fancy/dir' -e 'mr-fancy-pattern'
grep -rnw . -e 'mrs-fancy-pattern'

-r search recursively through sub directories -n print the line number for returned results -w match whole word

To search and print all markdown documents recursively for the word linux

find . -name "*.md" | xargs grep -i "linux"

grep Gotchas πŸ”

When searching for the phrase set showmode grep will not return any results because the search string will explicitly search for set showmode even though the config line within the .vimrc is set noshowmode

Working with GNU sed πŸ”

To iterate over all files in a directory and search for a pattern and remove all occurrences of the pattern within the files.

for i in *.rb
  sed -i -e 's/<<<<<<< HEAD//g' $i
end

The above command will remove all occurances of <<<<<<< HEAD within all the files defined in the for loop.

To verify the pattern has been erased

pt "<<<<<<< HEAD"

GNU sed > Useful Links

  • To see how I used GNU sed to process entries in my $PATH environment variable for fish shell, see

Working with GNU Core Utilities πŸ”

Working with GNU Core Utilties dd πŸ”

dd provided by GNU Core Utitlies can be used to backup an entire partition to a file if needed. For my particular use case I used dd provided by GNU Core Utilities on macOS installed via brew to install GNU Core Utilities which provides its variant of the dd command.

To backup an entire partition on macOS using dd

dd if=/dev/disk[NUM]s[PARTITION] of=./path/backedup.img status=progress bs=512 conv=noerror

The above command can take quite a bit of time if the partition is a large size, ie. > 100GB or more. Also, if using dd on macOS, Apple provides diskutil which is a CLI program for working with disks, partitions, and file systems.

To find the block size of a file system, which is useful for running the above dd command to backup a partition for a disk on macOS diskutil can be used.

diskutil info /dev/disk[NUM]

To create zero'd out file using dd which can be formatted using a file system such as ext2, ext3, ext4, or even HFS.

dd if=/dev/zero of=./path/to/zero.file bs=1024 count=1024000

The above command will create ~ 1GB file filled with zeros.

Working with GNU Core Utitilites split πŸ”

To split a large file on a system that has split installed which is like πŸ’ 99.9% all *nix systems.

split -b [SIZE_OF_SPLIT_FILE] "/path/to/large_file.ext" "/path/to/split_files.ext."

To join the split files back together

cat /path/to/split_files.ext.\* > /path/to/large_file.join.ext

To flash broken symlinks on macOS, GNU Core Utilities will need be installed, and setup for the shell environment.

Flash broken symlink with GNU Core Utilities

Flash broken symlink

brew install coreutils

Out of the box macOS supports $LSCOLORS

To configure BSD based $LSCOLORS online, see

Core Utilities uses a user directive for $LS_COLORS that is generated by running the dircolors command provided by Core Utilities.

Working with tree πŸ”

The tree command can be used to list a hierarchical structure of a directory(ies) and it's files.

The command I use to generate file structure for this git project / repo and the accompanying submodules, I run the below,

tree -a -I "tmux_resurrect_*|.git|undo|swap|target|tmp|*.pyc|*.vader|tests|.vscode|.netrwhist|*.weechatlog|*.gpg"

Working with du πŸ”

To display the to 10 largest files and directories within a given directory

\du -a $HOME | sort -n -r | head -n 10

Applying the -h flag jacks πŸ’ͺ shit up

To get a recursive size of a directory and all files and directories within that directory

du -sh /path/to/mr-fancy-dir

The above command should output a single line and not the typical du output shit.

Working with df

Working with df display free disk space on macOS

macOS can perform some heavy caching of files, ie. Spotlight.app indexes and what not, ie. local library caches for applications, ~/Library/Caches That said, rebuilding the spotlight index from time to time can free up a substantial amount of disk space. Also, note that df uses a different blocksize than does Finder.app and Disk Utility.app

Example

Finder.app

Finder.app-free-space

df -h

df

To compare the df -k to the free space calculated from Finder.app multiply the Available free space, ie. in my case 572163364 by 1024 the side of a 1K block which is what Finder.app Disk Utility.app appear to be using.

$$572163364 * 1024 = 585895284736$$

Convert 585895284736 to Gigabyte notation

$$585895284736 Γ· 1,073,741,824 = Gigabytes ~ 545.65GB$$
Understanding purgeable space on macOS

When Apple released macOS 10.13 and the APFS file system for High Sierra, APFS file systems take local snapshots of the local file system, ie. the internal disk that is running macOS. That said if a large file or directory is deleted df will not be able to reflect the new changes to file system because macOS can take a snapshot of the large deletion and supposedly recover the deleted files in a Recovery Mode. These local snapshots features are coupled with Time Machine.app ie. tmutil which allows several sub commands for working with localsnapshots. However, presently I can not figure out how to permenently disable local snapshots on a APFS volume.

For a visual understanding of what I am describing, click here

  • figure out where the 40GB of discrepency space is coming from

Working with the watch command πŸ”

To continously monitor a directory for file changes, ie. display the newest files at the top of the listing while continously listing results in real time.

watch -n 1 -d 'ls -lt /dev'

The above command is useful for monitoring when devices are connected / added to a system.

Legacy communication πŸ”

To write a message to another user on the system

write <user>
My friendly message 8)
EOF
Using mdfind

To search for all files in /opt/Code/dotfiles with a fish extension using one of the below search tools, and print an integer of all files contained within the search.

mdfind is a CLI utility that interfaces with Spotlight, ie. macOS search

mdfind -onlyin /opt/Code/dotfiles -name .fish -count

The above command can report false positives, ie. a file with the name of mr-fancy.fish=

Useful find commands

To change permissions for a type of file or directory recursively throughout a directory, ie. useful when working with rsync to backup directories.

find /Users/mr-fancy-42 -type d -exec chmod g+wrx {} +
find /Users/mr-fancy-42 -type f -exec chmod g+wr {} +

The above command will modify files and dirs for the user mr-fancy-42 to allow users in the same group to read, write, and execute files and dirs within mr-fancy-42's $HOME dir.

Using BSD locate πŸ” 
sudo /usr/libexec/locate.updatedb
locate '/opt/Code/dotfiles/**.fish' | wc -l

To update the native locate database on macOS

sudo /usr/libexec/locate.updatedb

Using GNU locate πŸ” 
/usr/local/bin/locate --version

GNU locate allows specifying a path to a database file that can be queried from the CLI, also GNU locate supports using the $LOCATE_PATH env var. However, the macOS version of BSD locate DB, ie. /var/db/locate.database is NOT compatible with the version of GNU locate installed by brew.

To create or update the locate database using updatedb provided by GNU locate

sudo /usr/local/bin/updatedb --localpaths='/' --output='/path/to/gnu-locate-database-file'

The database file will be created by the super user on the system,

To search for all files with a fish extension using GNU locate

/usr/local/bin/locate -c '/opt/Code/dotfiles/*ME.md' --database=/path/to/gnu-locate-database-file

Working with X11 on macOS

To print a list of available resolutions supported by X11

xrandr

To print the dots per inch, ie. DPI

xdpyinfo | grep -i resolution

To print the current keymap table

xmodmap -pke

To print the current keyboard layout configuration

setxkbmap -query

Working with Networking

To print a process ID related to a networking port

lsof -n -i :[PORT_NUM] | grep LISTEN

HOSTS files

Most if not all UNIX derivatives, ie. Linux, BSD's macOS support editing a hosts file, even Microsoft Windows supports a hosts file.

The hosts file on macOS is

/etc/hosts

and entries can be added to quickly navigate to a particular host on a intranet / LAN for experimenting with different services, ie. a web server and what not. A sample entry in /etc/hosts would look like the following.

10.0.1.42 the-meaning-of-life.exp # Local IP address domain name

Microsft Windows based OSes have hosts file at

C:\WINDOWS\SYSTEM32\DRIVERS\ETC\HOSTS

Working with OpenSSH

To suppress verbose output when logging into a OpenSSH server that is running on a GNU+Linux box, add a .hushlogin file in the $USER $HOME directory.

Troubleshooting Networking related issues

A fix for the below error message

sudo: unable to resolve host [NAME_OF_HOST]

There is no entry in the /etc/hosts file that corresponds to the name in /etc/hostname. see for more info.

Working with man and man pages

To print a list of paths that will be searched for man pages

manpath

Working with man pages on macOS

To print a list of directories that can be searched for man pages

man -d man

To print the directory where man would look for the MANPATH for a binary

man -d

My man page breakthrough

On macOS if binaries, ie. rvm, who, and mv etc etc, are contained within a bin directory, then if a sibling level man directory exists macOS will be able to find the accompanying man page that corresponds to the command.

./
bin/
man/

The parent paths aren't important, and editing of system level files contained with the /etc is not required.

Working with GNU Coreutils on macOS

If GNU Coreutils are installed with default names on macOS the binaries are located within

$brew_prefix/Cellar/coreutils/[MAJOR.MINOR]/libexec/gnubin
$brew_prefix/Cellar/coreutils/[MAJOR.MINOR]/libexec/gnuman

Copy the gnubin and gnuman to bin and man within the same libexec directory.

cd $brew_prefix/Cellar/coreutils/[MAJOR.MINOR]/libexec
ln -sf ./gnubin ./bin
ln -sf ./gnuman ./man

After the above directories have been copied the man command should be able to read the accomponying man pages for the commands.

Working with compressed files

To extract a password protected file

unzip -p 4242 /path/to/file.zip

If the below error occurs when trying to extract a zip file

unsupported compression method 99
7z x -p 4242 /path/to/file.zip

Linux πŸ”

Working with dkms

To print a list of added dkms modules in the various versions of linux kernels

sudo dkms status

❗️ there is a difference between a addedd and installed kernel module.

To remove a dkms kernel module for all kernels present on a system

dkms remove <module>/<module-version> --all

The --all flag should remove all versions of the specified module for all kernels on the system.

General Commands

To print the UUID for a hard disk or a partition of a hard disk

blkid

To print the major & minor release version of a Debian system

lsb_release -a

To mount a partition with an ext2 filesytem

mount -t ext2 /dev/sda1 /opt

To change the hostname of the system

hostnamectl set-hostname mr-fancy-pants-hostname-here

To extract a directory of rar files

find . -name "*.rar" -exec unrar x -o+ {} \;

Resizing a partition for a QEMU virtual disk

If a virtual hard disk for a QEMU VM needs to be increased then fdisk can be used to increase the partition after qemu-img has been used to increase the size of the virtual disk.

To increase the size a virtual qcow disk

qemu-img resize /path/to/virtual-disk.qcow2 10G # resize disk to 10G

To add 10 Gigabytes to a virtual disk

qemu-img resize /path/to/virtual-disk.qcow2 +10G # append 10G to virtual disk

Working with Users and Groups πŸ”

To change the login password for a user on a GNU+Linux system

sudo passwd $USER
sudo passwd mr-fancy

To change the UID for a user on a GNU+Linux system

usermod -u [NEW_UID] mr-fancy

❗️mr-fancy can not change his own UID and no processes can be associated with mr-fancy. If mr-fancy has associated processes run the below commands

pkill -u mr-fancy pid
pkill -9 -u mr-fancy

The same procedure is applicable for changing the GID of mr-fancy as well

groupmod -g [NEW_GID] mr-fancy

❗️ Be certain to update old files and dirs that were associated with the GID see

To print the GID for a particular group name ie. staff

getent group staff

To print the GID of a group name of macOS

dscl . -read /Group/staff | awk '($1 == "PrimaryGroupID:") { print $2 }'

Raspbian / Working with sudo

To set an empty / blank password for the user pi on Raspbian

  1. Make certain the pi user has sudo permissions.

❗️ The pi may require setting the default editor to run visudo otherwise the default editor will be set to nano

To set the default editor for visudo

sudo update-alternatives --config editor
  1. Add the below line to /etc/sudoers 
pi ALL=(ALL) NOPASSWD:ALL
  1. Delete the current password for the pi user
sudo passwd -d `whoami`

After the above settings have been made the pi user should be able to login without being prompted for a password.

Working with tty / pts from a CLI

A quick way to list tty's / pts's on a GNU+Linux distro

w

❗️ On macOS listing all active terminal sessions is a little different, and the w command only shows the initial console session when running w. However, a crude count can be obtained by listing all /dev/ttysXXX* devices within the dev directory. To print the active pseudo terminal / tty for a shell

tty

To kill a different tty from an existing tty session

pkill -9 -t pts/[NUMBER]
ps -ft pts/[NUMBER] pts/42 pts/24
kill [PID] [PID]

To list the default options that will be applied when creating a new user on the system

useradd -D

To delete a user from the system, for more info see

userdel mr_fancy_user

To create a new user with a specified shell, home directory, and user name

see

useradd -s /bin/bash -m -d /home/vanilla vanilla

To print a users home directory, ie. /home/mr-fancy-42-user

grep mr-fancy-42-user /etc/passwd | cut -d ":" -f6

Setting new files in a directory to belong to a particular group πŸ”

  1. Set the directory to have a umask of 007
umask 007

The above command will make all newly created files to

u=rwx,g=rwx,o=

To verify the above setting

umask -S

To remove the setuid ie. sticky bits from a directory

chmod ug-s /path/to/dir
echo "or try the below"
chmod 0755 /path/to/dir

To print all users on a Linux box

cut -d: -f1 /etc/passwd

To print all groups on a Linux system

cut -d: -f1 /etc/group

To print user and groups values of a particular user

id [user_name]

To print the group memberships for a particular user

groups [mr_fancy_user_name]

To create a new group on a Linux system

groupadd [mr-fancy-42-group]

To add an existing Linux system user to an existing group

usermod -a -G [name_of_group] [name_of_user]

If adding a $USER to a group on a remote system through ssh more than likely the $USER will have to logout and then log back in after adding the $USER to a group.

Ex

usermod -a -G prettyladies diana

Working with disks block devices and filesystems on Linux πŸ”

To list all the partitions for block devices, ie. disks on a Linux system

lsblk
echo "for more a descriptive listing"
lsblk -f

To create, modify, delete a partition on Linux system, gparted is a popular command

gparted only works with devices and volumes, ie disks and partitions.

To display the free space on a disk using gparted

gparted /dev/disk
(parted) print free

To display the partition labels for a particular device, ie. disk

e2label /dev/disk

To display a partition label for a particular volume / partition

e2label /dev/disk[Partiton#]

To set a label for a partition which can referenced ie in a /etc/fstab file for auto mounting a file system when present.

e2label /dev/disk[Parttion#] [mr-fancy-partition-label]

To remove a partition label

e2label /dev/disk[Parttion#] ""

To create FAT-32 file system on a removable USB device on the 3rd partition / volume of the device

mkdosfs -F 32 -I /dev/sda3

/dev/sda is dependent on the disk path!

To remove a removable USB disk from a Linux system, make sure all partitions have been unmounted, then use udisksctl

  • keywords power, off, turn, poweroff, turnoff, USB, powerdown, down
udiskctl power-off -b /dev/sd[A,B,C]

[A,B,C] refers to the disk that one would like to unmount.

To safely remove a USB mass storage, 1) unmount the disks, 2) then power down the drive

sudo udisks --unmount /dev/sd[???];
echo "the below command will power down the drive";
sudo udisks --detach /dev/[sd[???];

Working with autofs

To unmount all filesystems that have been mounted with autofs

umount -a -t autofs

Working with udev πŸ”

For an exhaustive guide to working with udev, see

To print the currently installed version of udev

udevadm version

To setup a custom network interface name using udev for a standard interface name, ie. eth0 or wlan0 create a 70-persistent-net.rules within /etc/udev/rules.d

SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="01:23:45:ab:cd:ef", NAME="eth-pi"

SELinux πŸ”

Stretch enabled support for SELinux on Debian, but is disabled on a default install.

To check whether SELinux is enabled / disabled on Stretch

grep FSC /etc/default/rcS

If the output from the above command returns

FSCKFIX=no

then SELinux has not been enabled on the box.

Another way to check if SELinux is enabled / disabled on a Debian 9.x box.

sestatus

Unsorted

Working with X11 > display managers

To list / show the current setup / active display manager on Debian

cat /etc/X11/default-display-manager

To display the setup active resolution for the display manager on Debian

xdpyinfo | grep -B 2 resolution

Debian πŸ”

Setting up backports on Debian Stretch πŸ”

To setup Debian Stretch 9.x to work with a backports-repo

  1. Edit /etc/apt/sources.list file or add a .list file, ie. a debian-strech-backports.list file within the /etc/apt/sources.list.d/ directory and add the below entry.
deb http://ftp.debian.org/debian stretch-backports main

To install a package from a backports repo

apt-get update
apt-get -t stretch-backports install [mr-fancy-backport-package]

For more info on working with backports see

Debian > Working with apt & apt-get

To generate a list of all packages installed with apt / apt-get

dpkg --get-selection

Debian 9.x and above provides apt-mark

To generate a list of all manually installed packages by a user, and not their dependencies

apt-mark showmanual > [mr-fancy-42-package.list.txt]

To reinstall all the manually installed packages provided by apt / apt-get

xargs < [mr-fancy-42-package.list.txt] apt-get install

To print a list of files associated with a package

dpkg -L [mr-fancy-42-package]

To show package records for a particular package

apt-cache show postgresql

To remove a specific version from the list

apt-get remove postgresql=9.6+181+deb9u1

To list all packages related to particular string, ie. [postgres]

dpkg --get-selections | grep postgres

To display the package version number of all packages currently installed on the system

dpkg-query -l

To find the package that provides a particular ".h" file

dpkg -S <name_of_h_file.h>

Example,

dpkg -S stdio.h

Output

libc6-dev: /usr/include/stdio.h
libc6-dev: /usr/include/bits/stdio.h
perl: /usr/lib/perl/5.6.0/CORE/nostdio.h

Adding a key signature to apt-get, see

To remove artifacts from apt-get installed packages

rm -rf /var/lib/apt/lists/*

then rerun the below command.

apt-get update

To see what packages can be upgraded

apt-get upgrade --dry-run

To search for a particular package from the apt-get sources

apt-cache search [mr-fancy-apt-package]

To show or list the dependencies a package requires using apt

apt-cache depends [mr-fancy-package]

ie.

apt-cache depends xsel

or

apt-cache depends xclip

Running apt-get update on Debian 9.x was giving an expired an key error.

See the following issue about getting the updated key in order to resolve the expired key issue for OpenSuse server, ie. it is related to fish shell.

To reinstall an app on Debian using apt-get

apt-get install --reinstall postgresql

To download the source for a particular package that would installed using apt-get command

apt-get source [packagename]

To pin / hold back a package from being upgraded via apt or apt-get

sudo apt-mark hold [PACKAGE_NAME]

To unpin / unhold a package

sudo apt-mark unhold [PACKAGE_NAME]

To add a path for every user on the system edit /etc/login.defs add the respected path entries for normal users, and the root user.

Troubleshooting apt on Debian
Debian > apt > troubleshooting partial install

Error message

apt 2 not fully installed or removed.

To list the partially installed or removed packages

dpkg -C

Then remove or reinstall the offending apt packages

apt install [OFFENDING_PKG]
apt install --force-yes [OFFENDING_PKG]
apt remove [OFFENDING_PKG]
apt remove --force-yes [OFFENDING_PKG]

From my empirical evidence apt performs without errors if there is only a single version of python in the PATH, ie. do not have pyenv to use both a 2.7.x and a 3.{6,7}.x variant of python with the below command

pyenv global system 3.6.5 2.7.15

Instead use

pyenv global system 3.6.5

Debian date, time, timezone πŸ”

The quickest way to setup the proper time + date, and correct timezone is to install ntp if a network connection is present.

apt-get install ntp

To configure the Debian system for a specific timezone, ie. US Centeral TZ

sudo dpkg-reconfigure tzdata

Follow the curses based menus for setting up tz data.

Debian Working with KVM πŸ”

Debian Stretch, ie. 9.x πŸ”

Setting up SELinux on Debian πŸ” 

apt-get install selinux-basics selinux-policy-default auditd

  • Download the _load_selinux_policy script.

  • Move the script into /usr/share/initramfs-tools/scripts/init-bottom

  • Run update-initramfs -u

  • Run selinux-activate to configure GRUB and PAM to work with SELinux

    1. Create /.autorelabel

  • Reboot the system.

    1. Run check-selinux-installation to see the errors / warnings about SELinux

  • Finally add enforcing=1 to /etc/default/grub

  • Reboot

Debian packaged kernels provide support for SELinux out the box.

SELinux Useful Links πŸ”

Working with apt πŸ”

To remove / purge old packages .deb files on a Debian system

sudo aptitude clean

Research missing packages with apt πŸ”

To get a list of files a package can provide install apt-file, generate a DB for apt-file and then query apt-file with search strings. Also, apt-file can take a list argument to see what files a package provides, ie.

apt-file list libglib2.0-dev | grep '.m4$'
sudo apt-get update; \
sudo apt-get sudo apt-get install apt-file \
sudo apt-file update
apt-file search [string]

Credit

Working with source code using apt-get πŸ”

Source code for Debian packages installed via apt-get is currently being maintained on salsa.debian.org

An account will need me made on the GitLab clone thingy, ie. server specifically designed for Debian packages, and then the apropriate package can be cloned to a local box.

To clone a package from salsa.debian.org

  1. sudo apt-get install git-buildpackage
  2. gbp clone git@salsa.debian.org:[mr-fancy-guest]/[mr-fancy-42-package].git

To build a debian package from source using a git clone

cd [mr-fancy-42-package]/
gbp buildpackage --git-pbuilder

Pbuilder Howto πŸ”

Working with source using apt-get Useful Links πŸ”

Troubleshooting apt πŸ”

If apt-get is having difficulty finding scripts to run, ie. /usr/sbin/update-info-dir make certain that /usr/sbin is in the current $USER's $PATH

Upgrading from Jessie to Stretch πŸ”

Make sure all Jessie updates have been applied to the system first. To make sure no updates need to be applied

apt-get update upgrade --dry-run

Then run the two below commands to make sure there are no issues pending

dpkg --audit
dpkg --get-selection | grep hold

Services such as postgres will need to be halted when performing a system upgrade because tools / libs ie, lib-c require them to be halted during the upgrade process.

To stop the postgres service on Debian jessie

/etc/init.d/postgresql-[version] stop

If postgres has been setup to work with systemd then the postgres server can be halted with the below command.

systemctl stop postgres

Then edit sources.list

nvim /etc/apt/sources.list

Remove or comment out source entries related to jessie and add the below entries related for stretch

deb http://mirrors.digitalocean.com/debian stretch main
deb-src http://mirrors.digitalocean.com/debian stretch main

deb http://security.debian.org/ stretch/updates main
deb-src http://security.debian.org/ stretch/updates main

deb http://mirrors.digitalocean.com/debian stretch-updates main
deb-src http://mirrors.digitalocean.com/debian stretch-updates main

Then run the below commands again

apt-get update
apt-get upgrade

Then run the below command

apt-get dist-upgrade

If the upgrade process halts for whatever reason it can be resumed using the below command

apt-get -f dist-upgrade

After everything has been installed, and the system has been rebooted, the old kernels can be purged.

dpkg --list | grep linux-image
aptitude purge ~ilinux-image-\[0-9\]\(\!`uname -r`\)

To purge / remove unneeded packages from the system

apt-get remove --purge

Working with cron πŸ”

To test the various cron tasks on a Debian system

run-parts /etc/cron.daily
run-parts /etc/cron.weekly
run-parts /etc/cron.monthly

To arbitrarily run a cron job with the bare metal cron setup see this

To display the particular cron jobs for a user on a Debian box.

crontab -l

The file printed from the above command is stored in the following in the following location on the system.

/var/spool/cron/cron/crontabs/

When running a cron job / shell script make sure the script does not contain any commands being run with sudo. Long story short, a password will have to be stored in plain text somewhere on the system. 😬 When using env vars in a shell script run by cron make sure the env vars are set within the script itself.

See this for more details.

Working with pam.d πŸ”

To restart the pam.d service

/usr/sbin/pam-auth-update

Working with locales πŸ”

Update February 17 2019

The below command resolved locale issues I was having after upgrading locales on Debian Stretch see for more info.

sudo sh -c 'echo en_US.UTF-8 UTF-8 >>/etc/locale.gen' && sudo /usr/sbin/locale-gen

To list the locales on a system

locale -a

To configure locales on a Debian based distro, ie. Raspbian

dpkg-reconfigure locales

Follow the instructions in the curses based menu program. ⚠ dpkg-reconfigure locales must be run as root and not used via sudo

🚨 When generating locales using locale-gen run as root do not use sudo if you value your sanity πŸ‘©β€βš•οΈ

To remove erronous locales from a system edit /etc/locale.gen and comment out the unnecessary locales.

The following link appears to have resolved the locale / perl warnings

To fix a broken locale system on a Debian box, see

To remove unused / extra locales on a Debian box

sudo apt-get install localepurge

Unselect all the locales that aren't being used.

When changing locale for a particular user, sometimes reloading the shell, ie. exec bash will not pick up on the locale change, so it's important to "logout" and "log" back in.

Useful Commands for Working with locales πŸ” 

apt-get install localepurge
bass . /etc/default/locale
dpkg-reconfigure localepurge

Postgresql πŸ”

General Notes about working with PostgreSQL πŸ”

To check the server version of PostgreSQL if it has been installed

postgres -V

To check the installed version the PostgreSQL client

psql -V

TL;DR

To restart PostgreSQL on Debian 8.x & 9.x

/usr/lib/postgresql/10/bin/pg_ctl -D /path/to/where/DBs/are/located -l /path/to/server/log/file.log start

pg_ctl can be located in different locations depending on how PostgreSQL was installed.

For my particular Digtal Ocean droplet the below command works πŸ‘

/usr/lib/postgresql/10/bin/pg_ctl -D /var/lib/postgresql/10_3_1 -l /usr/local/var/postgres/server.log start

The above command should be run as the postgres user on the system, DO NOT USE sudo or root to start the daemon unless you like headaches πŸ€• If the postgresql.service is being used in conjuction with PostgreSQL then the postgres can be controlled via Systemd

systemctl start postgresql.service

To check and see if postgresql server is running on the system

ps -ef | grep postgre

To reload postgresql

/etc/init.d/postgresql reload

To create a user for Postgres

createdb [mr-fancy-42-user]

The above command helps alleviate the common below error message

psql: FATAL: database [user] does not exist

Postgres Backing up databases πŸ”

To implement a backup solution for postgres, see

To setup a cron job to run the backup scripts daily copy or symlink pg_back* in the Linux dir within the jobs dir.

To test a daily cron job on Debain Linux

run-parts -v /etc/cron.daily -v

To back a Postgresql database, see for more details.

pg_dump [name_of_database] > name_of_backup_file.sql

Postgres Restoring a database πŸ”

As of June 30 2018 I have a couple of shell scripts running to automate the process of taking daily snapshots of my Postgres databases. That said the scripts produce two back up files, ie. a .gz and a .custom

To restore a Postgres database from a snapshot using the .custom file, use the following,

pg_restore --dbname=mr-fancy-42-db --verbose --host=localhost --username=mr-fancy-username --password ~/path/to/mr-fancy-42-db.custom

When restoring a DB that works with a rails web application, make sure to restore the DB, before running the migrations, and then run the migrations after restoring the database. πŸ‘Œ

If the schema.rb file is affected from importing data, and migrations are rerun, more than likely puma will need to be restarted after intitial deploy with new data in the database.

Working with Postgres on Debian πŸ”

pg_ctlcluster supersedes pg_ctl and in practice pg_ctlcluster should be used in favor of using pg_ctl when possible. That said, pg_ctl can be found at the below location

/usr/lib/postgresql/[MAJOR.MINOR]/bin/pg_ctl

The PostgreSQL configuration files, ie. pg_hba.con are located in the below location

/var/lib/postgresql/10_3_1/

The postgres binaries, ie. commands are located in the below location

/usr/lib/postgresql/10/bin

psql is located in both /usr/bin/psql and /usr/lib/postgresql/10/bin

Getting postgres to work with systemd

postgresql@.service acts a template for setting up postgres to work with systemd.

The units file that interacts with Postgres server is,

/lib/systemd/system/postgresql.service

As of March 21, 2018, all database records / data are stored in the below location

/var/lib/postgresql/10_3_1

Upgrading Postgres from 10.3.x to 10.4.x πŸ”

  1. Installed Postgresql 10.4.x using apt-get
  2. Dumped all existing data from 10.3.x
pg_dumpall > backup-may-30-2018
  1. Stop the current database server, ie. Postgresql 10.3.x
pg_ctl stop

if using systemd

systemctl stop postgresql
  1. Create a new database cluster 
initdb -D /var/lib/postgresql/10_4_1
  1. Start the new database server
postgres -D /var/lib/postgresql/10_4_1

systemd

systemctl start postgresql
  1. Restore the data from 10_3_1
psql -d postgres -f m backup-may-30-2018

For more detailed instructions see

Working with psql πŸ”

To connect to a particular database using psql

postgres=# \connect database_name

To list all the databases

postgres=# \list

To list all tables for a particular database

postgres=# \dt

To change the owner of a particular database

postgres=# ALTER DATABASE [mr_fancy_database] OWNER TO [mr_fancy_owner_name];

PostgreSQL Useful Links πŸ”

Working with PHP πŸ”

To restart the PHP service / process

/etc/init.d/php5-fpm restart

To find the the local user on the system associated with the PHP service / process

Create a php file with the following snippet

<?php
phpinfo();
?>

Then look for a heading of Environment in the rendered output from the snippet.

Nginx πŸ”

To restart nginx web server

service nginx restart

The above command won't work on Debian 9.4 out of the box because it does not contain the service binary, instead use systemd, ie. systemctl

Working with MySQL πŸ”

To install MySQL on a Debian based distro

apt-get install mysql-server

To complete the MySQL installation

mysql_secure_installation

To access the MySQL shell as the root user

mysql -u root -p

To list local MySQL databases on the system

show databases;

To show local permissions a particular user has on a system πŸ“¦

show grants for 'database_user'@'localhost';

Sometimes the below command will need to be run after making permission changes

flush privileges;

Useful Links MySQL πŸ”

MediaWiki πŸ”

Make sure the proper user ie, the php user associated with the system has write permissions to the images directory

To see if Google Analytics is installed within mediawiki

Special:Version

Useful Links MediaWiki

Working with SSH πŸ”

To properly set permissions of SSH directories and files, see πŸ™ˆ

Some useful settings for setting up SSH permissions

  1. $HOME/.ssh should be set to 700
  2. $HOME/.ssh/[encryption_type].pub should be set to 644
  3. $HOME/.ssh/[encryption_type] ie. id_rsa the private key should be set to 600
  4. The users $HOME directory should not have a group write permission, and should be set to 755

A useful command for debugging an SSH session, ie. debugging X11 Forwarding

ssh -v -X -l [mr_fancy_user_name] -p [mr_fancy_port] [hostname]

To suppress the last login from X.X.X.X IP address

  1. Edit /path/to/sshd_config

Add the below line to the file

PrintLastLog no

To suppress the connection from IP closed

  1. Edit $HOME/.ssh/config

Add the below setting

LogLevel QUIET

Working with systemd πŸ”

To restart a Systemd service

sudo systemctl restart [NAME_OF_SERVICE]

Ex

sudo systemctl restart sshd

systemd History πŸ”

Systemd was introduced by RedHat to replace / compliment SYSV for Linux distributions. Traditionally SYSV used syslogd or rsyslogd to log errors and output to log files for services and daemons running on the system. However systemd provides own logging facility which interacts for special journal files.

Systemd Working with journaling πŸ”

To control the journaling settings for systemd edit the below

/etc/systemd/journald.conf

To get journaling to work properly on Debian 9.x I had to explicity set compression for the journal files to off

To allow a standard user to access and manipulate log files controlled by the journaling capabilities provided by systemd, perform the below

  1. Add the standard user, ie. whoami to the systemd-journal group.

See notes above, about working with users and groups, if needed.

  1. Make sure the path to the journal directory and all sub folders and files are writeable by the systemd-journal group, ie. 775 for /var/log/journal

To rotate logs generated by systemd, one can use the --vacuum-time=2weeks flag / option to the journalctl command to trim / shrink the size of log files.

On Debian 9.x systemd stores system services in the below location

/lib/systemd/system/[mr-fancy-system.service]

systemd also stores $USER services in the below location

/usr/lib/systemd/user/[mr-fancy-user.service]

To start a systemd service for a particular $USER

systemctl --user start [service-name]

The full path and the .service extension can be omitted if the unit file, (ie. that is what systemd calls the .service files.) is either of the above mentioned paths.

To add a additionaly systemd unit file the preferred location to add a unit file is within the /etc/systemd/system directory on Debian.

The unit files located in the /etc/systemd/system will take precedence over the ones that are located in /lib/systemd/system. Also, if there is a already a Debian provided systemd unit file in /lib/systemd/system the user provided will be chosen if located in the /etc/systemd/system directory. πŸ‘

To install libpam-systemd

apt-get install libpam-systemd

See, πŸ™ˆ for more details

To list all enabled services from systemctl

systemctl list-unit-files | grep enabled

To analyze system boot time, ie. measure startup time of GNU+Linux using systemd performance

systemd-analyze

When modifying a systemd .service file, the below command will need to be executed in order for systemd to pick up on the changes.

systemctl daemon-reload

Troubleshooting systemd

To remove / delete old systemd log files

sudo journalctl --flush --rotate
sudo journalctl --vacuum-time=2d

The above command will remove all non active logs, and delete all active logs that 2 days or older.

Working with man pages πŸ”

man pages are divided into various groups, ie. man pages about using a specific command, ie. uname will be located within Section 1 where as getting a understanding of how the programming works for the uname command will be located within Section 2

man page directory layout πŸ”

Directory Section Name
man1 (1) User Commands
man2 (2) System Calls
man3 (3) Subroutines
man4 (4) Devices
man5 (5) File Formats
man6 (6) Games
man7 (7) Miscellaneous
man8 (8) Sys. Administration
manl (l) Local
mann (n) New
mano (o) Old

When working with man pages on macOS user added CLI commands can store their man page in /usr/local/share/man/man1. The man1 directory is reserved for user added CLI commands, but make certain the man page has a .1 or a .1.gz unless you don't want to read πŸ“– the man page ΰ² _ΰ² 

Useful Links man pages πŸ”

Working with ld πŸ”

To print the ld search path, see πŸ™ˆ

All Useful Links πŸ”

If you find any of this info helpful on your journey 🏍 click that πŸ‘† ⭐️ star button. It sure makes me feel warm and fuzzy 🐻 on the inside.

Frequent Flyers ✈️

❀️ ~/.πŸ› πŸˆ

Clone this wiki locally