TABLE OF CONTENTS tips + tricks - the bell trick - write to file from cmdline - 8-bit color tables bash + scripting - arrays - misc linux things - stuff to add boot - rd.break - custom install ISO - rebuild initramfs daemons + services - crontab - atq - systemd - auditing find + edit - find - sed auth - polkit - authconfig - PAM - NIS - FreeIPA - OpenLDAP storage - partition + format - SMART data - badblocks - LVM - NFS config - dconf packages - yum hardware + virtualization - hw info - drivers - KVM/libvirt/QEMU network - basics (ip a, nmtui, etc etc) - ssh - vnc - firewall-cmd - iptables wget ansible - config files - oneoffs - playbooks and vars security - selinux - OpenSCAP COOL SOFTWARE FOR RESCUE DISK smartmontools gsmartcontrol meld wget tree pstree htop iftop that thing like ip -a but older gnuplot nmtui THE BELL TRICK ssh while :; do echo -e '\a'; done # can also do with any soundfile, apparently locate -r '\.wav$' | xargs -n1 /usr/bin/play ALEX'S 8-BIT TERMINAL COLOR TABLES # thanks Alex for coding this! i=0; cols=54; for c in 48\;5\;{231..016}; do echo -ne "\033[${c}m${c}\033[00m"; if [[ "$(($cols - ( $i +9 ) ))" -lt 9 ]]; then echo; i=0; else echo -n '|'; let i+=9; fi; done; echo ARRAYS mapfile -t < <() # output of cmd becomes array, newline-separated(?) # do smth to each array item for i in ${arrayName[@]}; do ... done QUICKLY WRITE TO A FILE FROM THE COMMAND LINE cat << EOF > ``` EOF ``` SOFTLINKS AND HARDLINKS # create softlink ls -s # create hardlink ln # target must be file, no directories allowed # view inode count (increases with number of hardlinks) ls -i SET TIME AND DATE timedatectl # view a status page list-timezones # UTC is at end, if available on system set-timezone # set timezone TARBALLS tar -C -cvf # create a tarball -j # use bzip2 -z # use gzip tar -C -xvf # untar a tarball tar -tvf # list files in tarball SED # basic syntax sed -i # delete matching lines sed -i '// d' # convert linebreaks to spaces sed -z 's/\n/ /g' # select and comment out a range of lines = '/BBB/,$ s/^\(\s*[^#].*\)\?$/#\ \1/' /BBB/,$ # select range from string BBB to EOF s/// # within range, search and replace matches lines with: ^ # begins with \(\) # define group \s* # whitespace character, any number of times [^#].* # any character not '#', any number of times \? # unsure; match one or zero times?? comment blank lines too $ # ends with replaces matched lines with: #\ # a literal # character and then a space \1 # contents of previously defined group # locate + replace lines containing = '// c ' c # within range, replace lines with # whitespace between 'c' and '' is ignored NB: if range is set numerically or by file location markers such as $, will replace the entire range with one instance of NB: if range is set by a search, will replace every matching line (or set of lines) with one instance of # append string to end of file = '$ a ' $ # select end of file a # append on newline after each match # ciphers! = 'y/src/dst/' swap characters in "src" with corresponding chars in "dst": - s → d - r → s - c → t can make cool ciphers!! FIND AND LOCATE updatedb locate find where = -name '' # search for -iname '' # search case-insensitive -type d # find directories -mtime -7 # find files modded in the last 7 days -path '<**/system/**/repos/*.ext>' # find stuff matching the pattern -not-path '<*/site-packages/*>' # exclude results with site-packages -maxdepth X # 0 = search given startpoints only 1 = search contents of startpoint dir 2 = search subdirs too [...] = etc -mindepth X mandb # installs missing man pages or just updates database? man -k caiser@max:/test$ pwd /test caiser@max:/test$ find . -maxdepth 3 -path '*/4' -prune -o -name '*' . ./1 ./4 ./2 ./3 ./64 caiser@max:/test$ find . -maxdepth 3 -path '*' -prune -o -name '*' . caiser@max:/test$ tree . ├── 1 ├── 2 ├── 3 ├── 4 │   └── 5 └── 64 1 directory, 5 files INPUT-OUTPUT REDIRECTION cmd > file # redirect STDOUT to file cmd >> file # append STDOUT to file cmd | cmd2 # redirect STDOUT of cmd to STDIN of cmd2 cmd 2> file # redirect STDERR to file cmd > file 2>&1 # redirect STDOUT to file, and redirect STDERR to STDOUT's current target cmd < file # read STDIN from contents of file cmd | tee file # pipe STDOUT to tee, which writes it to file, and also writes it to the screen cmd 2> /dev/null # throws away STDERR STUFF TO ADD netstat -i # show network card activities -a # display socket activities nice -n <-20..19> renice +/- tuned # is a thing that exists tuned-adm # tool to adjust tuned config and apply and run tuned stuff # to preserve system journals, edit the journald config vi /etc/systemd/journald.conf # if you forget this filename, look in here to re-find man journalctl # who's using that file? fuser # why is bootup taking so long? systemd-analyze blame # to connect to an NFS share: yum install nfs-utils vi /etc/fstab # the _netdev option is good, stops some errors mount -a mount # to check mounts # check version of RHCSA machine!! cat /etc/redhat-release and snapshot often!!! when mount ext4 in fstab, end with 0 2 not 0 0 because disables fsck!! mount via fstab and ignore fail on error _netdev wont do that defaults,nofail is good put defaults first and adjustment after, because gets overridden by subsequent options! defaults unneeded if you have anything else to put in that column? only useful to avoid empty column and confusing whitespace-parser both filesystems and block devices have a UUID lsblk tends to show the filesystem's lvdisplay shows the lv block device's to mount it in fstab, you need the filesystem's for ldap, nis, auth etc: - know services to allow thru firewalld will have to try hard to remember this - know stuff to start/enable in systemctl mount nfs/cifs like mounting other stuff, but extraoptions, and optional config file for user/pass change away from default ntp server chrony config file reload after? check with timedatectl ssh config for key-based login ssh-keygen ssh-copy-id ...something else in config files? the user or server config? where are they each located? doublecheck grub output locations for bios and uefi and how to tell if you're booting bios or uefi ...just reread notes RD.BREAK reboot press "e" to edit GRUB add at end of kernel boot params: rd.break rw enforcing=0 # no need to rm other bits of line # if no "rw": mount -o remount,rw # if no "enforcing=0": touch /.autorelabel, no restorecon -- relabels entire filesystem, takes a long time Ctrl+X to boot chroot /sysroot passwd root # or whatever else you need to do Ctrl+D twice to exit and resume boot login restorecon -rv /etc # fix SELinux contexts damaged by passwd change setenforce what is rd.break? during normal boot: - machine loads rd.break env (initial ramdisk image, initrd) into RAM - mounts normal system disk at /sysroot - continues booting from there - therefore when you change boot settings you often have to regen the initrd during rd.break: - stop process in middle to access normal system disk without booting to it why SELinux relabel is required? - initrd is minimal, no SELinux active/started! (unless you use kernel boot param "enforcing=1" to specifically start it) - therefore, when files are edited, the security context is not updated! - if SELinux is normally enforcing, bad /etc/shadow security contexts will then cause login issues - therefore, fix security contexts before reenabling SELinux or rebooting EDIT GRUB cp /etc/default/grub /etc/default/grub.bak vi /etc/default/grub ``` [...] GRUB_CMDLINE_LINUX_DEFAULT="" GRUB_CMDLINE_LINUX="" [...] ``` # for Ubuntu update-grub # for RHEL grub2-mkconfig -o /boot/grub2/grub.cfg # for BIOS -o /boot/efi/EFI/redhat/grub.cfg # for UEFI reboot # check which bootparams used in last boot cat /proc/cmdline useful bootparams: quiet splash rd.break rw enforcing=0 intel_iommu=on pcie_acs_override=downstream systemd.unit= REBUILD INITRAMFS # rebuild initramfs the easy way?: dracut -f -v # or perl script? that gets run when installing kernel pkgs? see note from Alex on checking yum package scripts GENERATE CUSTOM KICKSTART ISO download CentOS 7 iso mount iso cp -pr mountdir workdir cd workdir edit files in workdir: - EFI/BOOT/grub.cfg ``` menuentry <...> linuxefi /images/pxeboot/vmlinuz inst.stage2 <...> quiet ks=cdrom:/ks.cfg initrdefi <...> ``` - isolinux/isolinux.cfg ``` label menu label ^ kernel vmlinuz menu default append initrd=initrd.img ks=cdrom:/ks.cfg ``` - ks.cfg ``` %post --nochroot --log=/mnt/sysimage/root/ks-post-nochroot.log cp /mnt/install/repo/ /mnt/sysimage/ ``` - any other custom file you want to place on the ISO: genisoimage -U -r -v -T -J -joliet-long -V "CentOS 7 x86_64" -volset "CentOS 7 x86_64" -A "CentOS 7 x86_64" -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -eltorito-alt-boot -e images/efiboot.img -no-emul-boot -o ../isokick.iso . implantisomd5 ../isokick.iso KICKSTART SERVER SETUP KICKSTART CENTOS 7 # edit kickstart partitioning during kickstart? info from Alex # make cmd output display on main screen during kickstart { } >/dev/pts/0 2>/dev/pts/0 # count down during kickstart, and hit specific key during countdown to do thing # incomplete, untested, needs review ``` CUSTOMIZE=false t=5 while [[ $t -gt 0 ]] && ! { read -n1 -t1 -rsp "^MPress enter to customize ($t) " && CUSTOMIZE=true; }; do let t-- done ``` # `^M` = `Ctrl+V then Ctrl+M`; a literal Ctrl+M character gets inserted KICKSTART CENTOS 8 CRONTAB crontab -l crontab -u -l files in: - /etc/cron* - /var/spool/cron/* - /var/log/cron - /var/adm/<...?> ATQ atq # list the at queue SYSTEMD TIMERS # start/enable the timer specifically, not the service systemctl enable --now example.timer # what NOT to do systemctl enable --now example # timer must have service file with matching name: - example.timer AND - example.service # alternatively, specify a differently named service file in the timer vi /etc/systemd/system/example.timer ``` [Unit] Description= [Timer] OnCalendar= Persistent=true [Install] WantedBy=timers.target ``` vi /etc/systemd/system/example.service ``` [Unit] Description= [Service] ExecStart=/bin/bash -c '' # OR ExecStart= # an executable file path ``` SYSTEMD TARGETS systemctl list-units --type target # list available targets systemctl get-default systemctl set-default && reboot systemctl isolate # switch targets # set systemd target via kernel boot parameter: systemd.unit= AUDITING AND LOGGING ausearch -f journalctl --utc -xe -xp3 # view errors only ls /var/log/messages also see - SElinux A NOTE ON AUTHORIZATION important files: - /etc/passwd - /etc/shadow # what service provides auth? cat /etc/nsswitch.conf # edit sudoers (w/ syntax checking!) sudo visudo PASSWORD HASHES AND DOVECOT dovecot pw -s sha512-crypt # hashing algorithm -p # otherwise reads STDIN -t # test if hash matches password # create password hash # ignore stuff in curly brackets {}! the rest is the good stuff that you need to use # (stuff in between dollar signs is the salt) use dovecot anywhere a password hash is required: - with passwd -p - in /etc/shadow - in kickstarts AUTHCONFIG authconfig-tui is good careful, it overwrites that PAM file # enable automatic homedir creation authconfig --enablemkhomedir --update # requires pkg: oddjob-mkhomedir, which allows creation of SELinux labels for new homedirs PAM.D all files in /etc/pam.d/ PAM starts with just a couple important files: system-auth, passwd-auth one controls remote-access logins, other controls local logins installing software (which?) causes it to behave more daemon-y, multiple files hooked together changes files to symlinks: system-auth >> system-auth-ac passwd-auth >> passwd-auth-ac creates/edits/clobbers: system-auth-ac passwd-auth-ac any changes you make to *-auth-ac will be clobbered next time you run authconfig!! many programs will run authconfig in the background even(?) to avoid clobber: create symlinks system-auth >> system-auth-includes passwd-auth >> passwd-auth-includes place content in files system-auth-includes ``` auth include system-auth-ac account include system-auth- password include system-auth-ac session include system-auth-ac ``` system-auth- ``` copy lines from system-auth-ac that must be read before yr new stuff add new stuff account include system-auth-ac # to ensure it also gets all the system-auth-ac config ``` passwd-auth-includes ``` auth include passwd-auth-ac account include passwd-auth- password include passwd-auth-ac session include passwd-auth-ac ``` passwd-auth- ``` ``` alternately, use substack command instead of include command causes it to return control to the next step in the file upon failure, instead of quitting? USERS AND GROUPS vi /etc/default/useradd # set new user defaults getent passwd # check if user exists userdel # remove user useradd # create user -m # create home directory -s # set default shell -g # set primary group -G ,... # set additional groups -u # use specific user ID number -c '' # add a comment usermod -aG ,[...] # add user to group -s /dev/null -p LOCKED # set login shell to /dev/null and password hash to LOCKED passwd # change user password -S # view account status info -u # unlock locked account -e # force password expiry, require change at next logon chage -E # set account expiry date to yesterday where = $(date -d yesterday +%F) # a nice useful trick getent group # check if group exists groupdel # remove group groupadd # add group -g # use specific group ID number gpasswd -M ,[...] # set group membership to listed users OWNERS AND PERMISSIONS # change owner/group of file or directory chown : -R # apply changes recursively to subdirs and files chmod 700 # standard syntax chmod 0655 chmod 4750 # special permissions replace x in "drwxr-x---" chmod +t # lowercase if "x" is also set chmod g+s .test always open needs to point to DNSSEC-capable DNS server OPENLDAP ldapsearch -x -H ldap:// -d "cn=,dc=,dc=" -W -v '(&)' # the '(&)' is the search string, currently will show all items # may need to upper/lowercase the -d, depending on LDAP version slapcat # run directly on LDAP server as local root user ldapsearch -H ldapi:// -Y EXTERNAL '(&)' # run directly on LDAP server as LDAP admin user; can use search strrings ldappasswd -x -H ldap:// -d "cn=,dc=,dc=" -W -S "uid=,ou=,dc=,dc=" ldapadd -x -H ldap:// -d "cn=,dc=,dc=" -W -f .diff # ldapdiff file contains userinfo in the format returned by by ldapsearch ldapdelete -x -H ldap:// -d "cn=,dc=,dc=" -W "uid=,ou=,dc=,dc=" ldapmodify -x -H ldap:// -d "cn=,dc=,dc=" -W Enter LDAP password: dn: changetype: modify replace: : Ctrl+C # you will have to type it out longform at the cmdline, or use clever scripting A NOTE ON PARTITIONING GPT is better DOS-style is less good 4 partitions max if want more, one must be an "extended" partition that takes up the rest of the room and you put the logical partitions on it # to partition fdisk or gdisk A NOTE ON FILESYSTEMS - ext2: good for flashdrives b/c non-journalling - ext4: Linux standard - ZFS: AWESOME - btrfs: like ZFS but with Linux-compatible license # to format mkfs -t /dev/sdx # view UUID or label lsblk -o +label,uuid blkid LUKS ENCRYPTION cryptsetup luksFormat /dev/sdx cryptsetup luksOpen /dev/sdx mkfs -t ext2 /dev/mapper/ mount it copy data unmount cryptsetup luksClose SMART DATA BADBLOCKS LABEL A FILESYSTEM e2label