Ubuntu server - my list of useful commands

Here is a collection of commands I find very useful when doing daily work on Ubuntu Linux. For more information, you can reference this course.

1. How to check what is the exact name of a package containing a command with a certain name? With: apt-file we can search inside of contents of a package. 1.1 sudo apt-file update, then 1.2 apt-file search gimp
   or dpkg -S filename
2. How about checking all the files that have been modified in the last 30 minutes? find -cmin -30 will do the job
3. or just to see the installed packages: cat /var/log/dpkg.log |grep installed is the command.
4. Sometimes I need to check what is happening to my disk space. One very good program for the command prompt is ncdu. Just try it, and you won't be disappointed: ncdu
5. Sometimes I don't know which applications are using the internet or if certain download/update is stalled: nethogs is best for those cases.
   
6. And what if I would like to be able to run a certain program without typing constantly sudo in front? Well adding our current user $USER to the program's group is all needed:  sudo usermod -aG docker $USER(here is for running docker without sudo) 
7. Permissions: you may encounter that there exist additional fine-grained permissions apart from the default for the user/group/and owner. Welcome the ACLs. If you see a + next to the ls -la listing of a file or directory just type getfacl /dir_name/ to see them. To a add a group ACLs: setfacl -m g:www-data:rwx /var/www/, to remove group ACLs: setfacl -x g:www-data /var/www. ACLs also have defaults with -d. This way a new directory/file will inherit the default permissions of the parent directory: setfacl -d -m g:www-data:rwx /var/www Another example: setfacl -Rd -m u:{$USER}:rwx /var/www/ will set rwx permissions to /var/www for a specific user - here we also use the recursive -R flag. Note that sometimes we need to do 2 passes to set permissions correctly: one with defaults -d for any newly created files/dirs, and one without the -d for the current files and directories!
8. Packages dependency clashes:
   I. We can have different versions of the same package inside of the apt repository for example for bionic and for focal releases. First, check and fix every URL to come from the same distribution inside: /etc/apt/sources.list and the inner directories. Then run apt update again to refresh the updated list of packages and use: dpkg --configure -a to configure the packages.
   II. Interrupted apt dist-upgrade while installing the packages, then later trying again to use apt update && apt dist-upgrade but in between a new version of some package has been released. In this case, you have unmet dependencies because you have an old version about to be installed (staying in the apt-cache list), then a new version comes and it cannot continue with the installation, because the old one is still not installed successfully. Circular dependencies may happen, or when the package version installed on the system is different than the required from another package. For example:
   libqtermwidget5-0:amd64 depends on qtermwidget5-data (= 0.14.1-2); however: The version of qtermwidget5-data on the system is 0.14.1-0ubuntu3
   1. remove the half-installed package that causes the problems and its configuration from your system:
   sudo dpkg -P --force-depends qtermwidget5-data_0.14.1-2_all 
   2. when we do apt update, the command saves a cached list of packages that will be downloaded from the Ubuntu servers in /var/lib/apt/lists/, so remove all apt caches: sudo find /var/lib/apt/lists -type f  |xargs rm -f >/dev/null and run apt update
   3. configure the rest of the packages to be installed and configured, without checking their dependency requirements (ignore): sudo dpkg --configure -a --force-depends
   4. continue installing the  packages with their correct dependencies: sudo apt-get -f install
   in another version of the problem a new package cannot be installed, because it asks to overwrite a file of an old one: dpkg is trying to overwrite 'file' which is also in package '...'. In this case issue: sudo dpkg -i --force-overwrite /var/cache/apt/archives/libffi8ubuntu1_3.4~20200819gead65ca871-0ubuntu3_amd64.deb (where you can place the name of the new package archive)
   or a more "safe option" is to remove the problem causing archive from /var/cache/apt/archives/
   
9. Network tools:
   list all the processes taking most of the CPU: ps -e --format=pid,rss,args | sort --numeric-sort --key=2 check the network connections inside of specific process: sudo lsof -aPi -p 3258 or trace them: sudo strace -p 3258 -e trace=network
   list all listening(open) ports on the current machine: sudo lsof -i | grep -i LISTEN
   list all the network connections of user 'nevyan': sudo lsof -aPi -u nevyan
   listening connections inside of a process:
   sudo lsof -ai -p 730
   
10. Working with text files:
    - replacing strings inside of a file:
    sudo sed -i 's/focal/groovy/g' /etc/apt/sources.list
    // s -substitute, g - global, apply to all matches
    - downloading a file and replacing its contents at the same time:
    curl -s https://raw.githubusercontent.com/istio/istio/release-1.6/samples/bookinfo/platform/kube/bookinfo.yaml | sed 's/app: reviews/app: reviews_test/'
    - adding information to text file (-a = append / the default action is overwrite):
    echo "deb https://download.sublimetext.com/ apt/stable/" | sudo tee -a /etc/apt/sources.list.d/sublime-text.list
11. Working with variables:
    - how to grep webpage tags:
    MYHOST="http://www.google.com";
    curl -s $MYHOST | grep -o "<title>*.*</title>";
    // -o shows only full matched lines with content
    - how to save an output of command:
    look for pattern line and display column 3: ip route | awk '/default/ { print $3 }'
    save output into variable: MYHOST=$(ip route | awk '/default/ { print $3 }')
    ping MYHOST
    - complex example for purging non-used kernels:
    dpkg --list linux-{headers,image,modules}-* | awk '{ if ($1=="ii") print $2 }' | grep -v -e "$(uname -r | cut -d "-" -f 1,2)" | sudo xargs apt purge -y
    // e regex match, v inverted match
    It there are any removed packages, they can be reinstalled with: apt install --reinstall linux-image-X.Y-ARCH5
12. Finding and deleting files recursively based on pattern: find ./mail -depth -path '*gmai*' -delete
13. How to do a default prompt shortening, when the shell gets too long: export PS1="\[\e]0;\u@\h: \w\a\]${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]\$ "
14. Ever wanted to be able to resume downloads, here is the curl option: curl -c http://url_to_download

Congratulations and enjoy the course!