What is UNIX?

• Single UNIX specification
• POSIX
• Unix shell
• bash
• UNIX philosophy

Navigating the file system

• Unix filesystem

• Filesystem Hierarchy Standard

• pwd, cd, ls, mkdir, ~, cp, rm

• Print working directory

pwd

## /home/bbaumer/Dropbox/git/bootcamp

• Move back one directory

cd ..
pwd

## /home/bbaumer/Dropbox/git

• Jump back to home directory

cd ~
pwd

## /home/bbaumer

• List contents of directory

ls
ls -l
ls -a
ls -alhS

## aggregating.html
## aggregating.Rmd
## bash.Rmd
## bootcamp.Rproj
## create.Rmd
## data-raw
## docs
## dplyr.Rmd
## git.Rmd
## include
## index.Rmd
## intro.Rmd
## join.Rmd
## makefile
## my_path.txt
## postgres.Rmd
## queries.Rmd
## README.md
## resources.Rmd
## site_libs
## _site.yml
## slides
## structure.Rmd
## total 208
## -rw-rw-r-- 1 bbaumer bbaumer 15932 Jun 18 09:16 aggregating.html
## -rw-rw-r-- 1 bbaumer bbaumer  6662 Jun 17 14:27 aggregating.Rmd
## -rw-rw-r-- 1 bbaumer bbaumer  4164 Jun 17 14:27 bash.Rmd
## -rw-rw-r-- 1 bbaumer bbaumer   224 Jun 17 14:35 bootcamp.Rproj
## -rw-rw-r-- 1 bbaumer bbaumer   622 Jun 17 14:27 create.Rmd
## drwxrwxr-x 3 bbaumer bbaumer  4096 Jun 16 20:34 data-raw
## drwxrwxr-x 5 bbaumer bbaumer  4096 Jun 17 10:51 docs
## -rw-rw-r-- 1 bbaumer bbaumer  1854 Jun 18 09:16 dplyr.Rmd
## -rw-rw-r-- 1 bbaumer bbaumer  1310 Jun 17 14:27 git.Rmd
## drwxrwxr-x 2 bbaumer bbaumer  4096 Jun 16 20:34 include
## -rw-rw-r-- 1 bbaumer bbaumer   879 Jun 17 14:27 index.Rmd
## -rw-rw-r-- 1 bbaumer bbaumer  1759 Jun 17 14:27 intro.Rmd
## -rw-rw-r-- 1 bbaumer bbaumer  2566 Jun 17 14:27 join.Rmd
## -rw-rw-r-- 1 bbaumer bbaumer   326 Jun 13 10:06 makefile
## -rw-rw-r-- 1 bbaumer bbaumer   185 Jun 17 10:51 my_path.txt
## -rw-rw-r-- 1 bbaumer bbaumer  1101 Jun 17 14:27 postgres.Rmd
## -rw-rw-r-- 1 bbaumer bbaumer  4957 Jun 17 14:27 queries.Rmd
## -rw-rw-r-- 1 bbaumer bbaumer   155 Jun 17 14:39 README.md
## -rw-rw-r-- 1 bbaumer bbaumer  1404 Jun 17 14:27 resources.Rmd
## drwxrwxr-x 9 bbaumer bbaumer  4096 Jun 18 09:16 site_libs
## -rw-rw-r-- 1 bbaumer bbaumer  1866 Jun 17 10:46 _site.yml
## drwxrwxr-x 3 bbaumer bbaumer  4096 Jun 16 20:34 slides
## -rw-rw-r-- 1 bbaumer bbaumer  2896 Jun 17 14:27 structure.Rmd
## .
## ..
## aggregating.html
## aggregating.Rmd
## bash.Rmd
## bootcamp.Rproj
## create.Rmd
## data-raw
## docs
## dplyr.Rmd
## .git
## .gitignore
## git.Rmd
## include
## index.Rmd
## intro.Rmd
## join.Rmd
## makefile
## my_path.txt
## postgres.Rmd
## queries.Rmd
## README.md
## resources.Rmd
## .Rhistory
## .Rproj.user
## site_libs
## _site.yml
## slides
## structure.Rmd
## total 256K
## -rw-rw-r--  1 bbaumer bbaumer  16K Jun 18 09:16 aggregating.html
## -rw-rw-r--  1 bbaumer bbaumer 6.6K Jun 17 14:27 aggregating.Rmd
## -rw-rw-r--  1 bbaumer bbaumer 4.9K Jun 17 14:27 queries.Rmd
## -rw-rw-r--  1 bbaumer bbaumer 4.1K Jun 17 14:27 bash.Rmd
## drwxrwxr-x  9 bbaumer bbaumer 4.0K Jun 18 09:16 .
## drwxrwxr-x 84 bbaumer bbaumer 4.0K Jun 16 20:34 ..
## drwxrwxr-x  3 bbaumer bbaumer 4.0K Jun 16 20:34 data-raw
## drwxrwxr-x  5 bbaumer bbaumer 4.0K Jun 17 10:51 docs
## drwxrwxr-x  8 bbaumer bbaumer 4.0K Jun 18 09:16 .git
## drwxrwxr-x  2 bbaumer bbaumer 4.0K Jun 16 20:34 include
## drwxrwxr-x  5 bbaumer bbaumer 4.0K Jun 16 21:31 .Rproj.user
## drwxrwxr-x  9 bbaumer bbaumer 4.0K Jun 18 09:16 site_libs
## drwxrwxr-x  3 bbaumer bbaumer 4.0K Jun 16 20:34 slides
## -rw-rw-r--  1 bbaumer bbaumer 2.9K Jun 17 14:27 structure.Rmd
## -rw-rw-r--  1 bbaumer bbaumer 2.6K Jun 17 14:27 join.Rmd
## -rw-rw-r--  1 bbaumer bbaumer 1.9K Jun 17 10:46 _site.yml
## -rw-rw-r--  1 bbaumer bbaumer 1.9K Jun 18 09:16 dplyr.Rmd
## -rw-rw-r--  1 bbaumer bbaumer 1.8K Jun 17 14:27 intro.Rmd
## -rw-rw-r--  1 bbaumer bbaumer 1.7K Jun 13 13:03 .Rhistory
## -rw-rw-r--  1 bbaumer bbaumer 1.4K Jun 17 14:27 resources.Rmd
## -rw-rw-r--  1 bbaumer bbaumer 1.3K Jun 17 14:27 git.Rmd
## -rw-rw-r--  1 bbaumer bbaumer 1.1K Jun 17 14:27 postgres.Rmd
## -rw-rw-r--  1 bbaumer bbaumer  879 Jun 17 14:27 index.Rmd
## -rw-rw-r--  1 bbaumer bbaumer  622 Jun 17 14:27 create.Rmd
## -rw-rw-r--  1 bbaumer bbaumer  581 Jun 10 15:22 .gitignore
## -rw-rw-r--  1 bbaumer bbaumer  326 Jun 13 10:06 makefile
## -rw-rw-r--  1 bbaumer bbaumer  224 Jun 17 14:35 bootcamp.Rproj
## -rw-rw-r--  1 bbaumer bbaumer  185 Jun 17 10:51 my_path.txt
## -rw-rw-r--  1 bbaumer bbaumer  155 Jun 17 14:39 README.md

• Symbolic links
• relative vs. absolute paths

What’s happening?

• who

• top, htop

• free, df

• ps, kill

• less

• Who am I?

who
whoami
w

## bbaumer  :0           2019-06-16 21:03 (:0)
## bbaumer
##  09:16:19 up 1 day, 12:12,  1 user,  load average: 1.49, 0.89, 0.61
## USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
## bbaumer  :0       :0               Sun21   ?xdm?  14:29   0.01s /usr/lib/gdm3/gdm-x-session --run-script env GNOME_SHELL_SESSION_MODE=ubuntu gnome-session --session=ubuntu

• How much memory does this computer have?

free -h

##               total        used        free      shared  buff/cache   available
## Mem:            15G        4.5G        1.5G        874M        9.6G        9.9G
## Swap:           31G          0B         31G

• How much disk space does this computer have?

df -h

## Filesystem      Size  Used Avail Use% Mounted on
## udev            7.8G     0  7.8G   0% /dev
## tmpfs           1.6G  2.3M  1.6G   1% /run
## /dev/nvme0n1p3  435G  174G  240G  43% /
## tmpfs           7.8G  194M  7.6G   3% /dev/shm
## tmpfs           5.0M  4.0K  5.0M   1% /run/lock
## tmpfs           7.8G     0  7.8G   0% /sys/fs/cgroup
## /dev/loop1      4.2M  4.2M     0 100% /snap/gnome-calculator/406
## /dev/loop2       15M   15M     0 100% /snap/gnome-characters/280
## /dev/loop3      141M  141M     0 100% /snap/gnome-3-26-1604/86
## /dev/loop4       36M   36M     0 100% /snap/gtk-common-themes/1198
## /dev/loop6      3.8M  3.8M     0 100% /snap/gnome-system-monitor/83
## /dev/loop7       54M   54M     0 100% /snap/core18/970
## /dev/loop8      141M  141M     0 100% /snap/gnome-3-26-1604/82
## /dev/nvme0n1p1  496M   41M  456M   9% /boot/efi
## /dev/loop9      152M  152M     0 100% /snap/gnome-3-28-1804/51
## /dev/loop10      89M   89M     0 100% /snap/core/6964
## /dev/loop12     1.0M  1.0M     0 100% /snap/gnome-logs/61
## tmpfs           1.6G   28K  1.6G   1% /run/user/125
## tmpfs           1.6G   80K  1.6G   1% /run/user/1000
## /dev/loop13     3.8M  3.8M     0 100% /snap/gnome-system-monitor/87
## /dev/loop14      15M   15M     0 100% /snap/gnome-characters/284
## /dev/loop15     152M  152M     0 100% /snap/gnome-3-28-1804/55

• What is this computer doing right now?

ps aux

Getting help

• man

Where is it?

• find, locate, which
• $PATH variable
• Environment Variables
• nano, watch out for vi!

echo $PATH

## /home/bbaumer/bin:/home/bbaumer/.local/bin:/home/bbaumer/bin:/home/bbaumer/.local/bin:/home/bbaumer/bin:/home/bbaumer/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

locate stuff

## /home/bbaumer/Dropbox/Academic/Smith/courses/mth220/StatEdCompute-2013/collaborations2013/poster/knitrstuff.tex
## /home/bbaumer/Dropbox/Academic/Smith/stationery/poster/knitrstuff.tex
## /home/bbaumer/Dropbox/Academic/modeling_fielding/knitrstuff.tex
## /home/bbaumer/Dropbox/Academic/modeling_fielding/Poster/knitrstuff.tex
## /home/bbaumer/Dropbox/Academic/poster-test/knitrstuff.tex
## /home/bbaumer/Dropbox/WAR/nessis2013/knitrstuff.tex
## /home/bbaumer/Dropbox/git/sds/to-do/poster/knitrstuff.tex
## /home/bbaumer/backup/home/bbaumer/.config/chromium/Default/Local Storage/http_stupidstuffihadtofix.blogspot.com_0.localstorage
## /home/bbaumer/backup/home/bbaumer/.config/chromium/Default/Local Storage/http_stupidstuffihadtofix.blogspot.com_0.localstorage-journal
## /snap/gnome-3-26-1604/82/usr/share/icons/Humanity/mimes/16/application-x-stuffit.svg
## /snap/gnome-3-26-1604/82/usr/share/icons/Humanity/mimes/16/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-26-1604/82/usr/share/icons/Humanity/mimes/22/application-x-stuffit.svg
## /snap/gnome-3-26-1604/82/usr/share/icons/Humanity/mimes/22/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-26-1604/82/usr/share/icons/Humanity/mimes/24/application-x-stuffit.svg
## /snap/gnome-3-26-1604/82/usr/share/icons/Humanity/mimes/24/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-26-1604/82/usr/share/icons/Humanity/mimes/32/application-x-stuffit.svg
## /snap/gnome-3-26-1604/82/usr/share/icons/Humanity/mimes/32/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-26-1604/82/usr/share/icons/Humanity/mimes/48/application-x-stuffit.svg
## /snap/gnome-3-26-1604/82/usr/share/icons/Humanity/mimes/48/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-26-1604/82/usr/share/mime/application/x-stuffit.xml
## /snap/gnome-3-26-1604/86/usr/share/icons/Humanity/mimes/16/application-x-stuffit.svg
## /snap/gnome-3-26-1604/86/usr/share/icons/Humanity/mimes/16/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-26-1604/86/usr/share/icons/Humanity/mimes/22/application-x-stuffit.svg
## /snap/gnome-3-26-1604/86/usr/share/icons/Humanity/mimes/22/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-26-1604/86/usr/share/icons/Humanity/mimes/24/application-x-stuffit.svg
## /snap/gnome-3-26-1604/86/usr/share/icons/Humanity/mimes/24/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-26-1604/86/usr/share/icons/Humanity/mimes/32/application-x-stuffit.svg
## /snap/gnome-3-26-1604/86/usr/share/icons/Humanity/mimes/32/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-26-1604/86/usr/share/icons/Humanity/mimes/48/application-x-stuffit.svg
## /snap/gnome-3-26-1604/86/usr/share/icons/Humanity/mimes/48/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-26-1604/86/usr/share/mime/application/x-stuffit.xml
## /snap/gnome-3-28-1804/51/usr/share/icons/Humanity/mimes/16/application-x-stuffit.svg
## /snap/gnome-3-28-1804/51/usr/share/icons/Humanity/mimes/16/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-28-1804/51/usr/share/icons/Humanity/mimes/22/application-x-stuffit.svg
## /snap/gnome-3-28-1804/51/usr/share/icons/Humanity/mimes/22/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-28-1804/51/usr/share/icons/Humanity/mimes/24/application-x-stuffit.svg
## /snap/gnome-3-28-1804/51/usr/share/icons/Humanity/mimes/24/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-28-1804/51/usr/share/icons/Humanity/mimes/32/application-x-stuffit.svg
## /snap/gnome-3-28-1804/51/usr/share/icons/Humanity/mimes/32/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-28-1804/51/usr/share/icons/Humanity/mimes/48/application-x-stuffit.svg
## /snap/gnome-3-28-1804/51/usr/share/icons/Humanity/mimes/48/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-28-1804/51/usr/share/mime/application/x-stuffit.xml
## /snap/gnome-3-28-1804/55/usr/share/icons/Humanity/mimes/16/application-x-stuffit.svg
## /snap/gnome-3-28-1804/55/usr/share/icons/Humanity/mimes/16/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-28-1804/55/usr/share/icons/Humanity/mimes/22/application-x-stuffit.svg
## /snap/gnome-3-28-1804/55/usr/share/icons/Humanity/mimes/22/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-28-1804/55/usr/share/icons/Humanity/mimes/24/application-x-stuffit.svg
## /snap/gnome-3-28-1804/55/usr/share/icons/Humanity/mimes/24/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-28-1804/55/usr/share/icons/Humanity/mimes/32/application-x-stuffit.svg
## /snap/gnome-3-28-1804/55/usr/share/icons/Humanity/mimes/32/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-28-1804/55/usr/share/icons/Humanity/mimes/48/application-x-stuffit.svg
## /snap/gnome-3-28-1804/55/usr/share/icons/Humanity/mimes/48/gnome-mime-application-x-stuffit.svg
## /snap/gnome-3-28-1804/55/usr/share/mime/application/x-stuffit.xml
## /snap/gtk-common-themes/1198/share/icons/Humanity/mimes/16/application-x-stuffit.svg
## /snap/gtk-common-themes/1198/share/icons/Humanity/mimes/16/gnome-mime-application-x-stuffit.svg
## /snap/gtk-common-themes/1198/share/icons/Humanity/mimes/22/application-x-stuffit.svg
## /snap/gtk-common-themes/1198/share/icons/Humanity/mimes/22/gnome-mime-application-x-stuffit.svg
## /snap/gtk-common-themes/1198/share/icons/Humanity/mimes/24/application-x-stuffit.svg
## /snap/gtk-common-themes/1198/share/icons/Humanity/mimes/24/gnome-mime-application-x-stuffit.svg
## /snap/gtk-common-themes/1198/share/icons/Humanity/mimes/32/application-x-stuffit.svg
## /snap/gtk-common-themes/1198/share/icons/Humanity/mimes/32/gnome-mime-application-x-stuffit.svg
## /snap/gtk-common-themes/1198/share/icons/Humanity/mimes/48/application-x-stuffit.svg
## /snap/gtk-common-themes/1198/share/icons/Humanity/mimes/48/gnome-mime-application-x-stuffit.svg
## /usr/lib/python3/dist-packages/reportlab/lib/boxstuff.py
## /usr/lib/python3/dist-packages/reportlab/lib/__pycache__/boxstuff.cpython-36.pyc
## /usr/share/icons/Humanity/mimes/16/application-x-stuffit.svg
## /usr/share/icons/Humanity/mimes/16/gnome-mime-application-x-stuffit.svg
## /usr/share/icons/Humanity/mimes/22/application-x-stuffit.svg
## /usr/share/icons/Humanity/mimes/22/gnome-mime-application-x-stuffit.svg
## /usr/share/icons/Humanity/mimes/24/application-x-stuffit.svg
## /usr/share/icons/Humanity/mimes/24/gnome-mime-application-x-stuffit.svg
## /usr/share/icons/Humanity/mimes/32/application-x-stuffit.svg
## /usr/share/icons/Humanity/mimes/32/gnome-mime-application-x-stuffit.svg
## /usr/share/icons/Humanity/mimes/48/application-x-stuffit.svg
## /usr/share/icons/Humanity/mimes/48/gnome-mime-application-x-stuffit.svg
## /usr/share/mime/application/x-stuffit.xml

find /usr -name "git"

## /usr/bin/git
## /usr/share/bash-completion/completions/git
## /usr/share/doc/git
## /usr/share/lintian/overrides/git
## /usr/lib/git-core/git

which git

## /usr/bin/git

I/O

• grep, sed
• Standard input/output
• Redirection, >, <
• Piping |
• xargs

echo $PATH > my_path.txt
ls
wc my_path.txt

## aggregating.html
## aggregating.Rmd
## bash.Rmd
## bootcamp.Rproj
## create.Rmd
## data-raw
## docs
## dplyr.Rmd
## git.Rmd
## include
## index.Rmd
## intro.Rmd
## join.Rmd
## makefile
## my_path.txt
## postgres.Rmd
## queries.Rmd
## README.md
## resources.Rmd
## site_libs
## _site.yml
## slides
## structure.Rmd
##   1   1 228 my_path.txt

echo $PATH | wc

##       1       1     228

find ~ -name "*.txt" -mtime -7
find ~ -name "*.txt" -mtime -7 -print0

echo $PATH > my_path.txt
sed 's/:/\n/g' my_path.txt > my_path.csv
cat my_path.csv | xargs ls

Tasks

1. Copy a bunch of files in nested directories to /tmp.

find /home/bbaumer/Dropbox/ -type f -name "*.Rmd" -mtime -7 -exec cp -t /tmp {} \;

find /home/bbaumer/Dropbox/ -type f -regex ".+Rproj\.user.+conflicted copy.+\.pper" -print0 | xargs -0 cp -t /tmp

1. Find and replace across files

grep "dplyr" *.Rmd
sed -i 's/dplyr/hello-dplyr/g' *.Rmd
grep "hello-dplyr" *.Rmd

2. Rename a large set of files in nested directories