LEARN TO LOVE THE COMMAND LINE

Thursday, 19 February, 2015

Google Montreal
Shopify Montreal

In love with...

the command line

http://cli-talk.bitballoon.com

Blanca Mancilla / bluciam / @Blanckus

LinuxCommand.org
LinuxCommand.org
You have Linux installed and running. The GUI is working fine, but you are getting tired of changing your desktop themes. You keep seeing this "terminal" thing.

What is the command line?

What is the command line?

Anybody with little or no experience?
What do you know about it?
What are you expecting to learn tonight?

The command line
and the shell


   A command-line interface (CLI) is a means of interacting with a
   computer program where the user issues commands to the program in
   the form of successive lines of text (command lines).
                
                https://en.wikipedia.org/wiki/Command-line_interface
Specifically, the computer program is the operating system (UNIX or nowadays Linux) and the interface is what we called the shell .

Types of shells in Linux

How do I know which shell?

The Inspiration

Down arrow
Data Science at the Command Line by Jeroen Janssens Data Science at the command line
The Unix philosophy of simple tools, each doing one job well, then cleverly piped together, is embodied by the command line. Jeroen expertly discusses how to bring that philosophy into your work in data science, illustrating how the command line is not only the world of file input/ output, but also the world of data manipulation, exploration, and even modeling.
—Chris H. Wiggins
Associate Professor in the Department of Applied Physics
and Applied Mathematics at Columbia University
and Chief Data Scientist at The New York Times

Appendix: commands

alias awk aws bash bc bigmler body cat cd chmod cols cowsay cp csvcut csvgrep csvjoin csvlook csvsort csvsql csvstack csvstat curl curlicue cut display drake dseq echo env export feedgnuplot fieldsplit find for git grep head header in2csv jq json2csv less ls man mkdir mv parallel paste pbc pip pwd python R Rio Rio-scatter rm

Some details now!

For loops?

Yes! Of course. 

for i in *pdf; do evince $i; done

For loops

For loops

In a script 


for i in *pdf
do
  evince $i
done

or even 


for i in `ls *pdf`
do
  evince $i
done

More info

Check if a file exists and if not ...


cli-talk: $ [ -f index.html ] && echo "Yes, we have a presentation!" \
|| echo "PANIC"

Check if a file exists and if not ...

Check if a file exists and if not ...

Let's try a demo 

cli-talk: $ locate *9768.JPG

Glob patterns:
real magic!

Strictly speaking, there are neither regular expressions nor patterns used in pattern recognition.
They are glob patterns .

Glob patterns:
real magic!


ls app/*.{erb,scss,rb,yml}

vi public_html/[iI]*.htm*

Glob patterns:
real magic!

My favorite: Move all image files for the years
2010 to 2012 into my_dir.


            mv {IMG,img}*201[012]*.{jpg,JPG,png,gif} my_dir

Find a resource anywhere in your hard drive

Find a resource ...

locate versus find 

LoriBiz: $ locate *.erb
LoriBiz: $ find . -name *.erb

Find a resource...

find . -name *.erb -exec ls {} \;

find . -name *.erb -exec grep mytableauthor {} \;

find . -name *.erb -exec grep -H mytableauthor {} \;

find . -name *.scss -exec grep -H mytableauthor {} \;

find . -name *.scss -exec grep -n -H mytableauthor {} \;

Find and replace trailing spaces in source code

Find ... trailing spaces in source code


            cli-talk $ grep "\s\+$" index.html

... replace trailing spaces in source code


LoriBiz$ find . -name *.erb -exec grep -Hn "\s\+$" {} \; 
LoriBiz$ find . -name *.erb -exec grep -Hn "\s\+$" {} \; -exec vi {} \;

# Once in vi find and replace using the command
:%s/\s\+$//

Making backups of
git repositories

superspreadsheet.wordpress.com

Making backups of
git repositories

Making backups of ...

  1. pull from the bare repository
  2. commit local changes
  3. push changes to the bare repository

Making backups of ...

Pull and Commit 


 1 #!/bin/sh
 2 #
 3 # Automatic pull and commit on the directories listed in dirs_to_pull
 4 
 5 CRONLOG="/complete/path/to/pullcommitpush.log"
 6 COMMITMSG="/complete/path/to/msgAutoCommit"
 7 echo >> $CRONLOG
 8 echo "Nightly git auto-commit for the absent minds: $(date)" >> $CRONLOG
 9 for i in `cat /complete/path/to/dirs_to_pull`
10   do (cd $i;
11       echo "PULL $i $(date)" >> $CRONLOG ;
12       git pull >> $CRONLOG 2>&1 ;
13       echo "COMMIT $i $(date)" >> $CRONLOG ;
14       git commit -a -F $COMMITMSG >> $CRONLOG 2>&1
15       echo >> $CRONLOG  )
16 done 
17 echo >> $CRONLOG ;

Making backups of ...

Push 

 1 #!/bin/sh
 2 #
 3 # Automatic push on the directories listed in dirs_to_push
 4
 5 CRONLOG="/complete/path/to/pullcommitpush.log"
 6 echo >> $CRONLOG
 7 echo "Nightly git auto-push for the absent minds: $(date)" >> $CRONLOG
 8 for i in `cat /complete/path/to/dirs_to_push`
 9  do (cd $i;
10    echo "PUSH: $i $(date)" >> $CRONLOG
11    git push >> $CRONLOG 2>&1
12    echo >> $CRONLOG
13   ) ;
14 done
15 echo >> $CRONLOG ;

Making backups using ...

cron: from the Greek word for time, χρόνος chronos.

Making backups using ...

Cron is a time-based job scheduler that Sysadmins use to schedule jobs to run periodically at fixed times, dates, or intervals (i.e. download email at regular intervals).

Making backups using ...


            crontab -e
minute hour dayOfMonth month dayOfWeek command 

3 2 * * * /complete/path/to/pull_commit 2> /complete/path/to/pull_commit.err
5 2 * * * /complete/path/to/push 2> /complete/path/to/push.err

Creating a copy of a git repository which is not quite a copy

Blog post in development!

THE END

Are you in love yet?

THE END

JavaScript source code for the slides taken from reveal.js.
I encourage you to try it!