Simplifying Your Linux Work

Note

Nothing in this note is required. You can type all those long commands over and over if you like. However, remember my mantra: “I am lazy, and proud of it!” I learned these tricks years ago! My first exposure to UNIX (what Linux is trying to be) was in 1980!

As apprentice Geeks, you owe it to yourself to lern the tricks that help you work more smoothly. These tricks are used by millions of developers today!

Now that you are working on the Linux Command Line, there are a couple of things you can do to streamline your work. The references I provided are a good way to learn more. For now, try here are a few topics that can hel simplify your work.

SUDO Mode

Before we gt going, you need to understand what this “sudo” stuff is all about.

In the “old days”, when you set up a Linux system, there was only one account created on the machine. The username was “root” and that user had “administrator” rights. That meant the user could do anything, and work anywhere in the system. Messing around with system files can break the operating system, so you better know what you are doing when operating as an administrator. The “root” user came to be called the “super user” (sort of like Superman!)

Normal users do not have the same rights at all. That provides a level of security for the system when normal users play with files. They can destroy their own areas, but not the system as a whole.

Modern Linux systems automatically set up a “normal” user when you install the OS. However this user has the ability ro “become” the super user” when needed. Doing this is dangerous, because you still need to know what you are doing, or risk destroying the system.

Many seemingly simple tasks, like installing common tools, need to write into protected system directories. In order you to do that, (and assuming you are that first user), you need to temporarily become the “super user”.

Welcome to the “sudo” command prefix!

“Sudo”, which stands for “Super User Do”, elevates you to a higher power level, but only for this one command. You add “sudo” in front of a normal command, and that command will run with the access rights of the “root” account.

Note

In any terminal session, the first time you use “sudo” you will be asked for your account password. If you have the “sudo” rights, the command will run. Otherwise, you get a nasty error message. After that, the “sudo” commands will run without additional prompts.

The safe way to use Linux, now, is to normally run commands without “sudo”. If you see error messages telling you there were problems accessing certain files, re-run the command with “sudo” in front of it (Remember that you can easily edit a previous command, using the arrow keys on your keyboard.)

We will use “sudo” in several of the commands in this note.

Git Global Configuration

Git “brands” each commit with the name and email of the user who made that commit. To do this, you need to set up your personal system:

Warning

Do not do this on the CS lab machines. Those are shared by many users!

Open up a terminal in your VM. Then type this (obviously, changing it so it is you, not me!):

$ git config --global "Roie R. Black"
$ git config --global user.email "rblack@austincc.edu"

Now, Git will know who you are. This is more useful in team projects where several developers will make changes.

No More GitHub Passwords!

Note

There is a guide on GitHub to help set this up. Look in their help section.

It gets annoying having to provide your user name and password to GitHub each time you interact with the server. There is a way to stop all of that, but you have a bit of setup work to do first!

SSH Credentials

You need to create two files on your Linux machine. One is called a “private key” and you should protect that one and NEVER let anyone, but you, see it. The other is called a “public key”. That one can be freely shared with anyone. Together those two keys set up an encryption system that will ensure you are who you say you are when you interact with a remote server.

Basically, your “public key” can be used to encrypt messages to be sent to you. Only you, using your “private key”, can decrypt those messages. Anyone can send you a message, but only you can decrypt it.

GitHub (actually, many sites that use this scheme) will use this scheme to check that you are the correct person to access your account. I create keys like this on each machine I use, and use these keys on every server I access! Saves a lot of typing (and you know I hate typing!)

Creating the Keys

First, verify that the needed tool is on your system. (It should be!)

If this does not work, run this command and try again

$ sudo apt-get install openssh-client

Now, issue this command:

$ ssh-keygen -t rsa -C "your@email-address"
Generating public/private rsa key pair.
Enter file in which to save the key (/home/vagrant/.ssh/id_rsa):

Just press the “Enter” key here. The keys will end up in the standard location.

Next, you will be asked if you want to add a “passphrase (another term for password) to your keys. This is a good idea, but it makes it a bit more complex to eliminate passwords on your GitHub interactions. Since this setup is inside of a VM, on your personal machine. You can leave this empty. (I never do, for extra security). YMMV!

Enter passphrase (empty for no passphrase):
Enter same passphrase again:

At this point the program will generate your keys.

Your identification has been saved in /home/vagrant/.ssh/id_rsa.
Your public key has been saved in /home/vagrant/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:uo9WgrGOgGPtpuW9jAiRX7jgwa6C8Wuzq8 rblack@austincc.edu
The key's randomart image is:
+---[RSA 2048]----+
|   .*ooo*        |
|   ..*.*o.       |
|    +.+=o .      |
| . ..+o..o .     |
|  +o.o. S o      |
| *. ....*        |
| + o   oo        |
| o=.o o.o        |
| *E+oo.+.*       |
+----[SHA256]-----+

This is just the program telling you that it did its job.

Your new “key” files are in the hidden ~/.ssh directory in your home directory (that “squiggle” is an alias for your home directory, you can use it a lot: it is identical to /home/username).

  • id_rsa is your “private key”
  • id_rsa.pub is your “public key”

Paste Public Key on GitHub

The key files are actually a single long string of random looking gibberish. But that gibberish is mathematically generated, so make sure you do not modify the contents of the file in any way.

Warning

That means do not open it up in an editor that folds long lines automatically. If you save that, the file is toast!

Create an “alias”

We need one additional tool to make this process easier on Ubuntu. (thank you Google!):

Macs have a nice tool, called pbcopy that copies a file to the clipboard. Unfortunately, that tool is not available on Ubuntu. However, a similar tool, called xclip is available. We can install that, and use a neat command line trick to create a pbcopy command in Ubuntu!

First, install xclip:

$ sudo apt-get install -y xclip

The problem with xclip is that it’s command line is way too long (for me). To streamline it, I am going to create a Linux “alias”. This is a neat way to set up a short name for a long command string, saving you some of that ugly typing.

Do this:

$ vim ~/.bash_aliases

This file may not exist on your system now, that is fine, we are creating it!

Add this line:

alias pbcopy='xclip -selection clipboard'

Save this file, then do this:

$ source ~/.bashrc

Note

Whenever you open up a new terminal window in Linux (or Macs) the system automatically runs a “script” in your home directory that is named .bashrc. That file normally loads a second file named .bash_aliases` where users can add more “alias” lines. (We will do this later)!

“source” causes the terminal program ot read the file again, since I just changed it.

Now, to copy a file onto the clipboard, and avoid that mouse, do this:

$ pbcopy ~/.ssh/id_rsa.pub

Note

The “alias turned that “pbcopy” command into the string I set up when I created the “alias”. The string was not all of the command, just everything I had to type that never changed. The alias lets you add more text after the alias name, then you hit “Enter”!

We will use the “alias” mechanism later to simplify your Git commands!

You now have your “public key” on your “clipboard.

Log onto GitHub

You need to get your “public key” file onto the GitHub server. To do this, log into GitHub and click on your account image. Then click on “Profile”. On the left side is a menu. Look for the “SSH and PGP keys” item and click on that.

A screen will open showing your current installed keys (probably empty). Click on the “New SSH Key” button. A page will appear with a place to paste in your new SSH Public Key.

Right-click in that box, and select “paste”. You should see the ugly text from your key file.

The “Old Fashioned” Way

Just for reference, this is how to get the key copied using just your mouse.

Open up the key file as follows:

$ vim ~/.ssh/id_rsa.pub

Now, carefully, select the text you see with your mouse.

Warning

Get all of the text, but be careful not to modify anything. I had to right click and select “copy” to get this to work in my VM.

Then move back to the browser with the area for pasting in it, and right-click and select “paste” to enter the text. Again, make sure it is all there (you can highlight and delete if it is messed up, then try again).

Add the SSH Key to GitHub

When you have the key file text in the box, click on “Add SSH Key”.

Note

My screen seemed to show the text folded, but it was fine.

Note

At the bottom of the window that appears after you add the key is a link to a guide on doing all this from GitHub. If you have problems, check that out.

Back on the Linux VM

Now, go back into your Linux VM. Do this:

$ ssh -T git@github.com
The authenticity of host 'github.com (192.30.253.112)' can't be established.
RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8.
Are you sure you want to continue connecting (yes/no)? yes

At this point, it asks you if you recognize GitHub. Just say “yes”.

This pop-up window will appear, asking for the password you set up when you created the keys.

../_images/github-ssh-check.png

Finally, you should see this:

Warning: Permanently added 'github.com,192.30.253.112' (RSA) to the list of known hosts.
Hi rblack42! You've successfully authenticated, but GitHub does not provide shell access.

This means that GitHub has checked your credentials. How this happens is interesting, but nothing we need to worry about.

Automating passphrase management

If you put a password on your key file, GitHub will ask for the password for that file. It will do this every time you try to use the key, which is not what we want!

Warning

I am still fighting with Ubuntu to get rid of passwords. What is shown here works in my VM, but it is a bit clumsy!

We can stop this by setting up a tool called ssh-agent so it runs every time you log in. Edit that .bashrc file again, and add this at the bottom of the existing file:

eval `ssh-agent -s`

Warning

Look closely, there are backward quotes on that line. That key is near the top left on your keyboard.

We need to get this running now, so do this:

$ source ~/.bashrc
Agent pid 4032

Now, set up this file (you may actually create it):

$ vim ~/.ssh/config

Add this text:

Host *
AddKeysToAgent yes
IdentityFile ~/.ssh/id_rsa

At this point, your system should ask for a password only on the first attempt to communicate with GitHub. Unfortunately, in my VM, this setup did not work. A bit more digging on Google turned up a solution.

When you clone a repo, you can use one of two different URLs. These use different secure protocols to communicate. One is a conventional web browser URL that starts with https://github.com. The other one uses SSH and looks like this: git@github.com:rblack42/. Both are available to copy to your clipboard on GitHub. When you browse to a repo on GitHub and click on the “Clone this repo” button, the default URL that displays uses HTTPS. At the top of the box showing that URL there is a link that says “Use SSH”. Selecting that, then copying the URL to the clipboard works for me.

$ git clone git@github.com:<username/<reponame>.git

The system will now only ask for your password one time, the first time you connect to GitHub after you launch a new terminal session. All future Git interactions shoudl run with no prompts! Great!

IMHO, this is worth the effort, especially if you try for the record number of commits in this class (currently 971!)

Automating Git Commands

Now that we know about the “alias” trick, let’s shorten up those git commands you will be typing (a lot):

Here are the settings in my ~/.bash_aliases file:

alias pbcopy='xclip -selection clipboard'
alias ga='git add \.'
alias gc='git commit'
alias gp='git push origin master'
alias gs='git status'

This file (minus that first line) is exactly what I set up on all my Linux/Mac systems. Macs are so similar to Linux, it is not worth worrying about it, at least on the command line!

Now, my “stop work” commands lopk like this:

$ cd ~/cosc2325/<project dir> - just make sure you are at the top
$ gs
$ ga
$ gc -m "This is what I did"
$ gp

That is far easier than before.

Warning

The alias hides what is happening. It is important that you know the commands, just for when you find yourself working on a machine that has not been “trained” with your “aliases”

I will show more tricks like this later.