Our Development Process

You are going to build several software products in this course. How you do that is as important as what you produce. Let’s look at the tools you need, and the development process.

In this course, you will get exposed to several new tools and software development procedures. Of course, I could ignore all of that, but doing so is a disservice to you. If you expect to work in any field where you will create software, you need to be prepared for what you will find there.

Warning

Some students complain that I ask too much for one course. Maybe so, but you need to learn this stuff, and the sooner you start, the better off you will be. My goal is simple, to turn you into developers worth hiring!

Development Environment

No, you will not sit down and fire up Dev-C++ and write code. Neither will you sit down and use the Python IDE you may have used in that course. You will be expected to work with the tools in use at that organization. Perhaps, you will be asked to work with no fancy idE at all.

You will also be expected to work with the software management tools that organization uses to protect the code while it is being developed. Let’s start off by setting up your work environment.

Linux Command Line

My answer to all of this is to ask you to work on the command line. Although this is admittedly an old fashioned way to work, it is an essential way to work when you need to work on computers somewhere out there in the “cloud”. More and More, the applications we use are tightly coupled to servers in that cloud. Are you ready for that?

As you have seen, we will be working in Linux. You will need to do a bit of reading and practice to help you with that. Here is a resource I recommend:

This site offers a free PDF version of the book with that title, and there is online tutorial associated with this book. I keep the PDF on my laptop as a reference.

Programmer’s Editor

You will be asked to use a good programmer’s editor. I recommend (but to not demand) that you learn Vim, a tool that has been an important part of computing for about 35 years. It is cross-platform, free, and powerful enough to be very useful. It is worth learning something about.

Here is a tutorial site that should help:

Code Management with Git

The last tool I am going to introduce you to is Git. This tool, created by Linus Torvalds to help him manage the thousand or so programmers who contribute to the Linux Kernel code. Git is a very powerful tool, but we will just touch the surface on what you can do with it.

Basicaly, Git watches the code (and other files) that make up your project. Git is not really interested in the actual program file you create from your code, that file can be generated as needed, and will surely change every time you modify your code. Instead, Git focuses on the source code needed to recreate that program.

Git is free, and is available for any platform. The installer on Windows even offers to install a set of tools that will let you run many Linux commands on your Windows PC. If you move between those two systems (which I do), that can keep you from muttering four letter words when you type a Linux command on your PC! YMMV!

The notes on installing Git are here:

Here is a free book (in PDF form) that you should download and browse. It gives very detailed instructions on how to use Git.

Project Timeline

Git is a tool that tracks changes to your project files. Those changes might be file additions, deletions, or, more often, modifications. How Git does that is cool, but not important now. Basically, what we have to understand is how we need to work using Git in the process.

We will march forward in time, working through several (many) development sessions! It is that simple. You probably already do that, but we will tune up your process and get you into a much better development workflow.

Essentially, we will view your development process as a sequence of work sessions over a period of time. At the end of each work session, we will mark the state of your code in a way that lets us “back up” if needed to a previous version.

Why would you need to do that?

Well, suppose you have a cool idea on how to proceed. You work on that idea for a few sessions and then discover your idea was really not so cool. Now what? Without a tool like Git you have to undo all those changes you made. How long is that going to take, and can you een do that?

With Git, it is a simple command and you are done. Cool!

Once we start managing your project with Git we call is a “repository, or “repo” for short. This is just to remond you that it is not just a normal directory. You need to think about what you are doing to keep this folder in sync with your work.

The changes to your workflow are few, but very important.

Note

If needed, create some note cards with the basic commands we will introduce. Use them until this gets to be part of how you work!

OK! Let’s start our project!

Starting the Project

Imagine you are starting a new project. You should start off by setting up a suitable home for that project. That home will be a single directory on your development system, hopefully in a place you can find easily. Everything needed to build and run this project should live under this one directory. Name your project now, and use that name for the directory name.

Create the Project Directory

This project is named “coolproject”.

(Boring, but suitable for now!)

Here is how we create our project directory:

$ mkdir coolproject
$ cd coolproject

We now have a home for our project, but no code yet. We need to tell Git to watch this directory:

$ git init
$ ls -al
total 0
drwxr-xr-x   3 rblack  staff   96 Jan 23 15:34 .
drwxr-xr-x  15 rblack  staff  480 Jan 23 15:33 ..
drwxr-xr-x   9 rblack  staff  288 Jan 23 15:34 .git

Note

The “.” and “..” are aliases for the current directory name, and the name of the one above this one. Those aliases are handy on “CD” commands!

The “.git” directory is where Git stores everything it needs to track your work. NEVER change anything in that directory, unless you know what you are doing! If you want to unhook your project from Git, all you need to do is delete this directory, and everything under it!

Warning

Watch out as you work. It is not a good idea to put one directory being managed by Git under another project directory that is also being managed by Git. It is possible to set up “subprojects” but I will leave that topic for your later learning. For now, just do not do that. I will be providing sample code for your projects on my public GitHub account. Do not place copies of those projects in your own project folders! More on that later.

Setting up GitHub

Although it is quite possible to work on your project just on your development system, it is far more common for a server to enter the picture here. That server will be able to keep a copy of your project directory on its file system. For our work here, we will use GitHub, a system set up for open-source developers to manage their code.

Warning

Do not put class work on your public account. Doing so is a violation of classroom rules!

Having your code on a server is a nice way to have a backup of your work, but it is much more than that! It lets us work on the project on multiple machines. Maybe you have a desktop at home, another at your work place, and still another that you can use at school. You can install the required build tools (compilers, etc) on each of there, and use a copy of your project code on each one to do your work. The trick to making this happen is synchronizing the files on all three systems (four, counting the server). Git can do this!

For now, let’s assume you have everything set up, including Git on multiple systems, and you want to start work!

GitHub Setup

Open up a web browser and go to GitHub. Log in with your account name and password. Then click on your “profile image” at the top right of the screen.

Note

For most of you, this is a funny square block with smaller blocks inside. For me it is my Gravatar image! (You should look into setting up a Gravatar image for your work now, it is part of your professional image on the Internet!

Click on your “Profile” and you will see a list of your current “repos”. If you have none yet, the list will be empty. Above that list is a menu item that says “repositories”. Click on that. You should see a button on the right side of the window. You click on that to create a new “repo”

../_images/new-repo.png

Click on this button and fill in the form that appears:

../_images/create-repo.png

Here, I named the repo, and made it “public”. If you try to make it “private” you will discover you need to sign up for a paid GitHub account. That is your choice. I have a paid account, because I do want to keep some of my work private. We will proceed with the public account.

Click on the Create Repository to complete this process

At this point, you have a repository on the GitHub. It has nothing in it, which is fine. (You can create a README file, more on that later!)

The next screen that comes up tells you what to do next. Pay attention here. i .. warning:

What follows in these notes is an example for you to use in your own
project, not those you do for my class. We will work through the class
process in our next lecture. This lecture is background. You should follow
along, doing this on your own development system, to get familiar with Git_
and GitHub_.
../_images/clone-guide.png

Connect GitHub to your Development Repo

Since we already started our project and have a repo sitting on our local workstation, we need to tie that repo to this new GitHub repo. THis is pretty simple.

When you are sitting on your development machine, we call the GitHub server a “remote”. What we need to to is tell Git which “remote” we want to connect our local repository to. You can hook your local repo to multiple “remotes”, but we will not explore that.

While working on the command line, in the project directory (the one with the .git directory in it), type this command. Note that the URL I use is the one we see on the image you got when you created your GitHub repo above:

$ git remote add origin https://github.com/rblack42/coolproject.git

Warning

Do not type exactly what you see here. Change your user name to your own (not rblack42) and your project name to whatever you set up.

The term “origin” above is another alias, this one refers to the remote server.

At any time after this has been done, you can check that your local repository is properly connected to GitHub by doing this:

$ git remote -v
origin      https://github.com/rblack42/coolproject.git (fetch)
origin      https://github.com/rblack42/coolproject.git (push)

This output confirms that we are properly connected. Now we can start work.

Let’s add some files to our local repository, and send them up to the server.

Add a README FIle

It is common to add a basic “README” file to your project. GitHub will display this file on your project page in a nice way.

Open up your favorite editor (Vim, of course) and create a file named README.rst that looks like this:

The file, which ends in .rst is written in a simple markup language known as reStructuredText. This markup is the standard way most Python programmers create documentation for their projects. It is so popular, many other developers use it as well. In fact, all of my lecture notes are written in this markup, and processed with a Python tool named Sphinx we will look at that tool later.

When you see this file on GitHub it will have been processed into a nice looking page, courtesy of a Python tool that can convert reStructuredText files into nice web pages.

Create a .gitignore

We need to set up one more file before we move on.

Git can manage just about any file you place in your project. We do not want every file to be tracked, though. Any file that can be built using our source code is definitely something I care about tracking. I really only want to watch the source code files, and any other files needed to build my program.

Set up this file now. It will be explained in class:

This will keep Git from tracking silly files your system may create in thie folder, and any files created by the compiler. The lines in this file are “patterns” used to see if Git should track them, or not. For example *.o tells Git to ignore any file than ends in .o. The compiler generates those files, as we will see later.

Now, let’s send our changes to GitHub

Push Changes to GitHub

The set of commands here is somehting you should learn. You will use them over and over.

There are four lines we will use. Each line does something different.

Git Status Check

The first command asks Git to tell us what it thinks has changed in our project.

$ git status
On branch master

Initial commit

Untracked files:
  (use "git add <file>..." to include in what will be committed)

    .gitignore
    README.rst

nothing added to commit but untracked files present (use "git add" to track)

There is a lot of stuff here, but it is pretty simple.

Git Branches

You do not think much about the path you move along as you develop code. You just work through a number of sessions until you are done. But that is not how you will always work. For example, say you want to try an experiment. You do not know if it will work or not. If it does, you will make it part of your project. If it does not, you will trash it. Git is ready for this.

We will go over branches later. For now, just note the name “master”. It refers to your local repository.

“Add” your Changes

Git has noticed that you have new files in your project, and you have not told Git to track them. Git does not automatically track anything, you have to tel it to. That .gitignore file keeps silly names from appearing in this list.

To have Git track these new files, we use this command:

$ git add .

There are variations to this command, but this is the one most often used. That “dot” at the end tells git to add every change if can fine to the project, which is most likely what you want to do.

Let’s see what Git thinks now:

$ git status
On branch master

Initial commit

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)

    new file:   .gitignore
    new file:   README.rst

Now, Git has marked these changes. They are not fully part of your project yet, that will come next. You can continue to work, and add again. The list of “new files” will grow.

“Commit” your Changes

What we do now is interesting. We will “commit” this new version of our project. The term “commit” means that Git will set up a copy of the entire project at this moment in time. It will save that copy in a very efficient way inside of that .git directory. You will now see that. Git will be able to recover this version at a later time if needed. in fact, you can move your directory backward in time to see earlier versions whenever you like. We will not explore that here. (Read the Pro Git book for details).

Here is how ww do this:

$ git commit -m "First commit"
[master (root-commit) 0bb7e47] First commit
 2 files changed, 27 insertions(+)
 create mode 100644 .gitignore
 create mode 100644 README.rst

The string after that -m is a log message that will be recorded with this version. The log is a great way to find a particular version later.

Note

I you fail to add the -m option, and a message string, Git will launch an editor so you can add the message. You need to get in the habit of recording something about the changes you just made. Keep it short, it will help you later.

Checking in with Git this time give is this:

$ git status
On branch master
nothing to commit, working tree clean

We can stop work now. Git has tracked everything we have done up to this point.

Well, do not walk away, we have one more step!

“Push” to GitHub

We need to get our new version on the GitHub server. That way if lightning strikes our system, we will not lose our work!

Here is what we do now:

$ git push -u origin master
Counting objects: 4, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 546 bytes | 0 bytes/s, done.
Total 4 (delta 0), reused 0 (delta 0)
To https://github.com/rblack42/coolproject.git
 * [new branch]      master -> master
Branch master set up to track remote branch master from origin.

Here the two terms “origin” and “master” are being used. You are “pushing” all of your changes from “master” to the remote server named “origin”. The -u is only used once in your work on this project. When you “push” other work, leave that part off. The messages you see here confirm that your modifications have been transmitted to GitHub.

Now what does Git think?

$ git status

Walk Away

Now, it is safe to stop work. To prove that, go back to your web browser and click on your project name in your list of repositories, You should see something like this:

../_images/project-repo.png

Nice!

This is enough for this session. Maybe your head hurts. Don’t worry about messing up. You can always navigate to the bottom of your project page on GitHub and delete this repository and start over if needed.

Final Note

Here is the “mantra” you will use at the end of any work session, where you are using Git and GitHub:

$ git status
$ git add .
$ git commit -m "log message"
$ git push origin master

At the end of this “mantra”, check your work on GitHub using your wen browser. If needed, refresh the page to see the current version. You should see all of the files you just modified or added, and not any files you may have deleted.

There is a lot of tutorial material on GitHub and on the Internet. Look around and learn n=more!