File Input/Output¶
We now have a decent set of tools for solving problems that are more realistic than the simple ones we have studied up to now. We have one more basic concept to get into, though - reading and writing data files.
In the real world (wherever that is!) data sets that you need to process can get huge. In my supercomputer days, we regularly processed data sets 10 gigabytes big (or more). Often, these data sets were created by other programs or by experimental systems, and needed to be analyzed to figure out what went on during a test. In order to understand how we can deal with problems like this, we need to understand how a computer can access that thing we call a file!
What is a File, anyway?¶
Basically, a file is just a sequence of bytes of data written onto a disk and given a name so the operating system can find it. We will stream those bytes into our programs, putting them someplace (like in an array, perhaps) and stream them back out to be written into another file when we are done.
What the stream of bytes represents is very problem dependent, and we will need to teach our programs how to break up the stream into chunks that make sense. For instance, a stream of integer numbers would be broken every four bytes (if integers are 32 bits long in our system). Such a file would be called a binary file since the data is really written out in binary form. Other files may be made up of character data. In this case, the stream is just a bunch of characters written one character per byte. Files contains streams of characters are called text files for obvious reasons.
Character steams get treated a bit differently in the computer world, since they get processed so often. We usually think of streams of characters in terms of lines of text broken up by some kind of end-line marker. Unfortunately, the exact end-line marker differs between systems (ask Microsoft why) but we will not need to worry about this unless we move a file from Windows to Linux, for example.
Accessing a File¶
So, how do we access a file? Well, we have been doing something like this all along, anytime we use the cout and cin features of the C++ language. These two constructs are more properly called character stream routines - one for pulling characters into a program, and one for pushing characters out of a program. The cin stream is pulled in from a weird file called the keyboard which is attached to YOU! Only you know what the stream will contain! The cout stream is attached to the console, and gets displayed on a funny black background window displayed when you run the program.
We call the process of accessing a file reading and writing according to which way the data will go relative to our programs. We read data into our programs from a file, and write data from our programs to a file. Before we can do either, we need to open the files!
Opening a file¶
The first step we need to perform to access a file is to ask the operating system to hook us up with one (or more). To do this, we will need to come up with a name for the file we want to work with. This file can be new - meaning we want to create it, or it might already be in the file system the operating system controls, meaning we need to know its name. The name for a file can be anything the operating system allows - including a full path name which on Windows systems could be something like c:\COSC1315\lab11\data.set. (For reasons known only to M$, you need to use backslashes for path separators in C++ - Unix uses single forward slashes). If you just enter a name with no path information, the file is assumed to live in the same folder as the program executable file. (Yikes - another file we could read - wonder what that one looks like!)
We also need to tell the operating system which way our stream will go - in or out. (It is possible to do both at the same time, but that is something you learn in a later course!)
Here is how we do this magic in C++:
#include <fstream>
ifstream inFile;
ofstream outFile;
inFile.open("test.dat", ios::in);
outFile.open("testcopy.dat", ios::out);
if(inFile.is_open()) {
// process the input file
}
if outFile.is_open()) {
// process the output file
}
Warning
The program will look for your data file in the same directory where the
program executable file lives. In some systems (like CLion), that folder
may not be where your code lives. In that case, you have to provide the
fill path to the data file in your “open” line. Something like
C:/Users/rblack/cosc1315/FileGraphics/robot.txt”. (Ugly, but this will
work!). Another, easier, solution is to copy that data file into the right
spot. On CLion that spot is in a folder named “cmake-build-debug” in your
project folder. Better yet, tell your program to look for the file in
“../robot.txt”. The “..” says “look up one directory” for the file. Notice
that C wants you to use forward slashes to specify where files live -
even on Windows!
In the above example, we create two variables which are very different critters from those we have been building up to now. These variables (more properly - objects) will be connected to files by the operating systems. They are weird kinds of boxes that can get really big! Notice the ifstream and ofstream data types. The if stands for input file, and the of stands for output file. Make sense?
Once we have created these variables, we hook them up to the operating system files by calling the open function associated with those variables (once again, this is advanced C++ magic, we have seen this before!) The open function interacts with the operating system and sends it the name of the file we want to work with, and information needed to say which way we want data to move. (In both of these open functions the ios::in and ios::out stuff is actually not needed, since we already said which way we want to go when we created the variables. We will need to include something here later, though - so stay tuned!
There is always the possibility that something can go wrong when you try to open files for access. If you want to open a file for input, you expect the file to already exist. What happens if it does not? The operating system should tell you that so you can handle that situation. The inFile.is_open() function is a boolean function telling you if the file was found and successfully opened for reading. The outFile.is_open() function says the same thing for an output file. In this case, maybe there is no room for a new file, or you are trying to write some place where you are not allowed to write.
Reading, Writing, and Arithmetic¶
Now that we have files available, reading and writing (and a little math) should look familiar to you:
int mydata;
inFile >> mydata;
mydata = mydata * 5;
outFile << mydata << endl;
What has changed here? Simply the name of the variable we need to use. We use inFile here, where we used cin before, and we use outFile here, where we used cout before. In both cases, we will be working with streams of characters. The inFile variable expects to find characters in the stream, and will convert what it finds to whatever data type you tell it to read (an integer in this case). On output the outFile variable will take whatever data container you specify, and convert the data contained in that container into a sequence of characters and send it to the output file. Pretty simple, huh?
What about the end of the file?¶
We do have a bit of a problem now, that we did not have earlier. What happens when the program runs out of stuff to read when asking for data. With cin, you were the source of the data, and you never run out of stuff to say! If the file we are reading from runs out of stuff - it does have a real end! - we need a way to figure this out. There is another function that we can use - eof(). We will see this in action in the next example:
Let’s copy a file!¶
Just to do something simple, lets open a file for input, another for output, and make a copy of the file!
#include <cstdlib>
#include <iostream>
#include <fstream>
using namespace std;
int main(int argc, char * argv[])
{
cout << "File Copy Program" << endl;
ifstream inFile;
ofstream outFile;
char ch;
inFile.open("main.cpp");
if (inFile.is_open()) {
// we have a file for reading
outFile.open("newcopy.cpp");
if(outFile.is_open()) {
//we have a file for writing
inFile.get(ch);
while(!inFile.eof()) {
outFile << ch;
inFile.get(ch);
}
outFile.close();
} else {
cout << "Cannot open output file" << endl;
}
inFile.close();
} else {
cout << "input file not found" << endl;
}
system("PAUSE");
return EXIT_SUCCESS;
}
Boy, that is a lot of work just to copy a file. Well, it is, but it shows several patterns we will want to use when we play with files!
Notice how I set this code up. For each of the files, I checked if the file was successfully opened before proceeding to do the copy. If not, I write out an error message on the console so I can figure out what happened. Here is what the program generates running as it stands now:
File Copy Program
input file not found
Press any key to continue . . .
That makes sense, I did not have a file called mymain.cpp in the current directory where this program ran! Let’s change the file name to main.cpp (assuming you saved your project file as main.cpp) and see what happens:
File Copy Program
Press any key to continue . . .
Hmmm, what happened? Check the directory where your project lives. Do you see a new file called newcopy.cpp? If so, check it to see what it contains!
File Read Access Patterns¶
The most important pattern we see in the above example is the one we use to safely read data from a file. In the example above, rather than use the funky >> notation to read data from a file, I used inFile.get(ch), which reads a single character from the input file. (This avoids a situation where C++ is too smart for its own good when we are just trying to read a bunch of characters!)
The pattern here wants to make sure we have something useful to read from the file. Think about what could happen when you read from the file. The file could be missing, it could be empty, or it could have good stuff to read. What will we do in each case.
Checking for file missing¶
We handle the first case by making sure we get a good file open action. The is_open function will tell us if we got the result we want.
Checking if we have a character to read¶
The only way we will be able to see if we have a character to read is to try it and see! C++ sets up an error condition if it is asked to read something from a file and fails. We can check this error condition to make sure we got what we were after. So, the pattern we want to use to read however much stuff is in the file looks like this:
inFile.get(ch);
while(!inFile.eof()) {
// do something with ch
inFile.get(ch);
}
This is called a seeded loop since we seed the loop by trying our first read outside the loop. If we read a good data item, the system will not set an error flag. The function eof() - End of File - will tell us if we hit the end (or never had anything to begin with!) If we have a good data item, we enter the loop and immediately process that data item. After we finish, we try for the next data item. Once again, if we run out of stuff to read, the end of file error flag will be set and we will see that when we loop back and check it with eof() again. You should see how this will handle both the nothing to read situation, and the general situation where we have an arbitrary amount of stuff to read.
The copy routine above just reads characters from the input file, then turns around and writes them back out to the output file. Everything gets copied - even the end of line markers (which we see internally as a n character. Once we hit the end of the data, we close both files, which disconnects them from the program and tells the operating system to finish up for us.
If we fail to close files, the C++ system will do that for us when the program ends. However, if your program bombs and you do not get to the normal end, you may end up with files in an undefined state. Normally, the operating system will keep you out of trouble, but you might need to clean up after yourself if things go really wrong!
File Write Access¶
Normally, you will not run into problems writing to a file that you succeed in opening. The operating system will create a new, empty, file with the name you specify when it opens the file. Once that has been done, you write data to that file until you are finished. The data is streamed to the output file, like writing on a tape. You cannot back up and undo anything. (It is possible to back up, but we will not worry about that here.)
If the file you name for output already exists in the operating system files, the old copy of that file gets zapped and a new one is set up just like before. For this reason, you need to be careful that you do not kill a file you really want to keep around. Editing programs never write to the file they are working on (they create a new working file to play with) so they do not kill the old file you are working on before they save the new changes safely away. In this way, you can always kill the editor and not lose the original file.
What about adding stuff to a file?¶
Sometimes, we want to add data to the end of an existing file. We can do this very easily by using a different kind of open command. It looks like this:
outFile.open('datafile.txt',ios::app);
Here the ios::app says open the file for appending, and anything you write to this file will be added to the end of the previous contents.
Sorting an Array of Names¶
As an example of using files, let’s read in a list of names, and sort them. Then, let’s write out the names in order. This example is going to use code we have discussed in the last few lectures.
As usual, we will build up the program in baby steps!
Step one - getting started:¶
Here is our standard starting point:
#include <cstdlib>
#include <iostream>
using namespace std;
int main(int argc, char * argv[])
{
cout << "Name Sorter Program" << endl;
system("PAUSE");
return EXIT_SUCCESS;
}
I am not going to run this, you know what it will do! Next, let’s build a data file to use. Create this file and save it as names.dat in the folder with your project (you can use Dev-C++ to do this).
Flintstone, Wilma
Flintstone, Fred
Rubble, Barney
Dinosaur, Dino
Hmmm, how am I going to read this? Well, we saw an example of reading a bunch of characters earlier, can we read a bunch of characters into some string variables (like an array?) Sure! We need to set up the input file and string variables first:
#include <string>
#include <fstream>
ifstream inFile;
string line;
...
inFile >> myline;
while(!inFile.eof()) {
cout << myline.length() << " " << myline << endl;
inFile >> myline;
}
inFile.close();
...
This fragment sets up the input file, then starts off by trying a line read using the normal >> operator. It then just prints out what it read, and tries again. The loop will keep this up until the inner attempt to read fails, at which time, we leave the loop.
Name Sorter Program
16 Flintstone,Wilma
15 Flintstone,Fred
13 Rubble,Barney
13 Dinosaur,Dino
Press any key to continue . . .
Now, we need to store the names somewhere - like in an array!
string names[25]
int i = 0;
...
cout << myline.length() << " " << myline << endl;
names[i++] = line;
}
cout << "You read " << i << " names." << endl;
This is an interesting pattern. We are in a loop, reading lines. We have set up an integer (initialized to zero on the declaration) and we use that integer to index into the names array. As we save the current line into the right spot, we use a post increment operator on the index. What does this do?
Well, the real meaning of the i++ notation is this:
- take the current value out of i and use it in this spot
That means we save the line string in index names[i]
- When you get through using that value, increment it by one and put it back
That means when the smoke clears, i will be one bigger, for the next pass!
Just the kind of behaviour we want. When the loop ends, i will be the number of names we read!
Note
Now you know what the name C++ really means - it is one better than the old language C!
Name Sorter Program
16 Flintstone,Wilma
15 Flintstone,Fred
13 Rubble,Barney
13 Dinosaur,Dino
You read 4 names.
Press any key to continue . . .
Looks like we have the input side handled. Let’s prove it by displaying the names in the string array:
for(int j=0;j<i;j++) {
cout << "Name " << j << " " << names[j] << endl;
}
Name Sorter Program
16 Flintstone,Wilma
15 Flintstone,Fred
13 Rubble,Barney
13 Dinosaur,Dino
You read 4 names.
Name 0 Flintstone,Wilma
Name 1 Flintstone,Fred
Name 2 Rubble,Barney
Name 3 Dinosaur,Dino
Press any key to continue . . .
We are making progress. We have read the names into the array, and we can print them out. Now, let’s change the output part to write to our final file:
ofstream outFile;
outFile.open("SortedNames.dat");
..
for(j=0;j<i;j++) {
outFile << names[j];
}
outFile.close();
This is what we get in the file after running the program:
Flintstone,WilmaFlintstone,FredRubble,BarneyDinosaur,Dino
Tilt! It looks like we lost the end of line markers. We need to place one in the output file ourself (by adding an endl to the end of the output line. Then we get what we want:
Flintstone,Wilma
Flintstone,Fred
Rubble,Barney
Dinosaur,Dino
That worked just fine. Only, we wanted to sort these lines, and all we did was copy them! On to the next part!
Now, for the sorting¶
Here is a chunk of code that can sort an array of strings. It is called BubbleSort since it works by looking at all the names in the array and swapping them if they are out of order. This is pretty fancy code, but you should understand what each line does, even if the exact logic of the sort is a bit fuzzy. (It might help to think about how you might do this if the names were written on a deck of cards - on the first pass, you could examine any two successive cards in the deck, and exchange them if they are out of order. As you work your way through the deck, high cards will “bubble” their way toward the bottom of the deck. Once the first pass is complete, you do the job again. You keep this up until all the cards are in the right order) Note that the routine has some output statements so you can see how it works. We would pull those lines from a real routine we wanted to use.
void BubbleSort(string A[], int count) {
for(int i=0;i<count-1;i++) {
cout << "Pass" << i << endl;
for(int j=0;j<count-i-1;j++) {
cout << A[j] << " " << A[j+1] << endl;
if(A[j] > A[j+1]) {
string temp = A[j];
A[j] = A[j+1];
A[j+1] = temp;
}
}
}
}
This code sweeps over the array of strings from top to bottom. On the first loop it compares the first two items to see if they are in order. If they are, it moves on to the next two items and check them. If the first two are out of order, it swaps them moving the first one down. It them moves down and repeats this check on the next two. As it works its way through the array, you might be able to see that the “bigger” (in an alphabetic sense) string “bubbles” to the bottom of the array. We go back up to the top and do this again. Since we already put the last string at the bottom, we can stop one item from the bottom on the second pass, effectively bubbling the second to the last string into place. Keep this up and eventually, all the strings are in place. Think about it!
Here is what we get:
Dinosaur,Dino
Flintstone,Fred
Flintstone,Wilma
Rubble,Barney
Cool! Looks like we got it done!
Here is the entire program:
#include <cstdlib>
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
void BubbleSort(string[], int);
int main(int argc, char * argv[])
{
string names[25];
string myline;
int i = 0;
ifstream inFile;
ofstream outFile;
cout << "Name Sorter Program" << endl;
inFile.open("names.dat");
inFile >> myline;
while(!inFile.eof()) {
cout << myline.length() << " " << myline << endl;
names[i++] = myline;
inFile >> myline;
}
inFile.close();
cout << "You read " << i << " names." << endl;
cout << "Sorting data" << endl;
BubbleSort(names, i);
outFile.open("SortedNames.dat");
for(int j=0;j<i;j++) {
outFile << names[j] << endl;
}
outFile.close();
system("PAUSE");
return EXIT_SUCCESS;
}
void BubbleSort(string A[], int count) {
for(int i=0;i<count-1;i++) {
cout << "Pass" << i << endl;
for(int j=0;j<count-i-1;j++) {
cout << A[j] << " " << A[j+1] << endl;
if(A[j] > A[j+1]) {
string temp = A[j];
A[j] = A[j+1];
A[j+1] = temp;
}
}
}
}
This weeks assignment¶
Before you try this week’s lab, I want you to set up the code in this lecture and make sure you can make it work. Change the list of names and make it bigger to make sure your code works the way it is supposed to! Then you should be in good shape to try the lab.
This is the last lecture on material you are responsible for in this class. I have two additional lectures that I will post to give you a look at what else is available in the world of computer programming, but that material is not anything you need to learn for the last exam
Be sure to check out the end of term guidance posted on the web!
