.. _file-input-output: ################# File Input/Output ################# .. include:: /references.inc We now have a decent set of tools for solving problems that are more realistic than the simple ones we have studied up to now. We have one more basic concept to get into, though - reading and writing *data files*. In the real world (wherever that is!) data sets that you need to process can get huge. In my supercomputer days, we regularly processed data sets 10 gigabytes big (or more). Often, these data sets were created by other programs or by experimental systems, and needed to be analyzed to figure out what went on during a test. In order to understand how we can deal with problems like this, we need to understand how a computer can access that thing we call a *file*! What is a File, anyway? ####################### Basically, a *file* is just a sequence of bytes of data written onto a disk and given a name so the operating system can find it. We will *stream* those bytes into our programs, putting them someplace (like in an array, perhaps) and *stream* them back out to be written into another file when we are done. What the *stream* of bytes represents is very problem dependent, and we will need to teach our programs how to break up the stream into chunks that make sense. For instance, a stream of integer numbers would be broken every four bytes (if integers are 32 bits long in our system). Such a file would be called a *binary* file since the data is really written out in binary form. Other files may be made up of *character* data. In this case, the stream is just a bunch of characters written one character per byte. Files contains streams of characters are called *text files* for obvious reasons. Character steams get treated a bit differently in the computer world, since they get processed so often. We usually think of streams of characters in terms of lines of text broken up by some kind of end-line marker. Unfortunately, the exact end-line marker differs between systems (ask Microsoft why) but we will not need to worry about this unless we move a file from Windows to Linux, for example. Accessing a File ################ So, how do we access a file? Well, we have been doing something like this all along, anytime we use the **cout** and **cin** features of the C++ language. These two constructs are more properly called *character stream* routines - one for pulling characters into a program, and one for pushing characters out of a program. The **cin** stream is pulled in from a weird file called the *keyboard* which is attached to YOU! Only you know what the stream will contain! The **cout** stream is attached to the console, and gets displayed on a funny black background window displayed when you run the program. We call the process of accessing a file reading and writing according to which way the data will go relative to our programs. We read data into our programs from a file, and write data from our programs to a file. Before we can do either, we need to open the files! Opening a file ============== The first step we need to perform to access a file is to ask the operating system to hook us up with one (or more). To do this, we will need to come up with a name for the file we want to work with. This file can be new - meaning we want to create it, or it might already be in the file system the operating system controls, meaning we need to know its name. The name for a file can be anything the operating system allows - including a full path name which on Windows systems could be something like c:\\COSC1315\\lab11\\data.set. (For reasons known only to M$, you need to use backslashes for path separators in C++ - Unix uses single forward slashes). If you just enter a name with no path information, the file is assumed to live in the same folder as the program executable file. (Yikes - another file we could read - wonder what that one looks like!) We also need to tell the operating system which way our stream will go - in or out. (It is possible to do both at the same time, but that is something you learn in a later course!) Here is how we do this magic in C++: .. code-block:: c #include ifstream inFile; ofstream outFile; inFile.open("test.dat", ios::in); outFile.open("testcopy.dat", ios::out); if(inFile.is_open()) { // process the input file } if outFile.is_open()) { // process the output file } .. warning:: The program will look for your data file in the same directory where the program executable file lives. In some systems (like CLion), that folder may not be where your code lives. In that case, you have to provide the fill path to the data file in your "open" line. Something like C:/Users/rblack/cosc1315/FileGraphics/robot.txt". (Ugly, but this will work!). Another, easier, solution is to copy that data file into the right spot. On CLion_ that spot is in a folder named "cmake-build-debug" in your project folder. Better yet, tell your program to look for the file in "../robot.txt". The ".." says "look up one directory" for the file. Notice that ``C`` wants you to use forward slashes to specify where files live - even on Windows! In the above example, we create two variables which are very different critters from those we have been building up to now. These variables (more properly - *objects*) will be connected to files by the operating systems. They are weird kinds of boxes that can get really big! Notice the **ifstream** and **ofstream** data types. The **if** stands for *input file*, and the **of** stands for *output file*. Make sense? Once we have created these variables, we hook them up to the operating system files by calling the **open** function associated with those variables (once again, this is advanced C++ magic, we have seen this before!) The **open** function interacts with the operating system and sends it the name of the file we want to work with, and information needed to say which way we want data to move. (In both of these open functions the **ios::in** and **ios::out** stuff is actually not needed, since we already said which way we want to go when we created the variables. We will need to include something here later, though - so stay tuned! There is always the possibility that something can go wrong when you try to open files for access. If you want to open a file for input, you expect the file to already exist. What happens if it does not? The operating system should tell you that so you can handle that situation. The **inFile.is_open()** function is a *boolean* function telling you if the file was found and successfully opened for reading. The **outFile.is_open()** function says the same thing for an output file. In this case, maybe there is no room for a new file, or you are trying to write some place where you are not allowed to write. Reading, Writing, and Arithmetic ================================ Now that we have files available, reading and writing (and a little math) should look familiar to you: .. code-block:: c int mydata; inFile >> mydata; mydata = mydata * 5; outFile << mydata << endl; What has changed here? Simply the name of the variable we need to use. We use **inFile** here, where we used **cin** before, and we use **outFile** here, where we used **cout** before. In both cases, we will be working with streams of characters. The **inFile** variable expects to find characters in the stream, and will convert what it finds to whatever data type you tell it to read (an integer in this case). On output the **outFile** variable will take whatever data container you specify, and convert the data contained in that container into a sequence of characters and send it to the output file. Pretty simple, huh? What about the end of the file? =============================== We do have a bit of a problem now, that we did not have earlier. What happens when the program runs out of stuff to read when asking for data. With **cin**, you were the source of the data, and you never run out of stuff to say! If the file we are reading from runs out of stuff - it does have a real end! - we need a way to figure this out. There is another function that we can use - **eof()**. We will see this in action in the next example: Let's copy a file! ################## Just to do something simple, lets open a file for input, another for output, and make a copy of the file! .. code-block:: c #include #include #include using namespace std; int main(int argc, char * argv[]) { cout << "File Copy Program" << endl; ifstream inFile; ofstream outFile; char ch; inFile.open("main.cpp"); if (inFile.is_open()) { // we have a file for reading outFile.open("newcopy.cpp"); if(outFile.is_open()) { //we have a file for writing inFile.get(ch); while(!inFile.eof()) { outFile << ch; inFile.get(ch); } outFile.close(); } else { cout << "Cannot open output file" << endl; } inFile.close(); } else { cout << "input file not found" << endl; } system("PAUSE"); return EXIT_SUCCESS; } Boy, that is a lot of work just to copy a file. Well, it is, but it shows several patterns we will want to use when we play with files! Notice how I set this code up. For each of the files, I checked if the file was successfully opened before proceeding to do the copy. If not, I write out an error message on the console so I can figure out what happened. Here is what the program generates running as it stands now: .. code-block:: bash File Copy Program input file not found Press any key to continue . . . That makes sense, I did not have a file called **mymain.cpp** in the current directory where this program ran! Let's change the file name to **main.cpp** (assuming you saved your project file as **main.cpp**) and see what happens: .. code-block:: bash File Copy Program Press any key to continue . . . Hmmm, what happened? Check the directory where your project lives. Do you see a new file called **newcopy.cpp**? If so, check it to see what it contains! File Read Access Patterns ######################### The most important pattern we see in the above example is the one we use to safely read data from a file. In the example above, rather than use the funky **>>** notation to read data from a file, I used **inFile.get(ch)**, which reads a single character from the input file. (This avoids a situation where C++ is too smart for its own good when we are just trying to read a bunch of characters!) The pattern here wants to make sure we have something useful to read from the file. Think about what could happen when you read from the file. The file could be missing, it could be empty, or it could have good stuff to read. What will we do in each case. Checking for file missing ========================= We handle the first case by making sure we get a good file open action. The **is_open** function will tell us if we got the result we want. Checking if we have a character to read ======================================= The only way we will be able to see if we have a character to read is to try it and see! C++ sets up an error condition if it is asked to read something from a file and fails. We can check this error condition to make sure we got what we were after. So, the pattern we want to use to read however much stuff is in the file looks like this: .. code-block:: c inFile.get(ch); while(!inFile.eof()) { // do something with ch inFile.get(ch); } This is called a *seeded loop* since we *seed* the loop by trying our first read outside the loop. If we read a good data item, the system will not set an error flag. The function **eof()** - End of File - will tell us if we hit the end (or never had anything to begin with!) If we have a good data item, we enter the loop and immediately process that data item. After we finish, we try for the next data item. Once again, if we run out of stuff to read, the *end of file* error flag will be set and we will see that when we loop back and check it with **eof()** again. You should see how this will handle both the nothing to read situation, and the general situation where we have an arbitrary amount of stuff to read. The copy routine above just reads characters from the input file, then turns around and writes them back out to the output file. Everything gets copied - even the end of line markers (which we see internally as a **\n** character. Once we hit the end of the data, we close both files, which disconnects them from the program and tells the operating system to finish up for us. If we fail to close files, the C++ system will do that for us when the program ends. However, if your program bombs and you do not get to the normal end, you may end up with files in an undefined state. Normally, the operating system will keep you out of trouble, but you might need to clean up after yourself if things go really wrong! File Write Access ================= Normally, you will not run into problems writing to a file that you succeed in opening. The operating system will create a new, empty, file with the name you specify when it opens the file. Once that has been done, you write data to that file until you are finished. The data is streamed to the output file, like writing on a tape. You cannot back up and undo anything. (It is possible to back up, but we will not worry about that here.) If the file you name for output already exists in the operating system files, the old copy of that file gets zapped and a new one is set up just like before. For this reason, you need to be careful that you do not kill a file you really want to keep around. Editing programs never write to the file they are working on (they create a new working file to play with) so they do not kill the old file you are working on before they save the new changes safely away. In this way, you can always kill the editor and not lose the original file. What about adding stuff to a file? ================================== Sometimes, we want to add data to the end of an existing file. We can do this very easily by using a different kind of open command. It looks like this: .. code-block:: text outFile.open('datafile.txt',ios::app); Here the **ios::app** says open the file for appending, and anything you write to this file will be added to the end of the previous contents. Sorting an Array of Names ########################## As an example of using files, let's read in a list of names, and sort them. Then, let's write out the names in order. This example is going to use code we have discussed in the last few lectures. As usual, we will build up the program in baby steps! Step one - getting started: =========================== Here is our standard starting point: .. code-block:: c #include #include using namespace std; int main(int argc, char * argv[]) { cout << "Name Sorter Program" << endl; system("PAUSE"); return EXIT_SUCCESS; } I am not going to run this, you know what it will do! Next, let's build a data file to use. Create this file and save it as **names.dat** in the folder with your project (you can use Dev-C++ to do this). .. code-block:: bash Flintstone, Wilma Flintstone, Fred Rubble, Barney Dinosaur, Dino Hmmm, how am I going to read this? Well, we saw an example of reading a bunch of characters earlier, can we read a bunch of characters into some string variables (like an array?) Sure! We need to set up the input file and string variables first: .. code-block:: c #include #include ifstream inFile; string line; ... inFile >> myline; while(!inFile.eof()) { cout << myline.length() << " " << myline << endl; inFile >> myline; } inFile.close(); ... This fragment sets up the input file, then starts off by trying a line read using the normal **>>** operator. It then just prints out what it read, and tries again. The loop will keep this up until the inner attempt to read fails, at which time, we leave the loop. .. code-block:: bash Name Sorter Program 16 Flintstone,Wilma 15 Flintstone,Fred 13 Rubble,Barney 13 Dinosaur,Dino Press any key to continue . . . Now, we need to store the names somewhere - like in an array! .. code-block:: c string names[25] int i = 0; ... cout << myline.length() << " " << myline << endl; names[i++] = line; } cout << "You read " << i << " names." << endl; This is an interesting pattern. We are in a loop, reading lines. We have set up an integer (initialized to zero on the declaration) and we use that integer to index into the **names** array. As we save the current line into the right spot, we use a *post increment* operator on the index. What does this do? Well, the real meaning of the i++ notation is this: * take the current value out of i and use it in this spot * That means we save the line string in index names[i] * When you get through using that value, increment it by one and put it back * That means when the smoke clears, i will be one bigger, for the next pass! Just the kind of behaviour we want. When the loop ends, **i** will be the number of names we read! .. note:: Now you know what the name C++ really means - it is one better than the old language `C`! .. code-block:: c Name Sorter Program 16 Flintstone,Wilma 15 Flintstone,Fred 13 Rubble,Barney 13 Dinosaur,Dino You read 4 names. Press any key to continue . . . Looks like we have the input side handled. Let's prove it by displaying the names in the string array: .. code-block:: c for(int j=0;j A[j+1]) { string temp = A[j]; A[j] = A[j+1]; A[j+1] = temp; } } } } This code sweeps over the array of strings from top to bottom. On the first loop it compares the first two items to see if they are in order. If they are, it moves on to the next two items and check them. If the first two are out of order, it swaps them moving the first one down. It them moves down and repeats this check on the next two. As it works its way through the array, you might be able to see that the "bigger" (in an alphabetic sense) string "bubbles" to the bottom of the array. We go back up to the top and do this again. Since we already put the last string at the bottom, we can stop one item from the bottom on the second pass, effectively bubbling the second to the last string into place. Keep this up and eventually, all the strings are in place. Think about it! Here is what we get: .. code-block:: bash Dinosaur,Dino Flintstone,Fred Flintstone,Wilma Rubble,Barney Cool! Looks like we got it done! Here is the entire program: .. code-block:: c #include #include #include #include using namespace std; void BubbleSort(string[], int); int main(int argc, char * argv[]) { string names[25]; string myline; int i = 0; ifstream inFile; ofstream outFile; cout << "Name Sorter Program" << endl; inFile.open("names.dat"); inFile >> myline; while(!inFile.eof()) { cout << myline.length() << " " << myline << endl; names[i++] = myline; inFile >> myline; } inFile.close(); cout << "You read " << i << " names." << endl; cout << "Sorting data" << endl; BubbleSort(names, i); outFile.open("SortedNames.dat"); for(int j=0;j A[j+1]) { string temp = A[j]; A[j] = A[j+1]; A[j+1] = temp; } } } } This weeks assignment ##################### Before you try this week's lab, I want you to set up the code in this lecture and make sure you can make it work. Change the list of names and make it bigger to make sure your code works the way it is supposed to! Then you should be in good shape to try the lab. This is the last lecture on material you are responsible for in this class. I have two additional lectures that I will post to give you a look at what else is available in the world of computer programming, but that material is not anything you need to learn for the last exam Be sure to check out the end of term guidance posted on the web!