Python provides an extensive suite of tools for interacting with CSV files. This article will discuss how to read and write CSV files when Python is calling the PSSE shots. For those unfamiliar with CSV files, each line is a record of information with the data within a record separated by a comma character.

CSV files can be used to store bus numbers along with their name and voltage to be read when ever necessary. Our example will focus on reading and writing a list of bus numbers and their corresponding name and voltage.

The following example will cover how to write out to a CSV file and then read the file back in.

First off, we need to import the csv module from the Python standard library:

1

import csv

Now we have access to the csv functionality. The most significant objects in this module are the csv.reader and csv.writer object. They do as their names suggest (read and write) very well.

Writing CSV

To write the data to file, we must open a file for writing, create a csv.writer object and pass it the file we wish it to write to.

1
2
3
4
5

# open a file for writing.
csv_out = open("mycsv.csv",'wb')

# create the csv writer object.
mywriter = csv.writer(csv_out)

The last call to csv.writer uses the default ‘excel’ dialect (more about dialect choices later).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

import csv

bus_numbers = ['101', '102', '36', '40']
bus_names = ['NUC_A', 'NUC_B', 'CATDOG', 'HYDRO_A']
voltage = [.99, 1.02, 1.01, 1.00]

# open a file for writing.
csv_out = open('mycsv.csv', 'wb')

# create the csv writer object.
mywriter = csv.writer(csv_out)

# writerow - one row of data at a time.
for row in zip(bus_numbers, bus_names, voltage):
    mywriter.writerow(row)

# always make sure that you close the file.
# otherwise you might find that it is empty.
csv_out.close()

or alternatively:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

import csv

bus_numbers = ['101', '102', '36', '40']
bus_names = ['NUC_A', 'NUC_B', 'CATDOG', 'HYDRO_A']
voltage = [.99, 1.02, 1.01, 1.00]

# open a file for writing.
csv_out = open('mycsv.csv', 'wb')

# create the csv writer object.
mywriter = csv.writer(csv_out)

# all rows at once.
rows = zip(bus_numbers, bus_names, voltage)
mywriter.writerows(rows)

# always make sure that you close the file.
# otherwise you might find that it is empty.
csv_out.close()

The difference is on lines 13 to 15. The for loop has been replaced with a single call to writerows.

Reading CSV

Reader objects are defined in the same way as the writer objects: using an open file object.

1
2

csv_in = open('mycsv.csv', 'rb')
myreader = csv.reader(csv_in)

We can then loop over the reader object to process one row at a time:

1
2
3
4
5
6
7
8
9
10

# Create the lists to store the columns.
bus_id = []
bus_names = []
voltage = []

for row in myreader:
    id, name, volt = row
    bus_id.append(id)
    bus_names.append(name)
    voltage.append(float(volt))

Notice that volt is converted to a float before it is appended to the list. All variables read in from a CSV file are read in as strings and must be manually converted to the appropriate type before use. The other two fields, id and name, were not converted because later they will be used as strings.

A more compact way of achieving the same result:

1
2
3
4
5

csv_in = open('mycsv.csv','rb')
myreader = csv.reader(csv_in)

bus_id, bus_name, voltage = zip(*myreader)
voltage = map(float, voltage)

The zip(* ) call above does some magic to transpose a list of rows into a list of columns. The map call applies the function passed in as the first argument to every item in the list passed in as the second argument and returns a new list of converted items. This completes the data type conversion from a string to floats.

Dialects

Because CSV is not a standardised format there are always small, incompatible changes between CSV files from different vendors. Fortunately, Python acknowledges this and handles these cases with a minimum of fuss by specifying the changes when creating the reading and writing objects. For instance, the following writer object will use a tab character \t, instead of a comma, to separate the data within a row.

1

mywriter = csv.writer(csv_out, delimiter='\t')

Several format options can be set at a time by listing them all in the initial definition of the object. A list of all modifiable properties can be found on the Python Standard Library website for the csv module. A complete group of these properties is called a dialect. The default dialect is the ‘excel’ dialect and represents the CSV format Excel uses to export its spreadsheets.

If you regularly need to specify many of these properties, it may be worthwhile defining a new dialect and using that to initialise the writer object. This process is covered on the csv entry on the Python Standard Library website.