Strings, Lists, and Files

Where We Just Were

• Python is a nimble language with:
  • Numbers, strings, and Booleans
  • Conditionals (if, elif, and else)
  • while loops (with break and continue)
• This lecture describes:
  • Lists, for storing collections of values
  • Functions, which reduce redundancy, and make programs easier to read
  • Reading and writing files
• Comment on this slide

But First, Strings

• A string is an immutable sequence of characters
• Immutable means that it cannot be modified once it has been created
  • I.e., you cannot change individual characters in place

  • str = 'abc'
    print 'str is', str
    str[0] = 'x'
    print 'str is now', str
    

    str is abc
    

    Traceback (most recent call last):
      File "immutable_err.py", line 3, in ?
        str[0] = 'x'
    TypeError: object does not support item assignment
    

  • Though you can of course assign a new string value to a variable

  • str = 'abc'
    print 'str is', str
    str = 'xyz'
    print 'str is now', str
    

    str is abc
    str is now xyz
    

• Sequence means that it can be indexed
  • Indices start at 0 (as they do in C)…
  • …so text[0] is the first character of the string text
  • The built-in function len returns the length of a string…
  • …so the index of the last character of text is len(text)-1
• Note: there is no separate data type for characters
  • A character is simply a string of length 1

element = "boron"
i = 0
while i < len(element):
    print element[i]
    i += 1

b
o
r
o
n

• Comment on this slide

Slicing, Bounds, and Negative Indices

• text[start:end] takes a slice out of text
  • Creates a new string containing the characters of text from start up to (but not including) end

  • val = "helium"
    print val[1:3], val[:2], val[4:]
    

    el he um
    

  • Sometimes helps to think of indices as being between elements

  • Figure 7.1: Visualizing Indices

• A few logical consequences:
  • text[1:2] is either the second character in text, or the empty string (if text doesn't have a second character)
  • So text[1:1] is always the empty string
    • From index 1, up to but not including index 1
  • And text[2:1] is always the empty string
• Negative indices count backward from the end of the string
  • x[-1] is the last character
    • A lot easier to read than x[len(x)-1]
  • x[-2] is the second-to-last character

  • Figure 7.2: Visualizing Negative Indices

  • val = "carbon"
    print val[-2], val[-4], val[-6]
    

    o r c
    

• Bounds checking rules:
  • Python always does an out-of-bounds check when you index a single item
  • But it truncates out-of-range indices when you take a slice

  • val = "helium"
    print val[1:22]
    x = val[22]
    

    elium
    

    Traceback (most recent call last):
      File "bounds.py", line 3, in ?
        x = val[22]
    IndexError: string index out of range
    

• Comment on this slide

String Methods

• A method is a function that's tied to a particular object
  • Invented to help programmers organize their code
  • To call method M of object X, type X.M()
  • We'll see how to create objects and methods of our own later
• Almost everything in Python has methods
  • Numbers are the only important exception
• String methods

  • Method	Purpose	Example	Result
    capitalize	Capitalize first letter of string	"text".capitalize()	"Text"
    lower	Convert all letters to lowercase.	"aBcD".lower()	"abcd"
    upper	Convert all letters to uppercase.	"aBcD".upper()	"ABCD"
    strip	Remove leading and trailing whitespace (blanks, tabs, newlines, etc.)	" a b ".strip()	"a b"
    lstrip	Remove whitespace at left (leading) edge of string.	" a b ".lstrip()	"a b "
    rstrip	Remove whitespace at right (trailing) edge of string.	" a b ".rstrip()	" a b"
    count	Count how many times one string appears in another.	"abracadabra".count("ra")	2
    find	Return the index of the first occurrence of one string in another, or -1.	"abracadabra".find("ra")	2
    		"abracadabra".find("xyz")	-1
    replace	Replace occurrences of one string with another.	"abracadabra".replace("ra", "-")	"ab-cadab-"

    Table 7.1: String Methods

• Notes
  • These methods don't have to be called on constant strings

    • text = "Hogwarts"
      print text.upper()
      print text.replace("rts", "sh")
      

      HOGWARTS
      Hogwash
      

  • These methods create new strings: they do not change the strings they're called on

    • text = "Hogwarts"
      print "method result is", text.replace("rts", "sh")
      print "variable text is", text
      

      method result is Hogwash
      variable text is Hogwarts
      

  • You can chain method calls together, and mix with indexing, concatenation, etc.
    • If the result of one method call is a string, you can immediately call a method on that result

    • text = "Hermione"
      print ':' + text.upper()[4:7].center(10) + ':'
      

      :   ION    :
      

    • Use this in moderation: long chains of method calls are hard to read and debug
  • You can use is to check whether one string appears in another

    • print "atta" in "gatcattcgat"
      print "tcga" in "gatcattcgat"
      

      False
      True
      

• Comment on this slide

Lists

• A list is a mutable sequence of objects
  • Mutable means that, unlike a string, it can be changed in place
  • Of objects means that lists can hold anything and everything
  • Think of it as a one-dimensional array, or vector, that automatically resizes itself as needed
• Write lists using square brackets: [], [3], [5, "b"]
  • The empty list is considered false
• Indexing and slicing work almost the same way for strings and lists:
  • Indexing or slicing a string always returns a string
  • Indexing a list returns a single item
  • Slicing a list returns a new list

  • x = ["a", 2, "bcd", 4]
    print x[0]
    print x[-1]
    print x[1:-1]
    

    a
    4
    [2, 'bcd']
    

  • As before, Python always checks bounds on element access, but truncates bounds on slices
• Assign a new value to a list element using x[i] = v

  • gases = ["helium", "argon", "neon"]
    print gases
    i = 0
    while i < len(gases):
        gases[i] = "changed"
        print gases
        i += 1
    

    ['helium', 'argon', 'neon']
    ['changed', 'argon', 'neon']
    ['changed', 'changed', 'neon']
    ['changed', 'changed', 'changed']
    

• Adding two (or more) lists concatenates their elements
  • Creates a new list

  • left = ['L', 'E', 'F', 'T']
    right = ['W', 'R', 'I', 'T', 'E']
    both = left + right
    print both
    

    ['L', 'E', 'F', 'T', 'W', 'R', 'I', 'T', 'E']
    

• Can't concatenate a string and a list
  • But list(text) creates a list whose elements are the characters of the string text

    • print list("Dumbledore")
      

      ['D', 'u', 'm', 'b', 'l', 'e', 'd', 'o', 'r', 'e']
      

• del deletes something from the list, and shortens the list
  • The “something” can be a slice, in which case several items are deleted

  • values = ['a', 'b', 'c', 'd', 'e']
    print 'original', values
    del values[2]
    print 'after deleting item 2', values
    del values[-2:]
    print 'after deleting last two remaining items', values
    

    original ['a', 'b', 'c', 'd', 'e']
    after deleting item 2 ['a', 'b', 'd', 'e']
    after deleting last two remaining items ['a', 'b']
    

  • Note: del modifies the list, and doesn't return anything
• Comment on this slide

List Methods

• Like strings, lists are objects, and have methods
  • In the examples below, assume x is initially ['c', 'a', 'b', 'c']

  • Method	Purpose	Example	Result
    append	Add to the end of the list.	x.append(99)	['c', 'a', 'b', 'c', 99]
    count	Count how many times something appears in the list.	x.count('c')	2
    index	Find the first occurrence of something in the list.	x.index('a')	1
    insert	Insert something into the list.	x.insert(2, 'diamond')	['c', 'a', 'diamond', 'b', 'c']
    remove	Remove the first occurrence of something from the list.	x.remove('c')	['a', 'b', 'c']
    reverse	Reverse the list in place.	x.reverse()	['c', 'b', 'a', 'c']
    sort	Sort the list in place.	x.sort()	['a', 'b', 'c', 'c']

    Table 7.2: List Methods

• Notes
  • It's an error if list.index can't find the item it's searching for
    • The next lecture describes how to handle errors
  • list.reverse and list.sort change the list, and return None
    • x = x.reverse() is a common error
    • It reverses x, but then sets x to None, so all data is lost
• Comment on this slide

For Loops and Ranges

• Python's for statement automatically loops over the content of a collection
  • for x in text sets x to each character of text in turn
  • for x in someList sets x to each element of someList in turn

  • for c in 'lead':
        print '+' + c + '+',
    print
    
    for x in ['helium', 'argon', 'neon']:
        print x.capitalize()
    

    +l+ +e+ +a+ +d+
    Helium
    Argon
    Neon
    

  • The trailing comma in the print statement suppresses the automatic newline
• The built-in function range creates the list [start, start+1, ..., end-1]
  • Note: end-1, just as x[start:end] is the elements of x up to, but not including, end
  • range(end) is the same as range(0, end)
  • range(start, end, step) goes in increments of step
  • May generate an empty list

  • print range(2, 5)
    print range(3)
    print range(2, 10, 3)
    print range(3, 1)
    

    [2, 3, 4]
    [0, 1, 2]
    [2, 5, 8]
    []
    

• So:
  • To loop from 0 to N-1, use for i in range(N)
  • To loop over the indices of a list or string, use for i in range(len(sequence))

  • chars = "abc"
    for i in range(len(chars)):
        print i, chars[i]
    

    0 a
    1 b
    2 c
    

• Comment on this slide

Membership

• x in c is True if the value x is in the collection c
  • Equivalent to a substring test on strings
    • 'x' in 'Harry' is False
    • 'ar' in 'Harry' is True
  • Works element-by-element on lists
    • 3 in [1, 2, 3, 4] is True
    • [2, 3] in [1, 2, 3, 4] is False

  • vowels = 'aeiou'
    for v in vowels:
      if v in 'uranium':
          print v
    

    a
    i
    u
    

• Comment on this slide

Nesting Lists

• Since lists can contain anything, you can build lists of lists of lists of…
  • Example: use a list of two values to represent a point, and a list of points to represent a line segment

    • Figure 7.3: Line Segment

• Write indices left to right to select elements from the outside in

  • x = [[13, 17, 19], [23, 29]]
    print x[1]
    print x[0][1:3]
    

    [23, 29]
    [17, 19]
    

• The nested lists are objects in their own right
  • The outer list stores a reference to the inner list
    • But the inner list does not know that it's being referred to
    • Object in Python (almost) never know who's pointing at them
  • When you index the outer list, you get an alias for the inner list
    • Another name for the same data
  • Changes made through either reference update the same data

    • x = [['a', 'b'], ['c', 'd', 'e']]
      y = x[0]
      print 'before: nested list is', x
      print '...and sublist is', y
      
      y[0] = 123
      
      print 'after: nested list is', x
      print '...and sublist is', y
      

      before: nested list is [['a', 'b'], ['c', 'd', 'e']]
      ...and sublist is ['a', 'b']
      after: nested list is [[123, 'b'], ['c', 'd', 'e']]
      ...and sublist is [123, 'b']
      

    • Figure 7.4: Aliasing In Action

• I lied to you earlier when I said that indexing and slicing does the same thing on lists that it does on strings
  • The technical term is “pedagogically motivated misdirection”
  • Indexing returns whatever the list itself contains
    • In the case of a nested list, this is a reference to the sublist
  • Slicing returns a new list containing the selected elements of the original list
    • So changes to a slice do not affect the original list
    • Can't tell the difference with strings, since there's no way to update one in place anyway

    • x = [['a', 'b'], ['c', 'd', 'e'], ['f', 'g']]
      y = x[0:2]
      print 'before: nested list is', x
      print '...and slice is', y
      
      y[0] = 123
      
      print 'after: nested list is', x
      print '...and modified slice is', y
      

      before: nested list is [['a', 'b'], ['c', 'd', 'e'], ['f', 'g']]
      ...and slice is [['a', 'b'], ['c', 'd', 'e']]
      after: nested list is [['a', 'b'], ['c', 'd', 'e'], ['f', 'g']]
      ...and modified slice is [123, ['c', 'd', 'e']]
      

    • Figure 7.5: Slicing Copies Data

  • Yes, it can be confusing…
    • …but anything else would be unacceptably slow…
    • …so draw diagrams when you need to…
    • …and don't write overly-complicated code in the first place
• Comment on this slide

Tuples

• Python actually has a second type of list, called a tuple
  • Just like a normal list, but immutable (i.e., can't be changed after creation)
  • Written using parentheses instead of square brackets: (1, 2, 3) instead of [1, 2, 3]
  • Empty tuple is ()
  • Tuple with one element must be written with a comma, as in (55,)
    • Because (55) has to be just the integer 55, or the mathematicians will get upset
• Why bother?
  • Performance: if you know the length of the sequence isn't going to change, you can do some things more quickly
  • There are places where Python needs immutable objects that contain other values
    • We'll meet them in the next lecture
  • I still think it's a wart…
• Tuples allow Python to provide multi-valued assignment
  • a, b = 2, 3 assigns 2 to a, and 3 to b
    • Equivalent to a, b = (2, 3)
  • a, b, c = [1, 2, 3] does what you'd expect
  • a, b = [1, 2, 3, 4] is an error
    • Number of targets on the left must match the number of values on the right
  • a, b = b, a swaps the values in a and b
• Use multi-valued assignment in for loops to unpack structures on the fly

  • corners = [[1, 3], [2, 7], [4, 0], [4, 9]]
    sumX, sumY = 0, 0
    for cX, cY in corners:
        sumX += cX
        sumY += cY
    sumX = float(sumX) / len(corners)
    sumY = float(sumY) / len(corners)
    print "center is", sumX, sumY
    

    center is 2.75 4.75
    

• Comment on this slide

Files

• Use the built-in function open() to open a file
  • First argument is path
  • Second is 'r' (for read) or 'w' for write
  • Result is a file object with methods for input and output

infile = open('file.txt', 'r')
outfile = open('copy.txt', 'w')

line = infile.readline()
while line:
    outfile.write(line)
    line = infile.readline()

infile.close()
outfile.close()

• Here's what just happened:
  • First statement opens file.txt for reading, and assigns the file object to input
  • Second statement opens copy.txt for writing, and assigns the file object to output
  • Program then tries to read a line from input
    • If the file is empty, line is assigned the value None
    • Like Null in C, or null in Java, it means “nothing here”, and is considered false
  • As long as there are lines in the input file, the program:
    • Writes the most recent line to output
    • Reads the next line
  • When the input file is exhausted, the program closes input and output
    • Python will close the files automatically when the program exits…
    • …but it's good practice to tidy up your toys when your done playing with them
• Comment on this slide

Other Ways to Do It

• Could also copy files like this:

  • infile = open('file.txt', 'r')
    outfile = open('copy.txt', 'w')
    for line in infile:
        outfile.write(line)
    infile.close()
    outfile.close()
    

• Or this:

  • infile = open('file.txt', 'r')
    contents = infile.readlines()
    infile.close()
    
    outfile = open('copy.txt', 'w')
    outfile.writelines(contents)
    outfile.close()
    

• Moral of the story: get to know the standard library of whatever language you're using
  • A week of hard work can sometimes save you an hour of reading, too…
• Comment on this slide

Exercises

Exercise 7.1:

What does "aaaaa".count("aaa") return? Why?

Exercise 7.2:

What does the built-in function enumerate do? Use it to write a function called findOver that takes a list of numbers called values, and a number called threshold, as arguments, and returns a list of the locations where items in values are greater than threshold. For example, findOver([1.1, 3.8, -1.6, 7.4], 2.0) should return [1, 3], since the values in the input list at locations 1 and 3 are greater than the threshold 2.0.

Exercise 7.3:

What do each of the following five code fragments do? Why?

x = ['a', 'b', 'c', 'd']
x[0:2] = []

x = ['a', 'b', 'c', 'd']
x[0:2] = ['q']

x = ['a', 'b', 'c', 'd']
x[0:2] = 'q'

x = ['a', 'b', 'c', 'd']
x[0:2] = 99

x = ['a', 'b', 'c', 'd']
x[0:2] = [99]

Exercise 7.4:

What does 'a'.join(['b', 'c', 'd']) return? If you have a list of strings, how can you concatenate them in a single statement? Why do you think join is written this way, rather than as ['b', 'c', 'd'].join('a')?