Using the Eclipse IDE to access and modify a Python persistent dictionary.

In the previous article, we introduced Python database programming , the concept of persistent dictionaries, and different database modules such as dbm. In this article, we will put it all together and use the dbm module to create, access and modify a persistent dictionary.

All of the dbm modules support an open function to create a new dbm object. Once opened, you can store data in the dictionary, read data, close the dbm object as well as the associated data file/files, remove items and test for the existence of a key in the dictionary.

Python Database Programming: Creating a Persistent Dictionary

To open a dbm persistent dictionary, use the open function on the module you choose. For example, we can use this code to create a persistent dictionary with the dbm module:

import dbm

db = dbm.open('payroll', 'c')

# Add on item
db['Orioles'] = '118'
db['Yankees'] = '211'
db['Blue Jays'] = '120'

print(db['Orioles'])

# Close and save to disk
db.close()

When you run this script, you will see output like the following:

b'118'

This example, which creates a ‘payroll’ dictionary with three entries, uses the recommended dbm module. The open function requires the name of the dictionary to create. The name gets translated into the name of the data file or files that may already be on the disk. The dbm module may create more than one file (usually a file for the data and one for the index of the keys), but it does not always do this. The name of the dictionary is treated as a base file name, including the path. Usually, the underlying dbm library will append a suffix such as .dat for data. You can find the file yourself by looking for the file named payroll, most likely in your current working directory.

There is also an optional flag. The following table lists the available flags:

You can also set another optional parameter, the mode. The mode holds a set of UNIX file permissions.

The above code is simple. First, we use the open method of the dbm module, which returns a new dbm object (db), which we can then use to store and retrieve data.

Once we open a persistent dictionary, we can write values as we normally would with Python dictionaries, as shown in this example:

db['Orioles'] = '118'

Both the key and value must be strings and cannot be other objects, like numbers or Python objects. But if you want to save an object, you can serialize it using the pickle module:

import pickle

data = {
        'Orioles' : ['118', 'Dan Duquette', 'Buck Showalter', 'Camden Yards'],
        'Yankees' : ['211', 'Brian Cashman', 'Joe Girardi', 'Yankee Stadium III'],
        'Blue Jays' : ['120', 'Alex Anthopoulos', 'John Gibbons', 'Rogers Centre']
        }

with open('data.pickle', 'wb') as f:
    pickle.dump(data, f, pickle.HIGHEST_PROTOCOL)
    
with open('data.pickle', 'rb') as f:
    data = pickle.load(f)

Finally, the close method closes the file or files and saves the data to disk.

Python Database Programming: Accessing and Modifying the Persistent Database

With the dbm modules, you can treat the object you get back from the open function as a dictionary object. You can get and set values using code like the following:

db['key'] = 'value'
value = db['key']

Remember that the key and the value must both be text strings.

You can delete a value in the dictionary using del:

del db['key']

As with a normal dictionary, the keys method returns a list of all the keys:

for key in db.keys():
	# do something else

The keys method may take a long time to execute if there are a huge number of keys in the file. Also, this method may require a lot of memory to store the potentially large list that it would create with a large file.

Here’s a script we can use to access the persistent dictionary we created with the first script:

import dbm

# Open existing file
db = dbm.open('payroll', 'w')

# Add another item
db['Rays'] = '67'

# Verify the previous item remains
if db['Blue Jays'] != None:
    print('Found Blue Jays')
else:
    print('Error: Missing item')
    
# Iterate over the keys...may be slow
# May use a lot of memory
for key in db.keys():
    print('Key = ', key, ' value = ', db[key])
    
del db['Rays']
print('After deleting Rays, we have:')

for key in db.keys():
    print('Key = ', key, ' value = ', db[key])
    
# Close and save to disk
db.close()

When you run this script, you should see output similar to the following:

Found Blue Jays
Key =  b'Rays'  value =  b'67'
Key =  b'Orioles'  value =  b'118'
Key =  b'Yankees'  value =  b'211'
Key =  b'Blue Jays'  value =  b'120'

After deleting Rays, we have:

Key =  b'Orioles'  value =  b'118'
Key =  b'Yankees'  value =  b'211'
Key =  b'Blue Jays'  value =  b'120'

This script works with a small database of major league baseball teams and their payrolls (in millions of dollars). You need to run the first script in this article first. That example creates the dbm file and stores data in the file. This script then opens the preexisting dbm file.

The script opens the persistent dictionary payroll in read/write mode. The call to the open function will generate an error if the necessary data file or files do not exist on disk in the current directory.

From the previous example, there should be three values in the dictionary (the new script tests to see if one of them exists). This example adds the Tampa Bay Rays, with a payroll of $67 million, as another key.

The script verifies that the ‘Blue Jays’ key exists in the dictionary, using the following code:

if db['Blue Jays'] != None:
    print('Found Blue Jays')
else:
    print('Error: Missing item')

Next, the script prints out all of the keys and values in the dictionary:

for key in db.keys():
    print('Key = ', key, ' value = ', db[key])

Note that there should now be four entries.

After printing out all the entries, the script removes one using del:

del db['Rays']

The script then prints out all the keys and values again, which should result in three entries, as show in the output. Finally, the close method closes the dictionary, which involves saving all the changes to disk, so the next time the file is opened, it will be in the state we left it.

As you can see from these examples, the API for working with persistent dictionaries is very simple because it works with files and like dictionaries.

External Links:

Python Database Programming at wiki.python.org

Python Database Programming at python.about.com

Databases at docs.python-guide.org