Interactive Session

🐼 Reading Data into pandas

A panda, reading. MidJourney 5

Yesterday we ran pd.read_csv() on the Toolik weather file and a DataFrame appeared. This morning we will slow that one line of code down and work out what it actually does, what it gives you back, and which of its settings you will want to change. Every workflow we write for the rest of the week starts by getting a file into python correctly, so we give the reading step a session of its own.

By the end of this session you will be able to:

Getting Started

  1. Create the file. In the Explorer, hover over the EDS217 heading and click New File…, then type the name in full, extension included: Session_2A_Reading_Data.ipynb

  2. Check the kernel. The Kernel Selector in the notebook’s action bar should read Python 3.11.15 (Conda: eds217). If it reads anything else, click it, choose Change Kernel, and pick the eds217 entry.

  3. Add a title cell. Click + Markdown in the action bar, and give it this content, with today’s date:

# Day 2: Session 2A - Reading Data into pandas

[Session Webpage](https://eds-217-essential-python.github.io/course-materials/interactive-sessions/2a_reading_data.html)

Date: 09/01/2026

  1. Save with Ctrl + S (Cmd + S on macOS), and keep saving as you go.

  2. Add a code cell below the title cell.

Importing pandas

Every notebook we write with pandas starts the same way:

Code 
import pandas as pd

import pandas makes the library available. as pd gives the pandas namespace a short nickname so you can write pd.read_csv() instead of pandas.read_csv(). pd is the nickname used throughout the pandas documentation, the data science user community, and in most of the examples you will find online, so we will use it here too.

Nothing breaks if you pick a different nickname, but everybody who reads your code afterwards (including you, in a week) would likely bump on the difference.

Our data for today

We’ll work with air quality measurements from an OpenAQ monitoring station in Goleta, just up the road from campus. We stay with the same file in the next session, so it is worth getting to know.

Code 
url = 'https://eds-217-essential-python.github.io/data/openaq_goleta_measurments.csv'

Notice we put the address in a variable first. Naming the URL is a habit worth forming: the address is long, you will use it more than once, and a name keeps your read_csv line short and easy to read.

pd.read_csv()

Here is the whole line:

Code 
goleta = pd.read_csv(url)

pd.read_csv() takes a location, fetches the file, parses the commas, and returns a DataFrame, which we named goleta. Let’s look at the first few rows:

Code 
goleta.head()
location_id location_name parameter value unit datetimeUtc datetimeLocal timezone latitude longitude country_iso isMobile isMonitor owner_name provider
0 1186 Goleta o3 0.025 ppm 2024-07-12T01:00:00+00:00 2024-07-11T18:00:00-07:00 America/Los_Angeles 34.445301 -119.827797 NaN NaN NaN Unknown Governmental Organization AirNow
1 1186 Goleta o3 0.028 ppm 2024-07-12T02:00:00+00:00 2024-07-11T19:00:00-07:00 America/Los_Angeles 34.445301 -119.827797 NaN NaN NaN Unknown Governmental Organization AirNow
2 1186 Goleta o3 0.029 ppm 2024-07-12T03:00:00+00:00 2024-07-11T20:00:00-07:00 America/Los_Angeles 34.445301 -119.827797 NaN NaN NaN Unknown Governmental Organization AirNow
3 1186 Goleta o3 0.027 ppm 2024-07-12T04:00:00+00:00 2024-07-11T21:00:00-07:00 America/Los_Angeles 34.445301 -119.827797 NaN NaN NaN Unknown Governmental Organization AirNow
4 1186 Goleta o3 0.026 ppm 2024-07-12T05:00:00+00:00 2024-07-11T22:00:00-07:00 America/Los_Angeles 34.445301 -119.827797 NaN NaN NaN Unknown Governmental Organization AirNow
Note

🐍 read_csv accepts either a web address or a path to a file on disk. Both of the lines below are valid, and the only difference between them is what sits inside the quotes:

df = pd.read_csv('https://example.org/data.csv')   # from the web
df = pd.read_csv('data/measurements.csv')          # from a folder next to your notebook; the file must exist!

This week we will mostly read from the web, because then everyone in the room has exactly the same file.

✏️ Test your knowledge

The Santa Barbara station’s file sits in the same folder on the course site, under the name openaq_santa_barbara_measurments.csv. Read it into a variable called santa_barbara and display its first few rows.

DataFrames and Series

A DataFrame is a table: indexed rows and named columns, like a well-behaved spreadsheet.

Pull a single column out and you get a Series, which is one column of values with the row index (sometimes called row labels) still attached:

Code 
goleta['value']
0        0.025
1        0.028
2        0.029
3        0.027
4        0.026
         ...  
1957    10.000
1958     8.000
1959     8.000
1960     9.000
1961     5.000
Name: value, Length: 1962, dtype: float64

Check the types to see the difference plainly:

Code 
print(type(goleta))
print(type(goleta['value']))
<class 'pandas.core.frame.DataFrame'>
<class 'pandas.core.series.Series'>

Knowing whether you have a DataFrame or a Series matters more often than it sounds like it should. A lot of pandas methods return a Series rather than a DataFrame, and the methods available to you next depend on which of these two objects you are working with.

Code 
goleta['value'].head()
0    0.025
1    0.028
2    0.029
3    0.027
4    0.026
Name: value, dtype: float64
Note

🐍 Square brackets with one column name give you a Series. Square brackets with a list of column names give you back a DataFrame, even if the list has one item:

goleta['value']       # Returns a Series
goleta[['value']]     # Returns a DataFrame with one column
✏️ Test your knowledge

Pull out the parameter column. Confirm with type() that you have a Series, then display its first five values.

Column names as a list

Before we do anything else with a table, we usually want to know which columns are in it. The .columns attribute gives you the names:

Code 
goleta.columns
Index(['location_id', 'location_name', 'parameter', 'value', 'unit',
       'datetimeUtc', 'datetimeLocal', 'timezone', 'latitude', 'longitude',
       'country_iso', 'isMobile', 'isMonitor', 'owner_name', 'provider'],
      dtype='object')

The result is a pandas object rather than a plain Python list. To get the list, which is easier to read and easier to work with, use .tolist():

Code 
column_names = goleta.columns.tolist()
column_names
['location_id',
 'location_name',
 'parameter',
 'value',
 'unit',
 'datetimeUtc',
 'datetimeLocal',
 'timezone',
 'latitude',
 'longitude',
 'country_iso',
 'isMobile',
 'isMonitor',
 'owner_name',
 'provider']

A list is Python’s basic container for an ordered collection of things. You write one with square brackets and commas:

Code 
parameters_we_care_about = ['o3', 'pm25']
parameters_we_care_about
['o3', 'pm25']

You get items out by position, counting from zero:

Code 
print(column_names[0])
print(column_names[1])
location_id
location_name

And len() tells you how many items there are:

Code 
len(column_names)
15
Note

🐍 Python counts from zero, and if you are arriving from R (which counts from one) it takes some practice. The first item is at position 0, the second at 1, and the last one is at len(x) - 1. You can also count backwards: [-1] is the last item.

Code 
column_names[-1]
'provider'

Selecting columns with a list

The job we need a list for today is selecting columns. Give a DataFrame a list of column names, and you get back a smaller DataFrame with just those columns in it:

Code 
keep = ['location_name', 'parameter', 'value', 'unit']
goleta[keep].head()
location_name parameter value unit
0 Goleta o3 0.025 ppm
1 Goleta o3 0.028 ppm
2 Goleta o3 0.029 ppm
3 Goleta o3 0.027 ppm
4 Goleta o3 0.026 ppm

You’ll see the same selection written both ways. The inner brackets are the list, and the outer brackets are the selection:

Code 
goleta[['location_name', 'parameter', 'value', 'unit']].head()
location_name parameter value unit
0 Goleta o3 0.025 ppm
1 Goleta o3 0.028 ppm
2 Goleta o3 0.029 ppm
3 Goleta o3 0.027 ppm
4 Goleta o3 0.026 ppm
✏️ Test your knowledge

Build a list called time_columns containing 'datetimeUtc' and 'datetimeLocal', then use it to display just those two columns from goleta.

Choosing the index with index_col=

Every DataFrame has an index: the labels down the left-hand side. By default pandas numbers the rows 0, 1, 2, .... You saw those numbers in head().

Sometimes a column in the file is a better label than a row number. index_col= tells read_csv to use one:

Code 
goleta_by_time = pd.read_csv(url, index_col='datetimeLocal')
goleta_by_time.head()
location_id location_name parameter value unit datetimeUtc timezone latitude longitude country_iso isMobile isMonitor owner_name provider
datetimeLocal
2024-07-11T18:00:00-07:00 1186 Goleta o3 0.025 ppm 2024-07-12T01:00:00+00:00 America/Los_Angeles 34.445301 -119.827797 NaN NaN NaN Unknown Governmental Organization AirNow
2024-07-11T19:00:00-07:00 1186 Goleta o3 0.028 ppm 2024-07-12T02:00:00+00:00 America/Los_Angeles 34.445301 -119.827797 NaN NaN NaN Unknown Governmental Organization AirNow
2024-07-11T20:00:00-07:00 1186 Goleta o3 0.029 ppm 2024-07-12T03:00:00+00:00 America/Los_Angeles 34.445301 -119.827797 NaN NaN NaN Unknown Governmental Organization AirNow
2024-07-11T21:00:00-07:00 1186 Goleta o3 0.027 ppm 2024-07-12T04:00:00+00:00 America/Los_Angeles 34.445301 -119.827797 NaN NaN NaN Unknown Governmental Organization AirNow
2024-07-11T22:00:00-07:00 1186 Goleta o3 0.026 ppm 2024-07-12T05:00:00+00:00 America/Los_Angeles 34.445301 -119.827797 NaN NaN NaN Unknown Governmental Organization AirNow

The local timestamp is now the row label, and it’s no longer one of the columns:

Code 
goleta_by_time.columns.tolist()
['location_id',
 'location_name',
 'parameter',
 'value',
 'unit',
 'datetimeUtc',
 'timezone',
 'latitude',
 'longitude',
 'country_iso',
 'isMobile',
 'isMonitor',
 'owner_name',
 'provider']
Note

🐍 index_col= is our first keyword argument: an argument you pass by name rather than by position. read_csv has dozens of them, and you’ll meet a few more this week. The pattern is always name=value, after the required arguments.

Choosing an index

An index is most useful when its values are meaningful, and ideally when they are unique too. location_name would be a poor choice here, because every row has the same station name. A timestamp is a better choice, because it tells you when the measurement was taken. Be warned, though: a timestamp is only unique if the file records one measurement per moment, and plenty of files (this one included!) do not.

✏️ Test your knowledge

Read the Goleta file again, this time using datetimeUtc as the index, into a variable called goleta_utc. How many columns does it have, and how many does the original goleta have? Then run goleta_utc.index.nunique() and compare it with the number of rows. The caution above said this file records more than one measurement per moment, so check what that did to your index.

Putting it together

You now have the four lines that open just about every analysis you will write from here on:

import pandas as pd

url = 'https://eds-217-essential-python.github.io/data/some_file.csv'
df = pd.read_csv(url)
df.head()

Getting the file in with read_csv is the easy half. In the next session we take the same Goleta table and explore it properly, before we compute a single number from it.

Key points

  • import pandas as pd once, at the top of every notebook.
  • pd.read_csv(location) reads a CSV from a URL or a file path and returns a DataFrame.
  • df['col'] gives a Series (one column). df[['a', 'b']] gives a DataFrame.
  • df.columns.tolist() gives you the column names as a plain Python list.
  • Lists are ordered, written with [], indexed from 0, and measured with len().
  • index_col='name' chooses which column becomes the row labels.

Resources