An introduction

Python for
Climate Science

From a CSV of station rainfall to global reanalysis maps, in one afternoon.

Sabina Abba Omar · Climate System Analysis Group, University of Cape Town

Today

What we will do

  1. Python basics: what it is, how it runs, variables, lists, loops, functions, modules
  2. Station data with pandas: reading a CSV, summarising, grouping
  3. Time series plots with matplotlib
  4. Gridded data with xarray: NetCDF, selecting, unit conversion, reductions
  5. Spatial plots: maps, colour maps, regions, multi-panel figures
  6. Saving results, the Climate Data Store, and where to go next

Everything in these slides is also in the notebook intro_python_climate_science.ipynb. Open it now and follow along.

Two data files sit next to the notebook:

  • Sampledata.csv: daily rainfall at three sites, 2005 to 2024
  • fnl_africa_202302-202303.nc: NCEP FNL analysis cut to Africa, Feb to Mar 2023
Setup

Before we start

1. Install Python and JupyterLab

Install Miniconda, open a terminal (Anaconda Prompt on Windows), then:

conda create -n climate -c conda-forge python=3.12 \
    xarray netcdf4 pandas matplotlib cartopy jupyterlab
conda activate climate

2. Get the files

Put the notebook and both data files in one folder. In the terminal, cd into that folder and run jupyter lab. It opens in your browser.

Jupyter in three keys

KeyDoes
Shift + Enterrun the cell, move to the next
Tabautocomplete after a dot
? after a nameshow its documentation

Stuck? Kernel → Restart, then run from the top.

No install possible? Google Colab runs notebooks in the browser. Upload the files and run !pip install xarray netcdf4 cartopy first.

Part 1

Python basics

What it is, how it runs, and just enough syntax to read climate data.

Part 1 · Background

What is Python?

A high-level programming language known for its simplicity and readability.

Created by Guido van Rossum, first released in 1991. Now on version 3.

Free and open source: anyone can read, modify and distribute it. That is why a huge scientific community has grown around it.

Why climate scientists use it

  • Readable: code looks like the maths it describes
  • One language for everything: reading files, statistics, maps, papers' figures
  • The ecosystem: NumPy, pandas, xarray, matplotlib, cartopy, SciPy, all free
  • Shared conventions: NetCDF and CF metadata work out of the box
  • The community: CMIP, ERA5, Copernicus tutorials are written in Python
Part 1 · How it works

How Python runs

Python is interpreted: a program called the interpreter reads your code line by line and executes it. No compiling step.

You can run it three ways:

  • Interactively, typing one line at a time
  • As a script, a .py file run top to bottom
  • In a notebook, cells of code and text mixed, run in any order

Today we use Jupyter notebooks. A kernel is the interpreter running behind the notebook. It remembers every variable you define until you restart it.

The notebook shows the value of the last line of each cell. Use print() to show anything else.

Order matters. If a cell fails with NameError, a cell above it has not been run yet.

Part 1 · Syntax

Variables and types

A variable is a name for a value. Create one with =.

n_years    = 30          # int
temp_mean  = 16.4        # float
station    = "Cape Town" # str
is_raining = False       # bool

temp_day = [18, 20, 24, 16]   # list
coords   = (-33.9, 18.4)     # tuple
TypeHoldsCan change?
int, floatnumbersreplace
strtextno
listordered valuesyes
tupleordered valuesno

Names: letters, numbers and underscores. Cannot start with a number. Temp and temp are different.

Part 1 · Syntax

Operations you will use daily

# Arithmetic:  +  -  *  /  **  %  //
temp_c = temp_k - 273.15
mm_day = prate * 86400     # kg m⁻² s⁻¹ → mm/day

# Strings and f-strings
print(f"{station}: {rain:.1f} mm")
Cape Town: 12.5 mm

# Lists count from 0
temp_day[0]      # 18, the first
temp_day[-1]     # 16, the last
temp_day[1:3]    # [20, 24], end not included
temp_day.append(22)

Two conversions you will need again and again today:
K − 273.15 = °C
kg m⁻² s⁻¹ × 86400 = mm/day

A precipitation rate in kg m⁻² s⁻¹ is the same as mm/s, because 1 kg of water over 1 m² is 1 mm deep.

Slicing with [start:stop] excludes stop. Remember this: xarray's slice() behaves differently.

Part 1 · Control flow

A hot day warning system

We are given today's temperature and the past mean for that day. If today is 4 degrees or more above the mean, issue a warning.

temp_mean = 16
temp_day  = 20

if temp_day >= temp_mean + 4:
    print("HOT - Issue a warning")
else:
    print("No warning necessary")

More than one temperature? Loop over the list.

temp_day = [18, 20, 24, 16]

for t in temp_day:
    if t >= temp_mean + 4:
        print(t, "HOT - Issue a warning")
    else:
        print(t, "No warning necessary")

Indentation is the syntax. The spaces at the start of a line tell Python what belongs inside the if or the for.

Part 1 · Functions and modules

Write it once, use it many times

A function is a named piece of code that does one task. It may take inputs and may return a value.

def warn_temp(temp_day, temp_mean):
    if temp_day >= temp_mean + 4:
        return "HOT - Issue a warning"
    return "No warning necessary"

for t in temp_day:
    print(t, warn_temp(t, temp_mean))

A module is a file of functions someone else wrote. Load it with import.

import numpy as np
import pandas as pd
import xarray as xr
import matplotlib.pyplot as plt

temps = np.array([18, 20, 24, 16])
temps.mean()        # 19.5
temps + 273.15      # whole array at once

These four import lines start almost every climate notebook. The short names are a convention everyone uses.

Part 1 · The scientific stack

Who does what

matplotlib
draws the figures: lines, bars, maps, contours. cartopy adds coastlines and projections on top.
pandas
tables with a row index: station records, CSV files, time series. Rows and columns.
xarray
labelled N-dimensional arrays: gridded fields with time, level, lat, lon. Reads NetCDF.
NumPy
the fast array underneath all of them. You rarely call it directly, but everything is built on it.

Rule of thumb: one station or one index over time → pandas. A field on a grid → xarray. Either way, plotting → matplotlib.

Part 1 · Exercise

Your turn

  1. Declare an integer variable and display it
  2. Declare a float variable and display it
  3. Declare a list of 5 temperatures and display the second element
  4. Write a function to_celsius(kelvin) and test it with 300
  5. Loop over [280.5, 295.0, 310.2] and print each in °C

The notebook has a hidden solution under each exercise. Try first, then compare.

Part 2

Station data with pandas

Twenty years of daily rainfall at three sites, in one table.

Part 2 · pandas

A DataFrame is a table

import pandas as pd

data = pd.read_csv("Sampledata.csv", sep=";")
data.head()

Rows are days. Columns are variables. The numbers on the left are the index.

sep=";" because this file uses semicolons. The default is a comma.

A single column, data["Site_A_Mediterranean_mm"], is a Series.

yearmonthdaySite_A_Mediterranean_mmSite_B_TropicalMonsoon_mmSite_C_TemperateMaritime_mm
02005110.00.00.9
12005120.00.05.0
22005130.00.01.3
Part 2 · pandas

First look, then summarise

data.head()       # first 5 rows
data.tail(3)      # last 3 rows
data.shape        # (7304, 6)
data.columns      # column names
data.describe()   # count, mean, std, min, max…

data["Site_A_Mediterranean_mm"].mean()
data["Site_B_TropicalMonsoon_mm"].max()

Methods need brackets (). Attributes such as shape and columns do not.

Build a real date from the three columns, so pandas and matplotlib understand time:

data["Date"] = pd.to_datetime(
    data[["year", "month", "day"]]
)

Keep only the rows that match a condition:

wet = data[data["Site_A_Mediterranean_mm"] > 20]
len(wet)
Part 2 · pandas

groupby: climatology and annual totals

groupby splits the table into groups, applies a statistic to each, and joins the results back into a table.

sites = ["Site_A_Mediterranean_mm",
         "Site_B_TropicalMonsoon_mm",
         "Site_C_TemperateMaritime_mm"]

# mean annual cycle: one row per month
monthly_mean = data.groupby("month")[sites].mean()

# annual totals: one row per year
annual = data.groupby("year")[sites].sum()
QuestionGroup byStatistic
Mean annual cyclemonth.mean()
Annual rainfallyear.sum()
Wettest day each yearyear.max()
Interannual variabilityyear, then.sum().std()

Anomaly = value − long-term mean:
anomaly = annual - annual.mean()

Part 2 · Exercise

Your turn

  1. What is the mean daily rainfall at Site B?
  2. How many days had no rain at all (0.0 mm) at Site A?
  3. Calculate the mean rainfall per year for Site C
  4. Which year had the highest annual total at Site B? Hint: .idxmax()
Part 3

Time series with matplotlib

One recipe covers almost every line plot you will ever make.

Part 3 · matplotlib

The recipe

import matplotlib.pyplot as plt

plt.figure(figsize=(12, 4))            # 1 figure

plt.plot(data["Date"],                   # 2 draw
         data["Site_A_Mediterranean_mm"])

plt.xlabel("Date")                       # 3 label
plt.ylabel("Rainfall (mm)")
plt.title("Daily rainfall - Site A")

plt.show()                               # 4 show

The figure is the whole image. Inside it, the axes is the drawing area. Every plt. call adds to the current axes.

  • Several lines: call plt.plot once per line with a label=, then plt.legend()
  • marker="o" puts a dot on each point
  • plt.grid(True) adds a grid
  • plt.axhline(0, linestyle="--") draws a zero line, useful for anomalies
  • plt.bar(names, values, yerr=std) for bars with error bars
Part 3 · matplotlib

Save it before you show it

plt.figure(figsize=(10, 5))

for site in sites:
    plt.plot(annual.index, annual[site],
             marker="o", label=site)

plt.xlabel("Year")
plt.ylabel("Annual rainfall (mm)")
plt.legend()
plt.grid(True)

plt.savefig("annual_rainfall.png",
            dpi=150, bbox_inches="tight")
plt.show()

savefig must come before show. After show, the figure is gone and you save a blank image.

  • dpi=150 for slides, dpi=300 for a paper
  • bbox_inches="tight" trims the white margins
  • .pdf or .svg give vector output for publication

The loop over sites replaces three nearly identical plt.plot lines. That is what loops are for.

Part 3 · Exercise

Your turn

  1. Plot the annual totals for all three sites on one figure, with labels, a legend and a grid
  2. Plot the daily rainfall at Site B for 2015 only. Hint: data[data["year"] == 2015]
  3. Save the second plot as site_b_2015.png
Part 4

Gridded data with xarray

A value at every latitude, longitude, level and time. Tables do not fit any more.

Part 4 · NetCDF

NetCDF: data that describes itself

A .nc file stores three things together:

  • Dimensions: time, plev, lat, lon
  • Coordinates: the actual values along each dimension
  • Variables with attributes: units, long_name, standard_name

The CF conventions say how to name these, so any tool can read any CF file. xarray follows them, which is why your plots label themselves.

xarray's two objects:

Dataset

The whole file. A dictionary of variables that share coordinates. ds

DataArray

One variable, with its dimensions, coordinates and attributes. ds["t"] or ds.t

Part 4 · Our file

fnl_africa_202302-202303.nc

NCEP FNL (Final) operational global analysis, February to March 2023, cut to Africa for today. The original global file is 1.3 GB with the same structure.

DimensionSizeValues
time236every 6 h, 2023-02-01 00:00 to 2023-03-31 18:00
plev3500, 850, 1000 hPa, stored in Pa
lat8640 → −45, 1° steps, descending
lon91−25 → 65, 1° steps
VariableMeaningUnits
ttemperatureK
ghgeopotential heightgpm
u_2, v_2eastward, northward windm s⁻¹
prateprecipitation rate (surface only)kg m⁻² s⁻¹

Three traps: levels are in Pa (1000 hPa = 100000), latitude runs north to south, and longitude here is −180 to 180 but many files use 0 to 359. Check the coordinates before you select.

Part 4 · xarray

Open it, look at it

import xarray as xr

ds = xr.open_dataset("fnl_africa_202302-202303.nc")
ds                     # interactive summary

t = ds["t"]            # one DataArray
t.dims                 # ('time','plev','lat','lon')
t.shape                # (236, 3, 86, 91)
t.attrs["units"]       # 'K'

open_dataset is lazy. It reads the metadata only. Even the 1.3 GB global file opens instantly, and the values stay on disk until you use them.

That means the order of work is:

  1. Select the variable, level, region, time you need
  2. Convert units
  3. Reduce: mean, max, sum along a dimension
  4. Plot
Part 4 · Selecting

.sel by label, .isel by position

# one level, first time step
t.sel(plev=100000).isel(time=0)

# nearest grid point to Cape Town, all times
t.sel(plev=100000, lat=-33.9, lon=18.4,
      method="nearest")

# a region: southern Africa
t.sel(lat=slice(0, -40), lon=slice(10, 52))

# a range of dates
t.sel(time=slice("2023-02-01", "2023-02-28"))

# one moment
t.sel(time="2023-02-15T12")
  • .sel uses coordinate values: a real latitude, a real date
  • .isel uses positions: 0, 1, 2 … like a list
  • method="nearest" when your value is off the grid
  • slice(start, stop) includes both ends, unlike a Python list

Latitude is stored 90 → −90, so a latitude slice goes from the larger value to the smaller: slice(0, -40). Write slice(-40, 0) and you get nothing, with no error.

Part 4 · Computing

Convert units, then reduce along a dimension

t_c = t - 273.15
t_c.attrs["units"] = "°C"

pr = ds["prate"] * 86400
pr.attrs["units"] = "mm/day"

# collapse time → a map
t_map = t_c.sel(plev=100000).mean(dim="time")

# collapse space → a time series
pr_sa = pr.sel(lat=slice(0, -40), lon=slice(10, 52))
pr_series = pr_sa.mean(dim=("lat", "lon"))

Arithmetic applies to every value at once. Attributes are dropped by arithmetic, so set units again and the plots stay labelled.

dim= names what you collapse. What is left decides what you get:

Left overYou get.plot() draws
lat, lona mapa filled field
timea time seriesa line
nothingone number
Part 4 · Time

resample and groupby on time

# 6-hourly → daily means
pr_daily = pr_series.resample(time="1D").mean()

# monthly mean maps: one per calendar month
pr_monthly = pr_sa.groupby("time.month").mean(dim="time")
pr_monthly.sel(month=3) - pr_monthly.sel(month=2)
  • resample changes the time step: "1D" daily, "MS" month start, "YS" year start
  • groupby("time.month") pools every February together, every March together: the way to make a climatology
  • Also time.year, time.season, time.dayofyear

Same ideas as pandas, because xarray borrowed them.

Part 4 · Exercise

Your turn

  1. Select the 500 hPa geopotential height on 15 February 2023 at 12:00
  2. Maximum 1000 hPa temperature over the two months at the grid point nearest Nairobi (−1.3, 36.8), in °C
  3. Mean 850 hPa wind speed over southern Africa. Speed is np.sqrt(u**2 + v**2)
Part 5

Spatial plots

Maps, colour maps, regions, panels, contours and wind.

Part 5 · .plot()

A DataArray knows how to draw itself

plt.figure(figsize=(12, 5))
t_map.plot()
plt.title("Mean 1000 hPa temperature, Feb–Mar 2023")
plt.show()

# control the colours
pr.mean(dim="time").plot(cmap="YlGnBu", vmin=0, vmax=15)

# a difference: diverging, centred on zero
pr_diff.plot(cmap="BrBG", center=0)

Two dimensions left → a filled map with a colour bar. One dimension left → a line. Axis labels and the colour bar label come from the long_name and units attributes.

  • cmap= the colour map
  • vmin=, vmax= fix the range, essential when comparing panels
  • center=0 puts the neutral colour at zero
  • ax= which panel to draw in
Part 5 · Colour

Match the colour map to the data

Sequential, one hue light to dark, for magnitudes: temperature, rainfall, height.

viridis default, colour-blind safe
YlGnBu rainfall
Oranges heat

Diverging, two hues around a neutral middle, for anomalies and differences. Always center=0.

RdBu_r temperature anomaly
BrBG dry ↔ wet

Avoid rainbow maps.

jet do not use
  • The bright bands create edges that are not in the data
  • Lightness goes up and down, so it does not read as "more"
  • Red-green readers cannot tell half the scale apart

Full list: matplotlib colormaps reference. Add _r to any name to reverse it.

Part 5 · Maps

Coastlines with cartopy

import cartopy.crs as ccrs
import cartopy.feature as cfeature

fig = plt.figure(figsize=(9, 7))
ax = plt.axes(projection=ccrs.PlateCarree())

t_map.sel(lat=slice(0, -40), lon=slice(10, 52)).plot(
    ax=ax, transform=ccrs.PlateCarree(), cmap="RdYlBu_r",
    cbar_kwargs={"label": "Temperature (°C)"},
)
ax.coastlines()
ax.add_feature(cfeature.BORDERS, linewidth=0.5)
ax.gridlines(draw_labels=True)
plt.show()

cartopy gives matplotlib a map projection. Two things change:

  • The axes gets a projection=
  • The data gets a transform= saying which coordinates it is in. Lat/lon data is always PlateCarree(), whatever the projection.

Other projections: ccrs.Robinson() for the globe, ccrs.LambertConformal() for a region, ccrs.SouthPolarStereo() for Antarctica.

The first run downloads coastline data, so it needs internet.

Part 5 · Panels

Several panels, one figure

fig, axes = plt.subplots(1, 2, figsize=(14, 5))

pr_monthly.sel(month=2).plot(
    ax=axes[0], cmap="YlGnBu", vmin=0, vmax=15)
axes[0].set_title("February 2023")

pr_monthly.sel(month=3).plot(
    ax=axes[1], cmap="YlGnBu", vmin=0, vmax=15)
axes[1].set_title("March 2023")

plt.tight_layout()
plt.show()
  • plt.subplots(rows, cols) returns the figure and a grid of axes
  • Pass ax=axes[i] to say where to draw
  • With an axes object the calls change name: ax.set_title, ax.set_xlabel, ax.grid
  • Same vmin/vmax in every panel, or the comparison lies
  • sharex=True when panels share a time axis
Part 5 · Synoptic

Contours and wind vectors

when = "2023-02-15T12"
region = dict(lat=slice(0, -40), lon=slice(10, 52))

gh500 = ds["gh"].sel(plev=50000, time=when, **region)
u850  = ds["u_2"].sel(plev=85000, time=when, **region)
v850  = ds["v_2"].sel(plev=85000, time=when, **region)

fig, ax = plt.subplots(figsize=(9, 7))
cs = gh500.plot.contour(ax=ax, levels=15, colors="black")
ax.clabel(cs, fontsize=8, fmt="%.0f")

ax.quiver(u850.lon[::3], u850.lat[::3],
          u850[::3, ::3], v850[::3, ::3])
plt.show()
  • .plot.contour() draws lines, .plot.contourf() fills between them
  • ax.clabel writes the value on each contour
  • quiver(x, y, u, v) draws arrows. [::3] keeps every third grid point so the arrows do not pile up

The 500 hPa height field with 850 hPa winds is the classic synoptic chart. You now know how to make one from any reanalysis file.

Part 5 · Exercise

Your turn

  1. Map the mean 500 hPa geopotential height for February 2023, whole domain
  2. Map the 1000 hPa temperature anomaly on 15 Feb 12:00 relative to the two-month mean, over southern Africa, with a diverging colour map centred on zero
  3. A 1 × 2 figure of mean rainfall over East Africa (lat 15 to −15, lon 25 to 55) for February and March, same colour scale
Part 6

Saving, the Climate Data Store, and what next

Getting results out, and getting more data in.

Part 6 · Saving

Write files as good as the ones you read

pr_monthly.name = "pr"
pr_monthly.to_netcdf("pr_sa_monthly_2023.nc")

pr_daily.to_dataframe(name="pr_mm_day") \
        .to_csv("pr_sa_daily_2023.csv")

plt.savefig("figure.png", dpi=150,
            bbox_inches="tight")
ObjectMethodFormat
DataArray, Dataset.to_netcdf()NetCDF
DataFrame, Series.to_csv()CSV
Figureplt.savefig()PNG, PDF, SVG

Give the array a name and keep its units and long_name. Your future self, and your colleagues, will open the file with no idea what it is.

Part 6 · Climate Data Store

Copernicus Climate Data Store

cds.climate.copernicus.eu holds ERA5 reanalysis, seasonal forecasts, CMIP6 projections, satellite records and more. Free, with an account.

One-time setup

  1. Register and log in
  2. Copy your Personal Access Token from your profile page
  3. Save it in ~/.cdsapirc:
url: https://cds.climate.copernicus.eu/api
key: YOUR-PERSONAL-ACCESS-TOKEN
  1. Accept the licence on each dataset's page
  2. pip install cdsapi

Requesting data from Python

import cdsapi
client = cdsapi.Client()

client.retrieve(
    "reanalysis-era5-single-levels",
    {
        "product_type": "reanalysis",
        "variable": "2m_temperature",
        "year": "2023", "month": "02",
        "day": ["01", "02", "03"],
        "time": "12:00",
        "area": [0, 10, -40, 52],  # N, W, S, E
        "format": "netcdf",
    },
    "era5_t2m.nc",
)

Do not write requests by hand. Fill in the download form on the dataset page and click Show API request.

Part 6 · JupyterLab

Working in JupyterLab at home

Everything today ran in JupyterLab on your own machine. The routine each time:

  1. Open a terminal, conda activate climate
  2. cd to the folder with your notebook and data
  3. jupyter lab, it opens in the browser
  4. File browser on the left, double-click a notebook
  5. Kernel name top right should say Python 3
  6. Finished: File → Shut Down, then close the tab

Notebooks save where you launched jupyter lab. Files a notebook reads or writes are relative to the notebook's own folder.

  • Share a result: File → Save and Export Notebook As → HTML. Anyone can open it, no Python needed.
  • Start clean: Kernel → Restart Kernel and Run All Cells. If it fails, the notebook depends on something you ran out of order.
  • Add a package: conda install -c conda-forge scipy in the terminal, then restart the kernel.
Part 6 · Links

Where to go next

When stuck

Read the error from the bottom up
xr.open_dataset? in a cell
Tab after a dot to see what exists
Search the exact error text
Ask: sabina@csag.uct.ac.za

Recap

What you can do now

  • Read a CSV into pandas, summarise it, group by month and year
  • Plot time series, anomalies and bars, and save the figure
  • Open a NetCDF file with xarray and understand its dimensions, coordinates and attributes
  • Select a level, a point, a region and a time range
  • Convert units and reduce along a dimension
  • Draw maps, pick a colour map on purpose, add coastlines, build panels
  • Fetch ERA5 from the Climate Data Store

The pattern is always the same:
select → convert → reduce → plot.
Everything else is looking up the argument name.

Slides and notebook: keep them, reuse the code. Every figure you make from now on starts from one of these cells.

← → space · click · Home/End