From a CSV of station rainfall to global reanalysis maps, in one afternoon.
Sabina Abba Omar · Climate System Analysis Group, University of Cape Town
Everything in these slides is also in the notebook intro_python_climate_science.ipynb. Open it now and follow along.
Two data files sit next to the notebook:
Sampledata.csv: daily rainfall at three sites, 2005 to 2024fnl_africa_202302-202303.nc: NCEP FNL analysis cut to Africa, Feb to Mar 2023Install Miniconda, open a terminal (Anaconda Prompt on Windows), then:
conda create -n climate -c conda-forge python=3.12 \
xarray netcdf4 pandas matplotlib cartopy jupyterlab
conda activate climate
Put the notebook and both data files in one folder. In the terminal, cd into that folder and run jupyter lab. It opens in your browser.
| Key | Does |
|---|---|
Shift + Enter | run the cell, move to the next |
Tab | autocomplete after a dot |
? after a name | show its documentation |
Stuck? Kernel → Restart, then run from the top.
No install possible? Google Colab runs notebooks in the browser. Upload the files and run !pip install xarray netcdf4 cartopy first.
What it is, how it runs, and just enough syntax to read climate data.
A high-level programming language known for its simplicity and readability.
Created by Guido van Rossum, first released in 1991. Now on version 3.
Free and open source: anyone can read, modify and distribute it. That is why a huge scientific community has grown around it.
Python is interpreted: a program called the interpreter reads your code line by line and executes it. No compiling step.
You can run it three ways:
.py file run top to bottomToday we use Jupyter notebooks. A kernel is the interpreter running behind the notebook. It remembers every variable you define until you restart it.
The notebook shows the value of the last line of each cell. Use print() to show anything else.
Order matters. If a cell fails with NameError, a cell above it has not been run yet.
A variable is a name for a value. Create one with =.
n_years = 30 # int temp_mean = 16.4 # float station = "Cape Town" # str is_raining = False # bool temp_day = [18, 20, 24, 16] # list coords = (-33.9, 18.4) # tuple
| Type | Holds | Can change? |
|---|---|---|
int, float | numbers | replace |
str | text | no |
list | ordered values | yes |
tuple | ordered values | no |
Names: letters, numbers and underscores. Cannot start with a number. Temp and temp are different.
# Arithmetic: + - * / ** % // temp_c = temp_k - 273.15 mm_day = prate * 86400 # kg m⁻² s⁻¹ → mm/day # Strings and f-strings print(f"{station}: {rain:.1f} mm") Cape Town: 12.5 mm # Lists count from 0 temp_day[0] # 18, the first temp_day[-1] # 16, the last temp_day[1:3] # [20, 24], end not included temp_day.append(22)
Two conversions you will need again and again today:
K − 273.15 = °C
kg m⁻² s⁻¹ × 86400 = mm/day
A precipitation rate in kg m⁻² s⁻¹ is the same as mm/s, because 1 kg of water over 1 m² is 1 mm deep.
Slicing with [start:stop] excludes stop. Remember this: xarray's slice() behaves differently.
We are given today's temperature and the past mean for that day. If today is 4 degrees or more above the mean, issue a warning.
temp_mean = 16 temp_day = 20 if temp_day >= temp_mean + 4: print("HOT - Issue a warning") else: print("No warning necessary")
More than one temperature? Loop over the list.
temp_day = [18, 20, 24, 16] for t in temp_day: if t >= temp_mean + 4: print(t, "HOT - Issue a warning") else: print(t, "No warning necessary")
Indentation is the syntax. The spaces at the start of a line tell Python what belongs inside the if or the for.
A function is a named piece of code that does one task. It may take inputs and may return a value.
def warn_temp(temp_day, temp_mean): if temp_day >= temp_mean + 4: return "HOT - Issue a warning" return "No warning necessary" for t in temp_day: print(t, warn_temp(t, temp_mean))
A module is a file of functions someone else wrote. Load it with import.
import numpy as np import pandas as pd import xarray as xr import matplotlib.pyplot as plt temps = np.array([18, 20, 24, 16]) temps.mean() # 19.5 temps + 273.15 # whole array at once
These four import lines start almost every climate notebook. The short names are a convention everyone uses.
Rule of thumb: one station or one index over time → pandas. A field on a grid → xarray. Either way, plotting → matplotlib.
to_celsius(kelvin) and test it with 300[280.5, 295.0, 310.2] and print each in °CThe notebook has a hidden solution under each exercise. Try first, then compare.
Twenty years of daily rainfall at three sites, in one table.
import pandas as pd data = pd.read_csv("Sampledata.csv", sep=";") data.head()
Rows are days. Columns are variables. The numbers on the left are the index.
sep=";" because this file uses semicolons. The default is a comma.
A single column, data["Site_A_Mediterranean_mm"], is a Series.
| year | month | day | Site_A_Mediterranean_mm | Site_B_TropicalMonsoon_mm | Site_C_TemperateMaritime_mm | |
|---|---|---|---|---|---|---|
| 0 | 2005 | 1 | 1 | 0.0 | 0.0 | 0.9 |
| 1 | 2005 | 1 | 2 | 0.0 | 0.0 | 5.0 |
| 2 | 2005 | 1 | 3 | 0.0 | 0.0 | 1.3 |
data.head() # first 5 rows data.tail(3) # last 3 rows data.shape # (7304, 6) data.columns # column names data.describe() # count, mean, std, min, max… data["Site_A_Mediterranean_mm"].mean() data["Site_B_TropicalMonsoon_mm"].max()
Methods need brackets (). Attributes such as shape and columns do not.
Build a real date from the three columns, so pandas and matplotlib understand time:
data["Date"] = pd.to_datetime( data[["year", "month", "day"]] )
Keep only the rows that match a condition:
wet = data[data["Site_A_Mediterranean_mm"] > 20] len(wet)
groupby splits the table into groups, applies a statistic to each, and joins the results back into a table.
sites = ["Site_A_Mediterranean_mm", "Site_B_TropicalMonsoon_mm", "Site_C_TemperateMaritime_mm"] # mean annual cycle: one row per month monthly_mean = data.groupby("month")[sites].mean() # annual totals: one row per year annual = data.groupby("year")[sites].sum()
| Question | Group by | Statistic |
|---|---|---|
| Mean annual cycle | month | .mean() |
| Annual rainfall | year | .sum() |
| Wettest day each year | year | .max() |
| Interannual variability | year, then | .sum().std() |
Anomaly = value − long-term mean:anomaly = annual - annual.mean()
.idxmax()One recipe covers almost every line plot you will ever make.
import matplotlib.pyplot as plt plt.figure(figsize=(12, 4)) # 1 figure plt.plot(data["Date"], # 2 draw data["Site_A_Mediterranean_mm"]) plt.xlabel("Date") # 3 label plt.ylabel("Rainfall (mm)") plt.title("Daily rainfall - Site A") plt.show() # 4 show
The figure is the whole image. Inside it, the axes is the drawing area. Every plt. call adds to the current axes.
plt.plot once per line with a label=, then plt.legend()marker="o" puts a dot on each pointplt.grid(True) adds a gridplt.axhline(0, linestyle="--") draws a zero line, useful for anomaliesplt.bar(names, values, yerr=std) for bars with error barsplt.figure(figsize=(10, 5)) for site in sites: plt.plot(annual.index, annual[site], marker="o", label=site) plt.xlabel("Year") plt.ylabel("Annual rainfall (mm)") plt.legend() plt.grid(True) plt.savefig("annual_rainfall.png", dpi=150, bbox_inches="tight") plt.show()
savefig must come before show. After show, the figure is gone and you save a blank image.
dpi=150 for slides, dpi=300 for a paperbbox_inches="tight" trims the white margins.pdf or .svg give vector output for publicationThe loop over sites replaces three nearly identical plt.plot lines. That is what loops are for.
data[data["year"] == 2015]site_b_2015.pngA value at every latitude, longitude, level and time. Tables do not fit any more.
A .nc file stores three things together:
time, plev, lat, lonunits, long_name, standard_nameThe CF conventions say how to name these, so any tool can read any CF file. xarray follows them, which is why your plots label themselves.
xarray's two objects:
The whole file. A dictionary of variables that share coordinates. ds
One variable, with its dimensions, coordinates and attributes. ds["t"] or ds.t
NCEP FNL (Final) operational global analysis, February to March 2023, cut to Africa for today. The original global file is 1.3 GB with the same structure.
| Dimension | Size | Values |
|---|---|---|
time | 236 | every 6 h, 2023-02-01 00:00 to 2023-03-31 18:00 |
plev | 3 | 500, 850, 1000 hPa, stored in Pa |
lat | 86 | 40 → −45, 1° steps, descending |
lon | 91 | −25 → 65, 1° steps |
| Variable | Meaning | Units |
|---|---|---|
t | temperature | K |
gh | geopotential height | gpm |
u_2, v_2 | eastward, northward wind | m s⁻¹ |
prate | precipitation rate (surface only) | kg m⁻² s⁻¹ |
Three traps: levels are in Pa (1000 hPa = 100000), latitude runs north to south, and longitude here is −180 to 180 but many files use 0 to 359. Check the coordinates before you select.
import xarray as xr ds = xr.open_dataset("fnl_africa_202302-202303.nc") ds # interactive summary t = ds["t"] # one DataArray t.dims # ('time','plev','lat','lon') t.shape # (236, 3, 86, 91) t.attrs["units"] # 'K'
open_dataset is lazy. It reads the metadata only. Even the 1.3 GB global file opens instantly, and the values stay on disk until you use them.
That means the order of work is:
# one level, first time step t.sel(plev=100000).isel(time=0) # nearest grid point to Cape Town, all times t.sel(plev=100000, lat=-33.9, lon=18.4, method="nearest") # a region: southern Africa t.sel(lat=slice(0, -40), lon=slice(10, 52)) # a range of dates t.sel(time=slice("2023-02-01", "2023-02-28")) # one moment t.sel(time="2023-02-15T12")
.sel uses coordinate values: a real latitude, a real date.isel uses positions: 0, 1, 2 … like a listmethod="nearest" when your value is off the gridslice(start, stop) includes both ends, unlike a Python listLatitude is stored 90 → −90, so a latitude slice goes from the larger value to the smaller: slice(0, -40). Write slice(-40, 0) and you get nothing, with no error.
t_c = t - 273.15 t_c.attrs["units"] = "°C" pr = ds["prate"] * 86400 pr.attrs["units"] = "mm/day" # collapse time → a map t_map = t_c.sel(plev=100000).mean(dim="time") # collapse space → a time series pr_sa = pr.sel(lat=slice(0, -40), lon=slice(10, 52)) pr_series = pr_sa.mean(dim=("lat", "lon"))
Arithmetic applies to every value at once. Attributes are dropped by arithmetic, so set units again and the plots stay labelled.
dim= names what you collapse. What is left decides what you get:
| Left over | You get | .plot() draws |
|---|---|---|
lat, lon | a map | a filled field |
time | a time series | a line |
| nothing | one number | — |
# 6-hourly → daily means pr_daily = pr_series.resample(time="1D").mean() # monthly mean maps: one per calendar month pr_monthly = pr_sa.groupby("time.month").mean(dim="time") pr_monthly.sel(month=3) - pr_monthly.sel(month=2)
resample changes the time step: "1D" daily, "MS" month start, "YS" year startgroupby("time.month") pools every February together, every March together: the way to make a climatologytime.year, time.season, time.dayofyearSame ideas as pandas, because xarray borrowed them.
np.sqrt(u**2 + v**2)Maps, colour maps, regions, panels, contours and wind.
plt.figure(figsize=(12, 5)) t_map.plot() plt.title("Mean 1000 hPa temperature, Feb–Mar 2023") plt.show() # control the colours pr.mean(dim="time").plot(cmap="YlGnBu", vmin=0, vmax=15) # a difference: diverging, centred on zero pr_diff.plot(cmap="BrBG", center=0)
Two dimensions left → a filled map with a colour bar. One dimension left → a line. Axis labels and the colour bar label come from the long_name and units attributes.
cmap= the colour mapvmin=, vmax= fix the range, essential when comparing panelscenter=0 puts the neutral colour at zeroax= which panel to draw inSequential, one hue light to dark, for magnitudes: temperature, rainfall, height.
Diverging, two hues around a neutral middle, for anomalies and differences. Always center=0.
Avoid rainbow maps.
Full list: matplotlib colormaps reference. Add _r to any name to reverse it.
import cartopy.crs as ccrs import cartopy.feature as cfeature fig = plt.figure(figsize=(9, 7)) ax = plt.axes(projection=ccrs.PlateCarree()) t_map.sel(lat=slice(0, -40), lon=slice(10, 52)).plot( ax=ax, transform=ccrs.PlateCarree(), cmap="RdYlBu_r", cbar_kwargs={"label": "Temperature (°C)"}, ) ax.coastlines() ax.add_feature(cfeature.BORDERS, linewidth=0.5) ax.gridlines(draw_labels=True) plt.show()
cartopy gives matplotlib a map projection. Two things change:
projection=transform= saying which coordinates it is in. Lat/lon data is always PlateCarree(), whatever the projection.Other projections: ccrs.Robinson() for the globe, ccrs.LambertConformal() for a region, ccrs.SouthPolarStereo() for Antarctica.
The first run downloads coastline data, so it needs internet.
fig, axes = plt.subplots(1, 2, figsize=(14, 5)) pr_monthly.sel(month=2).plot( ax=axes[0], cmap="YlGnBu", vmin=0, vmax=15) axes[0].set_title("February 2023") pr_monthly.sel(month=3).plot( ax=axes[1], cmap="YlGnBu", vmin=0, vmax=15) axes[1].set_title("March 2023") plt.tight_layout() plt.show()
plt.subplots(rows, cols) returns the figure and a grid of axesax=axes[i] to say where to drawax.set_title, ax.set_xlabel, ax.gridvmin/vmax in every panel, or the comparison liessharex=True when panels share a time axiswhen = "2023-02-15T12" region = dict(lat=slice(0, -40), lon=slice(10, 52)) gh500 = ds["gh"].sel(plev=50000, time=when, **region) u850 = ds["u_2"].sel(plev=85000, time=when, **region) v850 = ds["v_2"].sel(plev=85000, time=when, **region) fig, ax = plt.subplots(figsize=(9, 7)) cs = gh500.plot.contour(ax=ax, levels=15, colors="black") ax.clabel(cs, fontsize=8, fmt="%.0f") ax.quiver(u850.lon[::3], u850.lat[::3], u850[::3, ::3], v850[::3, ::3]) plt.show()
.plot.contour() draws lines, .plot.contourf() fills between themax.clabel writes the value on each contourquiver(x, y, u, v) draws arrows. [::3] keeps every third grid point so the arrows do not pile upThe 500 hPa height field with 850 hPa winds is the classic synoptic chart. You now know how to make one from any reanalysis file.
Getting results out, and getting more data in.
pr_monthly.name = "pr" pr_monthly.to_netcdf("pr_sa_monthly_2023.nc") pr_daily.to_dataframe(name="pr_mm_day") \ .to_csv("pr_sa_daily_2023.csv") plt.savefig("figure.png", dpi=150, bbox_inches="tight")
| Object | Method | Format |
|---|---|---|
| DataArray, Dataset | .to_netcdf() | NetCDF |
| DataFrame, Series | .to_csv() | CSV |
| Figure | plt.savefig() | PNG, PDF, SVG |
Give the array a name and keep its units and long_name. Your future self, and your colleagues, will open the file with no idea what it is.
cds.climate.copernicus.eu holds ERA5 reanalysis, seasonal forecasts, CMIP6 projections, satellite records and more. Free, with an account.
One-time setup
~/.cdsapirc:url: https://cds.climate.copernicus.eu/api key: YOUR-PERSONAL-ACCESS-TOKEN
pip install cdsapiRequesting data from Python
import cdsapi client = cdsapi.Client() client.retrieve( "reanalysis-era5-single-levels", { "product_type": "reanalysis", "variable": "2m_temperature", "year": "2023", "month": "02", "day": ["01", "02", "03"], "time": "12:00", "area": [0, 10, -40, 52], # N, W, S, E "format": "netcdf", }, "era5_t2m.nc", )
Do not write requests by hand. Fill in the download form on the dataset page and click Show API request.
Everything today ran in JupyterLab on your own machine. The routine each time:
conda activate climatecd to the folder with your notebook and datajupyter lab, it opens in the browserNotebooks save where you launched jupyter lab. Files a notebook reads or writes are relative to the notebook's own folder.
conda install -c conda-forge scipy in the terminal, then restart the kernel.
xarray tutorial
xarray user guide
pandas user guide
matplotlib gallery
cartopy
Project Pythia
Software Carpentry
Read the error from the bottom up
xr.open_dataset? in a cell
Tab after a dot to see what exists
Search the exact error text
Ask: sabina@csag.uct.ac.za
The pattern is always the same:
select → convert → reduce → plot.
Everything else is looking up the argument name.
Slides and notebook: keep them, reuse the code. Every figure you make from now on starts from one of these cells.