Exercise 2: Tidyverse

Author

HeaDS Data Science Lab, University of Copenhagen

Setting up

  1. Create new Quarto document. For working on the exercise, create a new Quarto document with a descriptive name and save it where you can find it again, i.e. in the folder where you downloaded the teaching material. You can use the commands shown in presentation2.qmd to solve this exercise. There is no shame in outright copying from the presentation2.qmd script, provided you understand what the command is doing.

  2. Load packages. You will need to load the packages tidyverse and readxl for this exercise.

Importing data and a first look at the dataset

The data set used in these exercises was compiled from data downloaded from the website of the UK’s national weather service, the Met Office. It is saved in the file climate.xlsx1 which can be found in the folder Exercises/Data/. The spreadsheet contains monthly data from five UK weather stations for the following variables:

Variable name Explanation
station Location of weather station
year Year
month Month
af Days of air frost
rain Rainfall in mm
sun Sunshine duration in hours
device Brand of sunshine recorder / sensor

  1. Load data. Start by importing the dataset using either the read_excel() function or the Import Dataset button and name it climate. If you load with Import Dataset it is a good idea to copy the command into your script so that the next time you run your script you can just execute that line instead of having to find the file again.

  2. First look at data. Write the name of the dataframe, i.e. climate, into the console and press enter to see the first rows of the dataset. You can also click on the climate object in the Environment panel.

  3. Explore your dataset and understand what data you have.

    1. How many observations, i.e. rows are there?

    2. How many data columns are there and what are their types?

    3. What is the information in each row and column?

    4. How many different stations are there?

    5. How many rows per station?

Working with the data

Before you proceed with the exercises in this document, make sure you load the tidyverse in order to use the functions from this package.

  1. Count the number of rows that did not have any days with air frost.

  2. Count the number of rows per station that did not have any days with air frost.

  3. Select from the climate dataset (remember to filter rows and select columns):

    1. all rows from the station in Oxford

    2. all rows from the station in Oxford when there were at least 100 hours of sunlight

    3. all rows from the stations in Oxford and Camborne when there were at least 100 hours of sunlight

    4. a subset that only contains the station, year and rain columns

The next few questions build on each other, each adding a piece of code:

  1. Compute the average rainfall over the full dataset by using the summarize function. You can look at the examples we did at the end of presentation 2.

  2. Now, compute the average rainfall, standard deviation of the rainfall and the total rainfall (the sum) on the full dataset. I.e. all three measures should be inside the same resulting table. Have a look at the tidyverse lecture if you have trouble with this.

  3. Now, use group_by before summarize in order to compute group summary statistics (average, standard deviation, and sum) but split up into each of the five weather stations.

  4. Include a column in the summary statistics which shows how many observations, i.e. rows, the data set contains for each station.

  5. Sort the rows in the output in descending order according to average annual rainfall.

Manipulating the data

  1. Create a new column in climate and save the new dataset in a different variable so you don’t overwrite your original climate data. The new column should count the number of days in each month without air frost, based on the existing af column. For this exercise, assume each month has 30 days. To find the number of days without air frost, subtract the value in the af column from 30.

  2. Add another column to your new dataset that says whether the weather this month was good. We consider a month to be good if it had at least 100 hours of sunshine and less than 100 mm of rain. Otherwise the weather was bad.

  3. How many months are there with good weather (use the column you made in 14) for each station? Find the station that has the most months with good weather.

Complex operations

The final questions require that you combine commands and variables of the type above.

  1. For each weather station apart from the one in Armagh, compute the total rainfall and sunshine duration for months that had no days of air frost. Present the totals in centimetres and days, respectively.

  2. Identify the weather station for which the median number of monthly sunshine hours over the months April to September was largest.

Wrapping up

  1. Like in the last exercise; imagine you need to send your code to a collaborator. Review your code to ensure it is clear and well-structured, so your collaborator can easily understand and follow your work. Render your Quarto document and look at the result.

Optional section

If you went through the exercises above and are ready for more challenges - you have come to the right place. You have not learn all the operations you are asked to do, so feel free to search for help online. If you are a bit overwhelmed and need a break, by all means chill!

Let’s simulate some climate data for year 2056 and merge it with the original climate data set from 2016.

climate_fake <- climate

climate_fake$year <- 2056
set.seed(101)
climate_fake$af <- sample(0:11, nrow(climate_fake), replace = TRUE)
climate_fake$rain <- rnorm(nrow(climate_fake), mean = mean(climate$rain)+150, sd = sd(climate$rain)+50)
climate_fake$sun <- rnorm(nrow(climate_fake), mean = mean(climate$sun), sd = sd(climate$sun))
climate_fake$device <- paste(climate_fake$device, ', New and Improved')

climate_change <- rbind(climate, climate_fake)
head(climate_change)

  1. Change the station names to begin with capital letters.

  2. Create a new column that contain the months in names. Could someone using R have needed the month names before you?

  3. Add a column that tells the season of the month (Winter, Spring, Summer, Fall). Have a look at the case_when function.

Have a look at the case_when function.

  1. Summarize the mean rain fall, air frost, and sun for each year. Evaluate the results.

  2. Summarize the mean rain fall, air frost, and sun for each season and year. Compare the seasons across the years. How will the seasons be different in year 2056?

  3. Summarize the mean rain fall, air frost, and sun for each station, season and year. Is this a good way to get an overview of the weather changes?

We can get a better overview of the weather changes from three plots. We are learning the ggplot method in the next presentation.

  1. Export your data

If you’re up for more, we can try some things that might not totally fit this dataset but will take you through some useful tidyverse operations

  1. Extract the unique stations and save it as a vector.

The unlist function will convert a one-column dataframe to a vector.

  1. Count the number of A’s in each station.

  2. Count the number of A’s and a’s in each station.

  3. In the climate change dataframe, add a column for each word in the device column.

  4. Change one of the new columns to only include lower case letter and the other to only include upper case.

  5. Add a new columns that contains the first three letters of each month in upper case.

Have a look at the str_sub function.

Footnotes

  1. Contains public sector information licensed under the Open Government Licence v3.0.β†©οΈŽ