EDL
Jacolien van Rij & Dorothée Hoppe
10/8/2018
Here we provide astep-by-step overview of the core functions of the package using an example data set.
Example data
Load the example data from the package edl:
## Shape Color Category Frequency2 Frequency1
## 1 cat brown animal 88 6
## 2 rabbit brown animal 88 1
## 3 flower brown plant 6 16
## 4 tree brown plant 6 56
## 5 car brown vehicle 1 88
## 6 bicycle brown vehicle 56 32
step 1: Prepare data
This data set lists all unique learning events (i.e., the types) and their associated frequencies. However, for a data set to function as input for the learning functions, the dataframe must include the columns Cues and Outcomes, and optionally Frequency. Note that if Frequency is not included, the frequency of each learning event is assumed to be 1.
First, we construct the columns Cues and Outcomes for this example simulation. Here we will simulate how two features Color and Shape may function as cues for their category Category. We will add a background cue “BG” to represent the learner. The different cues and outcomes are separated using an underscore (i.e., "_"). It is possible to another symbol, but then we will need to indicate this in the various functions (i.e., RWlearning) with the argument split.
dat$Cues <- paste("BG", dat$Shape, dat$Color, sep="_")
dat$Outcomes <- paste(dat$Category)
dat$Frequency <- dat$Frequency2
# remove remaining columns to simplify this example:
dat <- dat[, c("Cues", "Outcomes", "Frequency")]
# add ID for learning events:
dat$ID <- 1:nrow(dat)
head(dat)
## Cues Outcomes Frequency ID
## 1 BG_cat_brown animal 88 1
## 2 BG_rabbit_brown animal 88 2
## 3 BG_flower_brown plant 6 3
## 4 BG_tree_brown plant 6 4
## 5 BG_car_brown vehicle 1 5
## 6 BG_bicycle_brown vehicle 56 6
Now the data dat defines 36 unique learning events with their associated frequencies.
##
## animal plant vehicle
## 12 12 12
step 2: Create training data
The training data lists all learning events (i.e., the tokens) in their order of presentation, one learning event per row. If no column with frequencies is specificied or all frequencies are 1, the training data mirrors the data set of learning events. The training data also determines the order in which the learning events are presented to the learning network.
# by default 1 run, with tokens randomized:
train <- createTrainingData(dat)
head(train)
## Cues Outcomes Frequency ID Item Trial
## 1 BG_car_yellow vehicle 1 23 item_23 1
## 2 BG_car_blue vehicle 1 35 item_35 2
## 3 BG_flower_blue plant 1 33 item_33 3
## 4 BG_bicycle_gray vehicle 1 12 item_12 4
## 5 BG_bicycle_gray vehicle 1 12 item_12 5
## 6 BG_bicycle_gray vehicle 1 12 item_12 6
# Frequency is always 1:
unique(train$Frequency)
## [1] 1
# total counts per outcome match original frequencies:
table(train$Outcomes)
##
## animal plant vehicle
## 398 398 398
##
## 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
## 88 88 6 6 1 56 56 56 1 1 88 88 32 32 56 56 56 1 16 16 88 88 6 6 6 6
## 27 28 29 30 31 32 33 34 35 36
## 32 32 32 32 1 1 16 16 16 16
Note that the function createTrainingData could also be used to train the network on multiple (blocked or randomized) runs. We refer to the examples in the function helpfile.
step 3: Learning
The function RWlearning trains the error-driven learning network. The output wm is a list with weight matrices, a weight matrix for each learning event (a learning event is basically a row in the data frame dat). The last weight matrix shows the connections after processing all data.
Inspection:
## [1] 1194
# ... which is the same as the number of rows in the training data:
nrow(train)
## [1] 1194
step 4: Inspection
We can now inspect the changes in the weights for after each learning event. The last weight matrix shows the connections after processing all data:
# after the first learning event:
wm[[1]]
## vehicle
## BG 0.01
## car 0.01
## yellow 0.01
# the final state of the network:
wm[[length(wm)]]
## vehicle plant animal
## BG 0.25183879 0.26023902 0.23234787
## car 0.57869696 -0.22138222 -0.23654574
## yellow -0.05222424 0.19551856 -0.01538718
## blue 0.05640855 0.06106817 -0.02745376
## flower -0.20567025 0.53628639 -0.19586861
## bicycle 0.58427796 -0.18397074 -0.27369648
## gray 0.13856554 -0.08443580 0.08633106
## tree -0.20180690 0.52949356 -0.19595374
## white 0.01483705 0.09271535 0.02837224
## brown 0.02539575 -0.07375082 0.18870029
## rabbit -0.25318191 -0.19939852 0.56773558
## cat -0.25047708 -0.20078945 0.56667685
## red 0.06885614 0.06912355 -0.02821478
The function getWM retrieves the weight matrix after a specific event, and adds zero-weight connections for the not yet encountered cues and outcomes.
# after the first learning event:
getWM(wm,1)
## vehicle plant animal
## BG 0.01 0 0
## car 0.01 0 0
## yellow 0.01 0 0
## blue 0.00 0 0
## flower 0.00 0 0
## bicycle 0.00 0 0
## gray 0.00 0 0
## tree 0.00 0 0
## white 0.00 0 0
## brown 0.00 0 0
## rabbit 0.00 0 0
## cat 0.00 0 0
## red 0.00 0 0
We can use the functions sapply and getWM to add zero-weight connections to all states of the network:
wm2 <- sapply(1:length(wm), function(x){getWM(wm,x)}, simplify = FALSE)
# inspect the list of states:
length(wm2)
## [1] 1194
## vehicle plant animal
## BG 0.01 0 0
## car 0.01 0 0
## yellow 0.01 0 0
## blue 0.00 0 0
## flower 0.00 0 0
## bicycle 0.00 0 0
## gray 0.00 0 0
## tree 0.00 0 0
## white 0.00 0 0
## brown 0.00 0 0
## rabbit 0.00 0 0
## cat 0.00 0 0
## red 0.00 0 0
The functions getWeightsByCue and getWeightsByOutcome could be used to extract the weights per cue or per outcome.
# weights for outcome "plant"
weights <- getWeightsByOutcome(wm, outcome="plant")
head(weights)
## BG bicycle blue brown car cat flower gray rabbit red tree
## 1 0.00000000 0.00000000 0.00 0 0 0 0.00 0.00000000 0 0 0
## 2 0.00000000 0.00000000 0.00 0 0 0 0.00 0.00000000 0 0 0
## 3 0.01000000 0.00000000 0.01 0 0 0 0.01 0.00000000 0 0 0
## 4 0.00990000 -0.00010000 0.01 0 0 0 0.01 -0.00010000 0 0 0
## 5 0.00980300 -0.00019700 0.01 0 0 0 0.01 -0.00019700 0 0 0
## 6 0.00970891 -0.00029109 0.01 0 0 0 0.01 -0.00029109 0 0 0
## white yellow
## 1 0 0
## 2 0 0
## 3 0 0
## 4 0 0
## 5 0 0
## 6 0 0
## BG bicycle blue brown car cat
## 1189 0.2613355 -0.1839707 0.06106817 -0.07403495 -0.2213822 -0.1983300
## 1190 0.2587250 -0.1839707 0.06106817 -0.07403495 -0.2213822 -0.2009405
## 1191 0.2598715 -0.1839707 0.06106817 -0.07403495 -0.2213822 -0.2009405
## 1192 0.2600225 -0.1839707 0.06106817 -0.07388391 -0.2213822 -0.2007894
## 1193 0.2601059 -0.1839707 0.06106817 -0.07388391 -0.2213822 -0.2007894
## 1194 0.2602390 -0.1839707 0.06106817 -0.07375082 -0.2213822 -0.2007894
## flower gray rabbit red tree white yellow
## 1189 0.5350565 -0.0844358 -0.1995316 0.06912355 0.5294936 0.09156885 0.1980457
## 1190 0.5350565 -0.0844358 -0.1995316 0.06912355 0.5294936 0.09156885 0.1954352
## 1191 0.5362030 -0.0844358 -0.1995316 0.06912355 0.5294936 0.09271535 0.1954352
## 1192 0.5362030 -0.0844358 -0.1995316 0.06912355 0.5294936 0.09271535 0.1954352
## 1193 0.5362864 -0.0844358 -0.1995316 0.06912355 0.5294936 0.09271535 0.1955186
## 1194 0.5362864 -0.0844358 -0.1993985 0.06912355 0.5294936 0.09271535 0.1955186
# weights for cue "red"
weights <- getWeightsByCue(wm, cue="red")
head(weights)
## animal plant vehicle
## 1 0 0 0
## 2 0 0 0
## 3 0 0 0
## 4 0 0 0
## 5 0 0 0
## 6 0 0 0
## animal plant vehicle
## 1189 -0.02821478 0.06912355 0.06885614
## 1190 -0.02821478 0.06912355 0.06885614
## 1191 -0.02821478 0.06912355 0.06885614
## 1192 -0.02821478 0.06912355 0.06885614
## 1193 -0.02821478 0.06912355 0.06885614
## 1194 -0.02821478 0.06912355 0.06885614
In addition, there are various functions to extract the activations for each learning event. The function getActivations is a wrapper that captures most common calculations, but other, more specialistic functions are described below.
act <- getActivations(wm, data=train)
head(act)
## Cues Outcomes Activation
## 1 BG_car_yellow vehicle 0.03000000
## 2 BG_car_blue vehicle 0.04940000
## 3 BG_flower_blue plant 0.03000000
## 4 BG_bicycle_gray vehicle 0.04891888
## 5 BG_bicycle_gray vehicle 0.07745131
## 6 BG_bicycle_gray vehicle 0.10512777
Alternatively,the function getActivations can output all possible outcomes per learning event.
act <- getActivations(wm, data=train, select.outcomes = TRUE)
head(act)
## Cues Outcomes animal plant vehicle
## 1 BG_car_yellow vehicle 0 0.00000000 0.03000000
## 2 BG_car_blue vehicle 0 0.00000000 0.04940000
## 3 BG_flower_blue plant 0 0.03000000 0.02871200
## 4 BG_bicycle_gray vehicle 0 0.00970000 0.04891888
## 5 BG_bicycle_gray vehicle 0 0.00940900 0.07745131
## 6 BG_bicycle_gray vehicle 0 0.00912673 0.10512777
We may want to add the activation of observed outcome in separate column:
act$Activation <- apply(act, 1, function(x){
out <- x['Outcomes']
return(as.numeric(x[out]))
})
head(act)
## Cues Outcomes animal plant vehicle Activation
## 1 BG_car_yellow vehicle 0 0.00000000 0.03000000 0.03000000
## 2 BG_car_blue vehicle 0 0.00000000 0.04940000 0.04940000
## 3 BG_flower_blue plant 0 0.03000000 0.02871200 0.03000000
## 4 BG_bicycle_gray vehicle 0 0.00970000 0.04891888 0.04891888
## 5 BG_bicycle_gray vehicle 0 0.00940900 0.07745131 0.07745131
## 6 BG_bicycle_gray vehicle 0 0.00912673 0.10512777 0.10512780
Note that getActivations only works for a single outcome in each learning event. With multiple outcomes, please use one of the other activation functions.
step 5: Visualization
Visualizing the change in weights between cues and outcomes is facilitated by two functions: plotCueWeights and plotOutcomeWeights. The first function retrieves the weights between a specific cue and all outcomes (or a selection of outcomes) for each learning event. The second function retrieves the weights between a specific outcome and all cues (or a selection of cues) for each learning.
oldpar <- par(mfrow=c(1,2), cex=1.1)
# plot left:
plotCueWeights(wm, cue="brown")
# plot right:
plotOutcomeWeights(wm, outcome="animal")
Both plot functions have a range of arguments that can be used to change the layout, as illustrated for the same two plots below:
oldpar <- par(mfrow=c(1,2), cex=1.1)
# plot left:
# 1. get outcome values:
out <- getValues(train$Outcomes, unique=TRUE)
out <- out[out != "animal"]
# 2. plot all outcomes, except 'plural':
lab <- plotCueWeights(wm, cue="brown", select.outcomes = out,
col=1, add.labels=FALSE, xlab='', ylim=range(getWM(wm)))
# 3. add plural:
lab2 <- plotCueWeights(wm, cue="brown", select.outcomes = "animal", col=2, lwd=2, adj=0, add=TRUE, font=2)
# 4. add legend:
legend_margin('bottom', ncol=4,
legend=c(lab2$labels, lab$labels),
col=c(lab2$col, lab$col), lty=c(lab2$lty, lab$lty),
lwd=c(lab2$lwd, lab$lwd), bty='n', cex=.85)
# plot right, different layout variant:
out <- getValues(dat$Cues, unique=TRUE)
out <- out[out != "animal"]
lab <- plotOutcomeWeights(wm, outcome="animal", select.cues = out,
col=alpha(1, f=.25), lty=1, pos=4, ylim=c(-.02,.2), font=2, ylim=range(getWM(wm)))
lab2 <- plotOutcomeWeights(wm, outcome="animal", select.cues = "brown", col='red', lwd=2, pos=4, add=TRUE, font=2)
Both plotfunctions are a wrapper around the functions getWeightsByCue and getWeightsByOutcome. These values could be used to extract the weights per cue or per outcome.
weights <- getWeightsByCue(wm, cue="brown")
head(weights)
## animal plant vehicle
## 1 0 0 0
## 2 0 0 0
## 3 0 0 0
## 4 0 0 0
## 5 0 0 0
## 6 0 0 0
Similarly, we can visualize the change in activations using the function plotActivations, which is a wrapper around the function getActivations.
oldpar <- par(mfrow=c(1,2), cex=1.1)
# an observed cueset:
plotActivations(wm, cueset="BG_cat_brown")
# an un-observed cueset:
plotActivations(wm, cueset="BG_cat_yellow")
step 6: Activations
The activation of outcomes reflect the learner’s expectation that this outcome will occur, based on the present cues.
The edl package includes a series different functions to calculate the activativations for outcomes:
activationsMatrix: Calculate activation based on a static state of the network. The basic activation function that is called by the other activation functions.
# select weight matrix:
mat <- getWM(wm)
# for a cueset:
activationsMatrix(mat,cues="BG_cat_brown")
## animal plant vehicle
## 1 0.987725 -0.01430125 0.02675746
# for a specific outcome:
activationsMatrix(mat,cues="BG_cat_brown", select.outcomes = "animal")
## animal
## 1 0.987725
# for a group of cuesets (all connection weights will be added):
activationsMatrix(mat,cues=c("BG_cat_brown", "BG_cat_blue"))
## animal plant vehicle
## 1 1.759296 0.1062165 0.08452773
activationsEvents: Calculate activations for each learning event in the training data.
# new dummy data:
dat <- data.frame(Cues = c("noise", "noise", "light"),
Outcomes = c("food", "other", "food_other"),
Frequency = c(5, 10, 15) )
dat$Cues <- paste("BG", dat$Cues, sep="_")
train <- createTrainingData(dat)
wm <- RWlearning(train, progress = FALSE)
# list with activations for observed outcomes:
act <- activationsEvents(wm, data=train)
## Warning in activationsEvents(wm, data = train): Function will return list of
## activations, because in some events multiple outcomes occurred.
## [[1]]
## food other
## 1 0.02 0.02
##
## [[2]]
## food other
## 1 0.0396 0.0396
##
## [[3]]
## food
## 1 0.039404
##
## [[4]]
## other
## 1 0.03901592
##
## [[5]]
## other
## 1 0.0582356
##
## [[6]]
## food other
## 1 0.06764936 0.07764144
# calculate max activation:
maxact <- lapply(act, function(x){ return(max(x, na.rm=TRUE)) })
unlist(maxact)
## [1] 0.02000000 0.03960000 0.03940400 0.03901592 0.05823560 0.07764144
## [7] 0.09608862 0.09533359 0.11342692 0.13115838 0.14076765 0.15795230
## [13] 0.17479326 0.19129739 0.20747144 0.18981239 0.20601614 0.11159833
## [19] 0.23734477 0.25259787 0.14528315 0.16237749 0.22452532 0.17594734
## [25] 0.26888374 0.28350606 0.29783594 0.25713186 0.31930790 0.27879615
# Using argument 'fun':
act <- activationsEvents(wm, data=train, fun="max")
head(act)
## [1] 0.02000000 0.03960000 0.03940400 0.03901592 0.05823560 0.07764144
activationsCueSet: Calculate activations for one or multiple cuesets.
# list with activations for observed outcomes:
act <- activationsCueSet(wm, cueset=c("BG_noise", "BG_light", "BG_somethingelse"))
names(act)
## [1] "BG_noise" "BG_light" "BG_somethingelse"
## food other
## 1 0.01000000 0.01000000
## 2 0.01980000 0.01980000
## 3 0.03940400 0.01940400
## 4 0.03861592 0.03901592
## 5 0.03784360 0.05823560
## 6 0.04735738 0.06764742
# also activations for non-trained connections:
head(act[[3]])
## food other
## 1 0.01000000 0.01000000
## 2 0.01980000 0.01980000
## 3 0.02960200 0.01960200
## 4 0.02920796 0.02940796
## 5 0.02882180 0.03901780
## 6 0.03833558 0.04842962
activationsOutcomes: Calculate activations per learning event for all outcomes in the data.
# list with activations for observed outcomes:
act <- activationsOutcomes(wm, data=train)
head(act)
## food other
## 1 0.02000000 0.02000000
## 2 0.03960000 0.03960000
## 3 0.03940400 0.01940400
## 4 0.03861592 0.03901592
## 5 0.03784360 0.05823560
## 6 0.06764936 0.07764144
