Downloads Are Available at the Bottom of This Page

Introduction

OpenABM.org and the Computational Social Science Society (CSSS) will be holding a competition for agent-based models to determine a model that best predicts the behavior of subjects in a foraging experiment. The author(s) of the winning entry will receive a cash prize of $1000 and one (1) invitation to attend the annual conference of the Computational Social Science Society (CSSS) 2011.

This document contains a description of how the competition will be run, and technical details of how to enter. It is designed to provide all the necessary information needed by anyone wishing to enter the competition.

Experimental Environment.

Janssen et al. (2010) have been investigating a real-time dynamic resource-harvesting scenario in order to explore the conditions in which people are able to solve collective action problems. When individuals share a common resource, there can be a tension between individual and group motives. When individuals behave in a selfish rational manner, one might expect over-harvesting of the common resource. The experimental results show that groups indeed over-harvest their common resources. In their experimental research program, Janssen et al. have been exploring conditions in which individuals are able to avoid over-harvesting by using techniques such as communication and punishment.

The participants collect tokens from a shared renewable resource environment. Groups consist of five participants who share a 29 x 29 grid of cells. At the start of an experiment, 25% of the grid space is filled with tokens (thus 210 tokens are randomly allocated on the board.) Each participant is assigned an avatar, which they are able to control using the arrow keys on the computer keyboard to move up, left, right, and down (See the video below for a typical experiment.) The participants were permitted a maximum of 8 keystrokes per second.  The avatars are initially placed in the middle row of the screen with an equal distance between the avatars.  When the participant wishes to harvest a token, they must position their avatar on top of a token and press the spacebar.  Each token is worth a certain amount of money defined by the experimenter. Participants have complete information on the spatial position of tokens and can watch the harvesting actions of other group members in real time (the entire screen is visible to the participants.) Furthermore, they can see the total number of harvested tokens of all the participants at the top of the screen.

Every second, empty cells have a possibility of regenerating a new token. The probability, p_t, that a given empty cell will generate a new token is density-dependent on the number of adjacent cells containing tokens:

where n_t is the number of neighboring cells containing tokens, N is the total number of neighboring cells (typically N = 8 because a Moore neighborhood is used), and p = 0.01. If an empty cell is completely surrounded by eight tokens, it will generate a token at a higher rate than an empty cell that abuts only three tokens. The model space is not toroidal, so cells located at the edges of the screen have fewer neighbors than interior cells.  For example, a corner cell only has 3 neighbors (N = 3).

At least one adjacent cell must contain a token for new token generation to occur. Therefore, if participants appropriate all of the tokens on the screen, they have exhausted the resource and no new tokens will be generated. By designing the environment in this manner, the experiment captures a key characteristic of many spatially-dependent renewable resources. 

<

p>

Video: Example round of the foraging game.

Experimental Data

The experimental data was collected from five groups consisting of five participants each. The participants received instructions on the experiment and were tested in their understanding of those instructions.  They were then allowed to practice for 4 minutes on an individual 13 x 13 grid. They did not know at that time that they would be put together into groups for the actual experimental phase. In the group experiment, each participant saw their own avatar as a yellow dot, while the avatars of the other participants were blue. Each avatar also had a number 1, 2, 3, 4, or 5.

The experiments consisted of multiple 4-minute decision periods in which different treatments were used where the participants were allowed or disallowed from using punishment or communication during the decision period. The data set provided for this competition only includes the initial decision period of these five groups during which no communication or punishment was allowed. The data from this simplified situation was selected in order to minimize complications from the participants' growing familiarity with the model in subsequent periods as well as communication opportunities.

Figure 1 depicts the number of tokens still on screen throughout the experiment. The actual numbers are provided in the accompanying spreadsheet. Four of the five groups depleted their common resource within 90 seconds, and for the remainder of the decision period they had to watch their black screens. One group only depleted their resource after 2.5 minutes.

Figure 1. The amount of tokens in the common resource for five different groups.

Figure 2 shows the harvesting rate of each group measured in 5-second periods. Not surprisingly, group 3 had a lower harvesting rate during the first 60 seconds, allowing more regrowth of the resource and a longer duration of harvesting. The total number of harvested tokens per group is, in order of group number, 264, 255, 267, 357, and 280.

Figure 2. The harvesting rate of the five groups over time.

The next statistic concerns the harvesting event. An optimal strategy of harvesting would maintain a checkerboard pattern of tokens--four tokens around a cell that is harvested. If we look at Figure 3, we see that most tokens are harvested when only 2 or 3 tokens are on the neighboring cells.

Figure 3. The distribution of harvesting events defined by the number of tokens around a cell that is harvested. The distribution is defined by the harvesting events of all five groups.

In addition to the harvesting events, avatars move around but not as a random walk. As discussed in Janssen et al. (2009), participants are more likely to move in straight lines. This might be due to the cost of changing directions. Figure 4 shows the distribution over all movements for the 5 groups.

Figure 4. Distribution of number of straight moves of an avatar before changing direction.

The final measurements concern the distribution of earnings within groups. There is a significant difference between the number of tokens collected by individuals within each group (Figure 5). The average gini coefficient is 0.204 and there is only a modest variation of the gini coefficient between groups.

Figure 5. Distribution of individual earnings in the five groups.

Developing a Model of Foraging Behavior

Given the data provided and the description of the experimental environment it is possible to develop an agent-based model that simulates five agents sharing a common resource. In Janssen et al. 2009, a model is presented based on earlier data sets and a somewhat different design of the model. This model might be a good starting point, but we suggest you think broader. For example, using a fixed speed distribution might not be appropriate for predicting a new treatment (see next section). Hence, a winning model will include the decisions of agents when to move, where to move to, and when to harvest a token.

The model might be implemented in any platform and language of choice by the participant in the competition, as long as we can replicate the results. This means that the model needs to be properly documented and instructions need to be provided how to run the model.  

Determining the Winning Model.

Six experimental sessions will be performed with human subjects using the same experimental environment as before but with one important difference: participants will not be able to see the whole screen but can only observe their environment within a radius of 6 cells. Tokens and avatars outside the participant's 6-cell radius can not be observed.

The task of your model is to predict the outcomes of the adjusted experiment. The winner of the competition is the model that provides the best fit with the experimental data. Note that a model that has a perfect fit with the data of the 5 groups in the original experiment (the data provided here) may not necessarily generate a good fit with the adjusted experiment. One is advised to think about possible changes in behavior due to the limited vision.

The best fit metric is defined in the following way: The model will be run 100 times for each of 6 experiments to generate 600 observations, grouped into 100 observation sets. The following data needs to be stored for those 100 observation sets:

• The average amount of tokens of the resource over the six experiments, for time steps 0, 5, 10, ..., 240 (see Figure 6).
• The average number of tokens harvested by the six groups
• The distribution of harvesting events which is, like Figure 3, the number of tokens per number of tokens in neighboring cell.
• Distribution of number of straight moves of an avatar before changing direction (see Figure 4).
• The average gini coefficient of earnings in the six groups.

Figure 6. Example plot of average token measurements for the six experiments over time.

The data needs to be stored in a comma-separated values (CSV) text file in the following way:

Resource level
  Run
 1 2 3 4 .. 100
0
5
10
15
..
235
240

Total harvest
  Run
 1 2 3 4 .. 100

Distribution harvesting events
  Run
 1 2 3 4 .. 100
0
1
2
3
4
5
6
7
8

Distribution of straight moves
 Run
 1 2 3 4 .. 100
1
2
3
4
..
27
28

Gini
 Run
 1 2 3 4 .. 100

Based on this data file, which will be generated by your model, we will determine the closeness of fit. In order to compare the different metrics, each metric will be scaled to a similar level:

• Resource size values are divided by 210, the initial number of tokens on the screen.
• The number of tokens harvested is scaled by first subtracting 210 tokens, and then dividing it by 400.
• Division by 400 is based on the fact that the maximum number of tokens that can be harvested in a decision period of 4 minutes is just above 600 tokens.
• By dividing by the total number of tokens harvested, the relative distribution of harvested tokens is derived.
• By dividing by the total number of movements, the relative distribution of movements derived.
• The gini coefficient is already between 0 and 1.

Then, we will calculate x_r,i,j, the squared difference between the empirical data and the simulated data for each run and each metric.

Finally, the fit-closeness score S of the model is calculated:

The best score possible is 0. The model that results in the score closest to 0 among the submitted models will be the winner of the competition.

Rules of Submission:

Participants need to submit the following materials before April 1, 2011 to Nathan.Rollins@asu.edu:

• Text data file according to the specifications
• Model Description
• Documented code for the model. If the model is not implemented in Netlogo, please provide instructions how to run the model and generate the output that is used to create the text file.
• People affiliated with the research group of Marco Janssen or the management of the openabm.org portal can not participate in this competition. Students of the jury members can participate in the competition. 

References

Janssen, M.A., R. Holahan, A. Lee and E. Ostrom (2010), Lab Experiments to Study Social-Ecological Systems, Science 328: 613-617.

Janssen, M.A., N.P. Radtke, A. Lee (2009), Pattern-oriented modeling of commons dilemma experiments, Adaptive Behaviour 17:508-523. 

Instructions and Excel dataset available for download: