ARE 298 - Empirical Project, 2006

Guidelines for Referee Reports

The following indicates some of the kinds of things you should consider when doing a peer review. You almost certainly should not include all of these in your report, but you certainly should be sure to include any useful, critical insights you have. Doing so will not only help your own grade in this course, but will also be helpful to the author, which is the real point of a peer review. Try to use examples and specific references to insure clarity in your comments. Your report will be evaluated on its thoughtfulness, clarity, and helpfulness.

You should read the paper carefully, checking all the arguments, whether mathematical or not, for correctness. Point out any problems that you find, and feel free to comment more generally on the paper. You should not be mean but you should be critical; pointing out errors and suggesting improvements is your job. Reports which are vaguely laudatory will be judged very harshly.

Summary

Write a short summary of the paper using your own words. What is the question asked by the author? What is the modeling strategy? What data is used? How is the hypothesis formulated and tested? What are the results?

Completeness

Following is a checklist for you. Only report briefly on what is inadequately treated in the paper.

- Is the paper complete? It should have an introduction, a discussion of the hypothesis to be tested, an economic model that permits a precise formulation of the hypothesis, a description of the data, a discussion of results, and a conclusion.

- Is the paper well structured? Did the author clearly state an argument and develop it? Did you always have a clear sense of what the author was trying to do?

- Does the author's citation practice make it clear that he knows the literature reasonably well?

- Does the author cite other work in the relevant literature to good advantage, managing to simplify his own presentation without obscuring the issues he tries to address?

- If you are unfamiliar with previous work in this area, does the author provide enough information so that you could go to the library and quickly get an overview of the relevant literature? If you are familiar with the field, did the author's references help you understand the contribution the author is attempting to make and how it complements existing work?

- Are all figures and tables clearly labeled and easy to understand? Are axes labeled and variables defined? Are units given?

- Are all references given?

- Is all notation defined (and defined before it is used)?

- Is the paper well written? Are there errors of grammar or spelling?

The Question

- Is the topic clearly explained? Could the question be made more precise?

- Does the author do a good job of motivating the question in the introduction?

- Is the answer to the question obvious in advance?

- Is the question original? What is the contribution of the paper? Does the author pose a question of reasonable scope (i.e., can she reasonably hope to answer the question in a short empirical paper)?

The Model

The model need not be a formal structural model of optimizing behavior, but if it is a reduced form model, there should at least be a clear verbal description of the economic theory that is behind its specification.

- Does the model formalize the argument given by the author in the question?

- Does the model incorporate those aspects of reality that the author seems to think are important?

- Is it possible to answer the question posed by the author within the context of the model?

- Is the model elegant? Is it simple? If it is not simple, is it unnecessarily complex? Could the author attack the problem with a simpler model?

- Is the notation clean and intuitive?

- Is the model internally consistent?

- Is the estimating equation clearly related to (or preferably derived from) the model?

- Does the disturbance term have an interpretation within the model, or is it just tacked on?

- Are the econometric techniques well suited to the problem at hand?

- What are the properties of the estimators employed by the author? Are the issues regarding these properties adequately addressed in the paper?

- What test statistics does the author employ? Do they answer the question?

The Data

- Does the author present a clear description of the data?

- Does the author's choice of a dataset seem well suited to answering the question she poses?

- If you had to replicate the author's study five years from now, is there sufficient information in the paper about the source of the data and sample used in estimation that you could do it?

- Does the author discuss issues that may affect her estimation strategy: Is the data from a random sample? What are known sources of measurement error? If a panel, is there reason to believe that there may be cross-sectional dependence?

- Does the author present summary statistics, and make good use of them to motivate the question or some specific aspects of her analysis

The Econometric Analysis

- Is the econometric analysis carefully done and reported?

- Have alternative specifications been tried and compared, when necessary?

- Is the issue of robustness of the results addressed?

- What test statistics does the author employ? Do they answer the question?

Results and Conclusion

- Are the results clearly stated and presented?

- Are they used in some interesting way (beyond quoting the value of the parameters and their standard errors)?

- Are the results related back to the question?

- Are appropriate caveat mentioned?

- Do the conclusions concisely summarize the main points of the paper?

- Are the conclusions reached by the author well supported by the evidence?

- Are you convinced? What did you learn from this paper?