Internet Economics
‫כלכלת האינטרנט‬
Class 9 – predictions and scoring rules
1
Today
• Prediction markets and scoring rules.
• Some guidelines for the 2nd semester
2
Wisdom of the crowds
• The Internet is used to aggregate information from the
general population.
• Creation of “information markets”.
• Main use: calculating probabilities of one-time events
–
–
–
–
Winners in elections.
Outcomes of sports events.
Political changes.
Financial events (future exchange rates, future stock values)
• In large companies: elicit information from employees.
– Probability that our new smartphone will gain more than 10% of the
market.
3
Information markets
intrade
Contracts
• We need to sign a contract with the
market participants.
• Contract should be based on verifiable
data.
– Sometimes it is tricky….
– Israel will strike Iran by December 31st 2011.
– Robert De-Niro will win the 2011 academy
awards for best actor in main role.
5
6
Winner-takes-all contract
• A bond is issued:
– If event A occurs, it pays $100.
– Otherwise, pays 0.
• For example: Saddam security
Paid out $100 if Saddam is no longer in power by June 2003.
– Traded on website tradesports.com
– In the hour after President Bush's war "declaration", the Saddam-gone-byMarch bet shot up nearly 10 percentage points (odds went from about 60 to
69 percent)
– Price on Jan 1, 2003: $55
– Price on March 1, 2003: $70
7
Trading Contracts
• How are contracts traded?
– In most prediction markets, the mechanism used
is the continuous double auction.
– This is the mechanism used on the NYSE.
8
Winner-takes-all Contracts
•
US 2008 elections – price history (from Iowa Electronic Market)
9
Scoring rules
• We sometime aim to more delicate contracts
• For example, think about the following simple
scenario:
we do not want to aggregate the wisdom of the
crowds, but the wisdom of one individual.
10
Example: Weatherman
• Tomorrow can be either sunny or rainy.
• We would like to know the probability of a sunny
day.
• A weather forecaster knows, but he can lie.
• Can we incentivize him to tell us the truth?
75% for a sunny
day tomorrow
11
Contracts
• We need to sign a contract with the weatherman.
• Contract with the weatherman can depend on:
– The probability declared by the forecaster.
– The realization of the outcome (sunny/rainy)
• Contract cannot depend on the true probability
– It is unobservable, even in retrospect.
12
Proper scoring rules
• We would like that given the contract, a utilitymaximizing forecaster will report the true probability.
– Via a bonus system.
• Contracts where the forecaster strictly prefers to
report the truth are called proper scoring rules.
– It is easy to construct scoring rules that are not proper.
13
Model
• Consider a binary outcome: z=0 or z=1
– Example: z=1 means rain, z=0 means sun.
• p is the true probability of z=1.
– Pr(z=1)=p, Pr(z=0)=1-p
• q is the report of the forecaster.
• The contract:
– After realizing the outcome z, the forecaster pays
R(z,q).
• Are there proper scoring rules in this model?
14
First try: pay your prediction
• Consider the following contract:
– R(z,q) = q
• Pay the probability that you gave to the realized outcome.
– i.e., when z=0 forecaster is paid R(0,q) = 1-q
when z=1 forecaster is paid R(1,q) = q
• Expected surplus for forecaster:
– p*q + (1-p)*(1-q) = (1-p)+(2p-1)q
• Forecaster will optimize q.
– Report 1 if p>1/2, report 0 otherwise.
• Not a proper scoring rule.
15
Example: quadratic scoring rules
• Consider the following contract:
– R(z,q) = 1 - (z-q)2
– i.e., when z=0 forecaster is paid R(0,q) = 1-q2
when z=1 forecaster is paid R(1,q) = 1-(1-q)2 =q(2-q)
• Expected surplus for forecaster:
– p*R(1,q) + (1-p)*R(0,q) = pq(2-q)+(1-p)(1-q2)
• Forecaster will optimize q.
– FOC:
• p(2-2q)+(1-p)(-2q) = 2p-2pq-2q+2pq = 0  q=p
• A proper scoring rule (“quadratic scoring rule”)
16
Example: logarithmic scoring rule
• Consider the following contract:
– R(z,q) = zlogq + (1-z)log(1-q)
– i.e., when z=0 forecaster is paid R(0,q) = log(1-q)
when z=1 forecaster is paid R(1,q) = logq
• Expected surplus for forecaster:
– p*R(1,q)+(1-p)*R(0,q) = p* log(q) + (1-p)* log(1-q)
• FOC:
– p(1/q)-(1-p)(1/(1-q))=0  q=p
• A proper scoring rule (“logarithmic scoring rule”)
17
Finding proper scoring rule
A general result:
• n possible events (1,…,n)
• Probability estimate r=r1,…,rn
• Scoring rule for each event si(ri) (i=1,…,n)
n
• Expected score function: H (r1 ,..., rn )   ri  si (ri )
i 1
Theorem: (McCarthy ‘56, Hendrickson and Buehlera'71 )
Every homogeneous and convex expected score
function defines a proper scoring rule.
• Example: quadratic scoring rule H(q)=q2(2-q)+(1-q)(1-q2)
18
Summary
• Often one would like to estimate probabilities of
future events
– Aggregate information from the crowds
• Used in Information/prediction markets
• Strong ties with financial securities trade
– Options, etc.
• Some beautiful economic theory behind it.
19
Today
• Prediction markets and scoring rules.
• Some guidelines for the 2nd semester
20
Grade
• 30% presentation
• 58% seminar paper
• 12% participation
– You will get 12 points if you attend at least 80% of the classes in both
the 1st and the 2nd semester.
– If you miss the 80% in one of the semesters, you get 0 points.
– You will not get a grade in the course if you attend less than 50% of
the classes in at least one of the semesters.
• 12% problem set (“magen”)
Course duties: presentation
• In pairs.
– Let the better speaker talk, but two speakers possible.
• Practice
– Remember: presentations in real-life are much faster
than the practice talks.
• 25 minute talk. (~3 in each meeting)
– Schedule will be posted in the beginning of the semester break.
– You should be ready to present in the first week of the semester.
• Make it interesting to the class!
– Speak to class, not to instructor.
• Language: Slides in English (preferred, not mandatory),
Speak Hebrew.
22
Course duties: presentation
• Goal: present the paper to the class.
– Present what is written in the paper, no need for criticism
or your interpretation.
– Give the background, and make the main results clear
(do not present all results).
– If suitable, spend at least 5-10 minutes on technical
details (proof idea, analysis methods etc.)
– Survey related work very briefly, only if needed.
(sometimes it is good to read some of these…)
– Don’t follow the structure of the paper exactly.
– Explain in your own words.
23
Course duties: presentation
• Presentation grading will be based on:
– Clarity.
– Understanding.
– Identifying and understanding the main results.
– Quality of presentation.
– Conveying complicated ideas in a simple way.
– Presenting your own understanding (not just cut
and paste).
“Make everything as simple as possible, but not
simpler. ”
Albert Einstein
24
Some presentation tips.
• Use large fonts
– in general, rarely use fonts below 24.
• Don’t have dense slides – split them.
• A figure is worth a thousand words:
If x is an individually rational revenue maximizing auction
then there exists a Pure Nash equilibrium in the game.
X is individually
rational revenue
maximizing
auction
Pure Nash
equilibrium exists
• Of course: Be politically correct
– No offensive language, images, examples etc.
25
Some presentation tips.
• Use colors.
• Use images.
• Use animation
– But with care,
do not overuse it
please
• Make it interesting, but remember that this is an
academic duty. Content is the main thing.
– Most effort should go on understanding the paper you
read.
26
Some presentation tips.
• Basic presentation structure:
– Motivation (usually, real life problem, informal) +
Background
– Informal results (if possible)
– Model
– Results
– Technical details
– Summary
• Of course, you should fit the structure to your paper.
27
Course duties: Problem set
• Optional (Magen 12%)
• Strict deadline: February 1st
• By your own. (not in pairs.)
• Available tomorrow on the website.
28
Course duties: Seminar paper
• Should be on the same topic as your presentation
– With the same partner.
• Up to 10 pages, font 12, 1.5 spacing. (strict!)
• Submission time: May 30th.
29
Course duties: Seminar paper
In the paper:
Part 1: Summarize the paper.
Background, main results, main technical details .
– Some proofs (outline if needed), methods of data
analysis. (Tables with data should not be part of the 10 pages you submit,
can be in an appendix.)
– Go to depth regarding the main result/s.
Part 2: A new research direction,
– Phrase exact questions and challenges, one (small) step
toward to solution.
– Idea may be theoretical, empirical or other. You need to
define the framework of the research (model, goals,
questions, difficulties, data resources, suggested
theorems)
30
Course duties: Seminar paper
Originality:
• Cite all materials that you use.
• Statements that are not from the paper you read
should be given with a bibliographic reference.
• Especially, let me know if you intend writing any
project for another course on a similar subject.
• Do not copy text, even from the paper you read.
– Explain in your own language. Give new examples. Give
the intuition behind the proofs.
31
All seminar
topics are
chosen
and approved
(January 1st)
Presentation
outline + hard
scheduling
constraints
sent to Avi.
(January 12th)
Timeline
Problem set Students
due date
presentations
st
(Feb. 1 )
start
(February
16thth)
All seminar
papers are
submitted
(May 30th)
Presentation
schedule
posted
(January 20th)
presentations
Winter
vacation
starts
Winter
vacation
ends
Summer
vacation
starts
32
Good luck.
Don’t hesitate to come to me for questions or advise.
33