Comparison of Improved Estimators under Pitman Closeness
Criterion
M.DUBE
Department of Statistics
M.D.University, Rohtak 124001 INDIA
S.CHANDRA
Department of Mathematics and Statistics
Banasthali University, Rajasthan304022 INDIA
Abstract
The article analyzes the performance of various improved estimators under balanced loss proposed by
Zellner. The quantities involved in the balanced loss function have quadratic loss structure, therefore
they may fail to reveal some intrinsic properties of estimators. As an alternative it is recommended to
use the criterion like the probability of concentration around the true parametric value and closeness of
the estimators is the true parametric value, one such criterion proposed by Pitman (1937) called
Pitman closeness criterion. The present article derived the asymptotic approximation for the Pitman
closeness probability and compared the least squares and Stein rule estimators utilizing the exact prior
information binding the regression coefficient using Pitman closeness criterion under balanced loss
function using small sigma disturbance asymptotics and disturbance distribution to be not necessarily
normal.
INTRODUCTION: In regression analysis
we are often interested in using an estimator
which is precise and which simultaneously
provides a model with good fit. Zellner [11]
advocated that both the attributes should be
utilized in analyzing the performance of
estimators. Accordingly, he recommended the
use of (quadratic) balanced loss function which
maintains a good balance between the two by
measuring the precision of estimators and the
goodness of the fitted model simultaneously.
Many authors used the balanced loss function as
a comparison criterion to compare the
performance properties of competitive estimators
see [7].
The quantities involved in the balanced loss
function have quadratic loss structure, therefore
they may fail to reveal some intrinsic properties
of estimators as pointed out by Rao [3]. As an
alternative it is recommended to use the criterion
like the probability of concentration around the
true parametric value and closeness of the
estimators to the true parametric value. Rao [3]
suggested to use one such criterion proposed by
Pitman called Pitman closeness criterion.
Recently, Pitman closeness criterion received
attention of many researchers for a comparison
of alternative estimators. For instance Shalabh
[6] compared the least squares and Stein-rule
estimators using Pitman closeness criterion when
the disturbances are not necessarily normal. Van
Hoa et al [9] derived the conditions for the
dominance of the two stage estimators over the
least squares and Stein-rule estimators under
generalized Pitman closeness criterion.
Until now, the extraneous information about the
parameters of interest has not been utilized to
study the properties of various competitive
estimators. However, in most of the situations
some prior information, may or may not be
exact, is available in the form of linear
constraints on parameters. Therefore, using small
sigma asymptotic approximations and assuming
the disturbances to be not necessarily normal, the
present article derived the asymptotic
approximation for the Pitman closeness
probability and compared the least squares and
Stein rule estimators utilizing the exact prior
information binding the regression coefficient
using Pitman closeness criterion under balanced
loss function.
Q  XX '
and
~

  I 






y  XbR ' y  XbR
2k
X ' X bR

n p2
bR' X 'QXbR

(2.6)
2. THE MODEL, ESTIMATORS AND LOSS
FUNCTIONS:
They have obtained the dominance condition of
superiority of Stein – type estimators defined
above with that of restricted least squares under
general quadratic error loss function.
Let us consider the following linear
regression model,
The balanced loss function for the
estimation of  by an estimator is specified by
y  X β+ σu
(2.1)
where y is an n  1 vector of n observations on
the response variable, X is a full column rank
matrix of n observations on p explanatory
variables,  is a p  1 column vector of
regression coefficient associated with them, 
is an n  1 vector of disturbances whose
elements are independently and identically
distributed assuming finiteness of moments up to
order four such that E[ut ]  0,
E[ut 2 ]  1,
E[ut 3 ]  r1 ,
E[ut 4 ]  r2  3 , t=1,2,-------------,n
and  is an unknown positive scalar.
The least squares estimator of  is
'
b   X ' X  X ' y


(2.2)
In the presence of some prior information in the
form of restrictions on  as
(2.3)
r Rβ
where r is a J  1 vector of known elements and
R is a J  p known prior information design
matrix, the least squares estimator of  is given
by
'
'


. bR  b   X ' X  R '  R X ' X  R '  r  Rb  (2.4)


 


The Stein rule based estimators proposed by
Srivastava et al [8] and Mittlehammer et al [9]
are given by
2k
ˆ  bR 
n p2
y' p X y
X ' Xb
y 'Qy
'
where p x  I  X  X ' X  X '


(2.5)
 



 

'
'
L ˆ ,   1    ˆ   X ' X ˆ     y  Xˆ y  Xˆ
(2.7)
where  is a non stochastic layer lying between 0
and 1.
The first component on the right hand side of
(2.7) reflects goodness of fitted model while the
second component reflects the precision of
estimation. The choice of  highly depends on
the experimenter and objective of the
experiment.
Another interesting criterion is the Pitman
closeness, following Rao et al [4]., the formal
definition of the Pitman closeness criterion is :
For any two estimators 1 and 2 of  , under
the loss function(2.7) , the estimator 1 is said to
be Pitman closer to the estimator  as compared
to 2 if
PL 2 ,    L1,    0 
1
2
(2.8)
3. Risk Comparison Of Estimators
Under Balanced Loss Function:
Much of literature is available to study the
relative performances of least squares and Stein
rule estimators with respect to the risk under
quadratic error loss structure using Pitman
closeness under the assumption of non normality
of disturbances, Employing large sample
properties an attempt has been made to study the
relative performances of the same in the presence
of exact prior information available in the form
of linear constraints under balanced loss
function. Under the balanced loss function, the
risk of b and bR are given by
Rb    2  n  p   1    p 
(3.1)

RbR   Rb   2 J 2  1
(3.2)
respectively.
Now, for the comparison of the estimators, let us
consider the difference of loss functions as
(3.3)
L  LbR ,    L ˆ , 
where
 

 


1   bR   ' X ' X bR   
(3.4)
and
    
1   ˆ   ' X ' X ˆ   
'
L ˆ ,    y  Xˆ y  Xˆ 
(3.5)
bR and  are defined in (2.4) and (2.5)
respectively. According to the Pitman closeness
criterion Stein rule based estimators behaves
better-than the restricted least squares if
PL  0 exceeds 0.5.
For finding an asymptotic approximation of this
1
n

probability, let us assume that  X ' X 

to a finite and nonsingular matrix as
and consider the following notations
'
tends
n 
'
1

S   X 'X 
n

1 ' '
2
   X QX
n
W 
1
1
n2

(3.6)
1  u 'u 
1
1  n



n2



 b   ' X ' X b   
R

L  2  ˆ  bR X 'u    R

  ˆ

ˆ
    'X'X   




 

(3.8)
utilizing the notations (3.6), (3.8) can be
written as
L
1   y ' Qu  
y' p x y 
4k

4k
y' p x y 
n  p  2 y' Q y 
 n  p  2

(3.9)
where using (2.1), we can write
y ' Px y = P  2 u ' u  u ' X ( X ' X 1) X ' u 



w

n
or y ' Px y =  2n 1 
X 'u
  n
(3.7)
Proof : Substitution of (3.4) and (3.5) in (3.3)
leads to
L bR ,    y  XbR ' y  XbR 
V 
(3.7)
1 ' ' 
 1    
E[ D] 
 p  J  2    X Qe 

n 

 k
n
1
p  2
1
= 1 
n
n 
n p2

(3.10)
v' sv 

n 
1

 1 
1

 o

n
 n2 
(3.11)
and
and
and
n
D
L
4 3R
Therefore it can be said that Stein rule based
estimator is superior to restricted least squares
estimator compared under balanced loss function
if E[D>0]
Theorem 1: The asymptotic approximation for
E[D] to order O(n-1/2 ) is given by
1
1

y ' Qy n 2

  1 
2
 ' X ' Xv   o  (3.12)
1 
  2 n
  n 2 
substituting (3.10), (3.11), and (3.12) in
(3.9), we get
L=
4k 3
n 2




1
1    ' X ' X ' v 
n








v' ( 


 

 2

k 

 2 x' x ' xx)  w 
n



1


  ' xx ' v




(3.13)
E[ D]  E[ A] 
1
E[ B ]
(3.18)
n
Substitution of values of expectation from (3.16)
and (3.17) in (3.18) gives the result (3.7) of
theorem 1.
From the result (3.7) we observe that ̂ performs
better than bR according to the order of our
approximation, if
Now writing ,
D=
n
(3.14)
.L
3
4k
It is obvious from (3.14) that P[D>0]=P[L>0],
Therefore, Stein rule behaves better than
restricted least squares in restricted regression if
E[D]>0

0  k  (1   )[( p  J  2) 1  ' X ' Qe]

(3.19)
for the normal distribution (i.e.  1=0), the
condition (3.19) reduces to
0<k<(1-α) (p-J-2)
Rewrite D as
If no prior information is available regarding the
parameter vector (i.e. J=0), the condition reduces
to
1
 O 
n
n
B
DA
0<k<(1-α) (p-2)
where
A=
1

which indicate that the dominance condition for
the Stein rule based estimator over least squares
estimator shrinks when compared under balanced
loss function as compared to that of obtained
under quadratic loss structures measuring only
the precision of estimation.
 ' X ' X ' v
And

B = 1   

 
2
 
v'    2 XX '  ' X ' X v  

   k
 

 1
  n
  ' X ' Xv.w

 

(3.15)
Taking expectation of above as
E[A] = 0
E[ B] 
; p>2
(3.16)


1 
  p  J  2  1  ' X 'e  k
(3.17)
v



4.
GENERAL PITMAN CLOSENESS
UNDER
THE
BALANCED
LOSS
FUNCTION:
Following Rao et al.[4], Stein rule estimator
dominates least squares under general Pitman
closeness criterion when P[D>0] exceeds 0.5. As
it is difficult to derive the exact expressions for
probability
P[D>0],
we
consider
the
approximation when n is large.
Theorem 2: The asymptotic expressions for the

1 


profability P[D>0] to order O n 2  is given by


(4.1)
P[D>0] = 0.5 +
(1   )( p  J  1) 
1 

 m

2  1  k
 6


n
(4.1)
Stein rule estimator performs better restricted
least squares when
=
6 
1 
m 
  1  1   mo  
 
6 
n 
K4 = E[D- E(D)]4 – 3 (E(D)-E(D))2]2
= O(n-1)
Employing the above results, the cumulant
generating function of D, will be
 m

0<k< (1   )( p  J  1)  1 
6 

(4.2)
K(t) = 
it  j K
j
j
Proof: Suppose g(D) denotes the pdf of D,
=i t K1 +

therefore P[D>0] =  g ( D)dD
0
=(1-α)2
The method of cumulant generating function is
adopted to get the probability density function.
Before obtaining the cumulants of D, let us first
discuss the following results.
E[A]=0
E[A2] = (1-α)2
 m
E[A3] = 1
 n
1


  p  j  2  

1

 ' X ' Qe  k






(1   )  p  J  4  k   3 1 mo 
E[A B] =




2
E[A3B] = O (n-1)
m=

3
3
t2


ita  (it )3 b
2
n

1
 ' X ' Qe
n

a= (1-α)  p  J  1 


r1


1

 ' X 'e


  m
mo   1
  6
(4.7)

t2
]exp
2

 

ita  (it )3 6   O(n 1 )

 n

Inverting it to
probability density function of D
1   itD
 (t )dt
e
2  
along with the results
Using above expectations, the cumulants of D

1 


1 2
1   itD  2 t
dt  f ( D)
e
2  
can be derived upto order O n 2  as


K1 = E[D]
K2 = E[D-E(D)]2
= (1-α)2 + O(n-1)
K3 = E[D- E(D)]3
1 2
1   itD  2 t
dt  iDf ( D)
 t.e
2  
(4.4)
(4.6)
The characteristic function of D, upto the order
O(n-1/2) can be written as
g(D) =
1
 ' X ' Qe (I*QXββ’X’Q)e
n
(4.5)
where
 (t )  exp[ K (t )] =exp[-(1-α)2
E[AB] = O (n-1)
where mo =
2
b = (1-α)  1 
(4.3)
E[B] =
it 2 K 2  it 3 K
get
the
It is worth mentioning here that the same
conditions of dominance on characterizing scalar
is obtained when the estimator defined in (2.6)
and bR are compared to the order o n 2
1 2
1  3  itD  2 t
dt  i 3D  D3 f ( D)
 t .e
2  


 
where f(D) denotes the p.d.f of standard normal
variate, we get the following results for g (D) as





a  b( D3  3) D 
g(D) = f(D) 1 
n


 5
o n 2
However, if the terms of 


(4.8)
the difference in dominance conditions may
arise. The terms in the risk differences are

retained to order o  n
To the order O(n-1/2 )



P[D>0] =  g ( D)dD
(4.9)
0
Substitution of the values (4.6) and (4.7) in (4.8)
and therefore in (4.9) gives
P[D>0]=0.5 +

1 
 1m

(1   )( p  J  1)  6  k 
n 2 



 are considered


1
2


 , resulting no difference

in the dominance conditions.
At last, on the basis of superiority conditions
obtained in (4.11) it can be concluded that if the
number of regression coefficients to be estimated
are more than one, the least squares estimators
can be improved upon by using the Stein rule
estimator in the presence of exact prior
information binding the regression coefficients
using Pitman closeness criterion.
which is the result (4.1) of Theorem 2.
The condition of superiority of ̂ over bR under
balanced loss using Pitman closeness criterion
can be obtained as to the order of our
approximation as
 m

0  k  (1   )( p  J  1)  1 
6 

1.
Mittlehammer, R.C. and Conway, R.K.
On the Admissible of Restricted Least
Square Estimation, Communications in
statistics –Theory and Methods, 13.
(1984) 1135-1146.
2.
Pitman, E.J.G, The closest estimators of
statistical parameters. Proceedings of
the Cambridge Philosophical Society,
33 (1937) 212-222.
3.
Rao, C.R., Some comments on the
minimum MSE as a Criterion of
Estimation. In statistics and related
topics, eds. N.Esorgo, D.A. Dawson,
J.N.K. Rao and A.K. Md, E.Saleh,
Amsertdam; North Holland, (1981)
123-143.
4.
Rao, C.R., J.P. Keating and R.L.
Mason, The Pitman nearness criterion
and its determination, communication
statistics A – Theory Methods 15(11)
(1986) 3173-3191
5.
The Stein paradox in the sense of the
Pitman Measure of Closeness. The
(4.10)
provided the quantity on the right hand side is
positive. The effect of skewness of the
distribution can be seen here.
For the symmetrical distribution (i.e.
1  0 ) The condition of dominance reduces to
0<k<(1-α) (p-J-1)
REFERENCES
(4.11)
For the non systematic distribution, if m and
1
have opposite signs i.e m is positive for
negatively skewed distributions and is negative
for positively skewed distributions, the condition
(4.10) is satisfied as long as (4.11) holds true.
The dominance conditions obtained in (3.20) and
(4.1) when compared, suggest that dominance
region is expanded when compared using Pitman
closeness criterion.
annals of statistics, Vol. 17(3), 13751389.
6.
Shalabh, Pitman closeness comparison
of least squares and Stein rule
estimators in linear regression Models
with Non normal Disturbances,
American Journal of Mathematical and
Management Sciences, Vol 21(182)
(2001) 89-100.
7.
Shalabh and Chaturvedi, Risk and
Pitman closeness properties of feasible
generated double K-class stimators in
linear regression models with non
spherical disturbances under balanced
loss function, Journal of muttivariate
Analysis, 90 (2004) 229-256.
8.
Srivastava, V.K. and Srivastava, A.K.,
Improved estimation of coefficients in
regression models with incomplete prior
information , Biometrical Journal, 25, 8
(1983), 775-782.
9.
Van Hao, T. and Chaturvedi, A.
Performance of 2SHI estimator under
the generalized Pitman nearness
criterion,
Working
Paper
99-4,
Department of Enomics, University of
Wollong ong Australia. (1999).
10. Wan, A.T.K., Risk comparison of the
inequality constrained least squares and
other related Estimators Under balanced
loss , Economics Letters, 46, (1994)
203-210.
11. Zellner, A., Bayesian and non Bayesian
using balanced loss functions, in
Statistical Decision Theory and Related
Topics, V. ecls, S.S. Gupta and J.O.
Berger, (Springer- Verlag ), (1994)
377-390.