25/25 Investigations of Estimating Densities by Parzen Windows and k-Nearest-Neighbor J. R. Fairley Mississippi State University Electrical Engineering Department Mississippi State, MS 39762 Abstract The nonparametric methods presented are for estimating density functions when the underlying density function is unknown by means of Parzen Windows and k-nearestneighbor and also to classify a set of points belonging to one of the three categories in the training data set given samples of known features for the three classes. The use of the Parzen Window and k-nearest-neighbor techniques is to demonstrate the use of these techniques to attain a good representation of the density function in the training data for which an appropriate classifier can be determined. Each category contains three features to describe the category, and the use of the Parzen Window and k-nearest-neighbor techniques to classify a category is exercised for the three categories. 1. Introduction The Parzen Window approach uses a spherical Gaussian window function, φ(u), and hn = 1.0 for the training data in each of the three categories to classify a category correctly given a set of known points, and then it is repeated for hn = 0.1. This approach is used to estimate the density function, and then appropriately classify a category based on the results of the computed density given a set of points. The window functions do not have to be limited to the window function used in this work. The window function is just used to interpolate each sample contributing to the estimate with its distance from x. The decision regions for a Parzen-window classifier due depend on the choice of the window function. In this approach, the densities are estimated for each category and a test point is classified by the label corresponding to the maximum posterior. For the case when it is not readily apparent what the “best” window function is, the solution can be to let the cell volume be a function of the training data rather than some arbitrary function of the samples. In the work presented, the cell about x is grown until kn samples are captured within a volume Vn which is the nonparametric method knearest-neigbors of x. In this method, if the density is high around or near x, the cell will be small which leads to good resolution and vice versa. computed for k = 1, 3, and 5. Density estimates were 2. Brief Discussion of Theory In general, the Parzen Window method used to estimate the density is computed by the following equation given below p n x 1 n 1 x x i n i 1 Vn h n , where the volume Vn is of the form V n h nd , and d is the dimension of the hypercube and hn is the length of an edge of that hypercube. The window function φ(u) is a spherical Gaussian window function of the form given below (( x x i ) / h) exp[ (x x i )t (x x i ) /( 2h 2 )] . In general, the k-nearest-neighbor method used to estimate the density is computed by the following equation given below p n x kn /n V , n Where the volume Vn is of the form V n V1 / n , where V1 is determined by the nature of the data rather than by some arbitrary choice. 3. Analysis of Results The training data for the three categories that was used for investigations of the nonparametric methods, Parzen Windows and k-nearest neighbor, to estimate the densities are given below in table 1. Samples 1 2 3 4 5 6 7 8 9 10 ω1 ω2 Features of ω1 x1 x2 x3 0.28 1.31 -6.2 0.07 0.58 -0.78 1.54 2.01 -1.63 -0.44 1.18 -4.32 -0.81 0.21 5.73 1.52 3.16 2.77 2.20 2.42 -0.19 0.91 1.94 6.21 0.65 1.93 4.38 -0.26 0.82 -0.96 Features of ω2 x1 x2 x3 0.011 1.03 -0.21 1.27 1.28 0.08 0.13 3.12 0.16 -0.21 1.23 -0.11 -2.18 1.39 -0.19 0.34 1.96 -0.16 -1.38 0.94 0.45 -0.12 0.82 0.17 -1.44 2.31 0.14 0.26 1.94 0.08 ω3 x1 1.36 1.41 1.22 2.46 0.68 2.51 0.60 0.64 0.85 0.60 Features of ω3 x2 x3 2.17 0.14 1.45 -0.38 0.99 0.69 2.19 1.31 0.79 0.87 3.22 1.35 2.44 0.92 0.13 0.97 0.58 0.99 0.51 0.88 Table 1. Training Data The test points to classify using the Parzen Window method are shown below in table 2. Features Test Point 1 Test Point 2 Test Point 3 x1 0.50 0.31 -0.3 x2 1.0 1.51 0.44 x3 0.0 -0.50 -0.1 Table 2. Test Points used in the Parzen Window Classification Method Table 3 below shows the classification of an arbitrary test point x based on the Parzen window estimates for hn = 1. Test Point p(x) of ω1 p(x) of ω2 p(x) of ω3 1 2 3 0.1259 0.1534 0.1399 0.4711 0.4828 0.3783 0.3980 0.2260 0.1823 Table 3. Results from Classification of Test Points by Parzen Window Estimates Table 4 below shows the classification of an arbitrary test point x based on the Parzen window estimates for hn = 0.1. Test Point p(x) of ω1 p(x) of ω2 p(x) of ω3 1 2 3 8.775e-20 2.871e-20 3.637e-12 6.769e-5 1.200e-5 3.783e-4 8.000e-17 2.159e-25 1.064e-39 Table 4. Results from Classification of Test Points by Parzen Window Estimates Figure 1 below shows a plot of the density by the k-nearest neighbor method for k = 1 using the x1 values of category 3. Figure 1. Plot of Estimated Density for k = 1 using k-Nearest-Neighbor Method Figure 2 below shows a plot of the density by the k-nearest neighbor method for k = 3 using the x1 values of category 3. Figure 2. Plot of Estimated Density for k = 3 using k-Nearest-Neighbor Method Figure 3 below shows a plot of the density by the k-nearest neighbor method for k = 5 using the x1 values of category 3. Figure 3. Plot of Estimated Density for k = 5 using k-Nearest-Neighbor Method Figure 4 below shows a plot of the density by the k-nearest neighbor method for k = 1 using the x1 and x2 values of category 2. Figure 4. Plot of Estimated Density for k = 1 using k-Nearest-Neighbor Method for 2-Dimensions Figure 5 below shows a plot of the density by the k-nearest neighbor method for k = 3 using the x1 and x2 values of category 2. Figure 5. Plot of Estimated Density for k = 3 using k-Nearest-Neighbor Method for 2-Dimensions Figure 6 below shows a plot of the density by the k-nearest neighbor method for k = 5 using the x1 and x2 values of category 2. Figure 6. Plot of Estimated Density for k = 5 using k-Nearest-Neighbor Method for 2-Dimensions The test points to classify the three-dimensional data from three categories using the knearest-neighbor method are shown below in table 5. Features Test Point 1 Test Point 2 Test Point 3 x1 -0.41 0.14 -0.81 x2 0.82 0.72 0.61 x3 0.88 4.1 -0.38 Table 5. Test Points used in the k-Nearest-Neighbor Classification Method Table 6 below shows the classification of an arbitrary test point x based on the k-nearestneighbor estimates for k = 1. Test Point p(x) of ω1 p(x) of ω2 p(x) of ω3 1 2 3 0.0142 0.0312 0.1337 0.1673 0.0012 0.1022 0.0582 0.0023 0.0104 Table 6. Results from Classification of Test Points by k-Nearest-Neighbor Estimates for k = 1 Table 7 below shows the classification of an arbitrary test point x based on the k-nearestneighbor estimates for k = 3. Test Point p(x) of ω1 p(x) of ω2 p(x) of ω3 1 2 3 0.0065 0.0136 0.0083 0.1749 0.0031 0.2747 0.1133 0.0065 0.0267 Table 7. Results from Classification of Test Points by k-Nearest-Neighbor Estimates for k = 3 Table 8 below shows the classification of an arbitrary test point x based on the k-nearestneighbor estimates for k = 5. Test Point p(x) of ω1 p(x) of ω2 p(x) of ω3 1 2 3 0.0083 0.0032 0.0059 0.1052 0.0049 0.0943 0.0841 0.0082 0.0300 Table 8. Results from Classification of Test Points by k-Nearest-Neighbor Estimates for k = 5 4. Conclusion In conclusion, for the Parzen Window method, category 2 was classified solely for all three points using the three-dimensional data from the three categories for hn = 1 and 0.1. This is likely due to the random selection of points to classify as well as the use of the spherical Gaussian window function φ(u). If an alternative window function was used, or a different set of points to classify were used, the results most likely would have different. Overall, the Parzen Window method is an effective way to estimate the density function when the form of the function is unknown. It does pose some issues if the choice for the “best” window function is unknown or the sample size of the data set is limited or small. The use of the k-nearest-neighbor method classified categories in a seemingly random nature for k = 1, 3, and 5. This erratic behavior is most likely due to the size of the sample data set as well as the value for k since as k and n go to infinity this method will converge on the true density function of the data set. This technique did show its usefulness in the case data set where the form of the density function is unknown or not well mathematically understood in regards to one of the more general density forms (e.g. normal, uniform, exponential, etc.), and when the “best” window function for the Parzen Window method is difficult to ascertain. 5. MatLab Computer Code for the Parzen Window and k-NearestNeighbor %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Author: Josh Fairley % Course: ECE 8443 - Pattern Recognition % Assignment: Computer Exercise # 3 - Problems 2 and 3, Parzen Windows % and k-NN % Date: April 20, 2006 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% clear all; close all; clc; %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % X1w1 - X1 feature of W1 % X2w1 - X2 feature of W1 % X3w1 - X3 feature of W1 % % X1w2 - X1 feature of W2 % X2w2 - X2 feature of W2 % X3w2 - X3 feature of W2 % % X1w3 - X1 feature of W3 % X2w3 - X2 feature of W3 % X3w3 - X3 feature of W3 % % X_W1 - Matrix Construction for three-dimensional case of W1 % % X_W2 - Matrix Construction for three-dimensional case of W2 % % X_W3 - Matrix Construction for three-dimensional case of W3 % % XTP1 - First Test Point to Classify % XTP2 - Second Test Point to Classify % XTP3 - Third Test Point to Classify X1w1 = [0.28 0.07 1.54 -0.44 -0.81 1.52 2.20 0.91 0.65 -0.26]'; X2w1 = [1.31 0.58 2.01 1.18 0.21 3.16 2.42 1.94 1.93 0.82]'; X3w1 = [-6.2 -0.78 -1.63 -4.32 5.73 2.77 -0.19 6.21 4.38 -0.96]'; X1w2 = [0.011 1.27 0.13 -0.21 -2.18 0.34 -1.38 -0.12 -1.44 0.26]'; X2w2 = [1.03 1.28 3.12 1.23 1.39 1.96 0.94 0.82 2.31 1.94]'; X3w2 = [-0.21 0.08 0.16 -0.11 -0.19 -0.16 0.45 0.17 0.14 0.08]'; X1w3 = [1.36 1.41 1.22 2.46 0.68 2.51 0.60 0.64 0.85 0.66]'; X2w3 = [2.17 1.45 0.99 2.19 0.79 3.22 2.44 0.13 0.58 0.51]'; X3w3 = [0.14 -0.38 0.69 1.31 0.87 1.35 0.92 0.97 0.99 0.88]'; X_W1 = [X1w1 X2w1 X3w1]; X_W2 = [X1w2 X2w2 X3w2]; X_W3 = [X1w3 X2w3 X3w3]; XTP1a = [0.50 1.0 0]'; XTP2a = [0.31 1.51 -0.5]'; XTP3a = [-0.3 0.44 -0.1]'; XTPa = [XTP1a XTP2a XTP3a]; %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Parzen-window approach to estimate densities - Problem 2, Chapter 4 n = 3; s = 10; % Number of test points % Number of samples for each feature hn = 0.1; % Length of edge for the hypercube Vn = hn^3; % Volume of the hypercube for i = 1:s phi1(i) = (1/Vn)*exp(-((XTPa(:,1) - X_W1(i,:)')'*(XTPa(:,1) - X_W1(i,:)'))/(2*hn^2)); phi2(i) = (1/Vn)*exp(-((XTPa(:,2) - X_W1(i,:)')'*(XTPa(:,2) - X_W1(i,:)'))/(2*hn^2)); phi3(i) = (1/Vn)*exp(-((XTPa(:,3) - X_W1(i,:)')'*(XTPa(:,3) - X_W1(i,:)'))/(2*hn^2)); end pn1a = (1/s)*sum(phi1) pn1b = (1/s)*sum(phi2) pn1c = (1/s)*sum(phi3) for i = 1:s phi1(i) = (1/Vn)*exp(-((XTPa(:,1) - X_W2(i,:)')'*(XTPa(:,1) - X_W2(i,:)'))/(2*hn^2)); phi2(i) = (1/Vn)*exp(-((XTPa(:,2) - X_W2(i,:)')'*(XTPa(:,2) - X_W2(i,:)'))/(2*hn^2)); phi3(i) = (1/Vn)*exp(-((XTPa(:,3) - X_W2(i,:)')'*(XTPa(:,3) - X_W2(i,:)'))/(2*hn^2)); end pn2a = (1/s)*sum(phi1) pn2b = (1/s)*sum(phi2) pn2c = (1/s)*sum(phi3) for i = 1:s phi1(i) = (1/Vn)*exp(-((XTPa(:,1) - X_W3(i,:)')'*(XTPa(:,1) - X_W3(i,:)'))/(2*hn^2)); phi2(i) = (1/Vn)*exp(-((XTPa(:,2) - X_W3(i,:)')'*(XTPa(:,2) - X_W3(i,:)'))/(2*hn^2)); phi3(i) = (1/Vn)*exp(-((XTPa(:,3) - X_W3(i,:)')'*(XTPa(:,3) - X_W3(i,:)'))/(2*hn^2)); end pn3a = (1/s)*sum(phi1) pn3b = (1/s)*sum(phi2) pn3c = (1/s)*sum(phi3) % End of Parzen-window approach to estimate densities %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % k-nearest-neighbor density estimations - Problem 2, Chapter 4 % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Part A - Uses X1 feature of Category 3 for k = 1, 3, and 5 % a = min(X1w3) - 1; % b = max(X1w3) + 1; step_size = 0.1; % Start of boundary for density estimation % End of boundary for density estimation % Step size for going from a to b % number_of_iterations = round((b - a)/step_size); k1 = 1; k2 = 3; k3 = 5; % x = a; Y = zeros(10,1); % for i = 1:number_of_iterations % I am experimenting here in this step % for j = 1:s % Distance(j) = abs(x - X1w3(j)); % Computing Distance % end % Distance = sort(Distance); % Radius1 = Distance(k1); % Radius2 = Distance(k2); % Radius3 = Distance(k3); % V1a = pi*Radius1^2; % V1b = pi*Radius2^2; % V1c = pi*Radius3^2; % Vn1 = V1a/sqrt(s); % Vn2 = V1b/sqrt(s); % Vn3 = V1c/sqrt(s); % Px1(i) = (k1/s)/Vn1; % Px3(i) = (k2/s)/Vn2; % Px5(i) = (k3/s)/Vn3; % X(i) = x; % x = x + step_size; % end % % figure; % plot(X, Px1, 'b', X, Px3, 'r', X, Px5, 'c', X1w3, Y, 'k*'); % xlabel('X'); % ylabel('P(x)'); % title('Estimated Density Function P(x) vs. Training Data X'); % legend('k = 1', 'k = 3', 'k = 5', 'X1 of W3'); % grid on; % End of experimentation for i = 1:s for j = 1:s D = abs(X1w3(i) - X1w3(j)); if D > 0 Distance(j) = D; end end Distance = sort(Distance); Radius1 = Distance(k1); Radius2 = Distance(k2); Radius3 = Distance(k3); V1a = pi*Radius1^2; V1b = pi*Radius2^2; V1c = pi*Radius3^2; Vn1 = V1a/sqrt(s); Vn2 = V1b/sqrt(s); Vn3 = V1c/sqrt(s); P1(i) = (k1/s)/Vn1; P3(i) = (k2/s)/Vn2; P5(i) = (k3/s)/Vn3; end % Computing Distance P1 = P1'; P3 = P3'; P5 = P5'; Data = sortrows(cat(2,X1w3,P1,P3,P5), 1); x = Data(:,1); Px1 = Data(:,2); Px3 = Data(:,3); Px5 = Data(:,4); figure; plot(x, Px1, 'b', X1w3, Y, 'k*'); xlabel('X'); ylabel('P(x)'); title('Estimated Density Function P(x) vs. Training Data X'); legend('k = 1', 'X1 of W3'); grid on; figure; plot(x, Px3, 'r', X1w3, Y, 'k*'); xlabel('X'); ylabel('P(x)'); title('Estimated Density Function P(x) vs. Training Data X'); legend('k = 3', 'X1 of W3'); grid on; figure; plot(x, Px5, 'c', X1w3, Y, 'k*'); xlabel('X'); ylabel('P(x)'); title('Estimated Density Function P(x) vs. Training Data X'); legend('k = 5', 'X1 of W3'); grid on; % End of Part A %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Part B - Uses X1 and X2 features of W2 for k = 1, 3, and 5 x = [X1w2 X2w2]; a = min(x(:,1)) - 1; b = max(x(:,1)) + 1; c = min(x(:,2)) - 1; d = max(x(:,2)) + 1; number_of_iterations1 = round((b - a)/step_size)+ 1; number_of_iterations2 = round((d - c)/step_size)+ 1; for i = 1:number_of_iterations2 for j = 1:number_of_iterations1 X1_Position(i,j) = a + (j - 1) * step_size; % X1 position X2_Position(i,j) = c + (i - 1) * step_size; % X2 position for k = 1:s Distance(k)= sqrt((X2_Position(i,j) - x(k,2))^2 + (X1_Position(i,j) - x(k,1))^2); % Computing Distance end Distance = sort(Distance); Radius1 = Distance(k1); Radius2 = Distance(k2); Radius3 = Distance(k3); V1a = pi*Radius1^2; V1b = pi*Radius2^2; V1c = pi*Radius3^2; Vn1 = V1a/sqrt(s); Vn2 = V1b/sqrt(s); Vn3 = V1c/sqrt(s); Px(i,j,1) = (k1/s)/Vn1; Px(i,j,2) = (k2/s)/Vn2; Px(i,j,3) = (k3/s)/Vn3; end end figure; mesh(X1_Position, X2_Position, Px(:,:,1)); xlabel('X1'); ylabel('X2'); zlabel('P(x)'); title('k-Nearest-Neighbor Density Estimation - k = 1'); axis tight; figure; mesh(X1_Position, X2_Position, Px(:,:,2)); xlabel('X1'); ylabel('X2'); zlabel('P(x)'); title('k-Nearest-Neighbor Density Estimation - k = 3'); axis tight; figure; mesh(X1_Position, X2_Position, Px(:,:,3)); xlabel('X1'); ylabel('X2'); zlabel('P(x)'); title('k-Nearest-Neighbor Density Estimation - k = 5'); axis tight; % End of Part B %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Part C - Uses X1, X2, and X3 features of all categories for k = 1, 3, % and 5 and estimates the densities for three points. XTP1b = [-0.41 0.82 0.88]'; XTP2b = [0.14 0.72 4.10]'; XTP3b = [-0.81 0.61 -0.38]'; for i = 1:s Distance1(i)= sqrt((XTP1b(1) - X_W1(i,1))^2 + (XTP1b(2) - X_W1(i,2))^2 + (XTP1b(3) - X_W1(i,3))^2); % Computing Distance Distance2(i)= sqrt((XTP1b(1) - X_W2(i,1))^2 + (XTP1b(2) - X_W2(i,2))^2 + (XTP1b(3) - X_W2(i,3))^2); % Computing Distance Distance3(i)= sqrt((XTP1b(1) - X_W3(i,1))^2 + (XTP1b(2) - X_W3(i,2))^2 + (XTP1b(3) - X_W3(i,3))^2); % Computing Distance Distance4(i)= sqrt((XTP2b(1) - X_W1(i,1))^2 + (XTP2b(2) - X_W1(i,2))^2 + (XTP2b(3) - X_W1(i,3))^2); % Computing Distance Distance5(i)= sqrt((XTP2b(1) - X_W2(i,1))^2 + (XTP2b(2) - X_W2(i,2))^2 + (XTP2b(3) - X_W2(i,3))^2); % Computing Distance Distance6(i)= sqrt((XTP2b(1) - X_W3(i,1))^2 + (XTP2b(2) - X_W3(i,2))^2 + (XTP2b(3) - X_W3(i,3))^2); % Computing Distance Distance7(i)= sqrt((XTP3b(1) - X_W1(i,1))^2 + (XTP3b(2) - X_W1(i,2))^2 + (XTP3b(3) - X_W1(i,3))^2); % Computing Distance Distance8(i)= sqrt((XTP3b(1) - X_W2(i,1))^2 + (XTP3b(2) - X_W2(i,2))^2 + (XTP3b(3) - X_W2(i,3))^2); % Computing Distance Distance9(i)= sqrt((XTP3b(1) - X_W3(i,1))^2 + (XTP3b(2) - X_W3(i,2))^2 + (XTP3b(3) - X_W3(i,3))^2); % Computing Distance end Distance1 = sort(Distance1); Distance2 = sort(Distance2); Distance3 = sort(Distance3); Distance4 = sort(Distance4); Distance5 = sort(Distance5); Distance6 = sort(Distance6); Distance7 = sort(Distance7); Distance8 = sort(Distance8); Distance9 = sort(Distance9); R1 = Distance1(k1); R2 = Distance2(k1); R3 = Distance3(k1); R4 = Distance4(k1); R5 = Distance5(k1); R6 = Distance6(k1); R7 = Distance7(k1); R8 = Distance8(k1); R9 = Distance9(k1); R10 = Distance1(k2); R11 = Distance2(k2); R12 = Distance3(k2); R13 = Distance4(k2); R14 = Distance5(k2); R15 = Distance6(k2); R16 = Distance7(k2); R17 = Distance8(k2); R18 = Distance9(k2); R19 = Distance1(k3); R20 = Distance2(k3); R21 = Distance3(k3); R22 = Distance4(k3); R23 = Distance5(k3); R24 = Distance6(k3); R25 = Distance7(k3); R26 = Distance8(k3); R27 = Distance9(k3); Vn1 = ((4/3)*pi*R1^3)/sqrt(s); Vn2 = ((4/3)*pi*R2^3)/sqrt(s); Vn3 = ((4/3)*pi*R3^3)/sqrt(s); Vn4 = ((4/3)*pi*R4^3)/sqrt(s); Vn5 = ((4/3)*pi*R5^3)/sqrt(s); Vn6 = ((4/3)*pi*R6^3)/sqrt(s); Vn7 = ((4/3)*pi*R7^3)/sqrt(s); Vn8 = ((4/3)*pi*R8^3)/sqrt(s); Vn9 = ((4/3)*pi*R9^3)/sqrt(s); Vn10 = ((4/3)*pi*R10^3)/sqrt(s); Vn11 = ((4/3)*pi*R11^3)/sqrt(s); Vn12 = ((4/3)*pi*R12^3)/sqrt(s); Vn13 = ((4/3)*pi*R13^3)/sqrt(s); Vn14 = ((4/3)*pi*R14^3)/sqrt(s); Vn15 = ((4/3)*pi*R15^3)/sqrt(s); Vn16 = ((4/3)*pi*R16^3)/sqrt(s); Vn17 = ((4/3)*pi*R17^3)/sqrt(s); Vn18 = ((4/3)*pi*R18^3)/sqrt(s); Vn19 = ((4/3)*pi*R19^3)/sqrt(s); Vn20 = ((4/3)*pi*R20^3)/sqrt(s); Vn21 = ((4/3)*pi*R21^3)/sqrt(s); Vn22 = ((4/3)*pi*R22^3)/sqrt(s); Vn23 = ((4/3)*pi*R23^3)/sqrt(s); Vn24 = ((4/3)*pi*R24^3)/sqrt(s); Vn25 = ((4/3)*pi*R25^3)/sqrt(s); Vn26 = ((4/3)*pi*R26^3)/sqrt(s); Vn27 = ((4/3)*pi*R27^3)/sqrt(s); P1_1a = (k1/s)/Vn1; P1_2a = (k1/s)/Vn2; P1_3a = (k1/s)/Vn3; P1_1b = (k1/s)/Vn4; P1_2b = (k1/s)/Vn5; P1_3b = (k1/s)/Vn6; P1_1c = (k1/s)/Vn7; P1_2c = (k1/s)/Vn8; P1_3c = (k1/s)/Vn9; P3_1a = (k2/s)/Vn10; P3_2a = (k2/s)/Vn11; P3_3a = (k2/s)/Vn12; P3_1b = (k2/s)/Vn13; P3_2b = (k2/s)/Vn14; P3_3b = (k2/s)/Vn15; P3_1c = (k2/s)/Vn16; P3_2c = (k2/s)/Vn17; P3_3c = (k2/s)/Vn18; P5_1a = (k3/s)/Vn19; P5_2a = (k3/s)/Vn20; P5_3a = (k3/s)/Vn21; P5_1b = (k3/s)/Vn22; P5_2b = (k3/s)/Vn23; P5_3b = (k3/s)/Vn24; P5_1c = (k3/s)/Vn25; P5_2c = (k3/s)/Vn26; P5_3c = (k3/s)/Vn27; P1a = cat(2, P1_1a, P1_2a, P1_3a); P1b = cat(2, P1_1b, P1_2b, P1_3b); P1c = cat(2, P1_1c, P1_2c, P1_3c); P1 = cat(1,P1a,P1b,P1c) P3a = cat(2, P3_1a, P3_2a, P3_3a); P3b = cat(2, P3_1b, P3_2b, P3_3b); P3c = cat(2, P3_1c, P3_2c, P3_3c); P3 = cat(1,P3a,P3b,P3c) P5a = cat(2, P5_1a, P5_2a, P5_3a); P5b = cat(2, P5_1b, P5_2b, P5_3b); P5c = cat(2, P5_1c, P5_2c, P5_3c); P3 = cat(1,P5a,P5b,P5c) % End of Part C %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % End of k-nearest-neighbor density estimations %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%