├── LICENSE ├── Main.m ├── README.md ├── ionosphere.mat ├── jBHHO.m └── jFitnessFunction.m /LICENSE: -------------------------------------------------------------------------------- 1 | BSD 3-Clause License 2 | 3 | Copyright (c) 2020, Jingwei Too 4 | All rights reserved. 5 | 6 | Redistribution and use in source and binary forms, with or without 7 | modification, are permitted provided that the following conditions are met: 8 | 9 | 1. Redistributions of source code must retain the above copyright notice, this 10 | list of conditions and the following disclaimer. 11 | 12 | 2. Redistributions in binary form must reproduce the above copyright notice, 13 | this list of conditions and the following disclaimer in the documentation 14 | and/or other materials provided with the distribution. 15 | 16 | 3. Neither the name of the copyright holder nor the names of its 17 | contributors may be used to endorse or promote products derived from 18 | this software without specific prior written permission. 19 | 20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 21 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 22 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 23 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 24 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 25 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 26 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 27 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 28 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30 | -------------------------------------------------------------------------------- /Main.m: -------------------------------------------------------------------------------- 1 | %-------------------------------------------------------------------% 2 | % Binary Harris Hawk Optimization (BHHO) demo version % 3 | %-------------------------------------------------------------------% 4 | 5 | 6 | %---Inputs----------------------------------------------------------- 7 | % feat : feature vector (instances x features) 8 | % label : label vector (instance x 1) 9 | % N : Number of hawks 10 | % max_Iter : Maximum number of iterations 11 | 12 | %---Outputs---------------------------------------------------------- 13 | % sFeat : Selected features 14 | % Sf : Selected feature index 15 | % Nf : Number of selected features 16 | % curve : Convergence curve 17 | %-------------------------------------------------------------------- 18 | 19 | 20 | %% Binary Harris Hawk Optimization 21 | clc, clear, close; 22 | % Benchmark data set 23 | load ionosphere.mat; 24 | 25 | % Set 20% data as validation set 26 | ho = 0.2; 27 | % Hold-out method 28 | HO = cvpartition(label,'HoldOut',ho,'Stratify',false); 29 | 30 | % Parameter setting 31 | N = 10; 32 | max_Iter = 100; 33 | % Binary Harris Hawk Optimization 34 | [sFeat,Sf,Nf,curve] = jBHHO(feat,label,N,max_Iter,HO); 35 | 36 | % Plot convergence curve 37 | plot(1:max_Iter,curve); 38 | xlabel('Number of iterations'); 39 | ylabel('Fitness Value'); 40 | title('BHHO'); grid on; 41 | 42 | 43 | 44 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Binary Harris Hawk Optimization for Feature Selection 2 | 3 | [![View Binary Harris Hawk Optimization for Feature Selection on File Exchange](https://www.mathworks.com/matlabcentral/images/matlab-file-exchange.svg)](https://www.mathworks.com/matlabcentral/fileexchange/78534-binary-harris-hawk-optimization-for-feature-selection) 4 | [![License](https://img.shields.io/badge/license-BSD_3-yellow.svg)](https://github.com/JingweiToo/Binary-Harris-Hawk-Optimization-for-Feature-Selection/blob/master/LICENSE) 5 | [![GitHub release](https://img.shields.io/badge/release-1.3-green.svg)](https://github.com/JingweiToo/Binary-Harris-Hawk-Optimization-for-Feature-Selection) 6 | 7 | ![Wheel](https://www.mathworks.com/matlabcentral/mlc-downloads/downloads/7cf53ce5-1104-490a-97a7-b86aa2b1c17d/f5d1aef5-e893-4569-92ca-b269d967dd77/images/1595488674.JPG) 8 | 9 | 10 | ## Introduction 11 | * This toolbox offers Binary Harris Hawk Optimization ( BHHO ) 12 | * The `Main` file illustrates the example of how BHHO can solve the feature selection problem using benchmark data-set. 13 | 14 | ## Input 15 | * *`feat`* : feature vector ( Instances *x* Features ) 16 | * *`label`* : label vector ( Instances *x* 1 ) 17 | * *`N`* : number of hawks 18 | * *`max_Iter`* : maximum number of iterations 19 | 20 | 21 | ## Output 22 | * *`sFeat`* : selected features 23 | * *`Sf`* : selected feature index 24 | * *`Nf`* : number of selected features 25 | * *`curve`* : convergence curve 26 | 27 | 28 | ### Example 29 | ```code 30 | % Benchmark data set 31 | load ionosphere.mat; 32 | 33 | % Set 20% data as validation set 34 | ho = 0.2; 35 | % Hold-out method 36 | HO = cvpartition(label,'HoldOut',ho); 37 | 38 | % Parameter setting 39 | N = 10; 40 | max_Iter = 100; 41 | % Binary Harris Hawk Optimization 42 | [sFeat,Sf,Nf,curve] = jBHHO(feat,label,N,max_Iter,HO); 43 | 44 | % Plot convergence curve 45 | plot(1:max_Iter,curve); 46 | xlabel('Number of iterations'); 47 | ylabel('Fitness Value'); 48 | title('BHHO'); grid on; 49 | ``` 50 | 51 | ## Requirement 52 | * MATLAB 2014 or above 53 | * Statistics and Machine Learning Toolbox 54 | 55 | 56 | ## Cite As 57 | ```code 58 | @article{too2019new, 59 | title={A new quadratic binary harris hawk optimization for feature selection}, 60 | author={Too, Jingwei and Abdullah, Abdul Rahim and Mohd Saad, Norhashimah}, 61 | journal={Electronics}, 62 | volume={8}, 63 | number={10}, 64 | pages={1130}, 65 | year={2019}, 66 | publisher={Multidisciplinary Digital Publishing Institute} 67 | } 68 | ``` 69 | 70 | 71 | -------------------------------------------------------------------------------- /ionosphere.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/JingweiToo/Binary-Harris-Hawk-Optimization-for-Feature-Selection/6c20341ae62c926ae2b95ffe0a5c84ca95369feb/ionosphere.mat -------------------------------------------------------------------------------- /jBHHO.m: -------------------------------------------------------------------------------- 1 | function [sFeat,Sf,Nf,curve] = jBHHO(feat,label,N,max_Iter,HO) 2 | 3 | beta = 1.5; 4 | ub = 1; 5 | lb = 0; 6 | 7 | fun = @jFitnessFunction; 8 | dim = size(feat,2); 9 | X = zeros(N,dim); 10 | for i = 1:N 11 | for d = 1:dim 12 | if rand() > 0.5 13 | X(i,d) = 1; 14 | end 15 | end 16 | end 17 | fitR = inf; 18 | fit = zeros(1,N); 19 | Y = zeros(1,dim); 20 | Z = zeros(1,dim); 21 | 22 | curve = inf; 23 | t = 1; 24 | %---Iteration start------------------------------------------------- 25 | while t <= max_Iter 26 | for i = 1:N 27 | fit(i) = fun(feat,label,X(i,:),HO); 28 | if fit(i) < fitR 29 | fitR = fit(i); 30 | Xrb = X(i,:); 31 | end 32 | end 33 | X_mu = mean(X,1); 34 | for i = 1:N 35 | E0 = -1 + 2 * rand(); 36 | E = 2 * E0 * (1 - (t / max_Iter)); 37 | if abs(E) >= 1 38 | q = rand(); 39 | if q >= 0.5 40 | k = randi([1,N]); 41 | r1 = rand(); 42 | r2 = rand(); 43 | for d = 1:dim 44 | Xn = X(k,d) - r1 * abs(X(k,d) - 2 * r2 * X(i,d)); 45 | S = 1 / (1 + exp(-Xn)); 46 | if rand() < S 47 | X(i,d)= 1; 48 | else 49 | X(i,d) = 0; 50 | end 51 | end 52 | elseif q < 0.5 53 | r3 = rand(); 54 | r4 = rand(); 55 | for d = 1:dim 56 | Xn = (Xrb(d) - X_mu(d)) - r3 * (lb + r4 * (ub - lb)); 57 | S = 1 / (1 + exp(-Xn)); 58 | if rand() < S 59 | X(i,d) = 1; 60 | else 61 | X(i,d) = 0; 62 | end 63 | end 64 | end 65 | elseif abs(E) < 1 66 | J = 2 * (1 - rand()); 67 | r = rand(); 68 | if r >= 0.5 && abs(E) >= 0.5 69 | for d = 1:dim 70 | DX = Xrb(d) - X(i,d); 71 | Xn = DX - E * abs(J * Xrb(d) - X(i,d)); 72 | S = 1 / (1 + exp(-Xn)); 73 | if rand() < S 74 | X(i,d) = 1; 75 | else 76 | X(i,d) = 0; 77 | end 78 | end 79 | elseif r >= 0.5 && abs(E) < 0.5 80 | for d = 1:dim 81 | DX = Xrb(d) - X(i,d); 82 | Xn = Xrb(d) - E * abs(DX); 83 | S = 1 / (1 + exp(-Xn)); 84 | if rand() < S 85 | X(i,d) = 1; 86 | else 87 | X(i,d) = 0; 88 | end 89 | end 90 | elseif r < 0.5 && abs(E) >= 0.5 91 | LF = jLevyDistribution(beta,dim); 92 | for d = 1:dim 93 | Yn = Xrb(d) - E * abs(J * Xrb(d) - X(i,d)); 94 | S = 1 / (1 + exp(-Yn)); 95 | if rand() < S 96 | Y(d) = 1; 97 | else 98 | Y(d) = 0; 99 | end 100 | Zn = Y(d) + rand() * LF(d); 101 | S = 1 / (1 + exp(-Zn)); 102 | if rand() < S 103 | Z(d) = 1; 104 | else 105 | Z(d) = 0; 106 | end 107 | end 108 | fitY = fun(feat,label,Y,HO); 109 | fitZ = fun(feat,label,Z,HO); 110 | if fitY <= fit(i) 111 | fit(i) = fitY; 112 | X(i,:) = Y; 113 | end 114 | if fitZ <= fit(i) 115 | fit(i) = fitZ; 116 | X(i,:) = Z; 117 | end 118 | elseif r < 0.5 && abs(E) < 0.5 119 | LF = jLevyDistribution(beta,dim); 120 | for d = 1:dim 121 | Yn = Xrb(d) - E * abs(J * Xrb(d) - X_mu(d)); 122 | S = 1 / (1 + exp(-Yn)); 123 | if rand() < S 124 | Y(d) = 1; 125 | else 126 | Y(d) = 0; 127 | end 128 | Zn = Y(d) + rand() * LF(d); 129 | S = 1 / (1 + exp(-Zn)); 130 | if rand() < S 131 | Z(d) = 1; 132 | else 133 | Z(d) = 0; 134 | end 135 | end 136 | fitY = fun(feat,label,Y,HO); 137 | fitZ = fun(feat,label,Z,HO); 138 | if fitY <= fit(i) 139 | fit(i) = fitY; 140 | X(i,:) = Y; 141 | end 142 | if fitZ <= fit(i) 143 | fit(i) = fitZ; 144 | X(i,:) = Z; 145 | end 146 | end 147 | end 148 | end 149 | curve(t) = fitR; 150 | fprintf('\nIteration %d Best (BHHO)= %f',t,curve(t)) 151 | t = t + 1; 152 | end 153 | Pos = 1:dim; 154 | Sf = Pos(Xrb == 1); 155 | Nf = length(Sf); 156 | sFeat = feat(:,Sf); 157 | end 158 | 159 | 160 | function LF = jLevyDistribution(beta,dim) 161 | nume = gamma(1 + beta) * sin(pi * beta / 2); 162 | deno = gamma((1 + beta) / 2) * beta * 2 ^ ((beta - 1) / 2); 163 | sigma = (nume / deno) ^ (1 / beta); 164 | u = randn(1,dim) * sigma; 165 | v = randn(1,dim); 166 | step = u ./ abs(v) .^ (1 / beta); 167 | LF = 0.01 * step; 168 | end 169 | 170 | 171 | -------------------------------------------------------------------------------- /jFitnessFunction.m: -------------------------------------------------------------------------------- 1 | % Notation: This fitness function is for demonstration 2 | 3 | function cost = jFitnessFunction(feat,label,X,HO) 4 | if sum(X == 1) == 0 5 | cost = inf; 6 | else 7 | cost = jwrapperKNN(feat(:, X == 1),label,HO); 8 | end 9 | end 10 | 11 | 12 | function error = jwrapperKNN(sFeat,label,HO) 13 | %---// Parameter setting for k-value of KNN // 14 | k = 5; 15 | 16 | xtrain = sFeat(HO.training == 1,:); 17 | ytrain = label(HO.training == 1); 18 | xvalid = sFeat(HO.test == 1,:); 19 | yvalid = label(HO.test == 1); 20 | 21 | Model = fitcknn(xtrain,ytrain,'NumNeighbors',k); 22 | pred = predict(Model,xvalid); 23 | num_valid = length(yvalid); 24 | correct = 0; 25 | for i = 1:num_valid 26 | if isequal(yvalid(i),pred(i)) 27 | correct = correct + 1; 28 | end 29 | end 30 | Acc = correct / num_valid; 31 | error = 1 - Acc; 32 | end 33 | 34 | --------------------------------------------------------------------------------