Skip to content

Commit 4dc11e6

Browse files
author
lebesgue
committed
add isfs
1 parent 35866b5 commit 4dc11e6

File tree

314 files changed

+46412
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

314 files changed

+46412
-0
lines changed

‎iSFS/Modules/Main.m

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
%% Validate new Model. No parallel setting in Matlab
2+
3+
clc, clear
4+
repN = 10;
5+
acc = zeros (repN, 1); auc = zeros (repN, 1);
6+
acc_imsf = zeros (repN, 1); auc_imsf = zeros (repN, 1);
7+
bslAcc = zeros (repN, 1);
8+
9+
10+
paras = logspace(-5, -1, 5);
11+
config = [];
12+
config.alpha = 'l1';
13+
config.beta = 'default';
14+
15+
doption = [];
16+
doption.clustering = true;
17+
doption.ratio = .5;
18+
19+
for rep = 1 : repN
20+
[trainX, testX, trainY, testY, ind, S, pf, hasPf] = processData ('ADNC', doption);
21+
22+
model = iSFSLS (trainX, trainY, ind, 0.01, config);
23+
[our_lab, ~] = predict ('RandomForest', trainX, trainY, testX, testY, model);
24+
[ROCX, ROCY, ~, auc(rep)] = perfcurve(testY, our_lab, 1);
25+
acc(rep) = sum(our_lab==testY) / length (our_lab);
26+
27+
fprintf ('Iteration %d\n', rep);
28+
fprintf ('iSFS: %d / %d = %.5f with %d features, auc=%.5f\n', sum(our_lab == testY), length (our_lab), acc(rep), sum(model.feaIdx), auc(rep));
29+
end
30+
31+
32+
33+

‎iSFS/README.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
iSFS
2+
============
3+
4+
*incomplete Source and Feature Selection (iSFS) model*
5+
6+
> iSFS is a feature learning model for multi-modal block-wise missing data.
7+
Linear models on both of the feature-level and source-level are learned simultaneously in a joint formulation.
8+
9+
## Dependency
10+
- [SLEP](http://www.public.asu.edu/~jye02/Software/SLEP/)
11+
- [Random Forest](http://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm)
12+
13+
Both of them are included in `Tools`
14+
15+
## Setup
16+
- Compile Random Forest and SLEP according to their manual
17+
- Initialize iSFS by running `Init.m` under `Tools/iMAD/Function`
18+
- Put your (block-wise missing) data under folder `Data/`
19+
- Implement you own `processData` under folder `Tools/iMAD/Function/processData.m`
20+
21+
## Usage
22+
Run `Main.m` under `Modules/`
23+
24+
25+

‎iSFS/Tools/SLEP_4.1/Examples/ExamplesReadme.txt

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
In this folder, we provide examples for all the main functions implemented in this package.
2+
3+
The folder "fusedLasso" contains functions for the fused Lasso regularized optimization.
4+
5+
The folder "invCov" contains the functions for sparse inverse covariance estimation.
6+
7+
The folder "L1" contains functions for L1 regularized/constrained optimization.
8+
9+
The folder "L1Lq" contains functions for L1Lq regularized/constrained optimization.
10+
11+
The folder "order" contains the functions for optimization with an ordered tree structure--nonnegative max-heap.
12+
13+
The folder "overlapping" contains the functions for the general overlapping group Lasso.
14+
15+
The folder "sgLasso" contains the functions for the sparse group Lasso penalized optimization problem.
16+
17+
The folder "traceNorm" contains the functions for the trace norm minimization.
18+
19+
The folder "tree" contains the functions for tree-structured sparse learning.

‎iSFS/Tools/SLEP_4.1/Examples/L1/example_LeastC.m

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
clear, clc;
2+
3+
% This is an example for running the function LeastC
4+
%
5+
% min 1/2 || A x - y||^2 + 1/2 * rsL2 * ||x||_2^2
6+
% s.t. ||x||_1 <= z
7+
%
8+
% For detailed description of the function, please refer to the Manual.
9+
%
10+
%% Related papers
11+
%
12+
% [1] Jun Liu and Jieping Ye, Efficient Euclidean Projections
13+
% in Linear Time, ICML 2009.
14+
%
15+
% [2] Jun Liu and Jieping Ye, Sparse Learning with Efficient Euclidean
16+
% Projections onto the L1 Ball, Technical Report ASU, 2008.
17+
%
18+
% [3] Jun Liu, Jianhui Chen, and Jieping Ye,
19+
% Large-Scale Sparse Logistic Regression, KDD, 2009.
20+
%
21+
%% ------------ History --------------------
22+
%
23+
% First version on August 10, 2009.
24+
%
25+
% September 5, 2009: adaptive line search is added
26+
%
27+
% For any problem, please contact Jun Liu (j.liu@asu.edu)
28+
29+
cd ..
30+
cd ..
31+
32+
root=cd;
33+
addpath(genpath([root '/SLEP']));
34+
% add the functions in the folder SLEP to the path
35+
36+
% change to the original folder
37+
cd Examples/L1;
38+
39+
m=1000; n=1000; % The data matrix is of size m x n
40+
41+
% for reproducibility
42+
randNum=1;
43+
44+
% ---------------------- generate random data ----------------------
45+
randn('state',(randNum-1)*3+1);
46+
A=randn(m,n); % the data matrix
47+
48+
randn('state',(randNum-1)*3+2);
49+
xOrin=randn(n,1);
50+
51+
randn('state',(randNum-1)*3+3);
52+
noise=randn(m,1);
53+
y=A*xOrin +...
54+
noise*0.01; % the response
55+
56+
z=100; % the radius of the L1 ball
57+
58+
%----------------------- Set optional items -----------------------
59+
opts=[];
60+
61+
% Starting point
62+
opts.init=2; % starting from a zero point
63+
64+
% Termination criterion
65+
opts.tFlag=5; % run .maxIter iterations
66+
opts.maxIter=100; % maximum number of iterations
67+
68+
% Mormalization
69+
opts.nFlag=0; % without normalization
70+
71+
%opts.rsL2=0.1; % the two norm regularization
72+
73+
%----------------------- Run the code LeastC -----------------------
74+
fprintf('\n lFlag=0 \n');
75+
opts.lFlag=0; % Nemirovski's line search
76+
tic;
77+
[x1, funVal1, ValueL1]= LeastC(A, y, z, opts);
78+
toc;
79+
80+
fprintf('\n lFlag=1 \n');
81+
opts.lFlag=1; % adaptive line search
82+
opts.tFlag=2; opts.tol= funVal1(end);
83+
tic;
84+
[x2, funVal2, ValueL2]= LeastC(A, y, z, opts);
85+
toc;
86+
87+
figure;
88+
plot(funVal1,'-r');
89+
hold on;
90+
plot(funVal2,'--b');
91+
legend('lFlag=0', 'lFlag=1');
92+
xlabel('Iteration (i)');
93+
ylabel('The objective function value');
94+
95+
% --------------------- compute the pathwise solutions ----------------
96+
opts.fName='LeastC'; % set the function name to 'LeastC'
97+
Z=[10, 100, 200, 500]; % set the parameters
98+
99+
% run the function pathSolutionLeast
100+
fprintf('\n Compute the pathwise solutions, please wait...');
101+
X=pathSolutionLeast(A, y, Z, opts);
102+

‎iSFS/Tools/SLEP_4.1/Examples/L1/example_LeastR.m

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
clear, clc;
2+
3+
% This is an example for running the function LeastR
4+
%
5+
% min 1/2 || A x - y||^2 + 1/2 * rsL2 * ||x||_2^2 + rho * ||x||_1
6+
%
7+
% For detailed description of the function, please refer to the Manual.
8+
%
9+
%% Related papers
10+
%
11+
% [1] Jun Liu and Jieping Ye, Efficient Euclidean Projections
12+
% in Linear Time, ICML 2009.
13+
%
14+
% [2] Jun Liu and Jieping Ye, Sparse Learning with Efficient Euclidean
15+
% Projections onto the L1 Ball, Technical Report ASU, 2008.
16+
%
17+
% [3] Jun Liu, Jianhui Chen, and Jieping Ye,
18+
% Large-Scale Sparse Logistic Regression, KDD, 2009.
19+
%
20+
%% ------------ History --------------------
21+
%
22+
% First version on August 10, 2009.
23+
%
24+
% September 5, 2009: adaptive line search is added
25+
%
26+
% For any problem, please contact Jun Liu (j.liu@asu.edu)
27+
28+
cd ..
29+
cd ..
30+
31+
root=cd;
32+
addpath(genpath([root '/SLEP']));
33+
% add the functions in the folder SLEP to the path
34+
35+
% change to the original folder
36+
cd Examples/L1;
37+
38+
m=1000; n=1000; % The data matrix is of size m x n
39+
40+
% for reproducibility
41+
randNum=1;
42+
43+
% ---------------------- Generate random data ----------------------
44+
randn('state',(randNum-1)*3+1);
45+
A=randn(m,n); % the data matrix
46+
47+
randn('state',(randNum-1)*3+2);
48+
xOrin=randn(n,1);
49+
50+
randn('state',(randNum-1)*3+3);
51+
noise=randn(m,1);
52+
y=A*xOrin +...
53+
noise*0.01; % the response
54+
55+
rho=0.2; % the regularization parameter
56+
% it is a ratio between (0,1), if .rFlag=1
57+
58+
%----------------------- Set optional items ------------------------
59+
opts=[];
60+
61+
% Starting point
62+
opts.init=2; % starting from a zero point
63+
64+
% termination criterion
65+
opts.tFlag=5; % run .maxIter iterations
66+
opts.maxIter=100; % maximum number of iterations
67+
68+
% normalization
69+
opts.nFlag=0; % without normalization
70+
71+
% regularization
72+
opts.rFlag=1; % the input parameter 'rho' is a ratio in (0, 1)
73+
%opts.rsL2=0.01; % the squared two norm term
74+
75+
%----------------------- Run the code LeastR -----------------------
76+
fprintf('\n mFlag=0, lFlag=0 \n');
77+
opts.mFlag=0; % treating it as compositive function
78+
opts.lFlag=0; % Nemirovski's line search
79+
tic;
80+
[x1, funVal1, ValueL1]= LeastR(A, y, rho, opts);
81+
toc;
82+
83+
opts.maxIter=1000;
84+
85+
fprintf('\n mFlag=1, lFlag=0 \n');
86+
opts.mFlag=1; % smooth reformulation
87+
opts.lFlag=0; % Nemirovski's line search
88+
opts.tFlag=2; opts.tol= funVal1(end);
89+
tic;
90+
[x2, funVal2, ValueL2]= LeastR(A, y, rho, opts);
91+
toc;
92+
93+
fprintf('\n mFlag=1, lFlag=1 \n');
94+
opts.mFlag=1; % smooth reformulation
95+
opts.lFlag=1; % adaptive line search
96+
opts.tFlag=2; opts.tol= funVal1(end);
97+
tic;
98+
[x3, funVal3, ValueL3]= LeastR(A, y, rho, opts);
99+
toc;
100+
101+
figure;
102+
plot(funVal1,'-r');
103+
hold on;
104+
plot(funVal2,'--b');
105+
hold on;
106+
plot(funVal3,':g');
107+
legend('mFlag=0, lFlag=0', 'mFlag=1, lFlag=0', 'mFlag=1, lFlag=1');
108+
xlabel('Iteration (i)');
109+
ylabel('The objective function value');
110+
111+
% % --------------------- compute the pathwise solutions ----------------
112+
opts.fName='LeastR'; % set the function name to 'LeastR'
113+
Z=[0.5, 0.2, 0.1, 0.01]; % set the parameters
114+
115+
% run the function pathSolutionLeast
116+
fprintf('\n Compute the pathwise solutions, please wait...');
117+
X=pathSolutionLeast(A, y, Z, opts);

0 commit comments

Comments
 (0)