Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add expm operation #32

Open
seoanezonjic opened this issue Nov 29, 2019 · 7 comments
Open

Add expm operation #32

seoanezonjic opened this issue Nov 29, 2019 · 7 comments

Comments

@seoanezonjic
Copy link

Hi numo authors
I'm currently working with kernels and a couple of them needs the expm operation. in my project, I tried to implemented this operation using the Padé approximation using Nmatrix library. The implementation doesn't work (the error rate was huge) and several months after, I change to your library. Your library doesn't support the expm operation so I solve this problem using the Pycall library to use the expm operation from scipy.linalg. The problem is the huge consumption of memory and the time used copying the matrix ruby -> python -> ruby with very large matrix. Could you add the expm operation to your library?
I can give you as little reference my old code:
https://github.com/ElenaRojano/NetAnalyzer/blob/master/lib/NetAnalyzer/numo_expansion.rb
In line 79 is defined the expm operation, currently invoking python to solve it. But the commented code has a few implementations (with old Nmatrix code, but I thisk that you can guess easily the similar operations) that I tried to use with the cited resources. I think that the best reference is the scipy itself:
https://github.com/scipy/scipy/blob/master/scipy/sparse/linalg/matfuncs.py
The function is in the line 550.

Also, I would like to cite you in my papers. Do you have any citation that I could use?
Thank you very much in advance
Pedro Seoane

@yoshoku
Copy link
Contributor

yoshoku commented Dec 1, 2019

@masa16 I have implemented the expm method with the basic algorithm using the Padé approximation. Please review my pull request above.

@yoshoku
Copy link
Contributor

yoshoku commented Dec 2, 2019

@seoanezonjic Hi. The expm method has been added to the master branch. Please install and confirm that.

$ gem install specific_install
$ gem specific_install https://github.com/ruby-numo/numo-linalg.git master
Numo::Linalg.expm(a)

# The ord parameter value affects the approximation accuracy.
Numo::Linalg.expm(a, 20)

@seoanezonjic
Copy link
Author

Hi @yoshoku
Sorry for the delay in my answer. I have tested your exp method and it gives me a Segmentation fault. First, i show you the error log:

/mnt/home/soft/soft_bio_267/.gem/ruby/2.4.1/gems/numo-linalg-0.1.4/lib/numo/linalg/function.rb:42: [BUG] Segmentation fault at 0x00000000000000
ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0009 p:---- s:0089 e:000088 CFUNC  :dlange
c:0008 p:0065 s:0083 e:000082 METHOD /mnt/home/soft/soft_bio_267/.gem/ruby/2.4.1/gems/numo-linalg-0.1.4/lib/numo/linalg/function.rb:42
c:0007 p:0839 s:0076 e:000075 METHOD /mnt/home/soft/soft_bio_267/.gem/ruby/2.4.1/gems/numo-linalg-0.1.4/lib/numo/linalg/function.rb:825
c:0006 p:0059 s:0060 e:000059 METHOD /mnt/home/soft/soft_bio_267/.gem/ruby/2.4.1/gems/numo-linalg-0.1.4/lib/numo/linalg/function.rb:1181
c:0005 p:0500 s:0045 e:000044 METHOD /mnt/home/soft/soft_bio_267/.gem/ruby/2.4.1/gems/NetAnalyzer-0.4.9/lib/NetAnalyzer/network.rb:534
c:0004 p:0845 s:0021 E:0025b8 TOP    /mnt/home/soft/soft_bio_267/.gem/ruby/2.4.1/gems/NetAnalyzer-0.4.9/bin/NetAnalyzer.rb:174 [FINISH]
c:0003 p:---- s:0013 e:000012 CFUNC  :load
c:0002 p:0152 s:0008 E:000bf0 EVAL   /mnt/home/soft/soft_bio_267/.gem/ruby/2.4.1/bin/NetAnalyzer.rb:23 [FINISH]
c:0001 p:0000 s:0003 E:001e90 (none) [FINISH]

-- Ruby level backtrace information ----------------------------------------
/mnt/home/soft/soft_bio_267/.gem/ruby/2.4.1/bin/NetAnalyzer.rb:23:in `<main>'
/mnt/home/soft/soft_bio_267/.gem/ruby/2.4.1/bin/NetAnalyzer.rb:23:in `load'
/mnt/home/soft/soft_bio_267/.gem/ruby/2.4.1/gems/NetAnalyzer-0.4.9/bin/NetAnalyzer.rb:174:in `<top (required)>'
/mnt/home/soft/soft_bio_267/.gem/ruby/2.4.1/gems/NetAnalyzer-0.4.9/lib/NetAnalyzer/network.rb:534:in `get_kernel'
/mnt/home/soft/soft_bio_267/.gem/ruby/2.4.1/gems/numo-linalg-0.1.4/lib/numo/linalg/function.rb:1181:in `expm'
/mnt/home/soft/soft_bio_267/.gem/ruby/2.4.1/gems/numo-linalg-0.1.4/lib/numo/linalg/function.rb:825:in `norm'
/mnt/home/soft/soft_bio_267/.gem/ruby/2.4.1/gems/numo-linalg-0.1.4/lib/numo/linalg/function.rb:42:in `call'
/mnt/home/soft/soft_bio_267/.gem/ruby/2.4.1/gems/numo-linalg-0.1.4/lib/numo/linalg/function.rb:42:in `dlange'

-- Machine register context ------------------------------------------------
 RIP: 0x00002b36aa55ef62 RBP: 0x0000000000002710 RSP: 0x00007ffcf44c5bf0
 RAX: 0x0000000000000000 RBX: 0x00002b382ad88080 RCX: 0x0000000000004e20
 RDX: 0x0000000000000000 RDI: 0x0000000000000000 RSI: 0x0000000000013880
  R8: 0x0000000000013880  R9: 0x000000000000002f R10: 0x00000000f387dd79
 R11: 0x00002b36aa17e240 R12: 0x00007ffcf44c5c60 R13: 0x0000000000000000
 R14: 0x00007ffcf44c5cc8 R15: 0x00007ffcf44c5c68 EFL: 0x0000000000010202

-- C level backtrace information -------------------------------------------
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(rb_vm_bugreport+0x528) [0x2b36a7767048]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(rb_bug_context+0xd0) [0x2b36a75ea920]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(sigsegv+0x3e) [0x2b36a76dab8e]
[0x2b36a7a69c10]
[0x2b36aa55ef62]
[0x2b36ac3fccd9]
[0x2b36ac58c44b]
[0x2b36ac58c27d]
[0x2b36a956d1a9]
[0x2b36a8fff403]
[0x2b36a900191e]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(rb_ensure+0xc8) [0x2b36a75f36a8]
[0x2b36a9000fa4]
[0x2b36a9002dd7]
[0x2b36a956e967]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(vm_call_cfunc+0xf1) [0x2b36a7749841]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(vm_call_method+0xe3) [0x2b36a775a433]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(vm_call_opt_send+0x19a) [0x2b36a775a95a]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(vm_call_method+0xe3) [0x2b36a775a433]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(vm_exec_core+0x2179) [0x2b36a77538e9]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(vm_exec+0x87) [0x2b36a7757c77]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(rb_load_internal0+0xad) [0x2b36a763539d]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(rb_f_load+0x88) [0x2b36a7635b48]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(vm_call_cfunc+0xf1) [0x2b36a7749841]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(vm_call_method+0xe3) [0x2b36a775a433]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(vm_exec_core+0x2179) [0x2b36a77538e9]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(vm_exec+0x87) [0x2b36a7757c77]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(ruby_exec_internal+0xad) [0x2b36a75f049d]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(ruby_exec_node+0x1d) [0x2b36a75f223d]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/lib64/libruby.so.2.4(ruby_run_node+0x1c) [0x2b36a75f4e9c]
/mnt/home/soft/rvm/.rvm/rubies/ruby-2.4.1/bin/ruby(main+0x4b) [0x40096b] main.c:36

Two different chunks of code give me this error:

diagonal_matrix = matrix.sum(1).diag 	
matrix_L = diagonal_matrix - matrix
beta = 0.02
beta_product = matrix_L * -beta
matrix_result = Numo::Linalg.expm(beta_product, 12)
diagonal_matrix = matrix.sum(1).diag 	
matrix_L = diagonal_matrix - matrix
beta=0.04
id_mat = Numo::DFloat.eye(dimension_elements)
m_matrix = (id_mat * dimension_elements - diagonal_matrix + matrix ) * (beta/dimension_elements)
matrix_result = Numo::Linalg.expm(m_matrix, 12)

matrix has 10 000 x 10 000 random float elements ranged between 0 and 100 but it's a symmetric matrix. What is the problem tha causes this seg fault?
Thank you in advance
Pedro Seoane

@yoshoku
Copy link
Contributor

yoshoku commented Dec 5, 2019

@seoanezonjic I think that the segmentation fault error occurs depending on the execution environment. That error did not occur in my environment (OS: macOS 10.15.1, CPU: Dual-Core Intel Core m3, Memory: 8 GB). Perhaps the cause of the error is in the BLAS/LAPACK library.

I setup the environment using OpenBLAS for the BLAS/LAPACK library.

$ brew install openblas
$ rbenv install 2.4.1
$ rbenv global 2.4.1
$ gem install specific_install
$ gem specific_install https://github.com/ruby-numo/numo-linalg.git master

Then, I executed the following code.

require 'numo/linalg/autoloader'

puts 'generate symmetric matrix'
matrix = Numo::DFloat.new(10000, 10000).rand
matrix = 0.5 * (matrix + matrix.transpose)

puts 'calculate laplacian matrix'
diagonal_matrix = matrix.sum(1).diag
matrix_L = diagonal_matrix - matrix
beta = 0.02
beta_product = matrix_L * -beta

puts 'calculate matrix exponential'
matrix_result = Numo::Linalg.expm(beta_product, 12)
$ ruby foo.rb
generate symmetric matrix
calculate laplacian matrix
calculate matrix exponential
$

@seoanezonjic
Copy link
Author

Hi @yoshoku
Thank you by your quick check and the code. When I used your code, the seg fault arises again. i've reinstalled lapack and openmpi and the code works perfectly with small matrix. I'll launch the previous cited example and I'll comment hear the results when the executions are finished.
Thank you by your support
Pedro Seoane

@seoanezonjic
Copy link
Author

Hi @yoshoku
I've tested the expm method using the code that I've posted when the seg fault arises with n of 10000 and 15000. The execution were succes but the first used Pade orders give huge error when compared with the expm method of python. I've increased the orders and the second case that I've posted needed bigger increase. I use Pade 14 for the first case and 16 for the second case. There is a rule to estimate the aproximation order? Is very difficult to implement an automatic setting of this parameter?
Thank you very much by your attention and effort
Pedro Seoane

@yoshoku
Copy link
Contributor

yoshoku commented Dec 18, 2019

@seoanezonjic
It is difficult to implement an automatic parameter setting. Because the order parameter is affected by the matrix norm and the approximation error.

I implemented the expm method with reference to the book [1]. How to find the order q is described in the book.

Let q be the smallest nonnegative integer such that e(q, q) <= delta

The e(q, q) is the function defined as

e(q, q) = (2^(3 - 2 * q)) * (q!^2 / ((2 * q)! * (2 * q + 1)!))

The delta is an arbitrary constant with the following relationship

F=exp^(A+E),  ||E||_{max} <= delta ||A||_{max}.

If the matrix norm and the acceptable approximation error are clear, you may be able to find q. For reference, the e(q, q) function implementation is shown below.

def factorial(n)
 n.zero? ? 1 : n * factorial(n-1)
end

def eps(q)
  (2**(3 - 2 * q)) * (factorial(q)**2).fdiv(factorial(2 * q) * factorial(2 * q + 1))
end

puts eps(14) # 8.402086919997803e-47
puts eps(16) # 3.5687151066343315e-55

[1] Gene H. Golub and Charles F. Van Loan, "Matrix Computations 4th Edition: Algorithm 9.3.1," JHU Press, 2013.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants