-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Disable Variable Explorer auto-refresh feature by default #1958
Comments
From contrebasse on 2014-09-07T14:50:28Z Try to profile your code to see where exactly the computation is slow. You can use the profiler from spyder or line_profiler. I guess that the slow part is in Ptoid_Model. |
From lemoine....@gmail.com on 2014-09-08T09:54:38Z I profiled the code and the Ptoid_Model part did take up the most time (~95% of elapsed time taken up by running the model). I used both cProfile in Terminal and the spyder built-in profiler. Both returned roughly the same amount of elapsed time to run the code (~50 seconds for a reduced number of iterations). Ive tried running this code in the Spyder internal console, and it runs fine (~ 45 seconds). It also runs fine in the iPython console (~ 50 seconds). The same code in a regular" Python console/interpreter ran for about 15 - 20 minutes and didnt finish. So I guess the question is what is different about the Python interpreter in Spyder that it hangs on these kinds of problems, whereas both the internal console and iPython consoles work fine? |
From contrebasse on 2014-09-08T11:21:07Z Ptoid_Model uses which library ? What kind of computation does it do ? Sorry for my questions, but I think we need more indices. I run for loops daily in spyder without any problem ! The best would be to have a runnable minimum example of your problem. |
From lemoine....@gmail.com on 2014-09-08T11:27:58Z Sure! The Ptoid_Model is a recursive model that takes the previous time steps output and applies it to generate output at the current time step for 500 time steps (gen). Since some of the processes are stochastic, I do this for some number of simulations (nsim). Since Im interested in how the parameter mu affects model output, I repeat this simulations over a gradient of mu. The rest of the code (everything at the top) is simply setting up parameters to the right values, which are constant and never change in any run of the model. import numpy as np define the model functiondef Ptoid_Model(mu, N, P, Q):
def qConv(x): def pConv(x): lamb, delta = 20, 0.0015 def headToSearch(x): aP, aQ, ap, aq = headToSearch(P_head), headToSearch(Q_head), headToSearch(p_head), headToSearch(q_head) make a vector of muimm = range(1, 401, 2) Make a a vector the desired simulationsnsim = range(5) Number of generationsgen = 500 container to store the mu valuesmu_bif = [] set up containers to store the outputNbif = [] set initial conditionsNi, Pi, Qi = 100., 10., 10. for mu in imm: |
From ccordoba12 on 2014-09-10T18:35:40Z I think I found the problem. You need to go to
and deactivate the option called
This will avoid refreshing the Variable Explorer every 2 secs, which adds a lot of time to the simulation in the long run. The problem is you're creating very big lists, and building a view for them in such a short time is quite costly! :) For things to run even faster, you also need to not keep visible the Variable Explorer while the simulation is running because Spyder also refreshes it every time the main windows loses focus (e.g. when you move from Spyder to Chrome/Firefox and then back to Spyder again). For that you can close it or just give focus to another pane that is placed next to it. |
From lemoine....@gmail.com on 2014-09-10T19:12:13Z Yep that fixed it! Thanks! |
From contrebasse on 2014-09-14T12:25:13Z Should we disable the refresh when the corresponding console is busy ? It's not really useful to view variable contents while the script is still running (or is it ?). Maybe the correct thing to do is update only when a new prompt appears. |
From ccordoba12 on 2014-09-16T12:27:17Z Yeah, I think we can disable it. But we shouldn't remove the option because it can be useful for educational purposes. I think the variable explorer is refreshed automatically when the prompt comes back (at least that's what I've seen in our IPython consoles). Summary: Disable Variable Explorer auto-refresh feature by default (was: Spyder hangs in for loop (or takes a long time)) |
From ccordoba12 on 2014-09-21T10:37:13Z Labels: Easy |
From ccordoba12 on 2014-11-02T12:54:14Z This issue was closed by revision cfced284987e . Status: Fixed |
From lemoine....@gmail.com on 2014-09-06T14:29:24Z
Spyder Version: 2.2.5
Python Version: 2.7.5
Qt Version : 4.8.5, PyQt4 (API v2) 4.10.3 on Darwin
pyflakes >=0.5.0: 0.5.0 (OK)
pep8 >=0.6 : 1.4.6 (OK)
IPython >=0.13 : 1.1.0 (OK)
rope >=0.9.2 : 0.9.4 (OK)
sphinx >=0.6.6 : 1.1.3 (OK)
matplotlib >=1.0: 1.3.0 (OK)
sympy >=0.7.0 : 0.7.3 (OK)
pylint >=0.25 : 1.0.0 (OK)
I'm running a model with nested for loops, something that looks like this (for simplicity, I'm not showing the full code as it's not really important, I don't think):
make a vector of mu
imm = range(101)
Make a a vector the desired simulations
nsim = range(10)
Number of generations
gen = 500
container to store the mu values
mu_bif = []
set up containers to store the output
Nbif = []
Pbif = []
Qbif = []
set initial conditions
Ni, Pi, Qi = 100., 10., 10.
RUN MODEL
start = time.time()
for mu in imm:
for sim in nsim:
state = [Ni, Pi, Qi]
for i in range(gen):
state = Ptoid_Model(mu, state[0], state[1], state[2])
mu_bif.append(mu)
Nbif.append(state[0])
Pbif.append(state[1])
Qbif.append(state[2])
end = time.time()
print end - start
If I run this in Terminal, it only takes a minute or so. However, if I run this in Spyder's interactive interpreter, it can take anywhere between 10 - 20 minutes. If I increase nsim to range(100), Spyder can run for over 12 hours without finishing the task, whereas the task finishes in about 15 minutes.
Is there a reason the interactive interpreter in Spyder would be so much slower in for loops than running Python through the Terminal?
-Nate
Original issue: http://code.google.com/p/spyderlib/issues/detail?id=1958
The text was updated successfully, but these errors were encountered: