Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
9ffd55a
Experiments with orca and io module
jonmmease Aug 11, 2018
de10887
Added simple pyqt5 window backend
jonmmease Aug 11, 2018
38127fa
Added OrcaStatus singleton to hold status of orca server process
jonmmease Aug 11, 2018
fd32d64
More progress on the orca module.
jonmmease Aug 12, 2018
49130b1
Added update method to orca config object
jonmmease Aug 12, 2018
264d19b
Remove unused import
jonmmease Aug 12, 2018
9b6f8c8
Remove `autostart` and `hostname` configuration parameters
jonmmease Aug 13, 2018
8d31c9b
Docstrings and cleanup
jonmmease Aug 13, 2018
341d145
Kill the orca server process using psutil
jonmmease Aug 15, 2018
af10ff6
Don't auto import any of the show/backend logic
jonmmease Aug 15, 2018
eb21a70
auto import the plotly.io module
jonmmease Aug 15, 2018
0183d43
Capture server process output.
jonmmease Aug 15, 2018
f75652c
Merge branch 'master' into orca_integration
jonmmease Aug 18, 2018
fd3e65c
Initial Python 2 support
jonmmease Aug 18, 2018
15e62a0
Run orca auto-shutdown timer as daemon thread
jonmmease Aug 18, 2018
2d5a074
Added to_image tests against reference images
jonmmease Aug 18, 2018
2df38cc
Added image generation tests, converted orca server tests to use pytest
jonmmease Aug 18, 2018
68e6465
Point orca to packaged (offline) plotly.js and topojson and (online) …
jonmmease Aug 18, 2018
0d1e0eb
Add orca tests to 3.6 optional test suite
jonmmease Aug 19, 2018
f81e63c
Cannot install orca globally
jonmmease Aug 19, 2018
6f78063
Adds psutil dependency and try to add orca to path before running tox
jonmmease Aug 19, 2018
fc35eff
Forgot && in tox command
jonmmease Aug 19, 2018
48d4de5
Try global install of orca in tox environment
jonmmease Aug 19, 2018
88b293a
Local install in tox
jonmmease Aug 19, 2018
86fc540
Tox whitelist export command
jonmmease Aug 19, 2018
702dd59
whitelist ls and echo
jonmmease Aug 19, 2018
196b58b
Set extend PATH in tox setenv block
jonmmease Aug 19, 2018
2f08714
If search for orca fails, try orca.js. Also display search path
jonmmease Aug 19, 2018
2217628
Changing course. Try creating a conda environment
jonmmease Aug 19, 2018
a6a255a
Attempt to cache miniconda directory
jonmmease Aug 19, 2018
e8bc4eb
Activate conda environment through separate command
jonmmease Aug 19, 2018
1207e71
Make plotlyjs bundle read only
jonmmease Aug 19, 2018
7d3e244
Echo path in command. Why isn't pytest on the path?
jonmmease Aug 19, 2018
6fc596c
Remove debugging assertion
jonmmease Aug 19, 2018
bfd7ff8
Try to set environment variables in single block with tests
jonmmease Aug 19, 2018
b212564
Add nose and mock dependencies
jonmmease Aug 19, 2018
dc7cf36
disable warning in case this was causing exit code 2 failure
jonmmease Aug 19, 2018
192953b
Comment out optional tests for now.
jonmmease Aug 20, 2018
0a756f4
Don't shut down orca server during test image tests
jonmmease Aug 20, 2018
d29da80
Fix python specification (need ==)
jonmmease Aug 20, 2018
8ce2115
Removed dependency version constraints (something was forcing python …
jonmmease Aug 20, 2018
7a75478
FileNotFoundError not available on Python 2
jonmmease Aug 20, 2018
42be57e
Install all main repo dependencies in one command
jonmmease Aug 20, 2018
d213f86
Store failed image comparisons as artifacts
jonmmease Aug 20, 2018
3a73dfc
Remove geopandas for the moment, causing version conflict
jonmmease Aug 20, 2018
f9b0133
Update orca version constraints test
jonmmease Aug 20, 2018
62583c9
Try to make reference images more reproducible
jonmmease Aug 20, 2018
1830594
Try `python` as invalid executable for test
jonmmease Aug 20, 2018
4ce674d
Use separate reference images for Mac and Linux
jonmmease Aug 20, 2018
a50a5bf
Reinstate save/load settings with a few changes since last time:
jonmmease Aug 20, 2018
2c42fca
Don't run optional tests in orca workflow, just core and orca.
jonmmease Aug 20, 2018
37dcf76
Change cache key to include python version
jonmmease Aug 20, 2018
f30e247
Added back decorator dependency
jonmmease Aug 20, 2018
80f52c7
Added back pandas dependency
jonmmease Aug 20, 2018
6bfca05
Hide warnings from core tests
jonmmease Aug 20, 2018
c9d5832
Test orca on Python 2.7, 3.4, and 3.7
jonmmease Aug 20, 2018
79f89e1
Change 3.4 to 3.5 work around conda package dependency issues
jonmmease Aug 20, 2018
f44aea3
Fix workflow name
jonmmease Aug 20, 2018
32630a1
Single equals in conda python version specification
jonmmease Aug 20, 2018
aacc203
Reinstate all test workflows
jonmmease Aug 20, 2018
133cb24
Remove `orca` from the name of the functions in `plotly.io.orca`.
jonmmease Aug 22, 2018
1fa866b
Reorder properties in `config` and `status` repr to be a bit more log…
jonmmease Aug 22, 2018
cff8f1c
Merge branch 'master' into orca_integration
jonmmease Aug 22, 2018
9c70b3a
Add validate option to to_image and write_image
jonmmease Aug 22, 2018
8d97381
Added more helpful (at least friendlier) error message when plotly.py…
jonmmease Aug 23, 2018
c58d688
Add poppler dependency to circle conda environment
jonmmease Aug 23, 2018
8967568
Create failed directory and print failed image path
jonmmease Aug 23, 2018
c5dd2f3
Fix store_artifacts directory
jonmmease Aug 23, 2018
dab4e14
Update EPS images for CircleCI
jonmmease Aug 23, 2018
3137b0c
Re-enable all tests
jonmmease Aug 23, 2018
005299f
Extend time for orca startup to help CI robustness
jonmmease Aug 23, 2018
5f8b813
Remove explicit encoding when opening orca settings for Python 2 compat
jonmmease Aug 23, 2018
94c1bab
Fix executable version check error messages
jonmmease Aug 24, 2018
2b4fc0c
Make orca executable detection more specific.
jonmmease Aug 24, 2018
54d38d4
Python 2 test fix [ci skip]
jonmmease Aug 24, 2018
f38beb5
Use status.executable when building the orca command.
jonmmease Aug 24, 2018
e383c57
Use os.pathset so we don't mess up Windows paths
jonmmease Aug 24, 2018
88b37a0
Add helpful message when orca returns a 'plotly.js error' on mapbox
jonmmease Aug 25, 2018
0a0adac
Removed prototype plotly.io._show module.
jonmmease Aug 25, 2018
f37682f
revert tox changes (not using tox to test orca after all)
jonmmease Aug 25, 2018
d8958aa
Remove topojson from packagedata (not distributing it after all)
jonmmease Aug 25, 2018
25f1f81
Full PR review
jonmmease Aug 25, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Initial Python 2 support
  • Loading branch information
jonmmease committed Aug 18, 2018
commit fd3e65c5aad4700881c9018dca052d552c4c4e93
102 changes: 86 additions & 16 deletions plotly/io/_orca.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
import warnings
from copy import copy
from json import JSONDecodeError
from pprint import pformat
import requests
import subprocess
import socket
import json
import os
import sys
import threading
import retrying
import atexit
Expand Down Expand Up @@ -90,14 +90,81 @@ def _find_open_port():
int
An open port
"""
with socket.socket() as s:
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('', 0))
_, port = s.getsockname()
s = socket.socket()
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('', 0))
_, port = s.getsockname()
s.close()

return port


def which_py2(cmd, mode=os.F_OK | os.X_OK, path=None):
"""
Backport (unmodified) of shutil.which command from Python 3.6
Remove this when Python 2 support is dropped

Given a command, mode, and a PATH string, return the path which
conforms to the given mode on the PATH, or None if there is no such
file.

`mode` defaults to os.F_OK | os.X_OK. `path` defaults to the result
of os.environ.get("PATH"), or can be overridden with a custom search
path.
"""
# Check that a given file can be accessed with the correct mode.
# Additionally check that `file` is not a directory, as on Windows
# directories pass the os.access check.
def _access_check(fn, mode):
return (os.path.exists(fn) and os.access(fn, mode)
and not os.path.isdir(fn))

# If we're given a path with a directory part, look it up directly rather
# than referring to PATH directories. This includes checking relative to the
# current directory, e.g. ./script
if os.path.dirname(cmd):
if _access_check(cmd, mode):
return cmd
return None

if path is None:
path = os.environ.get("PATH", os.defpath)
if not path:
return None
path = path.split(os.pathsep)

if sys.platform == "win32":
# The current directory takes precedence on Windows.
if not os.curdir in path:
path.insert(0, os.curdir)

# PATHEXT is necessary to check on Windows.
pathext = os.environ.get("PATHEXT", "").split(os.pathsep)
# See if the given file matches any of the expected path extensions.
# This will allow us to short circuit when given "python.exe".
# If it does match, only test that one, otherwise we have to try
# others.
if any(cmd.lower().endswith(ext.lower()) for ext in pathext):
files = [cmd]
else:
files = [cmd + ext for ext in pathext]
else:
# On other platforms you don't have things like PATHEXT to tell you
# what file suffixes are executable, so just pass on cmd as-is.
files = [cmd]

seen = set()
for dir in path:
normdir = os.path.normcase(dir)
if not normdir in seen:
seen.add(normdir)
for thefile in files:
name = os.path.join(dir, thefile)
if _access_check(name, mode):
return name
return None


def which(cmd):
"""
Return the absolute path of the input executable string, based on the
Expand All @@ -117,9 +184,12 @@ def which(cmd):
the executable was not found.

"""
import shutil
# TODO: this doesn't exist on Python 2.7 :-(
return shutil.which(cmd)
if sys.version_info > (3, 0):
# Python 3 code in this block
import shutil
return shutil.which(cmd)
else:
return which_py2(cmd)


# Orca configuration class
Expand Down Expand Up @@ -183,7 +253,7 @@ def update(self, d={}, **kwargs):
"""
# Combine d and kwargs
if not isinstance(d, dict):
raise ValueError("""\
raise ValueError("""
The first argument to update must be a dict, \
but received value of type {typ}l
Received value: {val}""".format(typ=type(d), val=d))
Expand Down Expand Up @@ -218,7 +288,7 @@ def port(self):
@port.setter
def port(self, val):
if val is not None and not isinstance(val, int):
raise ValueError("""\
raise ValueError("""
The port value must be an integer, but received value of type {typ}.
Received value: {val}""".format(typ=type(val), val=val))

Expand Down Expand Up @@ -261,7 +331,7 @@ def executable(self, val):
# Validate val
# ------------
if not isinstance(val, str):
raise ValueError("""\
raise ValueError("""
The executable property must be a string, but received value of type {typ}.
Received value: {val}""".format(typ=type(val), val=val))
self._props['executable'] = val
Expand Down Expand Up @@ -423,7 +493,7 @@ def reload(self, warn=True):
# ### Parse as JSON ###
try:
orca_props = json.loads(orca_str)
except JSONDecodeError:
except ValueError:
if warn:
warnings.warn("""\
Orca configuration file at {path} is not valid JSON""".format(
Expand Down Expand Up @@ -625,7 +695,7 @@ def validate_orca_executable():
# Search for executable name or path in config.executable
executable = which(config.executable)
if executable is None:
raise ValueError("""\
raise ValueError("""
The orca executable is required in order to export figures as static images,
but it could not be found on the system path.

Expand All @@ -634,7 +704,7 @@ def validate_orca_executable():

# Run executable with --help and see if it's our orca
# ---------------------------------------------------
invalid_executable_msg = """\
invalid_executable_msg = """
The orca executable is required in order to export figures as static images,
but the executable that was found at '{executable}' does not seem to be a
valid plotly orca executable.
Expand All @@ -659,15 +729,15 @@ def validate_orca_executable():
try:
orca_version = subprocess.check_output([executable, '--version'])
except subprocess.CalledProcessError:
raise ValueError("""\
raise ValueError("""
An error occurred while trying to get the version of the orca executable.
Here is the command that plotly.py ran to request the version:

$ {executable} --version
""")

if not orca_version:
raise ValueError("""\
raise ValueError("""
No version was reported by the orca executable.

Here is the command that plotly.py ran to request the version:
Expand Down
17 changes: 17 additions & 0 deletions plotly/tests/test_optional/test_io/test_orca.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,29 @@ def ping_pongs(server_url):
class ValidateOrcaTest(TestCase):
def setUp(self):
pio.orca.reset_orca_status()
pio.orca.config.restore_defaults()

def test_validate_orca(self):
self.assertEqual(pio.orca.status.state, 'unvalidated')
pio.orca.validate_orca_executable()
self.assertEqual(pio.orca.status.state, 'validated')

def test_orca_not_found(self):
pio.orca.config.executable = 'bogus'
with self.assertRaises(ValueError) as err:
pio.orca.validate_orca_executable()

self.assertEqual(pio.orca.status.state, 'unvalidated')
self.assertIn('could not be found', err.exception.args[0])

def test_invalid_executable_found(self):
pio.orca.config.executable = 'cd'
with self.assertRaises(ValueError) as err:
pio.orca.validate_orca_executable()

self.assertEqual(pio.orca.status.state, 'unvalidated')
self.assertIn('executable that was found at', err.exception.args[0])

def test_orca_executable_path(self):
self.assertIsNone(pio.orca.status.executable)
if os.name == 'nt': # Windows
Expand Down