Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc: redis memory management / automatic flushing. #2344

Merged
merged 4 commits into from
Jul 6, 2018

Conversation

concretevitamin
Copy link
Contributor

No description provided.

@@ -0,0 +1,103 @@
Redis Memory Management (EXPERIMENTAL)
==============================================
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the line of ==== should have the same length as the title (this applies in other places as well)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, fascinating...

Redis Memory Management (EXPERIMENTAL)
==============================================

Ray stores metadata associated to tasks and objects in one or more Redis
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

associated "with"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

==============================================

Ray stores metadata associated to tasks and objects in one or more Redis
servers, as described in :doc:`internals-overview`. Applications that are
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This link doesn't seem to work. Maybe just use an absolute path to the URL.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does work in html. It does not work in raw .rst I think. Also it's the first recommended method when I googled how to link to another doc/section. Lastly it's used in for instance https://github.com/ray-project/ray/blob/master/doc/source/tune.rst as well.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see anything like this in https://github.com/ray-project/ray/blob/master/doc/source/tune.rst. Can you link to the relevant line?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

First, follow `instructions to build Ray from source
<installation.html#building-ray-from-source>`__ to install prerequisites. After
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This link doesn't seem to work. Maybe just use an absolute path to the URL.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my other comment?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I see this pattern in the tune documentation.


.. code-block:: bash

env RAY_USE_NEW_GCS=on RAY_USE_XRAY=1 pip install -e . --verbose
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The env shouldn't be necessary, I think. Also, RAY_USE_XRAY=1 won't have any effect.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks!


git clone https://github.com/ray-project/ray.git
cd ray/python
pip install -e . --verbose # Add --user if you see a permission denied error.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why include this line? You don't actually want people to run this line, right? You want people to run the line below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, good call

export RAY_USE_XRAY=1
python my_ray_script.py # Or launch python/ipython.

Flush Policy
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flushing Policy

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reworded

--------------------------------------

There are three `user-configurable parameters
<https://github.com/ray-project/ray/blob/master/python/ray/experimental/gcs_flush_policy.py#L31>`_
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use a permanent link because this will break if we modify that file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6459/
Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6489/
Test PASSed.

======================================

Ray stores metadata associated with tasks and objects in one or more Redis
servers, as described in :doc:`internals-overview`. Applications that are
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I just built the documentation locally and this seems to work. However, it seems like we're using two different link styles to accomplish the same thing.

Here we have

:doc:`internals-overview`

and below we have

<installation.html#building-ray-from-source>`__

Why not just stick with one style? Or am I missing something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to the same style now

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6498/
Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/6507/
Test PASSed.

@pcmoritz pcmoritz merged commit 23a98a2 into ray-project:master Jul 6, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants