Docs: Profiler Serialization by micdavis · Pull Request #928 · capitalone/DataProfiler

micdavis · 2023-06-28T21:20:36Z

updated the save and load section of the profiler.rst to reflect the new json serialization

taylorfturner · 2023-06-28T21:22:50Z

docs/source/profiler.rst

+* **pickle save/load**
+    * save a profile as a pkl file.
+    * load a pkl file as a profile object.
+
+* **json save/load**
+    * save a profile as a human-readable json file.
+    * load a json file as a profile object.


Suggested change

* **pickle save/load**

* save a profile as a pkl file.

* load a pkl file as a profile object.

* **json save/load**

* save a profile as a human-readable json file.

* load a json file as a profile object.

* **Pickle save/load**

* Save a profile as a `.pkl` file.

* Load a `.pkl` file as a profile object.

* **Json save/load**

* Save a profile as a human-readable json file.

* Load a json file as a profile object.

taylorfturner · 2023-06-28T21:23:14Z

docs/source/profiler.rst

+    * save a profile as a human-readable json file.
+    * load a json file as a profile object.
+
+**NOTE: json saving and loading isn't supported for unstructured profiles.**


Suggested change

**NOTE: json saving and loading isn't supported for unstructured profiles.**

**NOTE: Unstructured profiles do not support JSON saving and loading.**

taylorfturner · 2023-06-28T21:24:02Z

docs/source/profiler.rst

+
+**NOTE: json saving and loading isn't supported for unstructured profiles.**
+
 .. code-block:: python


I'd recommend another code-block too for JSON save / load example too ... could just write up an example since we know top-level API

taylorfturner · 2023-06-28T22:00:24Z

docs/source/profiler.rst

+
+      * unique_count - (UniqueCountOptions) Option to enable/disable unique row count calculations
+
+        * is_enabled - (Bool) Enables or disables options for unique row count
+        * hashing_method - (String) Property to specify row hashing method ("full" | "hll")
+        * hll - (HyperLogLogOptions) Options for alternative method of estimating unique row count (activated when `hll` is the selected hashing_method)
+
+          * seed - (Int) Used to set HLL hashing function seed
+          * register_count - (Int) Number of registers is equal to 2^register_count
+
+      * null_count - (Boolean) Option to enable/disable functionalities for row_has_null_ratio and row_is_null_ratio
    * **chi2_homogeneity** - Options for the chi-squared test matrix

      * is_enabled - (Boolean) Enables or disables performing chi-squared tests for homogeneity between the categorical columns of the dataset.
    * **null_replication_metrics** - Options for calculating null replication metrics

+      * is_enabled - (Boolean) Enables or disables calculation of null replication metrics
+  * **unstructured_options** - Options responsible for all unstructured data    * **chi2_homogeneity** - Options for the chi-squared test matrix
+
+      * is_enabled - (Boolean) Enables or disables performing chi-squared tests for homogeneity between the categorical columns of the dataset.
+    * **null_replication_metrics** - Options for calculating null replication metrics
+


this is okay to be in here ... I pushed this rendering fix to staging/dev-gh-pages/profile-serialization which is why it is not in feature/dev-gh-pages/profile-serialization

taylorfturner · 2023-06-28T22:01:54Z

docs/source/profiler.rst

+
+There are two save/load methods:
+
+* **Pickle save/load**


taylorfturner · 2023-06-28T22:02:01Z

docs/source/profiler.rst

+    print(json.dumps(loaded_pkl_profile.report(report_options={"output_format": "compact"}),
                                           indent=4))

+* **Json save/load**


docs/source/profiler.rst

Co-authored-by: Taylor Turner <taylorfturner@gmail.com>

docs/source/profiler.rst

taylorfturner · 2023-06-28T22:17:16Z

Pending #925

Co-authored-by: Taylor Turner <taylorfturner@gmail.com>

* added info to profile save and load func * addressed PR comments * fixed formatting * Update docs/source/profiler.rst Co-authored-by: Taylor Turner <taylorfturner@gmail.com> * Update docs/source/profiler.rst Co-authored-by: Taylor Turner <taylorfturner@gmail.com> --------- Co-authored-by: Taylor Turner <taylorfturner@gmail.com>

#937) * Docs: Profiler Serialization (#928) * added info to profile save and load func * addressed PR comments * fixed formatting * Update docs/source/profiler.rst Co-authored-by: Taylor Turner <taylorfturner@gmail.com> * Update docs/source/profiler.rst Co-authored-by: Taylor Turner <taylorfturner@gmail.com> --------- Co-authored-by: Taylor Turner <taylorfturner@gmail.com> * fixed formatting in docs (#936) --------- Co-authored-by: Taylor Turner <taylorfturner@gmail.com>

* documentation update for cms specific options to category (#917) * add cms specific options to category * address formatting requests * address formatting requests * Documentation: Update for Reservoir Sampling (#919) * update option for reservoir sampling * update for clarity rendering * Documentation: feature/options branch docs updates (#921) * reset ignore, update .gitignore, update documentation on presets (#874) * reset ignore * taylor's requested change * taylor's requested change * Fixed documentation for `sampling_ratio` option (#873) * Add documentation for `sampling_ratio` option * Update `sample_ratio` to `sampling_ratio` in documentations Co-authored-by: Taylor Turner <taylorfturner@gmail.com> * Update `sampling_ratio` documentation Co-authored-by: Taylor Turner <taylorfturner@gmail.com> * Update `sampling_ratio` documentation * Updated `sampling_ratio` documentation * Updated `sampling_ratio` documentation * Updated `sampling_ratio` documentation --------- Co-authored-by: Taylor Turner <taylorfturner@gmail.com> * update (#882) * Add documentation for `median_abs_deviation` option (#881) * Add documentation for `median_abs_deviation` option * Updated `median_abs_deviation` documentation * Row statistics option documentation (#883) * updated documentation on new row_statistic options * added documentation for row_statistics_options * fixed typing of hll and included the note that it activates when hll is chosen as the hashing method * removed space * fixed quotation mark * added micdavis comments * fixed doc descriptions for unique_count * small changes to docs --------- Co-authored-by: JGSweets <JGSweets@users.noreply.github.com> * rendering issue * rendering issue * update to fix rendering --------- Co-authored-by: Liz Smith <liz.smith@richmond.edu> Co-authored-by: clee1152 <72765198+clee1152@users.noreply.github.com> Co-authored-by: Richard Bann <87214439+drahc1R@users.noreply.github.com> Co-authored-by: JGSweets <JGSweets@users.noreply.github.com> * fix req tornado (#922) * Merge `staging/dev-gh-pages/profile-serialization` into `dev-gh-pages` (#937) * Docs: Profiler Serialization (#928) * added info to profile save and load func * addressed PR comments * fixed formatting * Update docs/source/profiler.rst Co-authored-by: Taylor Turner <taylorfturner@gmail.com> * Update docs/source/profiler.rst Co-authored-by: Taylor Turner <taylorfturner@gmail.com> --------- Co-authored-by: Taylor Turner <taylorfturner@gmail.com> * fixed formatting in docs (#936) --------- Co-authored-by: Taylor Turner <taylorfturner@gmail.com> * Staging update docs 0.10.0 (#945) * update or 0.10.0 docs version release * empty --------- Co-authored-by: Tyler <tfarnan@ucsd.edu> Co-authored-by: Liz Smith <liz.smith@richmond.edu> Co-authored-by: clee1152 <72765198+clee1152@users.noreply.github.com> Co-authored-by: Richard Bann <87214439+drahc1R@users.noreply.github.com> Co-authored-by: JGSweets <JGSweets@users.noreply.github.com> Co-authored-by: Michael Davis <36012613+micdavis@users.noreply.github.com>

added info to profile save and load func

af682fa

taylorfturner assigned micdavis Jun 28, 2023

taylorfturner added the Documentation Improvements or additions to documentation label Jun 28, 2023

taylorfturner enabled auto-merge (squash) June 28, 2023 21:21

taylorfturner reviewed Jun 28, 2023

View reviewed changes

addressed PR comments

040b6c3

taylorfturner reviewed Jun 28, 2023

View reviewed changes

taylorfturner previously approved these changes Jun 28, 2023

View reviewed changes

taylorfturner reviewed Jun 28, 2023

View reviewed changes

fixed formatting

a64a561

micdavis dismissed taylorfturner’s stale review via a64a561 June 28, 2023 22:06

taylorfturner suggested changes Jun 28, 2023

View reviewed changes

docs/source/profiler.rst Outdated Show resolved Hide resolved

Update docs/source/profiler.rst

c94749a

Co-authored-by: Taylor Turner <taylorfturner@gmail.com>

taylorfturner suggested changes Jun 28, 2023

View reviewed changes

docs/source/profiler.rst Outdated Show resolved Hide resolved

Update docs/source/profiler.rst

7a431f5

Co-authored-by: Taylor Turner <taylorfturner@gmail.com>

taylorfturner approved these changes Jun 28, 2023

View reviewed changes

taylorfturner closed this Jun 28, 2023

auto-merge was automatically disabled June 28, 2023 23:26
Pull request was closed

taylorfturner reopened this Jun 28, 2023

taylorfturner merged commit 2518dc5 into capitalone:feature/dev-gh-pages/profile-serialization Jun 28, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Docs: Profiler Serialization#928

Docs: Profiler Serialization#928
taylorfturner merged 5 commits intocapitalone:feature/dev-gh-pages/profile-serializationfrom
micdavis:profiler_serialization_docs

micdavis commented Jun 28, 2023

Uh oh!

taylorfturner Jun 28, 2023

Uh oh!

taylorfturner Jun 28, 2023

Uh oh!

taylorfturner Jun 28, 2023

Uh oh!

micdavis Jun 28, 2023

Uh oh!

taylorfturner Jun 28, 2023 •

edited

Loading

Uh oh!

taylorfturner Jun 28, 2023

Uh oh!

micdavis Jun 28, 2023

Uh oh!

taylorfturner Jun 28, 2023

Uh oh!

micdavis Jun 28, 2023

Uh oh!

Uh oh!

Uh oh!

taylorfturner commented Jun 28, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	NOTE: json saving and loading isn't supported for unstructured profiles.
	NOTE: Unstructured profiles do not support JSON saving and loading.


		NOTE: json saving and loading isn't supported for unstructured profiles.

		.. code-block:: python

Comments

Conversation

micdavis commented Jun 28, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

taylorfturner Jun 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

taylorfturner commented Jun 28, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

taylorfturner Jun 28, 2023 •

edited

Loading