[Documentation] sagemaker-debugger open source documentation pre-launch by mchoi8739 · Pull Request #506 · awslabs/sagemaker-debugger

mchoi8739 · 2021-06-23T23:32:29Z

Description of changes:

readthedocs build log: https://readthedocs.org/projects/sagemaker-debugger/builds/14082688/
pre-launched doc: https://sagemaker-debugger.readthedocs.io/en/website/

Style and formatting:

I have run pre-commit install to ensure that auto-formatting happens with every commit.

Issue number, if available

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

codecov-commenter · 2021-06-23T23:44:56Z

Codecov Report

Merging #506 (b170fd6) into master (d864bb4) will decrease coverage by 24.72%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master     #506       +/-   ##
===========================================
- Coverage   75.35%   50.62%   -24.73%     
===========================================
  Files         127      117       -10     
  Lines       11117    10590      -527     
===========================================
- Hits         8377     5361     -3016     
- Misses       2740     5229     +2489

Impacted Files	Coverage Δ
smdebug/analysis/utils.py	`23.52% <ø> (-50.99%)`	⬇️
smdebug/core/hook.py	`73.73% <ø> (-13.70%)`	⬇️
smdebug/exceptions.py	`47.61% <ø> (-16.67%)`	⬇️
...mdebug/profiler/analysis/notebook_utils/heatmap.py	`0.00% <ø> (-13.01%)`	⬇️
...debug/profiler/analysis/python_profile_analysis.py	`90.90% <ø> (ø)`
...profiler/analysis/utils/profiler_data_to_pandas.py	`36.07% <ø> (-0.92%)`	⬇️
...er/analysis/utils/python_profile_analysis_utils.py	`88.40% <ø> (ø)`
smdebug/rules/rule.py	`34.37% <ø> (-50.48%)`	⬇️
smdebug/rules/rule_invoker.py	`18.18% <ø> (ø)`
smdebug/trials/trial.py	`56.82% <ø> (-37.61%)`	⬇️
... and 79 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d864bb4...b170fd6. Read the comment docs.

NihalHarish · 2021-06-24T21:09:22Z

README.md


 | Framework | Version |
 | --- | --- |
-| [TensorFlow](docs/tensorflow.md) | 1.15, 2.1.0, 2.2.0, 2.3.0, 2.3.1 |


2.4 and 2.5 are also supported

incorporated

NihalHarish · 2021-06-24T21:10:57Z

README.md

-| [MXNet](docs/mxnet.md) | 1.6, 1.7 |
-| [PyTorch](docs/pytorch.md) | 1.4, 1.5, 1.6 |
-| [XGBoost](docs/xgboost.md) | 0.90-2, 1.0-1 ([As a built-in algorithm](docs/xgboost.md#use-xgboost-as-a-built-in-algorithm))|


Smdebug is supported on the latest versions of all available DLCs.

See page.

incorporated

NihalHarish · 2021-06-24T21:11:18Z

README.md

+| [TensorFlow](tensorflow.md) | 1.13, 1.14, 1.15, 2.1.0, 2.2.0, 2.3.0, 2.3.1 |
 | Keras (with TensorFlow backend) | 2.3 |
-| [MXNet](docs/mxnet.md) | 1.4, 1.5, 1.6, 1.7 |
-| [PyTorch](docs/pytorch.md) | 1.2, 1.3, 1.4, 1.5, 1.6 |
-| [XGBoost](docs/xgboost.md) |  0.90-2, 1.0-1 (As a framework)|
+| [MXNet](mxnet.md) | 1.4, 1.5, 1.6, 1.7 |
+| [PyTorch](pytorch.md) | 1.2, 1.3, 1.4, 1.5, 1.6 |
+| [XGBoost](xgboost.md) |  0.90-2, 1.0-1 (As a framework)|


See comment above.

incorporated

…into website

…ebsite

ndodda-amazon

Left mostly nits, otherwise looks good.

ndodda-amazon · 2021-07-30T18:43:50Z

.readthedocs.yml

+
+# Optionally set the version of Python and requirements required to build your docs
+python:
+  version: 3.6


nit: why Python 3.6? can we use Python 3.9?

ndodda-amazon · 2021-07-30T18:51:41Z

docs/README.rst

+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+You can use Debugger with your training script on your own container
+making only a minimal modification to your training script to add


nit: by making

ndodda-amazon · 2021-07-30T19:03:36Z

docs/README.rst

+Using SageMaker Debugger on custom containers
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Debugger is available for any deep learning models that you bring to


nit: "any deep learning model" or "all deep learning models"

ndodda-amazon · 2021-07-30T19:10:24Z

docs/collections.rst

+~~~~~~~~~~~~~~~~~~~~
+
+Below is a comprehensive list of the built-in collections that are
+managed by SageMaker Debugger. The Hook identifes the tensors that


nit: identifies

ndodda-amazon · 2021-07-30T19:12:46Z

docs/collections.rst

+``XGBoost``    METRICS
+============== ===========================
+
+If for some reason, you want to disable the saving of these collections,


We should tell customers to set debugger_hook_config=False in the estimator, this is a simpler alternative.

ndodda-amazon · 2021-07-30T19:16:46Z

docs/collections.rst

+       name="weights",
+       parameters={ "parameter": "value" })
+
+The parameters can be one of the following. The meaning of these


nit: The meaning of these parameters -> These parameters

ndodda-amazon · 2021-07-30T19:18:33Z

docs/environment.yml

+   - docutils==0.15.2
+   - bokeh
+   - ipython
+   - pandas


Should we pin the versions of bokeh, ipython, and pandas here?

ndodda-amazon · 2021-07-30T19:23:27Z

docs/hook-config-in-smestimator.rst

+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The available ``hook_parameters`` keys are listed in the following. The meaning
+of these parameters will be clear as you review the sections of


nit: The meaning of these parameters -> These parameters

ndodda-amazon · 2021-07-30T19:25:53Z

docs/hook-methods.rst

+.. method:: create_from_json_file(json_file_path (str)
+
+  Takes the path of a file which holds the json configuration of the hook,
+  and creates hook from that configuration. This is an optional parameter.


nit: creates a hook

leleamol · 2021-08-04T17:03:05Z

README.md

-### AWS training containers with script mode
+The following frameworks are available AWS Deep Learning Containers with
+the deep learning frameworks for the zero script change experience.



Are we explaining what is 'zero script change experience' in the doc? If yes, can we link it here?
In other locations, I am seeing the lines such as 'no changes to your training script'

leleamol · 2021-08-04T17:07:15Z

docs/hook-api.rst

+
+However, for some advanced use cases where you need access to customized
+tensors from targeted parts of a training script, you can manually
+construct the hook object. The SMDebug library provides hook classes to


Are we using the 'SMDebug' consistently? In other locations I am seeing it is mentioned as 'smdebug'

leleamol · 2021-08-04T17:52:46Z

docs/mxnet.rst

+Support
+-------
+
+-  Zero Script Change experience where you need no modifications to your


Same as above. If we are introducing a new term 'Zero script change experience', it needs to be explained some where.

leleamol · 2021-08-04T17:53:54Z

docs/release-notes.rst

+Migration to Deep Learning Containers
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+TBD


* add unified search to RTD website * configure rtd build environment to make it functional

* add licensing information * add licensing information

* add licensing information * add search filter

mchoi8739 and others added 30 commits August 9, 2020 23:47

update TF 2.2 smdebug features

aa789c7

add details

df74588

Update code samples/notes for new pySDK and smdebug/add and fix links

2fa0fdb

add 'New features' note

6857d6c

minor fix

8be632a

minor fix

d787f4b

fix formatting

6c00d2a

minor fix

4b6e0de

lint

54c12ce

lint

9e079dd

minor structure change

4afb5fc

minor fix

9c20ef2

minor fix

293f770

incorporate comments

4996feb

incorporate comments / lift limitation note

782e8c6

incorporate comments

aa7fcc5

include pypi links

83ad970

minor fix

3f2beff

incorporate comments

fd1b1c2

incorporate comments

463f0b4

incorporate comments

72e48df

version addition

557eae1

version addition

1eee9c6

Merge branch 'master' of https://github.com/awslabs/sagemaker-debugger

70a594b

add details

19754a1

add footnote

dd13c6c

Merge branch 'master' of https://github.com/awslabs/sagemaker-debugger

cefd9df

Merge branch 'master' of https://github.com/awslabs/sagemaker-debugger

f10a3a1

Merge branch 'master' of https://github.com/awslabs/sagemaker-debugger

fcc0236

Merge branch 'master' of https://github.com/awslabs/sagemaker-debugger

7778131

Miyoung Choi added 4 commits June 23, 2021 13:37

pre-launch smdebug readthedocs website

9edd714

add readthedocs yml

437d9d7

rm all warnings

84edbae

rm pip protobuf

c6a94ea

mchoi8739 changed the title ~~[Website] sagemaker-debugger open source documentation pre-launch~~ [Documentation] sagemaker-debugger open source documentation pre-launch Jun 23, 2021

NihalHarish suggested changes Jun 24, 2021

View reviewed changes

test: try /usr/local dir

5d32864

mchoi8739 force-pushed the website branch from e8c46d7 to 5d32864 Compare June 25, 2021 00:44

mchoi8739 added 6 commits June 29, 2021 13:41

Merge branch 'master' of https://github.com/awslabs/sagemaker-debugger …

b481d94

…into website

incorp comments

248de9e

Merge branch 'master' of https://github.com/awslabs/sagemaker-debugger …

f5051b5

…into website

Trigger Build for testing RTD PR builder

d11b76e

sync

2fe16db

rm smdebug from env.yml

d847ffc

mchoi8739 force-pushed the website branch from bfcb2bd to d847ffc Compare July 26, 2021 04:16

mchoi8739 added 4 commits July 25, 2021 21:20

Merge branch 'master' of github.com:awslabs/sagemaker-debugger into w…

90d84a5

…ebsite

Trigger Build

9e8fac5

Trigger Build

7bed697

Merge branch 'master' of github.com:awslabs/sagemaker-debugger into w…

98b8153

…ebsite

ndodda-amazon suggested changes Jul 30, 2021

View reviewed changes

leleamol reviewed Aug 4, 2021

View reviewed changes

mchoi8739 and others added 4 commits February 7, 2022 14:35

Merge branch 'master' into website

b170fd6

Add unified RTD search to RTD website (#610)

ccc802c

* add unified search to RTD website * configure rtd build environment to make it functional

add licensing information (#612)

4a2746e

* add licensing information * add licensing information

Add RTD Search Filters (#618)

9cb753d

* add licensing information * add search filter

Conversation

mchoi8739 commented Jun 23, 2021

Description of changes:

Style and formatting:

Issue number, if available

Uh oh!

codecov-commenter commented Jun 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ndodda-amazon left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

codecov-commenter commented Jun 23, 2021 •

edited

Loading