Revise Code - Update NCCL Example (#22)

Author: yzygitzh

README.md

@@ -10,6 +10,10 @@ Below is an example of NPKit timeline result. Green blocks are LL128 data transf
 
 ![NPKit Result Example](./npkit_result_example.png)
 
+## Quick Start
+
+Please check `nccl_samples` for NCCL quick start, `rccl_samples` for RCCL quick start and `msccl_samples` for MSCCL quick start.
+
 ## Build
 
 NPKit is a patches series of some version of NCCL/RCCL/MSCCL. Users need to apply these patches to correct NCCL/RCCL/MSCCL version and build NCCL/RCCL/MSCCL with expected profiling events specified. In this section, we take NCCL 2.10.3-1, RCCL develop branch commit 4643a17 and MSCCL master branch commit e52c525 as examples. Assume we want to jointly profile LL128 data transfer time in GPU and net send/recv time in CPU:

msccl_samples/npkit_runner.sh

@@ -9,10 +9,9 @@ set -x
 #   <npkit_dump_dir> <npkit_result_dir>
 function msccl_test() {
   mpirun --allow-run-as-root \
-    -np 8 -host localhost:8 \
+    -map-by ppr:8:node --bind-to numa \
     -x LD_PRELOAD=$2/build/lib/libnccl.so.2:$LD_PRELOAD \
-    -x NCCL_DEBUG=INFO \
-    -x NCCL_DEBUG_SUBSYS=INIT,GRAPH \
+    -x NCCL_DEBUG=WARN \
     -x NCCL_ALGO=$4 \
     -x NCCL_PROTO=$5 \
     -x NPKIT_DUMP_DIR=$8 \

nccl_samples/README.md

@@ -0,0 +1,19 @@
+## Introduction
+
+This folder contains scripts for NPKit sample workflow for NCCL. The sample workflow first builds NCCL with NPKit enabled, then runs nccl-tests to collect NPKit event dump files, and finally generates NPKit trace file.
+
+## Dependencies
+
+[NCCL 2.17.1-1](https://github.com/nvidia/nccl/tree/v2.17.1-1) and [nccl-tests](https://github.com/nvidia/nccl-tests).
+
+## Usage
+
+1) Get NCCL version 2.17.1-1 and apply `npkit-for-nccl-2.17.1-1.diff` to the source repo.
+
+2) Make sure parameters in `npkit_launcher.sh` are valid. Also note that currently NPKit only supports collecting non-overlapped events in GPU, and `NPKIT_FLAGS` should follow this rule.
+
+3) Make sure `nccl_test` function in `npkit_runner.sh` is a valid command to run `nccl-tests` binary. Also note that currently NPKit only supports 1 GPU per process, so `-g 1` mode is required in `nccl-tests` commands.
+
+4) Run command `bash npkit_launcher.sh`.
+
+5) The generated trace file `npkit_event_trace.json` (zipped in `npkit_result.tar.gz`) is in [Google Trace Event Format](https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/preview) and can be viewed by trace viewers.