Skip to content

Commit

Permalink
fix rendering problems
Browse files Browse the repository at this point in the history
  • Loading branch information
squito committed May 5, 2015
1 parent 5ae02ad commit 5e78b4f
Showing 1 changed file with 19 additions and 19 deletions.
38 changes: 19 additions & 19 deletions docs/monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,72 +179,72 @@ Note that the history server only displays completed Spark jobs. One way to sign
In addition to viewing the metrics in the UI, they are also available as JSON. This gives developers
an easy way to create new visualizations and monitoring tools for Spark. The JSON is available for
both running applications, an in the history server. The endpoints are mounted at `/json/v1`. Eg.,
for the history server, they would typically be accessible at `http://<server_url>:18080/json/v1`.
for the history server, they would typically be accessible at `http://<server-url>:18080/json/v1`.

<table class="table">
<tr><th>Endpoint</th><th>Meaning</th></tr>
<tr>
<td>`/applications`</td>
<td><code>/applications</code></td>
<td>A list of all applications</td>
</tr>
<tr>
<td>`/applications/<app_id>/jobs`</td>
<td><code>/applications/[app-id]/jobs</code></td>
<td>A list of all jobs for a given application</td>
</tr>
<tr>
<td>`/applications/<app_id>/jobs/<job_id>`</td>
<td><code>/applications/[app-id]/jobs/[job-id]</code></td>
<td>Details for one job</td>
</tr>
<tr>
<td>`/applications/<app_id>/stages`</td>
<td><code>/applications/[app-id]/stages</code></td>
<td>A list of all stages for a given application</td>
</tr>
<tr>
<td>`/applications/<app_id>/stages/<stage_id>`</td>
<td><code>/applications/[app-id]/stages/[stage-id]</code></td>
<td>A list of all attempts for a given stage</td>
</tr>
<tr>
<td>`/applications/<app_id>/stages/<stage_id>/<stage_attempt_id>`</td>
<td><code>/applications/[app-id]/stages/[stage-id]/[stage-attempt-id]</code></td>
<td>Details for the given stage attempt</td>
</tr>
<tr>
<td>`/applications/<app_id>/stages/<stage_id>/<stage_attempt_id>/taskSummary`</td>
<td><code>/applications/[app-id]/stages/[stage-id]/[stage-attempt-id]/taskSummary</code></td>
<td>Summary metrics of all tasks in a stage attempt</td>
</tr>
<tr>
<td>`/applications/<app_id>/stages/<stage_id>/<stage_attempt_id>/taskList`</td>
<td><code>/applications/[app-id]/stages/[stage-id]/[stage-attempt-id]/taskList</code></td>
<td>A list of all tasks for a given stage attempt</td>
</tr>
<tr>
<td>`/applications/<app_id>/executors`</td>
<td><code>/applications/[app-id]/executors</code></td>
<td>A list of all executors for the given application</td>
</tr>
<tr>
<td>`/applications/<app_id>/storage/rdd`</td>
<td><code>/applications/[app-id]/storage/rdd</code></td>
<td>A list of stored RDDs for the given application</td>
</tr>
<tr>
<td>`/applications/<app_id>/storage/rdd/<rdd_id>`</td>
<td><code>/applications/[app-id]/storage/rdd/[rdd-id]</code></td>
<td>Details for the storage status of a given RDD</td>
</tr>
</table>

When running on Yarn, each application has multiple attempts, so `<app_id>` is really a compound key,
with `<app_id>/<attempt_id>`.
When running on Yarn, each application has multiple attempts, so `[app-id]` is actually
`[app-id]/[attempt-id]` in all cases.

These endpoints have been strongly versioned to make it easier to develop applications on top of these
endpoints, without knowledge of spark internal classes. In particular, Spark guarantees:
These endpoints have been strongly versioned to make it easier to develop applications on top.
In particular, Spark guarantees:

* Endpoints will never be removed from one version
* Individual fields will never be removed for any given endpoint
* New endpoints may be added
* New fields may be added to existing endpoints
* New versions of the api may be added in the future, which are free to be completely incompatible with earlier versions
* New versions of the api may be added in the future at a separate endpoint (eg., `json/v2`). New versions are *not* required to be backwards compatible.
* Api versions may be dropped, but only after at least one minor release of existing beside a new api version

Note that even when examining the UI of a running applications, the `applications/<app_id>` portion is
Note that even when examining the UI of a running applications, the `applications/[app-id]` portion is
still required, though there is only one application available. Eg. to see the list of jobs for the
running app, you would go to `http://localhost:4040/json/v1/applications/<app_id>/jobs`. This is to
running app, you would go to `http://localhost:4040/json/v1/applications/[app-id]/jobs`. This is to
keep the paths consistent in both modes.

# Metrics
Expand Down

0 comments on commit 5e78b4f

Please sign in to comment.