Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Design and develop a couple of shell scripts for GitHub repository traffic history more than 14 days in README.md #29

Closed
BradleyA opened this issue Jul 29, 2019 · 5 comments
Assignees

Comments

@BradleyA
Copy link
Owner

BradleyA commented Jul 29, 2019

Present GitHub repository traffic history more than 14 days in README

Unable to locate available solution that did NOT require logging into a web site with my GitHub account for a service or installing more tools or sharing my GitHub token with people I do not know or a service that limit number of repositories.

Design and develop a couple of shell scripts to do the following:

  • download views, clones, popular/referrers, and popular/paths weekly
  • parse data to create clone table and view table

Solution: github-repository-traffic

@BradleyA BradleyA self-assigned this Jul 29, 2019
@BradleyA
Copy link
Owner Author

https://developer.github.com/v3/
terminal:
Basic authentication
curl -u "BradleyA" https://api.github.com

In this example, the 'BradleyA' and 'Search-docker-registry-v2-script.1.0' values are provided for the :owner and :repo parameters in the path while :state is passed in the query string.
curl -i "https://api.github.com/repos/BradleyA/Search-docker-registry-v2-script.1.0/issues?state=closed"

Conditional requests
Most responses return an ETag header. Many responses also return a Last-Modified header. You can use the values of these headers to make subsequent requests to those resources using the If-None-Match and If-Modified-Since headers, respectively. If the resource has not changed, the server will return a 304 Not Modified.

Note: Making a conditional request and receiving a 304 response does not count against your Rate Limit, so we encourage you to use it whenever possible.

curl -i https://api.github.com/users/BradleyA

. . .
{
"login": "BradleyA",
"id": 12129046,
"node_id": "MDQ6VXNlcjEyMTI5MDQ2",
"avatar_url": "https://avatars3.githubusercontent.com/u/12129046?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/BradleyA",
"html_url": "https://github.com/BradleyA",
"followers_url": "https://api.github.com/users/BradleyA/followers",
"following_url": "https://api.github.com/users/BradleyA/following{/other_user}",
"gists_url": "https://api.github.com/users/BradleyA/gists{/gist_id}",
"starred_url": "https://api.github.com/users/BradleyA/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/BradleyA/subscriptions",
"organizations_url": "https://api.github.com/users/BradleyA/orgs",
"repos_url": "https://api.github.com/users/BradleyA/repos",
"events_url": "https://api.github.com/users/BradleyA/events{/privacy}",
"received_events_url": "https://api.github.com/users/BradleyA/received_events",
"type": "User",
"site_admin": false,
"name": "Bradley Allen",
"company": "@bradleyaustintx",
"blog": "",
"location": "Cedar Park, Texas",
"email": null,
"hireable": true,
"bio": "Bradley Allen has over 35 years of experience in global IT management, data center operations, software development, UNIX administration, project management ...",
"public_repos": 28,
"public_gists": 0,
"followers": 3,
"following": 0,
"created_at": "2015-04-27T00:30:56Z",
"updated_at": "2019-06-21T23:40:13Z"
}

==>

curl -u "BradleyA" -i https://api.github.com/repos/BradleyA/Search-docker-registry-v2-script.1.0/traffic/clones
Enter host password for user 'BradleyA':
HTTP/1.1 200 OK
Server: GitHub.com
Date: Mon, 29 Jul 2019 01:40:32 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 330
Status: 200 OK
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4999
X-RateLimit-Reset: 1564368032
Cache-Control: private, max-age=60, s-maxage=60
Vary: Accept, Authorization, Cookie, X-GitHub-OTP
ETag: "76f2af46340196485a6316f8aec60f51"
X-GitHub-Media-Type: github.v3; format=json
Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type
Access-Control-Allow-Origin: *
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Frame-Options: deny
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
Content-Security-Policy: default-src 'none'
X-GitHub-Request-Id: D140:386C:215F816:3E29A00:5D3E4E90

{
"count": 5,
"uniques": 4,
"clones": [
{
"timestamp": "2019-07-16T00:00:00Z",
"count": 1,
"uniques": 1
},
{
"timestamp": "2019-07-19T00:00:00Z",
"count": 1,
"uniques": 1
},
{
"timestamp": "2019-07-24T00:00:00Z",
"count": 3,
"uniques": 2
}
]
}

===>
curl -u "BradleyA" -i https://api.github.com/repos/BradleyA/Search-docker-registry-v2-script.1.0/traffic/views
Enter host password for user 'BradleyA':
HTTP/1.1 200 OK
Date: Mon, 29 Jul 2019 01:46:19 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 1365
Server: GitHub.com
Status: 200 OK
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4999
X-RateLimit-Reset: 1564368379
Cache-Control: private, max-age=60, s-maxage=60
Vary: Accept, Authorization, Cookie, X-GitHub-OTP
ETag: "dc942f625f325768c2ed8cbda4a322e4"
X-GitHub-Media-Type: github.v3; format=json
Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type
Access-Control-Allow-Origin: *
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Frame-Options: deny
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
Content-Security-Policy: default-src 'none'
Vary: Accept-Encoding
X-GitHub-Request-Id: E812:1B0E:1F622A:3189F3:5D3E4FEA

{
"count": 157,
"uniques": 74,
"views": [
{
"timestamp": "2019-07-15T00:00:00Z",
"count": 7,
"uniques": 4
},
{
"timestamp": "2019-07-16T00:00:00Z",
"count": 4,
"uniques": 3
},
{
"timestamp": "2019-07-17T00:00:00Z",
"count": 15,
"uniques": 8
},
{
"timestamp": "2019-07-18T00:00:00Z",
"count": 10,
"uniques": 4
},
{
"timestamp": "2019-07-19T00:00:00Z",
"count": 12,
"uniques": 6
},
{
"timestamp": "2019-07-20T00:00:00Z",
"count": 4,
"uniques": 3
},
{
"timestamp": "2019-07-21T00:00:00Z",
"count": 3,
"uniques": 2
},
{
"timestamp": "2019-07-22T00:00:00Z",
"count": 15,
"uniques": 7
},
{
"timestamp": "2019-07-23T00:00:00Z",
"count": 13,
"uniques": 5
},
{
"timestamp": "2019-07-24T00:00:00Z",
"count": 22,
"uniques": 14
},
{
"timestamp": "2019-07-25T00:00:00Z",
"count": 22,
"uniques": 8
},
{
"timestamp": "2019-07-26T00:00:00Z",
"count": 18,
"uniques": 10
},
{
"timestamp": "2019-07-27T00:00:00Z",
"count": 3,
"uniques": 1
},
{
"timestamp": "2019-07-28T00:00:00Z",
"count": 9,
"uniques": 2
}
]
}

==>

curl -u "BradleyA" -i https://api.github.com/repos/BradleyA/Search-docker-registry-v2-script.1.0/traffic/clones?per=week
Enter host password for user 'BradleyA':
HTTP/1.1 200 OK
Server: GitHub.com
Date: Mon, 29 Jul 2019 01:53:05 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 237
Status: 200 OK
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4995
X-RateLimit-Reset: 1564368785
Cache-Control: private, max-age=60, s-maxage=60
Vary: Accept, Authorization, Cookie, X-GitHub-OTP
ETag: "fc945a70ebebd04d124270253b312d06"
X-GitHub-Media-Type: github.v3; format=json
Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type
Access-Control-Allow-Origin: *
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Frame-Options: deny
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
Content-Security-Policy: default-src 'none'
X-GitHub-Request-Id: CD1C:3989:16B780A:2F81BED:5D3E5180

{
"count": 5,
"uniques": 4,
"clones": [
{
"timestamp": "2019-07-15T00:00:00Z",
"count": 2,
"uniques": 2
},
{
"timestamp": "2019-07-22T00:00:00Z",
"count": 3,
"uniques": 2
}
]
}

==>
curl -u "BradleyA" -i https://api.github.com/repos/BradleyA/Search-docker-registry-v2-script.1.0/traffic/popular/referrers
curl -u "BradleyA" -i https://api.github.com/repos/BradleyA/Search-docker-registry-v2-script.1.0/traffic/popular/referrers
Enter host password for user 'BradleyA':
HTTP/1.1 200 OK
Date: Mon, 29 Jul 2019 01:58:46 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 381
Server: GitHub.com
Status: 200 OK
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4993
X-RateLimit-Reset: 1564369126
Cache-Control: private, max-age=60, s-maxage=60
Vary: Accept, Authorization, Cookie, X-GitHub-OTP
ETag: "5ca10814b2153bfc54ac51dc4b1fc536"
X-GitHub-Media-Type: github.v3; format=json
Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type
Access-Control-Allow-Origin: *
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Frame-Options: deny
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
Content-Security-Policy: default-src 'none'
Vary: Accept-Encoding
X-GitHub-Request-Id: C8EE:098C:6C8F2:A398D:5D3E52D6

[
{
"referrer": "forums.docker.com",
"count": 79,
"uniques": 37
},
{
"referrer": "github.com",
"count": 8,
"uniques": 5
},
{
"referrer": "stackoverflow.com",
"count": 7,
"uniques": 3
},
{
"referrer": "Google",
"count": 3,
"uniques": 1
},
{
"referrer": "docs.feedzai.com",
"count": 1,
"uniques": 1
}
]

==>
curl -u "BradleyA" -i https://api.github.com/repos/BradleyA/Search-docker-registry-v2-script.1.0/traffic/popular/paths
Enter host password for user 'BradleyA':
HTTP/1.1 200 OK
Server: GitHub.com
Date: Mon, 29 Jul 2019 02:00:13 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 2069
Status: 200 OK
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4992
X-RateLimit-Reset: 1564369213
Cache-Control: private, max-age=60, s-maxage=60
Vary: Accept, Authorization, Cookie, X-GitHub-OTP
ETag: "823aec79db3312e97bfea8873168b05e"
X-GitHub-Media-Type: github.v3; format=json
Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type
Access-Control-Allow-Origin: *
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Frame-Options: deny
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
Content-Security-Policy: default-src 'none'
X-GitHub-Request-Id: D2A4:6AB8:17C33C1:316241C:5D3E532D

[
{
"path": "/BradleyA/Search-docker-registry-v2-script.1.0",
"title": "GitHub - BradleyA/Search-docker-registry-v2-script.1.0: view-private-registry...",
"count": 81,
"uniques": 63
},
{
"path": "/BradleyA/Search-docker-registry-v2-script.1.0/blob/master/view-private-registry",
"title": "Search-docker-registry-v2-script.1.0/view-private-registry at master · Bradle...",
"count": 29,
"uniques": 25
},
{
"path": "/BradleyA/Search-docker-registry-v2-script.1.0/blob/master/view-private-registry-remote",
"title": "Search-docker-registry-v2-script.1.0/view-private-registry-remote at master ·...",
"count": 26,
"uniques": 21
},
{
"path": "/BradleyA/Search-docker-registry-v2-script.1.0/issues/8",
"title": "docker registry setup and IP SANs · Issue BradleyA/Search-docker-registry-v2-script.1.0#8 · BradleyA/Search-docker-registr...",
"count": 3,
"uniques": 3
},
{
"path": "/BradleyA/Search-docker-registry-v2-script.1.0/tree/master/images",
"title": "Search-docker-registry-v2-script.1.0/images at master · BradleyA/Search-docke...",
"count": 3,
"uniques": 3
},
{
"path": "/BradleyA/Search-docker-registry-v2-script.1.0/pulse",
"title": "Pulse · BradleyA/Search-docker-registry-v2-script.1.0 · GitHub",
"count": 3,
"uniques": 2
},
{
"path": "/BradleyA/Search-docker-registry-v2-script.1.0/graphs/traffic",
"title": "Traffic · BradleyA/Search-docker-registry-v2-script.1.0",
"count": 2,
"uniques": 1
},
{
"path": "/BradleyA/Search-docker-registry-v2-script.1.0/blob/master/README.md",
"title": "Search-docker-registry-v2-script.1.0/README.md at master · BradleyA/Search-do...",
"count": 1,
"uniques": 1
},
{
"path": "/BradleyA/Search-docker-registry-v2-script.1.0/issues",
"title": "Issues · BradleyA/Search-docker-registry-v2-script.1.0",
"count": 1,
"uniques": 1
},
{
"path": "/BradleyA/Search-docker-registry-v2-script.1.0/search",
"title": "Search · GET /BradleyA/repos",
"count": 1,
"uniques": 1
}
]

browser:
https://api.github.com/repos/BradleyA/Search-docker-registry-v2-script.1.0/releases

@BradleyA BradleyA changed the title add repository access data to README Include repository access data to README Jul 29, 2019
@BradleyA BradleyA changed the title Include repository access data to README Include repository access data in README Jul 29, 2019
@BradleyA BradleyA transferred this issue from BradleyA/Search-docker-registry-v2-script.1.0 Jul 30, 2019
BradleyA added a commit that referenced this issue Jul 30, 2019
BradleyA added a commit that referenced this issue Jul 30, 2019
@BradleyA
Copy link
Owner Author

BradleyA commented Aug 4, 2019

display Clones and Views with 👍

<img alt="Clones" src="https://img.shields.io/static/v1?label=CLones&message=26,24,1&color=blue"> <img alt="Views" src="https://img.shields.io/static/v1?label=Views&message=26,24,1&color=blue">

@BradleyA
Copy link
Owner Author

BradleyA commented Aug 9, 2019

#       column -t -s' ' filename
#       soffice --convert-to png ./clones
#       display ./clones.png
#
#       <img alt="Steam Views" src="https://img.shields.io/steam/views/100">
#
#       cd ~/github/BradleyA/automate/:repo
#       git pull
#       git commit -m '$DATE: automation the update of README table' README.md
#       git push README.md
#

@BradleyA
Copy link
Owner Author

BradleyA commented Aug 9, 2019

github-repository/parse.repository.data.sh  2.98.370  2019-08-08T23:47:37.538761-05:00 (CDT)  https://github.com/BradleyA/Linux-admin  uadmin  two-rpi3b.cptx86.com 2.97  
    github-repository/parse.repository.data.sh design complete, ready to create test cases

@BradleyA BradleyA added additional testing Test security & policy compliance and evaluate whether the solution satisfies the requirements. enhancement Increase or improve in quality, performance, business value, function, or user satisfaction. needs investigation What? How? new command/service labels Aug 9, 2019
@BradleyA BradleyA removed additional testing Test security & policy compliance and evaluate whether the solution satisfies the requirements. enhancement Increase or improve in quality, performance, business value, function, or user satisfaction. needs investigation What? How? labels Feb 17, 2020
@BradleyA BradleyA changed the title Include repository access data in README Include GitHub repository traffic data in README Feb 17, 2020
@BradleyA BradleyA changed the title Include GitHub repository traffic data in README Present GitHub repositories traffic history more than 14 days README Feb 18, 2020
@BradleyA BradleyA changed the title Present GitHub repositories traffic history more than 14 days README Present GitHub repositories traffic history more than 14 days in README Feb 18, 2020
@BradleyA BradleyA changed the title Present GitHub repositories traffic history more than 14 days in README Present GitHub repositories traffic history more than 14 days in README.md Feb 18, 2020
@BradleyA BradleyA changed the title Present GitHub repositories traffic history more than 14 days in README.md Present GitHub repository traffic history more than 14 days in README.md Feb 18, 2020
BradleyA added a commit that referenced this issue Feb 18, 2020
@BradleyA BradleyA changed the title Present GitHub repository traffic history more than 14 days in README.md Design and develop a couple of shell scripts for GitHub repository traffic history more than 14 days in README.md Feb 19, 2020
@BradleyA
Copy link
Owner Author

Solution: github-repository-traffic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant