Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 0.6.0: Augur, Taylor's Version #2563

Merged
merged 109 commits into from
Oct 30, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
109 commits
Select commit Hold shift + click to select a range
9c2fc32
Split ENTRYPOINT script in 2
mscherer Aug 24, 2023
ed27186
derive new status from hook
IsaacMilarky Aug 30, 2023
ada66b1
derive additional conditions from the passed in hook
IsaacMilarky Aug 31, 2023
7a442a0
partial progress
IsaacMilarky Aug 31, 2023
9085c7a
progress
IsaacMilarky Sep 1, 2023
604b50f
rework collection to use AugurTaskRoutine to encapsulate collection i…
IsaacMilarky Sep 1, 2023
5246ea5
update dei.py
IsaacMilarky Sep 1, 2023
4953cad
refactor based on Andrew's suggestions
IsaacMilarky Sep 1, 2023
88140f8
pass in session variable to method that assembles repos for the Colle…
IsaacMilarky Sep 3, 2023
04a4123
wrong
IsaacMilarky Sep 3, 2023
18b2d6a
iron out syntax errors
IsaacMilarky Sep 3, 2023
22a52c2
comment
IsaacMilarky Sep 3, 2023
eadfeb9
merge with dev
IsaacMilarky Sep 5, 2023
be12072
Graql api for augur
ABrain7710 Sep 16, 2023
2fbe7db
Fix tables on user settings page clipping past end of column
Ulincsys Sep 21, 2023
bc23dfd
Add cursor pagination for repos
ABrain7710 Sep 21, 2023
cfdbc04
Add issue pagination and make generic method to handle pagination logic
ABrain7710 Sep 21, 2023
752eac5
Organize classes and make prs paginateable
ABrain7710 Sep 21, 2023
64891e7
Make issues and prs paginateable from a repo object
ABrain7710 Sep 21, 2023
577e1a7
Make commits paginateable, and add max request limit
ABrain7710 Sep 21, 2023
61585d1
Make contributors and messages paginateable
ABrain7710 Sep 21, 2023
e10a5e9
Make messages paginateable from repo object
ABrain7710 Sep 21, 2023
c477e37
Fix bug where fields on the repo object weren't filtered by repo_id
ABrain7710 Sep 21, 2023
ae257d2
Andrew's suggestions
IsaacMilarky Sep 21, 2023
dc741b0
Merge pull request #2529 from chaoss/dev
ABrain7710 Sep 21, 2023
07415aa
adding graphene library to setup.py
sgoggins Sep 21, 2023
b1fdf6f
adding flask_graphql to setup.py
sgoggins Sep 21, 2023
e25c105
adding graphene_sqlalchemy library to setup.py
sgoggins Sep 21, 2023
e4b5a8b
Fix unpacking tuple issue
ABrain7710 Sep 25, 2023
3b1d63d
make facade trim commits with a bulk sql transaction instead of one a…
IsaacMilarky Sep 25, 2023
f2e15e4
version update
sgoggins Sep 26, 2023
a3ba33d
_finally_ use bulk insert for commit data
IsaacMilarky Sep 28, 2023
4adb1be
Stagger insert
IsaacMilarky Oct 2, 2023
b5df6cf
refresh materialized view bug fix for explorer_contributor_actions
sgoggins Oct 5, 2023
511720c
Add sql to update closed issue cntrbs with issue events'
ABrain7710 Oct 5, 2023
50a17e3
Merge branch 'dev' into add-closed-issue-cntrb
ABrain7710 Oct 5, 2023
a2acba5
Make api key required for graphql api
ABrain7710 Oct 5, 2023
74915e2
no unique constraint because of 7c164351e0d8f2848170e5e108661d11ed858…
IsaacMilarky Oct 5, 2023
6267dbe
Merge pull request #2528 from chaoss/fix-table-clipping
Ulincsys Oct 5, 2023
d717e21
sqlalchemy bulk insert
IsaacMilarky Oct 5, 2023
d0e75ad
Merge pull request #2541 from chaoss/add-closed-issue-cntrb
sgoggins Oct 6, 2023
aea9a8f
Merge pull request #2543 from chaoss/dev
sgoggins Oct 6, 2023
5fe3c94
working facade bulk insert
IsaacMilarky Oct 10, 2023
70346cf
Merge branch 'dev' into bulk-facade
IsaacMilarky Oct 10, 2023
1a9fa13
typo
IsaacMilarky Oct 10, 2023
4772191
Merge branch 'dev' into detangle-collection-further
IsaacMilarky Oct 10, 2023
9a3735c
minor syntax
IsaacMilarky Oct 10, 2023
bb8bc14
Merge pull request #2495 from mscherer/fix_docker
sgoggins Oct 15, 2023
ccbab39
fixer
sgoggins Oct 15, 2023
716619e
Merge pull request #2547 from chaoss/dev
sgoggins Oct 15, 2023
e8591c2
Temporarily slimming tasks not related to PR reviews in order to acce…
sgoggins Oct 15, 2023
1c5855a
Merge pull request #2548 from chaoss/graphql-api-experiment
sgoggins Oct 15, 2023
7494aa8
restoring functions
sgoggins Oct 16, 2023
fe2e4fb
returning to original state for secondary
sgoggins Oct 16, 2023
9fba99b
Merge branch 'dev' into bulk-facade
IsaacMilarky Oct 17, 2023
fe6cf49
Merge pull request #2522 from chaoss/detangle-collection-further
IsaacMilarky Oct 17, 2023
8f8bebe
fix missing files in commit analysis
IsaacMilarky Oct 17, 2023
d0dda13
recursive bulk insert to address flaws
IsaacMilarky Oct 17, 2023
432c5dd
indent error
IsaacMilarky Oct 17, 2023
8ad9953
import errors
IsaacMilarky Oct 17, 2023
30494be
typo
IsaacMilarky Oct 17, 2023
4299022
typo
IsaacMilarky Oct 17, 2023
8fb8ab3
typo
IsaacMilarky Oct 17, 2023
a386a11
Update recollection frequency
ABrain7710 Oct 18, 2023
b3ff92e
Merge pull request #2550 from chaoss/update-repo-collection-frequency
sgoggins Oct 18, 2023
fac9461
updated percentages
sgoggins Oct 18, 2023
8093724
Merge pull request #2549 from chaoss/data-fixer-reviews
sgoggins Oct 18, 2023
1af8fd5
Wrong indent
IsaacMilarky Oct 20, 2023
af2ab85
Andrew make bulk insert more readable
IsaacMilarky Oct 20, 2023
1962a81
Merge pull request #2546 from chaoss/bulk-facade
IsaacMilarky Oct 20, 2023
386a32c
Fix null cntrb id errors
ABrain7710 Oct 20, 2023
968aa0b
Merge pull request #2553 from chaoss/null-cntrb-id-fix
sgoggins Oct 20, 2023
826514a
Fix syntax error
ABrain7710 Oct 21, 2023
5797b4a
fix missing placeholder date in last facade change
IsaacMilarky Oct 21, 2023
786ce74
syntax
IsaacMilarky Oct 23, 2023
75f0fb0
refactor to be less nested
IsaacMilarky Oct 23, 2023
cbca6c5
fixed typos
kaxada Oct 24, 2023
80e0abb
Merge pull request #2554 from chaoss/fix-syntax-error
IsaacMilarky Oct 24, 2023
766fa23
Merge branch 'dev' into typo-fixes
sgoggins Oct 24, 2023
fb9b729
Merge pull request #2555 from kaxada/typo-fixes
sgoggins Oct 24, 2023
c534784
repo load message
sgoggins Oct 26, 2023
bb1c340
reposition message
sgoggins Oct 26, 2023
9f3913c
putting it back
sgoggins Oct 26, 2023
f74810f
Lighter Text
sgoggins Oct 26, 2023
3a796c3
refining layout
sgoggins Oct 26, 2023
3c34b6d
link cleanup
sgoggins Oct 26, 2023
4c90154
refinement
sgoggins Oct 26, 2023
e888453
more
sgoggins Oct 26, 2023
7b9cbf2
fixing spacing
sgoggins Oct 26, 2023
f378c14
heading
sgoggins Oct 26, 2023
350eaf4
more info
sgoggins Oct 26, 2023
dc24830
Imagery
sgoggins Oct 26, 2023
e8343a8
jets
sgoggins Oct 26, 2023
2c5b063
center
sgoggins Oct 26, 2023
f9590b1
circles
sgoggins Oct 26, 2023
a1638af
circle
sgoggins Oct 26, 2023
f098b94
css
sgoggins Oct 26, 2023
0ce7646
css2
sgoggins Oct 26, 2023
f50c0e3
jets
sgoggins Oct 26, 2023
fc5f5a8
d
sgoggins Oct 26, 2023
95055c1
simplify
sgoggins Oct 26, 2023
64bad41
simplify
sgoggins Oct 26, 2023
65c9137
Words
sgoggins Oct 26, 2023
0115410
style
sgoggins Oct 26, 2023
9c8c727
Merge pull request #2561 from chaoss/repo-load-message-patch
Ulincsys Oct 26, 2023
43ec217
fix wrong baseclass for process_ossf_dependency_metrics
IsaacMilarky Oct 26, 2023
6680f49
fix base class of facade tasks
IsaacMilarky Oct 26, 2023
33a1e5b
Merge pull request #2562 from chaoss/fix-wrong-task-baseclass
ABrain7710 Oct 26, 2023
a5e0d53
doocumentation of server update
sgoggins Oct 30, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Augur NEW Release v0.53.2
# Augur NEW Release v0.60.0

[![first-timers-only](https://img.shields.io/badge/first--timers--only-friendly-blue.svg?style=flat-square)](https://www.firsttimersonly.com/) We follow the [First Timers Only](https://www.firsttimersonly.com/) philosophy of tagging issues for first timers only, and walking one newcomer through the resolution process weekly. [You can find these issues tagged with "first timers only" on our issues list.](https://github.com/chaoss/augur/labels/first-timers-only).

Expand All @@ -7,8 +7,7 @@
## NEW RELEASE ALERT!
### [If you want to jump right in, updated docker build/compose and bare metal installation instructions are available here](docs/new-install.md)


Augur is now releasing a dramatically improved new version to the main branch. It is also available here: https://github.com/chaoss/augur/releases/tag/v0.53.2
Augur is now releasing a dramatically improved new version to the main branch. It is also available here: https://github.com/chaoss/augur/releases/tag/v0.60.0
- The `main` branch is a stable version of our new architecture, which features:
- Dramatic improvement in the speed of large scale data collection (100,000+ repos). All data is obtained for 100k+ repos within 2 weeks.
- A new job management architecture that uses Celery and Redis to manage queues, and enables users to run a Flower job monitoring dashboard
Expand Down
10 changes: 8 additions & 2 deletions augur/api/routes/dei.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
from augur.application.db.session import DatabaseSession
from augur.application.config import AugurConfig

from augur.tasks.util.collection_util import start_block_of_repos, get_enabled_phase_names_from_config, core_task_success_util
from augur.tasks.util.collection_util import CollectionRequest,AugurTaskRoutine, get_enabled_phase_names_from_config, core_task_success_util
from augur.tasks.start_tasks import prelim_phase, primary_repo_collect_phase
from augur.tasks.github.util.github_task_session import GithubTaskSession
from augur.tasks.init.redis_connection import redis_connection as redis
Expand Down Expand Up @@ -96,7 +96,13 @@ def core_task_success_util_gen(repo_git):

record = BadgingDEI(**record)
session.add(record)
start_block_of_repos(logger, session, [repo_url], primary_enabled_phases, "new")

deiHook = CollectionRequest("core",primary_enabled_phases)
deiHook.repo_list = [repo_url]

singleRoutine = AugurTaskRoutine(session,[deiHook])
singleRoutine.start_data_collection()
#start_block_of_repos(logger, session, [repo_url], primary_enabled_phases, "new")

session.close()

Expand Down
Loading
Loading