Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory scheduling deadlock #11

Merged
merged 31 commits into from
May 24, 2022
Merged
Changes from 1 commit
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
9e74f48
blocktask and evict tasks separated
jaewan Nov 17, 2021
2c89769
blocktasks when oom
jaewan Nov 18, 2021
cfc7a73
command to replay the bug
jaewan Nov 19, 2021
8176e99
Fix crash in raylet on duplicate object
stephanie-wang Nov 20, 2021
25136da
Fixed bug in blocktasks. Did not block when block task called only
jaewan Nov 22, 2021
020dc20
Merged blocktasks and evicttasks into a single callback and erased ev…
jaewan Nov 22, 2021
b90de27
get lowest pri, but obj does not have pri yet
jaewan Dec 20, 2021
5d281a1
debug logs to see where to put priority
jaewan Jan 6, 2022
2373c86
log without compile error
Jan 6, 2022
dc06d25
commit for migration
jaewan Jan 17, 2022
0887544
working version. But still need testing
jaewan Jan 19, 2022
840c853
Blocktasks but 2 new problems emerged
jaewan Feb 8, 2022
c195664
added microbechmark
jaewan Feb 8, 2022
c23030a
deadlock cases 1 and 2 induced by not spilling. 1 is when workers are…
jaewan Feb 10, 2022
4aaccc5
polished debug messages
jaewan Feb 17, 2022
74b6780
typo fix
jaewan Feb 17, 2022
4e8527b
removed scripts and included block, evict threshold triggered code
jaewan Feb 22, 2022
252c7c9
deadlock detection by checking #of leased workers and spinning tasks
jaewan Feb 24, 2022
c3a211f
reflected changes from branch -jae
jaewan Mar 1, 2022
7cd9734
reflected changes from branch -jae
jaewan Mar 1, 2022
f669e80
deadlock detection#1
jaewan Mar 2, 2022
a5eedc2
reflected changes from parent branch
jaewan Mar 2, 2022
c390637
.
jaewan Mar 3, 2022
0fecb16
Merge branch 'memory-scheduling' into memory-scheduling-deadlock
jaewan Mar 7, 2022
98d67e8
Merge branch 'memory-scheduling' into memory-scheduling-deadlock
jaewan Mar 10, 2022
0f8bab0
changed triggering spill from task_manager instead of giving #leased_…
jaewan Mar 30, 2022
8aedbae
Spinning task identificatino by queue_ version.
jaewan Mar 31, 2022
2bfe886
Merge branch 'memory-scheduling' into memory-scheduling-deadlock
jaewan Mar 31, 2022
72e3543
test rpc to core worker for object working set
May 6, 2022
d0daa0b
Revert "test rpc to core worker for object working set"
jaewan May 24, 2022
1498c20
test caese, will be updated to pytest
jaewan May 24, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
reflected changes from parent branch
jaewan committed Mar 2, 2022
commit a5eedc21a21a7488996649a2399ef7649f49f69d
7 changes: 5 additions & 2 deletions src/ray/object_manager/plasma/object_store.cc
Original file line number Diff line number Diff line change
@@ -53,8 +53,11 @@ const LocalObject *ObjectStore::CreateObject(const ray::ObjectInfo &object_info,

ray::Priority ObjectStore::GetLowestPriObject() {
// Return the lowest priority object in object_table
ray::Priority lowest_priority = ray::Priority({-1, -1});
for (auto it = object_table_.begin(); it != object_table_.end(); it++){
auto it = object_table_.begin();
if(it == object_table_.end())
return ray::Priority();
ray::Priority lowest_priority = it->second->GetPriority();
for (; it != object_table_.end(); it++){
ray::Priority p = it->second->GetPriority();
if(lowest_priority < p){
lowest_priority = p;