Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rsync: [sender] readdir("/data/config/auto/extensions/sd_dreambooth_extension"): Cannot allocate memory (12) #312

Closed
MithrilMan opened this issue Jan 25, 2023 · 3 comments · Fixed by #313
Closed

rsync: [sender] readdir("/data/config/auto/extensions/sd_dreambooth_extension"): Cannot allocate memory (12) #312

MithrilMan opened this issue Jan 25, 2023 · 3 comments · Fixed by #313
Labels
bug Something isn't working

Comments

@MithrilMan
Copy link

Has this issue been opened before?

  • [X ] It is not in the FAQ, I checked.
  • [X ] It is not in the issues, I searched.

Describe the bug

Tried to create a model with dreambooth extension but it failed with a "Cannot allocate memory (12)" error caused by rsync

image

Which UI

auto

Additional context

I've installed the dreambooth extensions, I've created a script in startup.sh to mount dreambooth folder
When I try to create a model, during the creation it error out complaining about rsync that's causing memory error

From my understanding there is an infinite loop caused by #309 that happens every 2 seconds and it seems causing problem to dreambooth model creation (and eventually training)

( I've 32GB of ram and a 3090TI GPU )
@MithrilMan MithrilMan added the bug Something isn't working label Jan 25, 2023
@MithrilMan MithrilMan changed the title rsync c rsync: [sender] readdir("/data/config/auto/extensions/sd_dreambooth_extension"): Cannot allocate memory (12) Jan 25, 2023
@AbdBarho
Copy link
Owner

@MithrilMan that is correct, I will remove it for now, but some other extensions will break.

@AbdBarho AbdBarho mentioned this issue Jan 25, 2023
Merged
AbdBarho added a commit that referenced this issue Jan 25, 2023
@AbdBarho
Revert #309 (#313)
95e489a 
Closes #312
@MithrilMan
Copy link
Author

I was wondering if maybe a solution could be to create an auto extension that enable/disable automatic rsync and/or execute rsync at will

@AbdBarho
Copy link
Owner

@MithrilMan this will push it to the user, and now the user has to understand what / why is being synced.

I have a new fix in #314, feel free to check if everything still works after its merged.

torun225 added a commit to torun225/stable-diffusion-webui-docker that referenced this issue Feb 11, 2023
@torun225
Squashed commit of the following:
b8f7801 
commit 3c860d6
Author: AbdBarho <[email protected]>
Date:   Thu Feb 2 19:53:48 2023 +0100

    Update versions (AbdBarho#320)

    - auto:
    AUTOMATIC1111/stable-diffusion-webui@226d840
    - invoke:
    invoke-ai/InvokeAI@80c5322

    Remove `accelerate` for startup. Refs AbdBarho#317

    Use `SIGINT` Fixes AbdBarho#319

commit 15e9841
Author: AbdBarho <[email protected]>
Date:   Sun Jan 29 20:07:41 2023 +0100

    Update versions (AbdBarho#316)

    - auto:
    AUTOMATIC1111/stable-diffusion-webui@00dab8f
    - invoke:
    invoke-ai/InvokeAI@07e03b3

commit eeb410f
Author: AbdBarho <[email protected]>
Date:   Fri Jan 27 07:55:22 2023 +0100

    Update versions (AbdBarho#315)

    auto:
    AUTOMATIC1111/stable-diffusion-webui@7a14c8a
    invoke:
    invoke-ai/InvokeAI@e4cd662

    Also, fix script mounting in auto AbdBarho#295

commit 9d2a338
Author: AbdBarho <[email protected]>
Date:   Wed Jan 25 18:17:10 2023 +0100

    Always serve static files (AbdBarho#314)

commit 95e489a
Author: AbdBarho <[email protected]>
Date:   Wed Jan 25 17:38:25 2023 +0100

    Revert AbdBarho#309 (AbdBarho#313)

    Closes AbdBarho#312

commit 00518a0
Author: AbdBarho <[email protected]>
Date:   Tue Jan 24 23:25:07 2023 +0100

    Add missing mkdir (AbdBarho#310)

commit f8de919
Author: AbdBarho <[email protected]>
Date:   Tue Jan 24 23:11:15 2023 +0100

    Sync Scripts (AbdBarho#309)

    Refs AbdBarho#308

commit 8713163
Author: AbdBarho <[email protected]>
Date:   Tue Jan 24 19:52:15 2023 +0100

    Update Auto (AbdBarho#307)

    AUTOMATIC1111/stable-diffusion-webui@93fad28

commit e2b0fb9
Author: AbdBarho <[email protected]>
Date:   Tue Jan 24 19:17:34 2023 +0100

    Add Lora support (AbdBarho#306)

    Refs AbdBarho#305 AbdBarho#303

    Co-authored-by: repligator <[email protected]>

commit 510f9fa
Author: AbdBarho <[email protected]>
Date:   Sun Jan 22 20:32:03 2023 +0100

    Update versions (AbdBarho#302)

    Closes AbdBarho#301

    - auto:
    AUTOMATIC1111/stable-diffusion-webui@c98cb0f
    - invoke:
    invoke-ai/InvokeAI@89791d9

commit 042d5c5
Author: AbdBarho <[email protected]>
Date:   Thu Jan 19 06:37:53 2023 +0100

    Fix link / mkdir order (AbdBarho#299)

    Fixes
    AbdBarho@c9153fa#commitcomment-97285124

commit 6684637
Author: AbdBarho <[email protected]>
Date:   Wed Jan 18 18:50:56 2023 +0100

    Mount scripts instead of copy (AbdBarho#298)

    Refs AbdBarho#295

commit c9153fa
Author: AbdBarho <[email protected]>
Date:   Wed Jan 18 18:48:34 2023 +0100

    Update versions (AbdBarho#297)

    - auto:
    AUTOMATIC1111/stable-diffusion-webui@6faae23
    - invoke:
    invoke-ai/InvokeAI@f232068

commit 13dfd4e
Author: AbdBarho <[email protected]>
Date:   Sun Jan 15 11:50:16 2023 +0100

    Update auto (AbdBarho#293)

    AUTOMATIC1111/stable-diffusion-webui@d97f467

    This version has a hash computation of models, so expect slower start up
    & loading times.

commit 9b1ea3c
Author: AbdBarho <[email protected]>
Date:   Fri Jan 13 21:27:55 2023 +0100

    [MAJOR] Update auto to torch 1.13.1 (AbdBarho#290)

    AUTOMATIC1111/stable-diffusion-webui@82725f0

    torch==1.13.1+cu117

    This might break your extensions!

commit b5cdf29
Author: AbdBarho <[email protected]>
Date:   Mon Jan 9 19:04:40 2023 +0100

    Update versions (AbdBarho#287)

    - auto:
    AUTOMATIC1111/stable-diffusion-webui@2b94ec7
    - sygil:
    Sygil-Dev/sygil-webui@571fb89

commit db831ec
Author: AbdBarho <[email protected]>
Date:   Sat Jan 7 18:58:50 2023 +0100

    Bump versions (AbdBarho#286)

    - auto:
    AUTOMATIC1111/stable-diffusion-webui@1512333
    - invoke:
    invoke-ai/InvokeAI@26e413a

commit ceeac61
Author: AbdBarho <[email protected]>
Date:   Fri Jan 6 10:24:20 2023 +0100

    Update auto (AbdBarho#282)

    AUTOMATIC1111/stable-diffusion-webui@683287d

commit 19972f3
Author: AbdBarho <[email protected]>
Date:   Wed Jan 4 20:36:49 2023 +0100

    Update Versions (AbdBarho#281)

    - auto:
    AUTOMATIC1111/stable-diffusion-webui@8149078
    - sygil:
    Sygil-Dev/sygil-webui@e484828
    - invoke:
    invoke-ai/InvokeAI@21bf512

commit 0c16c10
Author: AbdBarho <[email protected]>
Date:   Tue Jan 3 21:19:37 2023 +0100

    Update auto (AbdBarho#280)

    AUTOMATIC1111/stable-diffusion-webui@3e22e29

commit 78c90e5
Author: AbdBarho <[email protected]>
Date:   Sun Jan 1 16:46:41 2023 +0100

    Update auto & invoke (AbdBarho#277)

    - auto:
    AUTOMATIC1111/stable-diffusion-webui@524d532
    - invoke:
    invoke-ai/InvokeAI@524d532

    Happy new Year!!!

commit 6a3826c
Author: AbdBarho <[email protected]>
Date:   Mon Dec 26 09:50:29 2022 +0100

    Update auto (AbdBarho#274)

    AUTOMATIC1111/stable-diffusion-webui@4af3ca5

    Merry Christmas 🎄

commit 56d9763
Author: AbdBarho <[email protected]>
Date:   Thu Dec 22 09:19:39 2022 +0100

    Smaller xformers wheel (AbdBarho#271)

    Built with `Release` instead of `RelWithDebInfo`

    Related AbdBarho#270

commit 41e0dcc
Author: AbdBarho <[email protected]>
Date:   Sat Dec 17 12:22:45 2022 +0100

    Auto: clean up interrogators (AbdBarho#265)

commit 5a9d305
Author: AbdBarho <[email protected]>
Date:   Fri Dec 16 09:57:33 2022 +0100

    Update Invoke (AbdBarho#264)

    invoke-ai/InvokeAI@ffa54f4

commit d70e96d
Author: AbdBarho <[email protected]>
Date:   Wed Dec 14 09:47:36 2022 +0100

    Update Invoke (AbdBarho#261)

    invoke-ai/InvokeAI@5d20f47

commit d97d257
Author: AbdBarho <[email protected]>
Date:   Mon Dec 12 09:25:44 2022 +0100

    Update versions (AbdBarho#257)

    - auto:
    AUTOMATIC1111/stable-diffusion-webui@685f963
    - sygil:
    Sygil-Dev/sygil-webui@d3f9d05
    - invoke:
    invoke-ai/InvokeAI@e159bb3

commit c3cf812
Author: AbdBarho <[email protected]>
Date:   Sat Dec 10 12:06:37 2022 +0100

    Expose API per default

commit b8256cc
Author: AbdBarho <[email protected]>
Date:   Sat Dec 10 11:42:04 2022 +0100

    Update PR template

commit 4969906
Author: AbdBarho <[email protected]>
Date:   Sat Dec 10 11:40:25 2022 +0100

    Update Download URLs

commit 8201e36
Author: AbdBarho <[email protected]>
Date:   Fri Dec 9 18:14:48 2022 +0100

    Alpha support  for sd v2.x in auto (AbdBarho#244)

    AUTOMATIC1111/stable-diffusion-webui@44c46f0

    Install sd 2 from [these
    instructions](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features#stable-diffusion-20),
    put the downloaded files in `data/StableDiffusion/`, maybe create a
    subfolder `v2` and put the models in it, just to keep everything clean.

    There will be problems, so be warry.

commit 1423b27
Author: AbdBarho <[email protected]>
Date:   Fri Dec 9 16:50:46 2022 +0100

    Update Versions (AbdBarho#256)

    sygil:
    Sygil-Dev/sygil-webui@5291437
    Also, switch to pip instead of conda because it is really slow

    invoke:
    invoke-ai/InvokeAI@ed9186b

commit 87a51e9
Author: AbdBarho <[email protected]>
Date:   Sun Dec 4 11:00:28 2022 +0100

    [BREAKING] Rename UIs (AbdBarho#254)

    Rename the UIs in docker compose to their new names

    Changes folder names

    Changes output folder structure

    Closes issue AbdBarho#263

    Adds `sygil-sl` instead of docker compose flag.

commit bdee804
Author: AbdBarho <[email protected]>
Date:   Sun Dec 4 07:07:49 2022 +0100

    Fix mounting in Lstein (AbdBarho#253)

    Closes AbdBarho#251

commit f1a1641
Author: AbdBarho <[email protected]>
Date:   Sat Dec 3 13:36:18 2022 +0100

    Invoke 2.2.3 (AbdBarho#250)

    lstein:
    invoke-ai/InvokeAI@5c31feb

commit 8df9d10
Author: AbdBarho <[email protected]>
Date:   Wed Nov 30 19:33:44 2022 +0100

    Update versions (AbdBarho#246)

    - hlky:
    Sygil-Dev/sygil-webui@18a3b80
    - lstein:
    invoke-ai/InvokeAI@a9aa4e4

commit 7a1e52b
Author: AbdBarho <[email protected]>
Date:   Sat Nov 26 15:45:53 2022 +0100

    Enable extensions by default (AbdBarho#243)

    Closes AbdBarho#242

commit d20b873
Author: AbdBarho <[email protected]>
Date:   Fri Nov 25 13:25:10 2022 +0100

    Update xformers (AbdBarho#241)

    facebookresearch/xformers@8910bb5

commit 23757d2
Author: AbdBarho <[email protected]>
Date:   Sun Nov 20 11:35:56 2022 +0100

    Update invoke (AbdBarho#234)

    invoke-ai/InvokeAI@2b7e3ab

commit 9e7979b
Author: AbdBarho <[email protected]>
Date:   Sun Nov 20 11:05:39 2022 +0100

    Update Versions (AbdBarho#230)

    - auto:
    AUTOMATIC1111/stable-diffusion-webui@47a44c7
    - hlky:
    Sygil-Dev/sygil-webui@269107a

    Refs AbdBarho#216

commit 8623c73
Author: AbdBarho <[email protected]>
Date:   Sun Nov 13 08:42:27 2022 +0100

    v1.5 Inpainting (AbdBarho#221)

    Also remove v1.4

    Closes AbdBarho#217

commit 9b6750b
Author: AbdBarho <[email protected]>
Date:   Sun Nov 13 07:12:17 2022 +0100

    Use cuda 11.6 for auto (AbdBarho#220)

    auto:
    AUTOMATIC1111/stable-diffusion-webui@98947d1

    Closes AbdBarho#218 AbdBarho#219

commit 5e3f20b
Author: AbdBarho <[email protected]>
Date:   Sat Nov 12 18:33:59 2022 +0100

    Move contribution to the top
Jordan-Lambda pushed a commit to Jordan-Lambda/lambda-cloud-stable-diffusion-2.0-webui-easy that referenced this issue Aug 2, 2023
@AbdBarho @Jordan-Lambda
Revert AbdBarho#309 (AbdBarho#313)
3c440f3 
Closes AbdBarho#312
cloudaxes pushed a commit to cloudaxes/stable-diffusion-webui-docker that referenced this issue Sep 6, 2023
@AbdBarho
Revert AbdBarho#309 (AbdBarho#313)
9923aeb 
Closes AbdBarho#312
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Revert #309 AbdBarho/stable-diffusion-webui-docker
2 participants
@MithrilMan @AbdBarho