Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: compress pipeline graphs before sending to mermaid #8767

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

lbux
Copy link
Contributor

@lbux lbux commented Jan 24, 2025

Related Issues

Proposed Changes:

When pipelines get too big, the resulting base64 output is too long when we send a request to the mermaid server leading to a HTTP 400 error. Mermaid supports compressed URLS using Pako. With zlib, we are able to compress our input in the expected format and send it off to Mermaid.

How did you test it?

I wanted to do as close to a 1:1 change so that this has 0 impact to the user. The same tests were used which hopefully means the changes are 1:1 as they have all passed.

I am also asking for tests from Haystack users with big pipelines on Discord.

Notes for the reviewer

I will make another PR that will use this change as a foundation for:

  • Extension parameters (jpg, png, webp, svg, pdf)
  • Background color modifications
  • Themes
  • Image size

Aaaaand.... offline support! #7896

Checklist

  • I have read the contributors guidelines and the code of conduct
  • I have updated the related issue with new insights and changes
  • I added unit tests and updated the docstrings
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
  • I documented my code
  • I ran pre-commit hooks and fixed any issue

@lbux lbux requested review from a team as code owners January 24, 2025 06:11
@lbux lbux requested review from dfokina and davidsbatista and removed request for a team January 24, 2025 06:11
@lbux lbux changed the title Draw offline fix: compress pipeline graphs before sending to mermaid Jan 24, 2025
@coveralls
Copy link
Collaborator

Pull Request Test Coverage Report for Build 12944303830

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 2 unchanged lines in 1 file lost coverage.
  • Overall coverage increased (+0.002%) to 91.338%

Files with Coverage Reduction New Missed Lines %
core/pipeline/draw.py 2 97.1%
Totals Coverage Status
Change from base Build 12928411205: 0.002%
Covered Lines: 8858
Relevant Lines: 9698

💛 - Coveralls

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Mermaid Crashes If trying to draw a large pipeline
2 participants