Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create QA Kedro test projects for stress testing and performance and evaluation #3489

Closed
AhdraMeraliQB opened this issue Jan 5, 2024 · 0 comments

Comments

@AhdraMeraliQB
Copy link
Contributor

Description

There are several features across the Kedro organisation that could benefit from manual testing on large projects to evaluate performance. The proposal is to create several kedro projects of varying size that can be used to test and experiment with.

This could be particularly useful for testing Viz features, CC @rashidakanchwala, @NeroOkwa

These would ideally live in a separate repo within the organisation, and make use of many available features (namespaces, hooks, parameters, etc). I propose the following scales of project:

  • 10 nodes
  • 100 nodes
  • 1000 nodes
  • 10000 nodes

Numbers inspired by this comment

Note: After completion of this, it should become part of the release process to migrate these projects to the newest version. This provides the added opportunity to test project migration during major releases.

@merelcht merelcht changed the title Create scaling Kedro projects for stress testing and performance and evaluation Create QA Kedro test projects for stress testing and performance and evaluation Apr 8, 2024
@kedro-org kedro-org locked and limited conversation to collaborators Apr 8, 2024
@merelcht merelcht converted this issue into discussion #3790 Apr 8, 2024

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
None yet
Projects
Archived in project
Development

No branches or pull requests

1 participant