Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: ANALYZE messages query should be performed regularly #2986

Merged
merged 2 commits into from
Aug 21, 2024

Conversation

Ivansete-status
Copy link
Collaborator

@Ivansete-status Ivansete-status commented Aug 19, 2024

Description

In a pair session with @richard-ramos, thanks for it 🥳 🙌 !, we've discovered that the database wasn't using the proper indexes for certain query scenarios.

After applying ANALYZE messages to a database we've noticed that the query execution time went from ~1second to ~100ms.

Therefore, we considered that it is interesting to ANALYZE messages regularly in the database so that the stats are properly updated and the planner can pick up the optimum indexes for each case.

Changes

  • Run ANALYZE messages once a day.

More context

That was the query under analysis:

SELECT contentTopic, payload, pubsubTopic, version, timestamp, id, messageHash, meta FROM messages WHERE contentTopic IN ('/waku/1/0x9a3fcc34/rfc26','/waku/1/0xc9b649fe/rfc26','/waku/1/0x678be1ba/rfc26','/waku/1/0x87c713d2/rfc26','/waku/1/0x267ad6e5/rfc26','/waku/1/0xb50f4cd5/rfc26','/waku/1/0xe2509252/rfc26','/waku/1/0x8de76c97/rfc26') AND pubsubTopic = '/waku/2/rs/16/32' AND (timestamp, id) < ('1724046108468541771','ed019e15afef1273dabd74e9cc4091fad4a5f0fa054230a8e121f81fcec278c7') AND timestamp >= '1723971035000000000' AND timestamp <= '1724057435000000000' ORDER BY timestamp DESC, id DESC LIMIT '21';

@Ivansete-status Ivansete-status changed the title ANALYZE messages query should be performed regularly for better performance chore: ANALYZE messages query should be performed regularly for better performance Aug 19, 2024
Copy link

This PR may contain changes to database schema of one of the drivers.

If you are introducing any changes to the schema, make sure the upgrade from the latest release to this change passes without any errors/issues.

Please make sure the label release-notes is added to make sure upgrade instructions properly highlight this change.

@Ivansete-status Ivansete-status changed the title chore: ANALYZE messages query should be performed regularly for better performance chore: ANALYZE messages query should be performed regularly Aug 19, 2024
@Ivansete-status Ivansete-status marked this pull request as ready for review August 19, 2024 17:14
Copy link

You can find the image built from this PR at

quay.io/wakuorg/nwaku-pr:2986

Built from eea8972

Copy link
Member

@richard-ramos richard-ramos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking fwd to see this in action on the fleets!

Copy link
Contributor

@gabrielmer gabrielmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing! Thanks so much!

@Ivansete-status Ivansete-status merged commit 32f2d85 into master Aug 21, 2024
16 of 18 checks passed
@Ivansete-status Ivansete-status deleted the run-analyze-in-postgres branch August 21, 2024 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants