Huge amount of reads for within. Is this normal?

[mmmoli]

Issue

Worried about ongoing costs 😟

Observation

12k READ queries over a few hours

Expected

Maybe an order of magnitude fewer.

Fixes

Either reset expectation and prepare to pay or fix crazy code?

---

Context

I've developed a graphql API to list 27k Projects stored in Firestore. I'm using geoFirex to find Projects within a certain radius.

As part of development, I ran this function 30 times or so:

export const getProjectsNearPoint = async (
  point: Location,
  searchRadiusKm: number = 10,
  queryFn: QueryFn = ref => ref.limit(3) // Bug: This is not working
) => {
  const center = geo.point(point.latitude, point.longitude);

  const field = "location";

  const query = geo
    .collection(COLLECTION, queryFn)
    .within(center, searchRadiusKm, field);

  const nearbyProjects = await get(query);

  const result = nearbyProjects.map(
    ({ queryMetadata, ...project }: GeoQueryDocument) => ({
      distance: queryMetadata.distance,
      ...project
    })
  );

  return result;
};

---

Questions

• Is this normal?
• Would an index help?
• does anyone suspect my graphql resolvers are doing anything necessarily recursive?

Thanks.

[danieltanasec]

Did you find any fix to this ?. I also tested the code and when I receive 4 points on my map I have 800 reads in firebase.

[codediodeio]

Additional reads beyond the radius result is expected, but a bug was fixed in 0.1.0 where get was not unsubscribing from the underlying callback - I suspect that was the root of your issue.

Limit does not work as you might expect. A regular firestore query is made for 9 different square hashes, so your worst case limit would actually be applied as 9 x 3. I don't recommend using limit, but rather a smaller query radius or where with ==.

Also, you can now enable logging to track the total documents returned in a query within(c, r, f, { log: true }) to monitor the results clientside.

[alexmasita]

@mmmoli I am looking to using this library but I am curious to know if this issue is resolved for you as of this moment as per update suggested by @codediodeio ?

[mmmoli]

To be honest, I freaked out.

Between that and spending $30k in 72 hours I decided it wasn't worth it and moved away.

I think that's more of reflection of Fb's pricing model that this library though 🙏

[mmmoli]

@VincentDufour I didn't. I wasn't part of that engineering team.

From my link to the blog post describing their disaster:

This means that every session to our site read the same number of documents as we have of number of payments. #UnaVacaPorDeLaCalle received more than 16,000 supporters, so: 2 million sessions x 16,000 documents = more than 40 Billion requests to Firestore on less than 48 hours.

In summary: think, hard.