Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[7.x] bfetch (2) (#53711) #55039

Merged
merged 1 commit into from
Jan 16, 2020
Merged

[7.x] bfetch (2) (#53711) #55039

merged 1 commit into from
Jan 16, 2020

Conversation

streamich
Copy link
Contributor

@streamich streamich commented Jan 16, 2020

Backports the following commits to 7.x:


Dev Docs

Request batching and response streaming functionality of legacy Interpreter plugin has been moved out into a separate bfetch new platform plugin. Now every plugin can create server endpoints and browser wrappers that can batch HTTP requests and stream responses back.

As an example, we will create a batch processing endpoint that receives a number then doubles it
and streams it back. We will also consider the number to be time in milliseconds
and before streaming the number back the server will wait for the specified number of
milliseconds.

To do that, first create server-side batch processing route using addBatchProcessingRoute.

plugins.bfetch.addBatchProcessingRoute<{ num: number }, { num: number }>(
  '/my-plugin/double',
  () => ({
    onBatchItem: async ({ num }) => {
      // Validate inputs.
      if (num < 0) throw new Error('Invalid number');
      // Wait number of specified milliseconds.
      await new Promise(r => setTimeout(r, num));
      // Double the number and send it back.
      return { num: 2 * num };
    },
  })
);

Now on client-side create double function using batchedFunction.
The newly created double function can be called many times and it
will package individual calls into batches and send them to the server.

const double = plugins.bfetch.batchedFunction<{ num: number }, { num: number }>({
  url: '/my-plugin/double',
});

Note: the created double must accept a single object argument ({ num: number } in this case)
and it will return a promise that resolves into an object, too (also { num: number } in this case).

Use the double function.

double({ num: 1 }).then(console.log, console.error); // { num: 2 }
double({ num: 2 }).then(console.log, console.error); // { num: 4 }
double({ num: 3 }).then(console.log, console.error); // { num: 6 }

* feat: 🎸 implement ItemBuffer

* test: 💍 add tests for ItemBuffer

* feat: 🎸 add TimedItemBuffer

* test: 💍 add TimedItemBuffer tests

* feat: 🎸 add createBatchedFunction

* chore: 🤖 save wip on higher level batching

* test: 💍 add createBatchedFunction tests

* feat: 🎸 implement createStreamingBatchedFunction() method

* refactor: 💡 rename "data" key to "result"

* feat: 🎸 return error in "error" key in legacy protocol

* feat: 🎸 add server-side to Expressions plugin

* refactor: 💡 move interpreter server-side registries to NP

* feat: 🎸 implement bfetch.addBatchProcessingRoute

* feat: 🎸 improve streaming and batching func to pass request

* feat: 🎸 initial setup of new expressions batching endpoint

* feat: 🎸 expose bfetch.batchedFunction() function

* feat: 🎸 add of() function

of() function awaits a promise and converts it to a 3-tuple representing
its state.

* refactor: 💡 move normalizeError() to /common

* feat: 🎸 improve createStreamingBatchedFunction() function

* refactor: 💡 move GET /api/interpreter/fns to the New Platform

* feat: 🎸 move batched_fetch to the New Platform

* feat: 🎸 implement legacy interpreter batching on server in NP

* feat: 🎸 switch legacy interpreter server functions to NP

* chore: 🤖 remove unused import

* fix: 🐛 correct expressions mocks

* test: 💍 fix batching tests after refactor

* test: 💍 stub bfetch plugin explorer

* test: 💍 add routing and app structure to bfetch_explorer

* test: 💍 add server-side to bfetch_explorer

* test: 💍 create <DoubleInteger> component in bfetch_explorer

* test: 💍 improve bfetch_explorer

* test: 💍 add <CountUntil> demo to bfetch_explorer

* test: 💍 by default redirect to first bfetch_explorer example

* test: 💍 add error example to bfetch_explorer

* docs: ✏️ improve bfetch docs

* docs: ✏️ improve bfetch server-side docs

* chore: 🤖 address self-review comments

* fix: 🐛 use new core ES data client, remove unuseed import

* fix: 🐛 remove unused interface import

* Update examples/bfetch_explorer/public/components/count_until/index.tsx

Co-Authored-By: Lukas Olson <[email protected]>

* Update examples/bfetch_explorer/public/components/double_integers/index.tsx

Co-Authored-By: Lukas Olson <[email protected]>

* Update src/plugins/bfetch/common/buffer/item_buffer.ts

Co-Authored-By: Lukas Olson <[email protected]>

* Update src/plugins/kibana_utils/common/of.ts

Co-Authored-By: Lukas Olson <[email protected]>

* docs: ✏️ add batchedFunction params to README

* refactor: 💡 rename onflush to onFlush

* feat: 🎸 make maxItemAge optional in TimedItemBuffer

* refactor: 💡 remove promise from fetchStreaming

* test: 💍 add bfetch_explorer functional tests

* test: 💍 rename test plugin to kbn_tp_bfetch_explorer

* fix: 🐛 use stream instead of removed promise

* fix: 🐛 use correct tsconfig.json in bfetch test plugin

* feat: 🎸 add kbn_tp_bfetch_explorer server-side files to tsconfi

Co-authored-by: Lukas Olson <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
@kibanamachine
Copy link
Contributor

💚 Build Succeeded

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@streamich streamich added the release_note:plugin_api_changes Contains a Plugin API changes section for the breaking plugin API changes section. label Jan 16, 2020
@streamich streamich merged commit c16a1b6 into elastic:7.x Jan 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport release_note:plugin_api_changes Contains a Plugin API changes section for the breaking plugin API changes section.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants