[admin] config_dump scaffolding and routes endpoint (#2771)

This is a step towards dumping configs in proto form. There is a new config_dump endpoint, which spits out all registered configs. It's pretty-printed JSON for now; obviously we can play with format/serialization. Anyone can add a config via the ConfigTracker object that hangs off Admin. ConfigTracker makes the whole thing RAII-y, so that the list of config providers is maintained opaquely. This means the end user doesn't have to worry about ownership, capturing this, etc. Contrast with current unenforced pair of addHandler/removeHandler. This sort of managed weak ownership can be useful all over envoy; just in this diff I left a comment where something like ConfigTracker would delete a bunch of fragile code and confusion. I will genericize it in a future diff though, so let's focus on the functionality here and try to punt on extended discussion of this component and its possibilities. To demonstrate all the structured admin/config dumping scaffolding, I implemented it all for routes. See the data-plane-api PR (envoyproxy/data-plane-api#532) for context. But you can see how other config_dump handlers and, hopefully, all admin endpoints can be turned into structured, arbitrarily consumable proto via this setup. Risk Level: Medium. New feature on admin subsystem, intended to be minimally intrusive into rest of server Testing: New + existing unit. Tested locally with sudo ./bazel-bin/source/exe/envoy-static -c examples/front-proxy/front-envoy.yaml Output looks like: https://gist.github.com/jsedgwick/3a46408a9728ccef9a63d32b4c463c2b Docs Changes: I put TODO doxme everywhere I plan to add docs once the code/API is agreed upon. I won't merge this PR until docs are in. Release Notes: TODO pending consensus on this PR Issues: Makes progress on #2421, #2172 API Changes:] envoyproxy/data-plane-api#532 Deprecated: Current routes endpoint should be considered marked for deletion, but I'll do that in a subsequent PR. There should be some nice cleanup in addition to code deletion with this change. Signed-off-by: James Sedgwick <[email protected]>
envoyproxy · Mar 27, 2018 · 2429797 · 2429797
1 parent 5c1f951
commit 2429797
Show file tree

Hide file tree

Showing 24 changed files with 473 additions and 336 deletions.
diff --git a/include/envoy/router/rds.h b/include/envoy/router/rds.h
@@ -2,6 +2,7 @@
 
 #include <memory>
 
+#include "envoy/api/v2/rds.pb.h"
 #include "envoy/router/router.h"
 
 namespace Envoy {
@@ -22,6 +23,12 @@ class RouteConfigProvider {
    */
   virtual Router::ConfigConstSharedPtr config() PURE;
 
+  /**
+   * @return envoy::api::v2::RouteConfiguration the underlying RouteConfiguration object associated
+   * with this provider.
+   */
+  virtual const envoy::api::v2::RouteConfiguration& configAsProto() const PURE;
+
   /**
    * @return const std::string version info from last accepted config.
    *
@@ -33,32 +40,7 @@ class RouteConfigProvider {
   virtual const std::string versionInfo() const PURE;
 };
 
-/**
- * A provider for dynamic route configurations.
- */
-class RdsRouteConfigProvider : public RouteConfigProvider {
-public:
-  virtual ~RdsRouteConfigProvider() {}
-
-  /**
-   * @return std::string the loaded route table in JSON format.
-   */
-  virtual std::string configAsJson() const PURE;
-
-  /**
-   * @return const std::string& the name of the configured route table.
-   */
-  virtual const std::string& routeConfigName() const PURE;
-
-  /**
-   * @return const std::string& the configuration of the service the RdsRouteConfigProvider is
-   * issuing RDS requests to.
-   */
-  virtual const std::string& configSource() const PURE;
-};
-
 typedef std::shared_ptr<RouteConfigProvider> RouteConfigProviderSharedPtr;
-typedef std::shared_ptr<RdsRouteConfigProvider> RdsRouteConfigProviderSharedPtr;
 
 } // namespace Router
 } // namespace Envoy
diff --git a/include/envoy/router/route_config_provider_manager.h b/include/envoy/router/route_config_provider_manager.h
@@ -26,37 +26,46 @@ class RouteConfigProviderManager {
   virtual ~RouteConfigProviderManager() {}
 
   /**
-   * Get a RouteConfigProviderSharedPtr. Ownership of the RouteConfigProvider is shared by
-   * all the HttpConnectionManagers who own a RouteConfigProviderSharedPtr. The
+   * Get a RouteConfigProviderSharedPtr for a route from RDS. Ownership of the RouteConfigProvider
+   * is shared by all the HttpConnectionManagers who own a RouteConfigProviderSharedPtr. The
    * RouteConfigProviderManager holds weak_ptrs to the RouteConfigProviders. Clean up of the weak
    * ptrs happen from the destructor of the RouteConfigProvider. This function creates a
    * RouteConfigProvider if there isn't one with the same (route_config_name, cluster) already.
    * Otherwise, it returns a RouteConfigProviderSharedPtr created from the manager held weak_ptr.
-   * @param rds supplies the proto configuration of an RdsRouteConfigProvider.
+   * @param rds supplies the proto configuration of an RDS-configured RouteConfigProvider.
+   * @param cm supplies the ClusterManager.
    * @param scope supplies the scope to use for the route config provider.
    * @param stat_prefix supplies the stat_prefix to use for the provider stats.
    * @param init_manager supplies the init manager.
    */
-  virtual RouteConfigProviderSharedPtr getRouteConfigProvider(
+  virtual RouteConfigProviderSharedPtr getRdsRouteConfigProvider(
       const envoy::config::filter::network::http_connection_manager::v2::Rds& rds,
       Upstream::ClusterManager& cm, Stats::Scope& scope, const std::string& stat_prefix,
       Init::Manager& init_manager) PURE;
-};
 
-/**
- * The ServerRouteConfigProviderManager additionally allows listing all of the currently managed
- * RouteConfigProviders.
- */
-class ServerRouteConfigProviderManager : public RouteConfigProviderManager {
-public:
-  virtual ~ServerRouteConfigProviderManager() {}
+  /**
+   * Get a RouteConfigSharedPtr for a statically defined route. Ownership is as described for
+   * getRdsRouteConfigProvider above. Unlike getRdsRouteConfigProvider(), this method always creates
+   * a new RouteConfigProvider.
+   * @param route_config supplies the RouteConfiguration for this route
+   * @param runtime supplies the runtime loader.
+   * @param cm supplies the ClusterManager.
+   */
+  virtual RouteConfigProviderSharedPtr
+  getStaticRouteConfigProvider(const envoy::api::v2::RouteConfiguration& route_config,
+                               Runtime::Loader& runtime, Upstream::ClusterManager& cm) PURE;
+
+  /**
+   * @return std::vector<Router::RouteConfigProviderSharedPtr> a list of all the
+   * dynamic (RDS) RouteConfigProviders currently loaded.
+   */
+  virtual std::vector<RouteConfigProviderSharedPtr> getRdsRouteConfigProviders() PURE;
 
   /**
-   * @return std::vector<Router::RdsRouteConfigProviderSharedPtr> a list of all the
-   * RdsRouteConfigProviders currently loaded. This means that the manager does not provide
-   * pointers to StaticRouteConfigProviders.
+   * @return std::vector<Router::RouteConfigProviderSharedPtr> a list of all the
+   * static RouteConfigProviders currently loaded.
    */
-  virtual std::vector<RdsRouteConfigProviderSharedPtr> rdsRouteConfigProviders() PURE;
+  virtual std::vector<RouteConfigProviderSharedPtr> getStaticRouteConfigProviders() PURE;
 };
 
 } // namespace Router

diff --git a/include/envoy/server/BUILD b/include/envoy/server/BUILD
@@ -22,6 +22,7 @@ envoy_cc_library(
     name = "admin_interface",
     hdrs = ["admin.h"],
     deps = [
+        ":config_tracker_interface",
         "//include/envoy/buffer:buffer_interface",
         "//include/envoy/http:codes_interface",
         "//include/envoy/network:listen_socket_interface",
@@ -39,6 +40,15 @@ envoy_cc_library(
     ],
 )
 
+envoy_cc_library(
+    name = "config_tracker_interface",
+    hdrs = ["config_tracker.h"],
+    deps = [
+        "//source/common/common:non_copyable",
+        "//source/common/protobuf",
+    ],
+)
+
 envoy_cc_library(
     name = "drain_manager_interface",
     hdrs = ["drain_manager.h"],

diff --git a/include/envoy/server/admin.h b/include/envoy/server/admin.h
@@ -8,6 +8,7 @@
 #include "envoy/http/codes.h"
 #include "envoy/http/header_map.h"
 #include "envoy/network/listen_socket.h"
+#include "envoy/server/config_tracker.h"
 
 #include "absl/strings/string_view.h"
 
@@ -69,6 +70,11 @@ class Admin {
    * @return Network::Socket& socket reference.
    */
   virtual const Network::Socket& socket() PURE;
+
+  /**
+   * @return ConfigTracker& tracker for /config_dump endpoint.
+   */
+  virtual ConfigTracker& getConfigTracker() PURE;
 };
 
 } // namespace Server

diff --git a/include/envoy/server/config_tracker.h b/include/envoy/server/config_tracker.h
@@ -0,0 +1,59 @@
+#pragma once
+
+#include <functional>
+#include <map>
+#include <memory>
+
+#include "envoy/common/pure.h"
+
+#include "common/common/non_copyable.h"
+#include "common/protobuf/protobuf.h"
+
+namespace Envoy {
+namespace Server {
+
+/**
+ * ConfigTracker is used by the `/config_dump` admin endpoint to manage storage of config-providing
+ * callbacks with weak ownership semantics. Callbacks added to ConfigTracker only live as long as
+ * the returned EntryOwner object (or ConfigTracker itself, if shorter). Keys should be descriptors
+ * of the configs provided by the corresponding callback. They must be unique.
+ * ConfigTracker is *not* threadsafe.
+ */
+class ConfigTracker {
+public:
+  typedef std::function<ProtobufTypes::MessagePtr()> Cb;
+  typedef std::map<std::string, Cb> CbsMap;
+
+  /**
+   * EntryOwner supplies RAII semantics for entries in the map.
+   * The entry is not removed until the EntryOwner or the ConfigTracker itself is destroyed,
+   * whichever happens first. When you add() an entry, you must hold onto the returned
+   * owner object for as long as you want the entry to stay in the map.
+   */
+  class EntryOwner {
+  public:
+    virtual ~EntryOwner() {}
+
+  protected:
+    EntryOwner(){}; // A sly way to make this class "abstract."
+  };
+  typedef std::unique_ptr<EntryOwner> EntryOwnerPtr;
+
+  virtual ~ConfigTracker(){};
+
+  /**
+   * @return const CbsMap& The map of string keys to tracked callbacks.
+   */
+  virtual const CbsMap& getCallbacksMap() const PURE;
+
+  /**
+   * Add a new callback to the map under the given key
+   * @param key the map key for the new callback.
+   * @param cb the callback to add. *must not* return nullptr.
+   * @return EntryOwnerPtr the new entry's owner object. nullptr if the key is already present.
+   */
+  virtual EntryOwnerPtr add(const std::string& key, Cb cb) PURE;
+};
+
+} // namespace Server
+} // namespace Envoy
diff --git a/source/common/router/BUILD b/source/common/router/BUILD
@@ -76,6 +76,7 @@ envoy_cc_library(
         "//source/common/config:subscription_factory_lib",
         "//source/common/config:utility_lib",
         "//source/common/protobuf:utility_lib",
+        "@envoy_api//envoy/admin/v2:config_dump_cc",
         "@envoy_api//envoy/api/v2:rds_cc",
         "@envoy_api//envoy/config/filter/network/http_connection_manager/v2:http_connection_manager_cc",
     ],

diff --git a/source/common/router/rds_impl.cc b/source/common/router/rds_impl.cc
@@ -5,6 +5,7 @@
 #include <memory>
 #include <string>
 
+#include "envoy/admin/v2/config_dump.pb.h"
 #include "envoy/api/v2/rds.pb.validate.h"
 #include "envoy/api/v2/route/route.pb.validate.h"
 
@@ -29,11 +30,11 @@ RouteConfigProviderSharedPtr RouteConfigProviderUtil::create(
   switch (config.route_specifier_case()) {
   case envoy::config::filter::network::http_connection_manager::v2::HttpConnectionManager::
       kRouteConfig:
-    return RouteConfigProviderSharedPtr{
-        new StaticRouteConfigProviderImpl(config.route_config(), runtime, cm)};
+    return route_config_provider_manager.getStaticRouteConfigProvider(config.route_config(),
+                                                                      runtime, cm);
   case envoy::config::filter::network::http_connection_manager::v2::HttpConnectionManager::kRds:
-    return route_config_provider_manager.getRouteConfigProvider(config.rds(), cm, scope,
-                                                                stat_prefix, init_manager);
+    return route_config_provider_manager.getRdsRouteConfigProvider(config.rds(), cm, scope,
+                                                                   stat_prefix, init_manager);
   default:
     NOT_REACHED;
   }
@@ -42,7 +43,7 @@ RouteConfigProviderSharedPtr RouteConfigProviderUtil::create(
 StaticRouteConfigProviderImpl::StaticRouteConfigProviderImpl(
     const envoy::api::v2::RouteConfiguration& config, Runtime::Loader& runtime,
     Upstream::ClusterManager& cm)
-    : config_(new ConfigImpl(config, runtime, cm, true)) {}
+    : config_(new ConfigImpl(config, runtime, cm, true)), route_config_proto_{config} {}
 
 // TODO(htuch): If support for multiple clusters is added per #1170 cluster_name_
 // initialization needs to be fixed.
@@ -146,30 +147,42 @@ RouteConfigProviderManagerImpl::RouteConfigProviderManagerImpl(
     const LocalInfo::LocalInfo& local_info, ThreadLocal::SlotAllocator& tls, Server::Admin& admin)
     : runtime_(runtime), dispatcher_(dispatcher), random_(random), local_info_(local_info),
       tls_(tls), admin_(admin) {
-  admin_.addHandler("/routes", "print out currently loaded dynamic HTTP route tables",
-                    MAKE_ADMIN_HANDLER(RouteConfigProviderManagerImpl::handlerRoutes), true, false);
+  config_tracker_entry_ =
+      admin_.getConfigTracker().add("routes", [this] { return dumpRouteConfigs(); });
+  // ConfigTracker keys must be unique. We are asserting that no one has stolen the "routes" key
+  // from us, since the returned entry will be nullptr if the key already exists.
+  RELEASE_ASSERT(config_tracker_entry_);
 }
 
-RouteConfigProviderManagerImpl::~RouteConfigProviderManagerImpl() {
-  admin_.removeHandler("/routes");
-}
-
-std::vector<RdsRouteConfigProviderSharedPtr>
-RouteConfigProviderManagerImpl::rdsRouteConfigProviders() {
-  std::vector<RdsRouteConfigProviderSharedPtr> ret;
+std::vector<RouteConfigProviderSharedPtr>
+RouteConfigProviderManagerImpl::getRdsRouteConfigProviders() {
+  std::vector<RouteConfigProviderSharedPtr> ret;
   ret.reserve(route_config_providers_.size());
   for (const auto& element : route_config_providers_) {
     // Because the RouteConfigProviderManager's weak_ptrs only get cleaned up
     // in the RdsRouteConfigProviderImpl destructor, and the single threaded nature
     // of this code, locking the weak_ptr will not fail.
-    RdsRouteConfigProviderSharedPtr provider = element.second.lock();
+    RouteConfigProviderSharedPtr provider = element.second.lock();
     ASSERT(provider)
     ret.push_back(provider);
   }
   return ret;
 };
 
-Router::RouteConfigProviderSharedPtr RouteConfigProviderManagerImpl::getRouteConfigProvider(
+std::vector<RouteConfigProviderSharedPtr>
+RouteConfigProviderManagerImpl::getStaticRouteConfigProviders() {
+  std::vector<RouteConfigProviderSharedPtr> providers_strong;
+  // Collect non-expired providers.
+  std::transform(static_route_config_providers_.begin(), static_route_config_providers_.end(),
+                 providers_strong.begin(), [](auto&& weak) { return weak.lock(); });
+
+  // Replace our stored list of weak_ptrs with the filtered list.
+  static_route_config_providers_.assign(providers_strong.begin(), providers_strong.begin());
+
+  return providers_strong;
+};
+
+Router::RouteConfigProviderSharedPtr RouteConfigProviderManagerImpl::getRdsRouteConfigProvider(
     const envoy::config::filter::network::http_connection_manager::v2::Rds& rds,
     Upstream::ClusterManager& cm, Stats::Scope& scope, const std::string& stat_prefix,
     Init::Manager& init_manager) {
@@ -203,55 +216,27 @@ Router::RouteConfigProviderSharedPtr RouteConfigProviderManagerImpl::getRouteCon
   return new_provider;
 };
 
-Http::Code RouteConfigProviderManagerImpl::handlerRoutes(absl::string_view url, Http::HeaderMap&,
-                                                         Buffer::Instance& response) {
-  Http::Utility::QueryParams query_params = Http::Utility::parseQueryString(url);
-  // If there are no query params, print out all the configured route tables.
-  if (query_params.size() == 0) {
-    return handlerRoutesLoop(response, rdsRouteConfigProviders());
-  }
-
-  // If there are query params, make sure it is only the route_config_name param.
-  const auto it = query_params.find("route_config_name");
-  if (query_params.size() == 1 && it != query_params.end()) {
-    // Create a vector with all the providers that have the queried route_config_name.
-    std::vector<RdsRouteConfigProviderSharedPtr> selected_providers;
-    for (const auto& provider : rdsRouteConfigProviders()) {
-      if (provider->routeConfigName() == it->second) {
-        selected_providers.push_back(provider);
-      }
-    }
-    return handlerRoutesLoop(response, selected_providers);
-  }
-  response.add("{\n");
-  response.add("    \"general_usage\": \"/routes (dump all dynamic HTTP route tables).\",\n");
-  response.add("    \"specify_name_usage\": \"/routes?route_config_name=<name> (dump all dynamic "
-               "HTTP route tables with the "
-               "<name> if any).\"\n");
-  response.add("}");
-  return Http::Code::NotFound;
+RouteConfigProviderSharedPtr RouteConfigProviderManagerImpl::getStaticRouteConfigProvider(
+    const envoy::api::v2::RouteConfiguration& route_config, Runtime::Loader& runtime,
+    Upstream::ClusterManager& cm) {
+  auto provider =
+      std::make_shared<StaticRouteConfigProviderImpl>(std::move(route_config), runtime, cm);
+  static_route_config_providers_.push_back(provider);
+  return provider;
 }
 
-Http::Code RouteConfigProviderManagerImpl::handlerRoutesLoop(
-    Buffer::Instance& response, const std::vector<RdsRouteConfigProviderSharedPtr> providers) {
-  bool first_item = true;
-  response.add("[\n");
-  for (const auto& provider : providers) {
-    if (!first_item) {
-      response.add(",");
-    } else {
-      first_item = false;
-    }
-    response.add("{\n");
-    response.add(fmt::format("\"version_info\": \"{}\",\n", provider->versionInfo()));
-    response.add(fmt::format("\"route_config_name\": \"{}\",\n", provider->routeConfigName()));
-    response.add(fmt::format("\"config_source\": {},\n", provider->configSource()));
-    response.add("\"route_table_dump\": ");
-    response.add(fmt::format("{}\n", provider->configAsJson()));
-    response.add("}\n");
+ProtobufTypes::MessagePtr RouteConfigProviderManagerImpl::dumpRouteConfigs() {
+  auto config_dump = std::make_unique<envoy::admin::v2::RouteConfigDump>();
+  auto* const dynamic_configs = config_dump->mutable_dynamic_route_configs();
+  for (const auto& provider : getRdsRouteConfigProviders()) {
+    dynamic_configs->Add()->MergeFrom(provider->configAsProto());
   }
-  response.add("]\n");
-  return Http::Code::OK;
+  auto* const static_configs = config_dump->mutable_static_route_configs();
+  for (const auto& provider : getStaticRouteConfigProviders()) {
+    static_configs->Add()->MergeFrom(provider->configAsProto());
+  }
+  return ProtobufTypes::MessagePtr{std::move(config_dump)};
 }
+
 } // namespace Router
 } // namespace Envoy