Implement core Smithy endpoint support (#183)

* Implement core Smithy endpoint support

This commit adds `Endpoint` to smithy-http & records our design decisions in `endpoint.md`. This provides support for the endpoint trait in Smithy. A design for endpoint discovery is proposed but is not currently implemented.

* Apply suggestions from code review

Co-authored-by: Lucio Franco <[email protected]>

* More cleanups

Co-authored-by: Lucio Franco <[email protected]>

design/src/SUMMARY.md

@@ -1,4 +1,6 @@
 # Summary
 
 - [Http Operations](./operation.md)
+- [Endpoint Resolution](./endpoint.md)
 - [HTTP middleware](./middleware.md)
+

design/src/endpoint.md

@@ -0,0 +1,51 @@
+# Endpoint Resolution
+
+## Requirements
+The core codegen generates HTTP requests that do not contain an authority, scheme or post. These properties must be set later based on configuration. Existing AWS services have a number of requirements that increase the complexity:
+
+1. Endpoints must support manual configuration by end users:
+```rust
+let config = dynamodb::Config::builder()
+    .endpoint(StaticEndpoint::for_uri("http://localhost:8000"))
+```
+
+When a user specifies a custom endpoint URI, _typically_ they will want to avoid having this URI mutated by other endpoint discovery machinery.
+
+2. Endpoints must support being customized on a per-operation basis by the endpoint trait. This will prefix the base endpoint, potentially driven by fields of the operation. [Docs](https://awslabs.github.io/smithy/1.0/spec/core/endpoint-traits.html#endpoint-trait)
+
+3. Endpoints must support being customized by [endpoint discovery](https://awslabs.github.io/smithy/1.0/spec/aws/aws-core.html#client-endpoint-discovery). A request, customized by a predefined set of fields from the input operation is dispatched to a specific URI. That operation returns the endpoint that should be used. Endpoints must be cached by a cache key containing:
+```
+(access_key_id, [all input fields], operation)
+```
+Endpoints retrieved in this way specify a TTL.
+
+4. Endpoints must be able to customize the signing (and other phases of the operation). For example, requests sent to a global region will have a region set by the endpoint provider.
+
+
+## Design
+
+Configuration objects for services _must_ contain an `Endpoint`. This endpoint may be set by a user or it will default to the `endpointPrefix` from the service definition. In the case of endpoint discovery, _this_ is the endpoint that we will start with.
+
+During operation construction (see [Operation Construction](operation.md#operation-construction)) an `EndpointPrefix` may be set on the property bag. The eventual endpoint middleware will search for this in the property bag and (depending on the URI mutability) utilize this prefix when setting the endpoint.
+
+In the case of endpoint discovery, we envision a different pattern:
+```rust
+// EndpointClient manages the endpoint cache
+let (tx, rx) = dynamodb::EndpointClient::new();
+let client = aws_hyper::Client::new();
+// `endpoint_req` is an operation that can be dispatched to retrieve endpoints
+// During operation construction, the endpoint resolver is configured to be `rx` instead static endpoint
+// resolver provided by the service.
+let (endpoint_req, req) = GetRecord::builder().endpoint_disco(rx).build_with_endpoint();
+// depending on the duration of endpoint expiration, this may be spawned into a separate task to continuously
+// refresh endpoints.
+if tx.needs(endpoint_req) {
+    let new_endpoint = client.
+        call(endpoint_req)
+        .await;
+    tx.send(new_endpoint)
+}
+let rsp = client.call(req).await?;
+```
+
+We believe that this design results in an SDK that both offers customers more control & reduces the likelihood of bugs from nested operation dispatch. Endpoint resolution is currently extremely rare in AWS services so this design may remain a prototype while we solidify other behaviors.

rust-runtime/smithy-http/src/endpoint.rs

@@ -0,0 +1,156 @@
+/*
+ * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ * SPDX-License-Identifier: Apache-2.0.
+ */
+
+use http::uri::{Authority, InvalidUri, Uri};
+use std::str::FromStr;
+
+/// API Endpoint
+///
+/// This implements an API endpoint as specified in the
+/// [Smithy Endpoint Specification](https://awslabs.github.io/smithy/1.0/spec/core/endpoint-traits.html)
+pub struct Endpoint {
+    uri: http::Uri,
+
+    /// If true, endpointPrefix does ignored when setting the endpoint on a request
+    immutable: bool,
+}
+
+pub struct EndpointPrefix(String);
+impl EndpointPrefix {
+    pub fn new(prefix: impl Into<String>) -> Result<Self, InvalidUri> {
+        let prefix = prefix.into();
+        let _ = Authority::from_str(&prefix)?;
+        Ok(EndpointPrefix(prefix))
+    }
+}
+
+#[non_exhaustive]
+#[derive(Debug, Eq, PartialEq, Clone)]
+pub enum InvalidEndpoint {
+    EndpointMustHaveAuthority,
+}
+
+// user gives: Endpoint Provider. Give endpoint. Is this endpoint extensible?
+// where endpoint discovery starts from.
+// endpoints can mutate a request, potentially with a
+
+impl Endpoint {
+    /// Create a new endpoint from a URI
+    ///
+    /// Certain protocols will attempt to prefix additional information onto an endpoint. If you
+    /// wish to ignore these prefixes (for example, when communicating with localhost), set `immutable` to `true`.
+    pub fn new(uri: Uri, immutable: bool) -> Result<Self, InvalidEndpoint> {
+        Ok(Endpoint { uri, immutable })
+    }
+
+    /// Create a new immutable endpoint from a URI
+    ///
+    /// ```rust
+    /// # use smithy_http::endpoint::Endpoint;
+    /// use http::Uri;
+    /// let endpoint = Endpoint::from_uri(Uri::from_static("http://localhost:8000"));
+    /// ```
+    pub fn from_uri(uri: Uri) -> Self {
+        Endpoint {
+            uri,
+            immutable: true,
+        }
+    }
+
+    /// Sets the endpoint on `uri`, potentially applying the specified `prefix` in the process.
+    pub fn set_endpoint(&self, uri: &mut http::Uri, prefix: Option<&EndpointPrefix>) {
+        let prefix = prefix.map(|p| p.0.as_str()).unwrap_or("");
+        let authority = self
+            .uri
+            .authority()
+            .as_ref()
+            .map(|auth| auth.as_str())
+            .unwrap_or("");
+        let authority = if !self.immutable && !prefix.is_empty() {
+            Authority::from_str(&format!("{}{}", prefix, authority)).expect("parts must be valid")
+        } else {
+            Authority::from_str(authority).expect("authority is valid")
+        };
+        let scheme = *self.uri.scheme().as_ref().expect("scheme must be provided");
+        let new_uri = Uri::builder()
+            .authority(authority)
+            .scheme(scheme.clone())
+            .path_and_query(uri.path_and_query().unwrap().clone())
+            .build()
+            .expect("valid uri");
+        *uri = new_uri;
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use crate::endpoint::{Endpoint, EndpointPrefix};
+    use http::Uri;
+
+    #[test]
+    fn prefix_endpoint() {
+        let ep = Endpoint::new(
+            Uri::from_static("https://us-east-1.dynamo.amazonaws.com"),
+            false,
+        )
+        .expect("valid endpoint");
+        let mut uri = Uri::from_static("/list_tables?k=v");
+        ep.set_endpoint(
+            &mut uri,
+            Some(&EndpointPrefix::new("subregion.").expect("valid prefix")),
+        );
+        assert_eq!(
+            uri,
+            Uri::from_static("https://subregion.us-east-1.dynamo.amazonaws.com/list_tables?k=v")
+        );
+    }
+
+    #[test]
+    fn prefix_endpoint_custom_port() {
+        let ep = Endpoint::new(
+            Uri::from_static("https://us-east-1.dynamo.amazonaws.com:6443"),
+            false,
+        )
+        .expect("valid endpoint");
+        let mut uri = Uri::from_static("/list_tables?k=v");
+        ep.set_endpoint(
+            &mut uri,
+            Some(&EndpointPrefix::new("subregion.").expect("valid prefix")),
+        );
+        assert_eq!(
+            uri,
+            Uri::from_static(
+                "https://subregion.us-east-1.dynamo.amazonaws.com:6443/list_tables?k=v"
+            )
+        );
+    }
+
+    #[test]
+    fn prefix_immutable_endpoint() {
+        let ep = Endpoint::new(
+            Uri::from_static("https://us-east-1.dynamo.amazonaws.com"),
+            true,
+        )
+        .expect("valid endpoint");
+        let mut uri = Uri::from_static("/list_tables?k=v");
+        ep.set_endpoint(
+            &mut uri,
+            Some(&EndpointPrefix::new("subregion.").expect("valid prefix")),
+        );
+        assert_eq!(
+            uri,
+            Uri::from_static("https://us-east-1.dynamo.amazonaws.com/list_tables?k=v")
+        );
+    }
+
+    #[test]
+    fn set_endpoint_empty_path() {
+        let ep =
+            Endpoint::new(Uri::from_static("http://localhost:8000"), true).expect("valid endpoint");
+        let mut uri = Uri::from_static("/");
+        ep.set_endpoint(&mut uri, None);
+        assert_eq!(uri, Uri::from_static("http://localhost:8000/"))
+    }
+}

rust-runtime/smithy-http/src/lib.rs

@@ -5,6 +5,7 @@
 
 pub mod base64;
 pub mod body;
+pub mod endpoint;
 pub mod label;
 pub mod middleware;
 pub mod operation;