Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor PostPolicy implementation #1023

Merged
merged 3 commits into from
Dec 1, 2020
Merged

Refactor PostPolicy implementation #1023

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d622819
Refactor PostPolicy implementation
balamurugana Nov 27, 2020
edc5751
pythonic way to construct form-data and address review comments
balamurugana Nov 28, 2020
bae13be
Merge branch 'master' into Refactor-PostPolicy-implementation
balamurugana Dec 1, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 43 additions & 57 deletions docs/API.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,37 +30,37 @@ s3Client = Minio(
)
```

| Bucket operations | Object operations | Presigned operations |
|:------------------------------------------------------------|:----------------------------------------------------------------|:--------------------------------------------------|
| [`make_bucket`](#make_bucket) | [`get_object`](#get_object) | [`presigned_get_object`](#presigned_get_object) |
| [`list_buckets`](#list_buckets) | [`put_object`](#put_object) | [`presigned_put_object`](#presigned_put_object) |
| [`bucket_exists`](#bucket_exists) | [`copy_object`](#copy_object) | [`presigned_post_policy`](#presigned_post_policy) |
| [`remove_bucket`](#remove_bucket) | [`stat_object`](#stat_object) | |
| [`list_objects`](#list_objects) | [`remove_object`](#remove_object) | |
| [`get_bucket_versioning`](#get_bucket_versioning) | [`remove_objects`](#remove_objects) | |
| [`set_bucket_versioning`](#set_bucket_versioning) | [`fput_object`](#fput_object) | |
| [`delete_bucket_replication`](#delete_bucket_replication) | [`fget_object`](#fget_object) | |
| [`get_bucket_replication`](#get_bucket_replication) | [`select_object_content`](#select_object_content) | |
| [`set_bucket_replication`](#set_bucket_replication) | [`delete_object_tags`](#delete_object_tags) | |
| [`delete_bucket_lifecycle`](#delete_bucket_lifecycle) | [`get_object_tags`](#get_object_tags) | |
| [`get_bucket_lifecycle`](#get_bucket_lifecycle) | [`set_object_tags`](#set_object_tags) | |
| [`set_bucket_lifecycle`](#set_bucket_lifecycle) | [`enable_object_legal_hold`](#enable_object_legal_hold) | |
| [`delete_bucket_tags`](#delete_bucket_tags) | [`disable_object_legal_hold`](#disable_object_legal_hold) | |
| [`get_bucket_tags`](#get_bucket_tags) | [`is_object_legal_hold_enabled`](#is_object_legal_hold_enabled) | |
| [`set_bucket_tags`](#set_bucket_tags) | [`get_object_retention`](#get_object_retention) | |
| [`delete_bucket_policy`](#delete_bucket_policy) | [`set_object_retention`](#set_object_retention) | |
| [`get_bucket_policy`](#get_bucket_policy) | | |
| [`set_bucket_policy`](#set_bucket_policy) | | |
| [`delete_bucket_notification`](#delete_bucket_notification) | | |
| [`get_bucket_notification`](#get_bucket_notification) | | |
| [`set_bucket_notification`](#set_bucket_notification) | | |
| [`listen_bucket_notification`](#listen_bucket_notification) | | |
| [`delete_bucket_encryption`](#delete_bucket_encryption) | | |
| [`get_bucket_encryption`](#get_bucket_encryption) | | |
| [`set_bucket_encryption`](#set_bucket_encryption) | | |
| [`delete_object_lock_config`](#delete_object_lock_config) | | |
| [`get_object_lock_config`](#get_object_lock_config) | | |
| [`set_object_lock_config`](#set_object_lock_config) | | |
| Bucket operations | Object operations |
|:------------------------------------------------------------|:----------------------------------------------------------------|
| [`make_bucket`](#make_bucket) | [`get_object`](#get_object) |
| [`list_buckets`](#list_buckets) | [`put_object`](#put_object) |
| [`bucket_exists`](#bucket_exists) | [`copy_object`](#copy_object) |
| [`remove_bucket`](#remove_bucket) | [`stat_object`](#stat_object) |
| [`list_objects`](#list_objects) | [`remove_object`](#remove_object) |
| [`get_bucket_versioning`](#get_bucket_versioning) | [`remove_objects`](#remove_objects) |
| [`set_bucket_versioning`](#set_bucket_versioning) | [`fput_object`](#fput_object) |
| [`delete_bucket_replication`](#delete_bucket_replication) | [`fget_object`](#fget_object) |
| [`get_bucket_replication`](#get_bucket_replication) | [`select_object_content`](#select_object_content) |
| [`set_bucket_replication`](#set_bucket_replication) | [`delete_object_tags`](#delete_object_tags) |
| [`delete_bucket_lifecycle`](#delete_bucket_lifecycle) | [`get_object_tags`](#get_object_tags) |
| [`get_bucket_lifecycle`](#get_bucket_lifecycle) | [`set_object_tags`](#set_object_tags) |
| [`set_bucket_lifecycle`](#set_bucket_lifecycle) | [`enable_object_legal_hold`](#enable_object_legal_hold) |
| [`delete_bucket_tags`](#delete_bucket_tags) | [`disable_object_legal_hold`](#disable_object_legal_hold) |
| [`get_bucket_tags`](#get_bucket_tags) | [`is_object_legal_hold_enabled`](#is_object_legal_hold_enabled) |
| [`set_bucket_tags`](#set_bucket_tags) | [`get_object_retention`](#get_object_retention) |
| [`delete_bucket_policy`](#delete_bucket_policy) | [`set_object_retention`](#set_object_retention) |
| [`get_bucket_policy`](#get_bucket_policy) | [`presigned_get_object`](#presigned_get_object) |
| [`set_bucket_policy`](#set_bucket_policy) | [`presigned_put_object`](#presigned_put_object) |
| [`delete_bucket_notification`](#delete_bucket_notification) | [`presigned_post_policy`](#presigned_post_policy) |
| [`get_bucket_notification`](#get_bucket_notification) | |
| [`set_bucket_notification`](#set_bucket_notification) | |
| [`listen_bucket_notification`](#listen_bucket_notification) | |
| [`delete_bucket_encryption`](#delete_bucket_encryption) | |
| [`get_bucket_encryption`](#get_bucket_encryption) | |
| [`set_bucket_encryption`](#set_bucket_encryption) | |
| [`delete_object_lock_config`](#delete_object_lock_config) | |
| [`get_object_lock_config`](#get_object_lock_config) | |
| [`set_object_lock_config`](#set_object_lock_config) | |

## 1. Constructor

Expand Down Expand Up @@ -1338,8 +1338,6 @@ config = Retention(GOVERNANCE, datetime.utcnow() + timedelta(days=10))
minio.set_object_retention("my-bucketname", "my-objectname", config)
```

## 4. Presigned operations

<a name="presigned_get_object"></a>

### presigned_get_object(bucket_name, object_name, expires=timedelta(days=7), response_headers=None, request_date=None, version_id=None, extra_query_params=None)
Expand Down Expand Up @@ -1418,15 +1416,15 @@ print(url)

<a name="presigned_post_policy"></a>

### presigned_post_policy(post_policy)
### presigned_post_policy(policy)

Get form-data of PostPolicy of an object to upload its data using POST method.

__Parameters__

| Param | Type | Description |
|:--------------|:-------------|:-------------|
| `post_policy` | _PostPolicy_ | Post policy. |
| Param | Type | Description |
|:---------|:-------------|:-------------|
| `policy` | _PostPolicy_ | Post policy. |

__Return Value__

Expand All @@ -1436,27 +1434,15 @@ __Return Value__

__Example__

Create policy:

```py
post_policy = PostPolicy()
post_policy.set_bucket_name('bucket_name')

# set object prefix for object upload.
post_policy.set_key_startswith('objectPrefix/')

# set expiry to 10 days.
expires_date = datetime.utcnow()+timedelta(days=10)
post_policy.set_expires(expires_date)

# set content length for incoming uploads.
post_policy.set_content_length_range(10, 1024)

# set content-type to allow only text.
post_policy.set_content_type('text/plain')

form_data = presigned_post_policy(post_policy)
print(form_data)
policy = PostPolicy(
"my-bucket", datetime.utcnow() + timedelta(days=10),
)
policy.add_starts_with_condition("key", "my/object/prefix/")
policy.add_content_length_range_condition(
1*1024*1024, 10*1024*1024,
)
form_data = client.presigned_post_policy(policy)
```

## 5. Explore Further
Expand Down
61 changes: 25 additions & 36 deletions examples/presigned_post_policy.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,41 +14,30 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# Note: my-bucketname, my-objectname, YOUR-ACCESSKEYID, and
# YOUR-SECRETACCESSKEY are dummy values, please replace them with original
# values.

from datetime import datetime, timedelta

from minio import Minio, PostPolicy
from minio.error import ResponseError

post_policy = PostPolicy()
# set bucket name location for uploads.
post_policy.set_bucket_name('my-bucketname')
# set key prefix for all incoming uploads.
post_policy.set_key_startswith('my-objectname')
# set content length for incoming uploads.
post_policy.set_content_length_range(10, 1024)

# set expiry 10 days into future.
expires_date = datetime.utcnow() + timedelta(days=10)
post_policy.set_expires(expires_date)

client = Minio('s3.amazonaws.com',
access_key='YOUR-ACCESSKEYID',
secret_key='YOUR-SECRETACCESSKEY')

try:
url, signed_form_data = client.presigned_post_policy(post_policy)

curl_cmd = (
['curl -X POST {0}'.format(url)] +
['-F {0}={1}'.format(k, v) for k, v in signed_form_data.items()] +
['-F file=@<FILE>']
)

# print curl command to upload files.
print(' '.join(curl_cmd))
except ResponseError as err:
print(err)
from minio import Minio
from minio.datatypes import PostPolicy

client = Minio(
"s3.amazonaws.com",
access_key="YOUR-ACCESSKEYID",
secret_key="YOUR-SECRETACCESSKEY",
)

policy = PostPolicy(
"my-bucket", datetime.utcnow() + timedelta(days=10),
)
policy.add_starts_with_condition("key", "my/object/prefix/")
policy.add_content_length_range_condition(1*1024*1024, 10*1024*1024)

form_data = client.presigned_post_policy(policy)

curl_cmd = (
"curl -X POST "
"https://s3.amazonaws.com/my-bucket "
"{0} -F file=@<FILE>"
).format(
" ".join(["-F {0}={1}".format(k, v) for k, v in form_data.items()]),
)
print(curl_cmd)
1 change: 0 additions & 1 deletion minio/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,4 +41,3 @@
from .api import Minio
from .copy_conditions import CopyConditions
from .error import InvalidResponseError, S3Error, ServerError
from .post_policy import PostPolicy
65 changes: 17 additions & 48 deletions minio/api.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@
from .credentials import StaticProvider
from .datatypes import (CompleteMultipartUploadResult, ListAllMyBucketsResult,
ListMultipartUploadsResult, ListPartsResult, Object,
Part, parse_list_objects)
Part, PostPolicy, parse_list_objects)
from .deleteobjects import DeleteError, DeleteRequest, DeleteResult
from .error import InvalidResponseError, S3Error, ServerError
from .helpers import (BaseURL, ObjectWriteResult, ThreadPool,
Expand All @@ -64,8 +64,7 @@
from .retention import Retention
from .select import SelectObjectReader
from .selectrequest import SelectRequest
from .signer import (SIGN_V4_ALGORITHM, get_credential_string,
post_presign_v4, presign_v4, sign_v4_s3)
from .signer import presign_v4, sign_v4_s3
from .sse import SseCustomerKey
from .sseconfig import SSEConfig
from .tagging import Tagging
Expand Down Expand Up @@ -1767,63 +1766,33 @@ def presigned_put_object(self, bucket_name, object_name,
object_name,
expires)

def presigned_post_policy(self, post_policy):
def presigned_post_policy(self, policy):
"""
Get form-data of PostPolicy of an object to upload its data using POST
method.

:param post_policy: :class:`PostPolicy <PostPolicy>`.
:param policy: :class:`PostPolicy <PostPolicy>`.
:return: :dict: contains form-data.

Example::
post_policy = PostPolicy()
post_policy.set_bucket_name('bucket_name')
post_policy.set_key_startswith('objectPrefix/')
expires_date = datetime.utcnow() + timedelta(days=10)
post_policy.set_expires(expires_date)

form_data = presigned_post_policy(post_policy)
print(form_data)
policy = PostPolicy(
"my-bucket", datetime.utcnow() + timedelta(days=10),
)
policy.add_starts_with_condition("key", "my/object/prefix/")
policy.add_content_length_range_condition(
1*1024*1024, 10*1024*1024,
)
form_data = client.presigned_post_policy(policy)
"""
post_policy.is_valid()
if not isinstance(policy, PostPolicy):
raise ValueError("policy must be PostPolicy type")
if not self._provider:
raise ValueError(
"anonymous access does not require presigned post form-data",
)

bucket_name = post_policy.form_data['bucket']
region = self._get_region(bucket_name, None)
credentials = self._provider.retrieve()
date = time.utcnow()
credential_string = get_credential_string(
credentials.access_key, date, region,
)
policy = [
('eq', '$x-amz-date', time.to_amz_date(date)),
('eq', '$x-amz-algorithm', SIGN_V4_ALGORITHM),
('eq', '$x-amz-credential', credential_string),
]
if credentials.session_token:
policy.append(
('eq', '$x-amz-security-token', credentials.session_token),
)
post_policy_base64 = post_policy.base64(extras=policy)
signature = post_presign_v4(
post_policy_base64, credentials, date, region,
)
form_data = {
'policy': post_policy_base64,
'x-amz-algorithm': SIGN_V4_ALGORITHM,
'x-amz-credential': credential_string,
'x-amz-date': time.to_amz_date(date),
'x-amz-signature': signature,
}
if credentials.session_token:
form_data['x-amz-security-token'] = credentials.session_token
post_policy.form_data.update(form_data)
return (
self._base_url.build("POST", region, bucket_name),
post_policy.form_data,
return policy.form_data(
self._provider.retrieve(),
self._get_region(policy.bucket_name, None),
)

def delete_bucket_replication(self, bucket_name):
Expand Down
Loading