Skip to content

Commit

Permalink
Refactor PostPolicy implementation
Browse files Browse the repository at this point in the history
  • Loading branch information
@balamurugana
balamurugana committed Nov 27, 2020
1 parent d266a15 commit 7c4904a
Show file tree
Hide file tree
Showing 10 changed files with 326 additions and 334 deletions.
94 changes: 40 additions & 54 deletions docs/API.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,37 +30,37 @@ s3Client = Minio(
)
```

| Bucket operations | Object operations | Presigned operations |
|:------------------------------------------------------------|:----------------------------------------------------------------|:--------------------------------------------------|
| [`make_bucket`](#make_bucket) | [`get_object`](#get_object) | [`presigned_get_object`](#presigned_get_object) |
| [`list_buckets`](#list_buckets) | [`put_object`](#put_object) | [`presigned_put_object`](#presigned_put_object) |
| [`bucket_exists`](#bucket_exists) | [`copy_object`](#copy_object) | [`presigned_post_policy`](#presigned_post_policy) |
| [`remove_bucket`](#remove_bucket) | [`stat_object`](#stat_object) | |
| [`list_objects`](#list_objects) | [`remove_object`](#remove_object) | |
| [`get_bucket_versioning`](#get_bucket_versioning) | [`remove_objects`](#remove_objects) | |
| [`set_bucket_versioning`](#set_bucket_versioning) | [`fput_object`](#fput_object) | |
| [`delete_bucket_replication`](#delete_bucket_replication) | [`fget_object`](#fget_object) | |
| [`get_bucket_replication`](#get_bucket_replication) | [`select_object_content`](#select_object_content) | |
| [`set_bucket_replication`](#set_bucket_replication) | [`delete_object_tags`](#delete_object_tags) | |
| [`delete_bucket_lifecycle`](#delete_bucket_lifecycle) | [`get_object_tags`](#get_object_tags) | |
| [`get_bucket_lifecycle`](#get_bucket_lifecycle) | [`set_object_tags`](#set_object_tags) | |
| [`set_bucket_lifecycle`](#set_bucket_lifecycle) | [`enable_object_legal_hold`](#enable_object_legal_hold) | |
| [`delete_bucket_tags`](#delete_bucket_tags) | [`disable_object_legal_hold`](#disable_object_legal_hold) | |
| [`get_bucket_tags`](#get_bucket_tags) | [`is_object_legal_hold_enabled`](#is_object_legal_hold_enabled) | |
| [`set_bucket_tags`](#set_bucket_tags) | [`get_object_retention`](#get_object_retention) | |
| [`delete_bucket_policy`](#delete_bucket_policy) | [`set_object_retention`](#set_object_retention) | |
| [`get_bucket_policy`](#get_bucket_policy) | | |
| [`set_bucket_policy`](#set_bucket_policy) | | |
| [`delete_bucket_notification`](#delete_bucket_notification) | | |
| [`get_bucket_notification`](#get_bucket_notification) | | |
| [`set_bucket_notification`](#set_bucket_notification) | | |
| [`listen_bucket_notification`](#listen_bucket_notification) | | |
| [`delete_bucket_encryption`](#delete_bucket_encryption) | | |
| [`get_bucket_encryption`](#get_bucket_encryption) | | |
| [`set_bucket_encryption`](#set_bucket_encryption) | | |
| [`delete_object_lock_config`](#delete_object_lock_config) | | |
| [`get_object_lock_config`](#get_object_lock_config) | | |
| [`set_object_lock_config`](#set_object_lock_config) | | |
| Bucket operations | Object operations |
|:------------------------------------------------------------|:----------------------------------------------------------------|
| [`make_bucket`](#make_bucket) | [`get_object`](#get_object) |
| [`list_buckets`](#list_buckets) | [`put_object`](#put_object) |
| [`bucket_exists`](#bucket_exists) | [`copy_object`](#copy_object) |
| [`remove_bucket`](#remove_bucket) | [`stat_object`](#stat_object) |
| [`list_objects`](#list_objects) | [`remove_object`](#remove_object) |
| [`get_bucket_versioning`](#get_bucket_versioning) | [`remove_objects`](#remove_objects) |
| [`set_bucket_versioning`](#set_bucket_versioning) | [`fput_object`](#fput_object) |
| [`delete_bucket_replication`](#delete_bucket_replication) | [`fget_object`](#fget_object) |
| [`get_bucket_replication`](#get_bucket_replication) | [`select_object_content`](#select_object_content) |
| [`set_bucket_replication`](#set_bucket_replication) | [`delete_object_tags`](#delete_object_tags) |
| [`delete_bucket_lifecycle`](#delete_bucket_lifecycle) | [`get_object_tags`](#get_object_tags) |
| [`get_bucket_lifecycle`](#get_bucket_lifecycle) | [`set_object_tags`](#set_object_tags) |
| [`set_bucket_lifecycle`](#set_bucket_lifecycle) | [`enable_object_legal_hold`](#enable_object_legal_hold) |
| [`delete_bucket_tags`](#delete_bucket_tags) | [`disable_object_legal_hold`](#disable_object_legal_hold) |
| [`get_bucket_tags`](#get_bucket_tags) | [`is_object_legal_hold_enabled`](#is_object_legal_hold_enabled) |
| [`set_bucket_tags`](#set_bucket_tags) | [`get_object_retention`](#get_object_retention) |
| [`delete_bucket_policy`](#delete_bucket_policy) | [`set_object_retention`](#set_object_retention) |
| [`get_bucket_policy`](#get_bucket_policy) | [`presigned_get_object`](#presigned_get_object) |
| [`set_bucket_policy`](#set_bucket_policy) | [`presigned_put_object`](#presigned_put_object) |
| [`delete_bucket_notification`](#delete_bucket_notification) | [`presigned_post_policy`](#presigned_post_policy) |
| [`get_bucket_notification`](#get_bucket_notification) | |
| [`set_bucket_notification`](#set_bucket_notification) | |
| [`listen_bucket_notification`](#listen_bucket_notification) | |
| [`delete_bucket_encryption`](#delete_bucket_encryption) | |
| [`get_bucket_encryption`](#get_bucket_encryption) | |
| [`set_bucket_encryption`](#set_bucket_encryption) | |
| [`delete_object_lock_config`](#delete_object_lock_config) | |
| [`get_object_lock_config`](#get_object_lock_config) | |
| [`set_object_lock_config`](#set_object_lock_config) | |

## 1. Constructor

Expand Down Expand Up @@ -1418,15 +1418,15 @@ print(url)

<a name="presigned_post_policy"></a>

### presigned_post_policy(post_policy)
### presigned_post_policy(policy)

Get form-data of PostPolicy of an object to upload its data using POST method.

__Parameters__

| Param | Type | Description |
|:--------------|:-------------|:-------------|
| `post_policy` | _PostPolicy_ | Post policy. |
| Param | Type | Description |
|:---------|:-------------|:-------------|
| `policy` | _PostPolicy_ | Post policy. |

__Return Value__

Expand All @@ -1436,26 +1436,12 @@ __Return Value__

__Example__

Create policy:

```py
post_policy = PostPolicy()
post_policy.set_bucket_name('bucket_name')

# set object prefix for object upload.
post_policy.set_key_startswith('objectPrefix/')

# set expiry to 10 days.
expires_date = datetime.utcnow()+timedelta(days=10)
post_policy.set_expires(expires_date)

# set content length for incoming uploads.
post_policy.set_content_length_range(10, 1024)

# set content-type to allow only text.
post_policy.set_content_type('text/plain')

form_data = presigned_post_policy(post_policy)
policy = PostPolicy(
"bucket_name", datetime.utcnow() + timedelta(days=10),
)
policy.add_starts_with_condition("key", "objectPrefix/")
form_data = presigned_post_policy(policy)
print(form_data)
```

Expand Down
29 changes: 18 additions & 11 deletions examples/presigned_get_object.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,24 @@
# Note: YOUR-ACCESSKEYID, YOUR-SECRETACCESSKEY, my-bucketname and my-objectname
# are dummy values, please replace them with original values.

from datetime import timedelta

from minio import Minio
from minio.error import ResponseError

client = Minio('s3.amazonaws.com',
access_key='YOUR-ACCESSKEYID',
secret_key='YOUR-SECRETACCESSKEY')
client = Minio(
"s3.amazonaws.com",
access_key="YOUR-ACCESSKEYID",
secret_key="YOUR-SECRETACCESSKEY",
)

# Get presigned URL string to download 'my-objectname' in
# 'my-bucketname' with default expiry (i.e. 7 days).
url = client.presigned_get_object("my-bucketname", "my-objectname")
print(url)

# presigned get object URL for object name, expires in 7 days.
try:
print(client.presigned_get_object('my-bucketname', 'my-objectname'))
# Response error is still possible since internally presigned does get
# bucket location.
except ResponseError as err:
print(err)
# Get presigned URL string to download 'my-objectname' in
# 'my-bucketname' with two hours expiry.
url = client.presigned_get_object(
"my-bucketname", "my-objectname", expires=timedelta(hours=2),
)
print(url)
57 changes: 25 additions & 32 deletions examples/presigned_post_policy.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,35 +20,28 @@

from datetime import datetime, timedelta

from minio import Minio, PostPolicy
from minio.error import ResponseError

post_policy = PostPolicy()
# set bucket name location for uploads.
post_policy.set_bucket_name('my-bucketname')
# set key prefix for all incoming uploads.
post_policy.set_key_startswith('my-objectname')
# set content length for incoming uploads.
post_policy.set_content_length_range(10, 1024)

# set expiry 10 days into future.
expires_date = datetime.utcnow() + timedelta(days=10)
post_policy.set_expires(expires_date)

client = Minio('s3.amazonaws.com',
access_key='YOUR-ACCESSKEYID',
secret_key='YOUR-SECRETACCESSKEY')

try:
url, signed_form_data = client.presigned_post_policy(post_policy)

curl_cmd = (
['curl -X POST {0}'.format(url)] +
['-F {0}={1}'.format(k, v) for k, v in signed_form_data.items()] +
['-F file=@<FILE>']
)

# print curl command to upload files.
print(' '.join(curl_cmd))
except ResponseError as err:
print(err)
from minio import Minio
from minio.datatypes import PostPolicy

client = Minio(
"s3.amazonaws.com",
access_key="YOUR-ACCESSKEYID",
secret_key="YOUR-SECRETACCESSKEY",
)

policy = PostPolicy(
"bucket_name", datetime.utcnow() + timedelta(days=10),
)
policy.add_starts_with_condition("key", "objectPrefix/")
policy.add_content_length_range_condition(1*1024*1024, 10*1024*1024)

form_data = client.presigned_post_policy(policy)

curl_cmd = (
"curl -X POST "
"https://s3.amazonaws.com/bucket_name "
"{0} -F file=@<FILE>"
).format(
" ".join(["-F {0}={1}".format(k, v) for k, v in form_data.items()]),
)
print(curl_cmd)
31 changes: 17 additions & 14 deletions examples/presigned_put_object.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,21 +17,24 @@
# Note: YOUR-ACCESSKEYID, YOUR-SECRETACCESSKEY, my-bucketname and my-objectname
# are dummy values, please replace them with original values.

import datetime
from datetime import timedelta

from minio import Minio
from minio.error import ResponseError

client = Minio('s3.amazonaws.com',
access_key='YOUR-ACCESSKEYID',
secret_key='YOUR-SECRETACCESSKEY')
client = Minio(
"s3.amazonaws.com",
access_key="YOUR-ACCESSKEYID",
secret_key="YOUR-SECRETACCESSKEY",
)

# presigned Put object URL for an object name, expires in 3 days.
try:
print(client.presigned_put_object('my-bucketname',
'my-objectname',
datetime.timedelta(days=3)))
# Response error is still possible since internally presigned does get
# bucket location.
except ResponseError as err:
print(err)
# Get presigned URL string to upload data to 'my-objectname' in
# 'my-bucketname' with default expiry (i.e. 7 days).
url = client.presigned_put_object("my-bucketname", "my-objectname")
print(url)

# Get presigned URL string to upload data to 'my-objectname' in
# 'my-bucketname' with two hours expiry.
url = client.presigned_put_object(
"my-bucketname", "my-objectname", expires=timedelta(hours=2),
)
print(url)
1 change: 0 additions & 1 deletion minio/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -41,4 +41,3 @@
from .api import Minio
from .copy_conditions import CopyConditions
from .error import InvalidResponseError, S3Error, ServerError
from .post_policy import PostPolicy
61 changes: 14 additions & 47 deletions minio/api.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@
from .credentials import StaticProvider
from .datatypes import (CompleteMultipartUploadResult, ListAllMyBucketsResult,
ListMultipartUploadsResult, ListPartsResult, Object,
Part, parse_list_objects)
Part, PostPolicy, parse_list_objects)
from .deleteobjects import DeleteError, DeleteRequest, DeleteResult
from .error import InvalidResponseError, S3Error, ServerError
from .helpers import (BaseURL, ObjectWriteResult, ThreadPool,
Expand All @@ -64,8 +64,7 @@
from .retention import Retention
from .select import SelectObjectReader
from .selectrequest import SelectRequest
from .signer import (SIGN_V4_ALGORITHM, get_credential_string,
post_presign_v4, presign_v4, sign_v4_s3)
from .signer import presign_v4, sign_v4_s3
from .sse import SseCustomerKey
from .sseconfig import SSEConfig
from .tagging import Tagging
Expand Down Expand Up @@ -1763,63 +1762,31 @@ def presigned_put_object(self, bucket_name, object_name,
object_name,
expires)

def presigned_post_policy(self, post_policy):
def presigned_post_policy(self, policy):
"""
Get form-data of PostPolicy of an object to upload its data using POST
method.
:param post_policy: :class:`PostPolicy <PostPolicy>`.
:param policy: :class:`PostPolicy <PostPolicy>`.
:return: :dict: contains form-data.
Example::
post_policy = PostPolicy()
post_policy.set_bucket_name('bucket_name')
post_policy.set_key_startswith('objectPrefix/')
expires_date = datetime.utcnow() + timedelta(days=10)
post_policy.set_expires(expires_date)
form_data = presigned_post_policy(post_policy)
policy = PostPolicy(
"bucket_name", datetime.utcnow() + timedelta(days=10),
)
policy.add_starts_with_condition("key", "objectPrefix/")
form_data = presigned_post_policy(policy)
print(form_data)
"""
post_policy.is_valid()
if not isinstance(policy, PostPolicy):
raise ValueError("policy must be PostPolicy type")
if not self._provider:
raise ValueError(
"anonymous access does not require presigned post form-data",
)

bucket_name = post_policy.form_data['bucket']
region = self._get_region(bucket_name, None)
credentials = self._provider.retrieve()
date = time.utcnow()
credential_string = get_credential_string(
credentials.access_key, date, region,
)
policy = [
('eq', '$x-amz-date', time.to_amz_date(date)),
('eq', '$x-amz-algorithm', SIGN_V4_ALGORITHM),
('eq', '$x-amz-credential', credential_string),
]
if credentials.session_token:
policy.append(
('eq', '$x-amz-security-token', credentials.session_token),
)
post_policy_base64 = post_policy.base64(extras=policy)
signature = post_presign_v4(
post_policy_base64, credentials, date, region,
)
form_data = {
'policy': post_policy_base64,
'x-amz-algorithm': SIGN_V4_ALGORITHM,
'x-amz-credential': credential_string,
'x-amz-date': time.to_amz_date(date),
'x-amz-signature': signature,
}
if credentials.session_token:
form_data['x-amz-security-token'] = credentials.session_token
post_policy.form_data.update(form_data)
return (
self._base_url.build("POST", region, bucket_name),
post_policy.form_data,
return policy.form_data(
self._provider.retrieve(),
self._get_region(policy.bucket_name, None),
)

def delete_bucket_replication(self, bucket_name):
Expand Down
Loading

0 comments on commit 7c4904a

Please sign in to comment.