Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Object retention put/get APIs #923

Merged
merged 5 commits into from
May 13, 2021

Conversation

prakashsvmx
Copy link
Member

@prakashsvmx prakashsvmx commented Apr 22, 2021

Object retention APIs.

Note: this depends on #919

Test Script

var Minio = require("../../dist/main/minio")

var minioClient = new Minio.Client({
endPoint:"127.0.0.1",
port:9000,
useSSL:false,
accessKey:"minio",
secretKey :"minio123"
})
const file = '/home/prakash/Downloads/HouseDesign/E00000000833.jpg'

function makeBucketWithObjectLock (buckName,makeOpts,cb){
const bucketTaggingPromise = minioClient.makeBucket(buckName,makeOpts)
bucketTaggingPromise.then(() => {
console.log("makeBucketWithObjectLock success...")
cb && cb()
})
.catch((e)=>{
console.log("makeBucketWithObjectLock Error",e)
})
}

function setObjectLockConfig (buckName,lockOpts,cb){
const bucketTaggingPromise = minioClient.setObjectLockConfig(buckName,lockOpts)
bucketTaggingPromise.then(() => {
console.log("setObjectLockConfig success...")
cb && cb()
})
.catch((e)=>{
console.log("setObjectLockConfig Error",e)
})

}

function getObjectLockConfig (buckName,cb){
const bucketTaggingPromise = minioClient.getObjectLockConfig(buckName)
bucketTaggingPromise.then((e, data) => {
console.log("getObjectLockConfig success...", e, JSON.stringify(data))
cb && cb()
})
.catch((e)=>{
console.log("getObjectLockConfig Error",e)
})
}

function putObject (buckName,objName) {
var metaData = {
'Content-Type': 'application/octet-stream',
'X-Amz-Meta-Testing': 1234,
'example': 5678
}

const fPromise = minioClient.fPutObject(buckName, objName, file,metaData)

fPromise.then(() => {
console.log("Put object Success...")
})
.catch(()=>{
console.log("Put object Error")
})

}

function putObjectRetention (buckName,objectName, retentionOpts,cb){

const bucketTaggingPromise = minioClient.putObjectRetention(buckName,objectName,retentionOpts)
bucketTaggingPromise.then(() => {
console.log("Promise working...")
cb && cb()
})
.catch((e)=>{
console.log("Promise Error",e)
})

}

function getObjectRetention(buckName,objectName,getOpts){

const bucketTaggingPromise = minioClient.getObjectRetention(buckName,objectName,getOpts, )
bucketTaggingPromise.then((data) => {
console.log("Get retention working...", data)
})
.catch((e)=>{
console.log("Promise Error",e)
})
}

function removeObject (bucketName, objectName, removeOpts){
const delObj= minioClient.removeObject(bucketName, objectName, removeOpts)
delObj.then(()=>{
console.log("Removed object")
})
}

function removeBucket (bucketName){
const delObj= minioClient.removeBucket(bucketName)
delObj.then(()=>{
console.log("Removed bucket")
})
}
const bucketName = 'sph-my-bucket'
const objName ="E00000000833"

const expirationDate = new Date('2021-08-02')
expirationDate.setUTCHours(0,0,0,0)//Should be start of the day.(midnight)
const versionId ="OjYzf_hoYoAfeEYO3Ymc8_bToDH6F0Cb"
//makeBucketWithObjectLock(bucketName,{ObjectLocking:true})
//putObject(bucketName,objName)
/putObjectRetention(bucketName,objName,{
governanceBypass:true,
mode:"GOVERNANCE",
retainUntilDate:expirationDate.toISOString(),
versionId:versionId
})
/
//getObjectRetention(bucketName,objName,{versionId:versionId})
//removeObject(bucketName,objName,{versionId:versionId, governanceBypass:true})
//removeBucket(bucketName)

Signed-off-by: prakashsvmx <[email protected]>
docs/API.md Outdated Show resolved Hide resolved
docs/API.md Outdated Show resolved Hide resolved
docs/API.md Outdated Show resolved Hide resolved
docs/API.md Outdated Show resolved Hide resolved
docs/API.md Outdated Show resolved Hide resolved
docs/API.md Outdated Show resolved Hide resolved
docs/API.md Outdated Show resolved Hide resolved
expirationDate.setUTCHours(0,0,0,0)//Should be start of the day.(midnight)
const versionId ="e67b4b08-144d-4fc4-ba15-43c3f7f9ba74"

const objRetPromise = s3Client.putObjectRetention(bucketName,objectName,{governanceBypass:true,Mode:"GOVERNANCE",retainUntilDate:expirationDate.toISOString(),versionID:versionId} )
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of having the governanceBypass: true parameter here in this test?

Are we going to bypass an existing retention lock, ("governance" mode) on an object to do ... what?

If so, please provide some comments to make the test's purpose clear.

If there is no reason, then you can remove the bypass parameter from the examples.

Additionally, I get the following error when I run the put-object-retention test against minio or s3:

s3

 Error S3Error: The XML you provided was not well-formed or did not validate against our published schema
    at Object.parseError (/home/ersan/work/src/github.com/minio/minio-js/dist/main/xml-parsers.js:84:11)
    at /home/ersan/work/src/github.com/minio/minio-js/dist/main/transformers.js:161:22
    at DestroyableTransform._flush (/home/ersan/work/src/github.com/minio/minio-js/dist/main/transformers.js:83:10)
    at DestroyableTransform.prefinish (/home/ersan/work/src/github.com/minio/minio-js/node_modules/readable-stream/lib/_stream_transform.js:138:10)
    at DestroyableTransform.emit (events.js:315:20)
    at prefinish (/home/ersan/work/src/github.com/minio/minio-js/node_modules/readable-stream/lib/_stream_writable.js:619:14)
    at finishMaybe (/home/ersan/work/src/github.com/minio/minio-js/node_modules/readable-stream/lib/_stream_writable.js:627:5)
    at endWritable (/home/ersan/work/src/github.com/minio/minio-js/node_modules/readable-stream/lib/_stream_writable.js:638:3)
    at DestroyableTransform.Writable.end (/home/ersan/work/src/github.com/minio/minio-js/node_modules/readable-stream/lib/_stream_writable.js:594:41)
    at IncomingMessage.onend (_stream_readable.js:678:10) {
  code: 'MalformedXML',
  requestid: '84KHKDN9Q4Y8HJSE',
  hostid: 'eJtN9a+BOsVa7N4v/KEYuaaTFL+eyLvV77quPeJp2dXlCvF8/jqfJ2EROn/WkixhNY100XgODzY=',
  amzRequestid: null,
  amzId2: null,
  amzBucketRegion: null
}

minio

 Error S3Error: the XML you provided was not well-formed or did not validate against our published schema
    at Object.parseError (/home/ersan/work/src/github.com/minio/minio-js/dist/main/xml-parsers.js:84:11)
    at /home/ersan/work/src/github.com/minio/minio-js/dist/main/transformers.js:161:22
    at DestroyableTransform._flush (/home/ersan/work/src/github.com/minio/minio-js/dist/main/transformers.js:83:10)
    at DestroyableTransform.prefinish (/home/ersan/work/src/github.com/minio/minio-js/node_modules/readable-stream/lib/_stream_transform.js:138:10)
    at DestroyableTransform.emit (events.js:315:20)
    at prefinish (/home/ersan/work/src/github.com/minio/minio-js/node_modules/readable-stream/lib/_stream_writable.js:619:14)
    at finishMaybe (/home/ersan/work/src/github.com/minio/minio-js/node_modules/readable-stream/lib/_stream_writable.js:627:5)
    at endWritable (/home/ersan/work/src/github.com/minio/minio-js/node_modules/readable-stream/lib/_stream_writable.js:638:3)
    at DestroyableTransform.Writable.end (/home/ersan/work/src/github.com/minio/minio-js/node_modules/readable-stream/lib/_stream_writable.js:594:41)
    at IncomingMessage.onend (_stream_readable.js:678:10) {
  code: 'MalformedXML',
  key: 'file-locked-gov',
  bucketname: 'erslock',
  resource: '/erslock/file-locked-gov',
  region: 'us-east-1',
  requestid: '167953A1D590685F',
  hostid: 'a1556fb7-9cf2-409c-be38-ca6aa7f10df9',
  amzRequestid: null,
  amzId2: null,
  amzBucketRegion: null
}

I also would like to see what happens when the object is already locked when the bypass parameter is provided and set to true, or when it is not provided or set to false.

Copy link
Member Author

@prakashsvmx prakashsvmx Apr 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ebozduman The examples had typos (case sensitivity) of object keys. I have updated and tested against MinIO and S3.
however. still there is a response code mismatch as commented on the code.

Line 2426: (minio.js)

//FIXME minio Server returns 204 but AWS returns 200. So in aws, even though the operation is success, error is thrown.

@prakashsvmx
Copy link
Member Author

@ebozduman thank you so much for the review. I will address them .

Signed-off-by: prakashsvmx <[email protected]>
ebozduman
ebozduman previously approved these changes Apr 29, 2021
Copy link
Contributor

@ebozduman ebozduman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

kannappanr
kannappanr previously approved these changes May 2, 2021
Copy link
Contributor

@kannappanr kannappanr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

examples/put-object-retention.js Outdated Show resolved Hide resolved
Copy link
Contributor

@kannappanr kannappanr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@ebozduman ebozduman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kannappanr kannappanr merged commit 64386cb into minio:master May 13, 2021
@prakashsvmx prakashsvmx deleted the obj-retention-apis branch February 10, 2022 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants