Merge pull request Azure#14 from esummers-msft/master

Java Storage Client Private 4.3.0

CONTRIBUTING.md

@@ -0,0 +1,50 @@
+If you intend to contribute to the project, please make sure you've followed the instructions provided in the [Azure Projects Contribution Guidelines](http://azure.github.io/guidelines/).
+## Project Setup
+The Azure Storage development team uses Eclipse so instructions will be tailored to that preference. However, any preferred IDE or other toolset should be usable.
+
+### Install
+* Java SE 6+
+* [Eclipse](https://eclipse.org/downloads/)
+* [Maven plugin for Eclipse](http://www.eclipse.org/m2e/index.html). Some Eclipse packages (ex Eclipse IDE for Java Developers) may come with this plugin already installed.
+* [Maven](https://maven.apache.org/install.html)
+* Clone the source code from GitHub
+
+### Open Solution
+Open the project from Eclipse using File->Import->Maven->Existing Maven Projects and navigating to the azure-storage-java folder. Select the listed pom. This imports the source and the test files and downloads the required dependencies via Maven. If you'd like to import the samples, follow the same procedure but navigate to the azure-storage-java\microsoft-azure-storage-samples folder and select that pom. Both projects can be opened at the same time and will be shown in the Package Explorer.
+
+## Tests
+
+### Configuration
+The only step to configure testing is to setup a configuration file or connection string via environment variables. To use the connection string route, create an environment variable named "storageConnection". To use the configuration file route, create an environment variable named "storageTestConfiguration" with the path to a TestConfigurations.xml file with this [template](https://github.com/Azure/azure-storage-java/blob/master/microsoft-azure-storage-test/res/TestConfigurations.xml).
+
+### Running
+To actually run tests, right click on the test class in the Package Explorer or the individual test in the Outline and select Run As->JUnitTest. All tests or tests grouped by service can be run using the test runners in the com.microsoft.azure.storage package TestRunners file. Running all tests from the top of the package explorer will result in each test being run multiple times as the package explorer will also run every test runner.
+
+### Testing Features
+As you develop a feature, you'll need to write tests to ensure quality. You should also run existing tests related to your change to address any unexpected breaks.
+
+## Pull Requests
+
+### Guidelines
+The following are the minimum requirements for any pull request that must be met before contributions can be accepted.
+* Make sure you've signed the CLA before you start working on any change.
+* Discuss any proposed contribution with the team via a GitHub issue **before** starting development.
+* Code must be professional quality
+	* No style issues
+	* You should strive to mimic the style with which we have written the library
+	* Clean, well-commented, well-designed code
+	* Try to limit the number of commits for a feature to 1-2. If you end up having too many we may ask you to squash your changes into fewer commits.
+* [ChangeLog.md](ChangeLog.md) needs to be updated describing the new change
+* Thoroughly test your feature
+
+### Branching Policy
+Changes should be based on the **dev** branch, not master as master is considered publicly released code. If after discussion with us breaking changes are considered for the library, we will create a **dev_breaking** branch based on dev which can be used to store these changes until the next breaking release. Each breaking change should be recorded in [BreakingChanges.md](BreakingChanges.md). 
+
+### Adding Features for Java 6+
+We strive to release each new feature in a backward compatible manner. Therefore, we ask that all contributions be written to work in Java 6, 7 and 8.
+
+### Review Process
+We expect all guidelines to be met before accepting a pull request. As such, we will work with you to address issues we find by leaving comments in your code. Please understand that it may take a few iterations before the code is accepted as we maintain high standards on code quality. Once we feel comfortable with a contribution, we will validate the change and accept the pull request.
+
+
+Thank you for any contributions! Please let the team know if you have any questions or concerns about our contribution policy.

ChangeLog.txt

@@ -1,3 +1,8 @@
+2016.07.06 Version 4.3.0
+ * Added support for server-side encryption.
+ * Added support for getBlobReferenceFromServer methods on CloudBlobContainer to support retrieving a blob without knowing its type.
+ * Fixed a bug in the retry policies where 300 status codes were being retried when they shouldn't be.
+
 2016.04.07 Version 4.2.0
  * Added support for setting a library-wide proxy. The default proxy can be set on OperationContext.
  * Added support in Page Blob for getting a list of differing page ranges between snapshot versions.

microsoft-azure-storage-test/res/TestConfigurations.xml

@@ -21,6 +21,7 @@
     <BlobServiceEndpoint>http://[ACCOUNT].blob.core.windows.net</BlobServiceEndpoint>
     <QueueServiceEndpoint>http://[ACCOUNT].queue.core.windows.net</QueueServiceEndpoint>
     <TableServiceEndpoint>http://[ACCOUNT].table.core.windows.net</TableServiceEndpoint>
+    <FileServiceEndpoint>http://[ACCOUNT].file.core.windows.net</FileServiceEndpoint>
     <BlobServiceSecondaryEndpoint>http://[ACCOUNT]-secondary.blob.core.windows.net</BlobServiceSecondaryEndpoint>
     <QueueServiceSecondaryEndpoint>http://[ACCOUNT]-secondary.queue.core.windows.net</QueueServiceSecondaryEndpoint>
     <TableServiceSecondaryEndpoint>http://[ACCOUNT]-secondary.table.core.windows.net</TableServiceSecondaryEndpoint>

microsoft-azure-storage-test/src/com/microsoft/azure/storage/SecondaryTests.java

@@ -16,6 +16,8 @@
 
 import static org.junit.Assert.*;
 
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
 import java.net.HttpURLConnection;
 import java.net.URISyntaxException;
 import java.util.ArrayList;
@@ -30,6 +32,7 @@
 import com.microsoft.azure.storage.TestRunners.SlowTests;
 import com.microsoft.azure.storage.blob.BlobRequestOptions;
 import com.microsoft.azure.storage.blob.BlobTestHelper;
+import com.microsoft.azure.storage.blob.BlobType;
 import com.microsoft.azure.storage.blob.CloudBlobClient;
 import com.microsoft.azure.storage.blob.CloudBlobContainer;
 import com.microsoft.azure.storage.blob.CloudBlockBlob;
@@ -639,6 +642,32 @@ public void testMultiLocationRetriesTable() throws URISyntaxException, StorageEx
         testTableDownloadPermissions(null, LocationMode.SECONDARY_THEN_PRIMARY, StorageLocation.SECONDARY,
                 retryContextList, retryInfoList);
     }
+
+    @Test
+    public void testRetryOn304() throws StorageException, IOException, URISyntaxException {
+        OperationContext operationContext = new OperationContext();
+        operationContext.getRetryingEventHandler().addListener(new StorageEvent<RetryingEvent>() {
+            @Override
+            public void eventOccurred(RetryingEvent eventArg) {
+                fail("Request should not be retried.");
+            }
+        });
+
+        CloudBlobContainer container = BlobTestHelper.getRandomContainerReference();
+        try {
+            container.create();
+            CloudBlockBlob blockBlobRef = (CloudBlockBlob) BlobTestHelper.uploadNewBlob(container, BlobType.BLOCK_BLOB,
+                    "originalBlob", 1024, null);
+            AccessCondition accessCondition = AccessCondition.generateIfNoneMatchCondition(blockBlobRef.getProperties().getEtag());
+            blockBlobRef.download(new ByteArrayOutputStream(), accessCondition, null, operationContext);
+
+            fail("Download should fail with a 304.");
+        } catch (StorageException ex) {
+            assertEquals("The condition specified using HTTP conditional header(s) is not met.", ex.getMessage());
+        } finally {
+            container.deleteIfExists();
+        }
+    }
 
     private static void AddUpdatedLocationModes(List<RetryContext> retryContextList, List<RetryInfo> retryInfoList) {
         retryInfoList

microsoft-azure-storage-test/src/com/microsoft/azure/storage/TestRunners.java

@@ -13,7 +13,8 @@
 import com.microsoft.azure.storage.blob.CloudBlobClientTests;
 import com.microsoft.azure.storage.blob.CloudBlobContainerTests;
 import com.microsoft.azure.storage.blob.CloudBlobDirectoryTests;
-import com.microsoft.azure.storage.blob.CloudBlobEncryptionTests;
+import com.microsoft.azure.storage.blob.CloudBlobClientEncryptionTests;
+import com.microsoft.azure.storage.blob.CloudBlobServerEncryptionTests;
 import com.microsoft.azure.storage.blob.CloudBlockBlobTests;
 import com.microsoft.azure.storage.blob.CloudPageBlobTests;
 import com.microsoft.azure.storage.blob.LeaseTests;
@@ -100,8 +101,8 @@ public static class CoreTestSuite {
 
     @RunWith(Suite.class)
     @SuiteClasses({ BlobOutputStreamTests.class, CloudBlobClientTests.class, CloudBlobContainerTests.class, 
-        CloudBlobDirectoryTests.class, CloudAppendBlobTests.class, CloudBlockBlobTests.class, CloudPageBlobTests.class,
-        CloudBlobEncryptionTests.class, LeaseTests.class, SasTests.class })
+            CloudBlobDirectoryTests.class, CloudAppendBlobTests.class, CloudBlockBlobTests.class, CloudPageBlobTests.class,
+            CloudBlobClientEncryptionTests.class, CloudBlobServerEncryptionTests.class, LeaseTests.class, SasTests.class })
     public static class BlobTestSuite {
     }
 
@@ -130,7 +131,8 @@ public static class AnalyticsTestSuite {
     }
 
     @RunWith(Suite.class)
-    @SuiteClasses({ CloudBlobEncryptionTests.class, CloudQueueEncryptionTests.class, TableEncryptionTests.class })
+    @SuiteClasses({ CloudBlobClientEncryptionTests.class, CloudBlobServerEncryptionTests.class,
+            CloudQueueEncryptionTests.class, TableEncryptionTests.class })
     public static class EncryptionTestSuite {
     }
 
@@ -161,4 +163,4 @@ public static class DevFabricNoSecondarySuite {
     @SuiteClasses(AllTestSuite.class)
     public static class FastTestSuite {
     }
-}
+}

microsoft-azure-storage-test/src/com/microsoft/azure/storage/blob/BlobTestHelper.java

@@ -384,6 +384,7 @@ public static void assertAreEqual(CloudBlob blob1, CloudBlob blob2) throws URISy
         }
         else {
             assertNotNull(blob2);
+            assertEquals(blob1.getClass(), blob2.getClass());
             assertEquals(blob1.getUri(), blob2.getUri());
             assertEquals(blob1.getSnapshotID(), blob2.getSnapshotID());
             assertEquals(blob1.isSnapshot(), blob2.isSnapshot());
@@ -399,6 +400,7 @@ public static void assertAreEqual(BlobProperties prop1, BlobProperties prop2) {
         }
         else {
             assertNotNull(prop2);
+            assertEquals(prop1.getBlobType(), prop2.getBlobType());
             assertEquals(prop1.getCacheControl(), prop2.getCacheControl());
             assertEquals(prop1.getContentDisposition(), prop2.getContentDisposition());
             assertEquals(prop1.getContentEncoding(), prop2.getContentEncoding());

microsoft-azure-storage-test/src/com/microsoft/azure/storage/blob/CloudBlobEncryptionTests.java → microsoft-azure-storage-test/src/com/microsoft/azure/storage/blob/CloudBlobClientEncryptionTests.java

@@ -58,18 +58,18 @@
 import com.microsoft.azure.storage.core.SR;
 
 @Category({ CloudTests.class, DevFabricTests.class, DevStoreTests.class })
-public class CloudBlobEncryptionTests {
+public class CloudBlobClientEncryptionTests {
 
     protected CloudBlobContainer container;
 
     @Before
-    public void blockBlobTestMethodSetup() throws URISyntaxException, StorageException {
+    public void blobEncryptionTestMethodSetup() throws URISyntaxException, StorageException {
         this.container = BlobTestHelper.getRandomContainerReference();
         this.container.create();
     }
 
     @After
-    public void blockBlobTestMethodTearDown() throws StorageException {
+    public void blobEncryptionTestMethodTearDown() throws StorageException {
         this.container.deleteIfExists();
     }
 
@@ -1010,4 +1010,4 @@ public void eventOccurred(SendingRequestEvent eventArg) {
         TestHelper.assertStreamsAreEqualAtIndex(stream, new ByteArrayInputStream(outputStream.toByteArray()), 0, 0, 
                 size, 2 * 1024);
     }
-}
+}

microsoft-azure-storage-test/src/com/microsoft/azure/storage/blob/CloudBlobContainerTests.java

@@ -19,6 +19,7 @@
 import java.io.IOException;
 import java.net.HttpURLConnection;
 import java.net.URISyntaxException;
+import java.security.InvalidKeyException;
 import java.util.Calendar;
 import java.util.Date;
 import java.util.EnumSet;
@@ -41,6 +42,7 @@
 import com.microsoft.azure.storage.ResultContinuation;
 import com.microsoft.azure.storage.ResultSegment;
 import com.microsoft.azure.storage.SendingRequestEvent;
+import com.microsoft.azure.storage.StorageCredentialsSharedAccessSignature;
 import com.microsoft.azure.storage.StorageErrorCodeStrings;
 import com.microsoft.azure.storage.StorageEvent;
 import com.microsoft.azure.storage.StorageException;
@@ -134,6 +136,86 @@ public void testCloudBlobContainerReference() throws StorageException, URISyntax
                 .toString());
     }
 
+    @Test
+    @Category({ DevFabricTests.class, DevStoreTests.class })
+    public void testCloudBlobContainerReferenceFromServer() throws StorageException, URISyntaxException, IOException {
+        this.container.create();
+
+        CloudBlob blob = BlobTestHelper.uploadNewBlob(this.container, BlobType.BLOCK_BLOB, null, 1024, null);
+        blob.getProperties().setContentType("application/octet-stream");
+        blob.getProperties().setLength(1024);
+
+        CloudBlob blobRef = this.container.getBlobReferenceFromServer(blob.getName());
+        BlobTestHelper.assertAreEqual(blob, blobRef);
+
+        blob = BlobTestHelper.uploadNewBlob(this.container, BlobType.PAGE_BLOB, null, 1024, null);
+        blob.getProperties().setContentType("application/octet-stream");
+        blob.getProperties().setLength(1024);
+
+        blobRef = this.container.getBlobReferenceFromServer(blob.getName());
+        BlobTestHelper.assertAreEqual(blob, blobRef);
+
+        blob = BlobTestHelper.uploadNewBlob(this.container, BlobType.APPEND_BLOB, null, 1024, null);
+        blob.getProperties().setContentType("application/octet-stream");
+        blob.getProperties().setLength(1024);
+
+        blobRef = this.container.getBlobReferenceFromServer(blob.getName());
+        BlobTestHelper.assertAreEqual(blob, blobRef);
+    }
+
+    @Test
+    @Category({ DevFabricTests.class, DevStoreTests.class })
+    public void testCloudBlobContainerReferenceFromServerSnapshot() throws StorageException, URISyntaxException,
+            IOException {
+        this.container.create();
+
+        CloudBlob blob = BlobTestHelper.uploadNewBlob(this.container, BlobType.BLOCK_BLOB, null, 1024, null);
+        CloudBlob snapshot = blob.createSnapshot();
+        snapshot.getProperties().setContentType("application/octet-stream");
+        snapshot.getProperties().setLength(1024);
+
+        CloudBlob blobRef = this.container.getBlobReferenceFromServer(snapshot.getName(), snapshot.getSnapshotID(),
+                null, null, null);
+        BlobTestHelper.assertAreEqual(snapshot, blobRef);
+    }
+
+    @Test
+    @Category({ DevFabricTests.class, DevStoreTests.class })
+    public void testCloudBlobContainerReferenceFromServerSAS() throws StorageException, URISyntaxException,
+            IOException, InvalidKeyException {
+        this.container.create();
+        CloudBlob blob = BlobTestHelper.uploadNewBlob(this.container, BlobType.BLOCK_BLOB, null, 1024, null);
+
+        SharedAccessBlobPolicy policy = new SharedAccessBlobPolicy();
+        Calendar now = Calendar.getInstance();
+        now.add(Calendar.MINUTE, 10);
+        policy.setSharedAccessExpiryTime(now.getTime());
+        policy.setPermissions(EnumSet.of(SharedAccessBlobPermissions.READ));
+        String token = this.container.generateSharedAccessSignature(policy, null);
+
+        CloudBlobContainer containerSAS = new CloudBlobContainer(this.container.getStorageUri(),
+                new StorageCredentialsSharedAccessSignature(token));
+        CloudBlob blobRef = containerSAS.getBlobReferenceFromServer(blob.getName());
+        assertEquals(blob.getClass(), blobRef.getClass());
+        assertEquals(blob.getUri(), blobRef.getUri());
+    }
+
+    @Test
+    @Category({ DevFabricTests.class, DevStoreTests.class })
+    public void testCloudBlobContainerReferenceFromServerMissingBlob() throws StorageException, URISyntaxException,
+            IOException {
+        this.container.create();
+
+        String blobName = BlobTestHelper.generateRandomBlobNameWithPrefix("missing");
+
+        try {
+            this.container.getBlobReferenceFromServer(blobName);
+            fail("Get reference from server should fail.");
+        } catch (StorageException ex) {
+            assertEquals(404, ex.getHttpStatusCode());
+        }
+    }
+
     /**
      * Create a container
      *