Merge remote-tracking branch 'upstream/master' into 28547

.github/CODE_OF_CONDUCT.md

@@ -54,10 +54,10 @@ incident.
 
 This Code of Conduct is adapted from the [Contributor Covenant][homepage],
 version 1.3.0, available at
-[http://contributor-covenant.org/version/1/3/0/][version],
+[https://www.contributor-covenant.org/version/1/3/0/][version],
 and the [Swift Code of Conduct][swift].
 
-[homepage]: http://contributor-covenant.org
-[version]: http://contributor-covenant.org/version/1/3/0/
+[homepage]: https://www.contributor-covenant.org
+[version]: https://www.contributor-covenant.org/version/1/3/0/
 [swift]: https://swift.org/community/#code-of-conduct
 

.github/CONTRIBUTING.md

@@ -16,7 +16,7 @@ If you notice a bug in the code or documentation, or have suggestions for how we
 
 ## Contributing to the Codebase
 
-The code is hosted on [GitHub](https://www.github.com/pandas-dev/pandas), so you will need to use [Git](http://git-scm.com/) to clone the project and make changes to the codebase. Once you have obtained a copy of the code, you should create a development environment that is separate from your existing Python environment so that you can make and test changes without compromising your own work environment. For more information, please refer to the "[Working with the code](https://github.com/pandas-dev/pandas/blob/master/doc/source/development/contributing.rst#working-with-the-code)" section.
+The code is hosted on [GitHub](https://www.github.com/pandas-dev/pandas), so you will need to use [Git](https://git-scm.com/) to clone the project and make changes to the codebase. Once you have obtained a copy of the code, you should create a development environment that is separate from your existing Python environment so that you can make and test changes without compromising your own work environment. For more information, please refer to the "[Working with the code](https://github.com/pandas-dev/pandas/blob/master/doc/source/development/contributing.rst#working-with-the-code)" section.
 
 Before submitting your changes for review, make sure to check that your changes do not break any tests. You can find more information about our test suites in the "[Test-driven development/code writing](https://github.com/pandas-dev/pandas/blob/master/doc/source/development/contributing.rst#test-driven-development-code-writing)" section. We also have guidelines regarding coding style that will be enforced during testing, which can be found in the "[Code standards](https://github.com/pandas-dev/pandas/blob/master/doc/source/development/contributing.rst#code-standards)" section.
 

.github/workflows/assign.yml

@@ -7,9 +7,8 @@ jobs:
   one:
     runs-on: ubuntu-latest
     steps:
-      - name:
-        run: |
-            if [[ "${{ github.event.comment.body }}" == "take" ]]; then
-                echo "Assigning issue ${{ github.event.issue.number }} to ${{ github.event.comment.user.login }}"
-                curl -H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" -d '{"assignees": ["${{ github.event.comment.user.login }}"]}' https://api.github.com/repos/${{ github.repository }}/issues/${{ github.event.issue.number }}/assignees
-            fi
+    - if: github.event.comment.body == 'take'
+      name:
+      run: |
+        echo "Assigning issue ${{ github.event.issue.number }} to ${{ github.event.comment.user.login }}"
+        curl -H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" -d '{"assignees": ["${{ github.event.comment.user.login }}"]}' https://api.github.com/repos/${{ github.repository }}/issues/${{ github.event.issue.number }}/assignees

AUTHORS.md

@@ -14,7 +14,7 @@ About the Copyright Holders
     The PyData Development Team is the collection of developers of the PyData
     project. This includes all of the PyData sub-projects, including pandas. The
     core team that coordinates development on GitHub can be found here:
-    http://github.com/pydata.
+    https://github.com/pydata.
 
 Full credits for pandas contributors can be found in the documentation.
 

RELEASE.md

@@ -3,4 +3,4 @@ Release Notes
 
 The list of changes to Pandas between each release can be found
 [here](https://pandas.pydata.org/pandas-docs/stable/whatsnew/index.html). For full
-details, see the commit logs at http://github.com/pandas-dev/pandas.
+details, see the commit logs at https://github.com/pandas-dev/pandas.

asv_bench/benchmarks/attrs_caching.py

@@ -1,12 +1,18 @@
 import numpy as np
 
+import pandas as pd
 from pandas import DataFrame
 
 try:
     from pandas.util import cache_readonly
 except ImportError:
     from pandas.util.decorators import cache_readonly
 
+try:
+    from pandas.core.construction import extract_array
+except ImportError:
+    extract_array = None
+
 
 class DataFrameAttributes:
     def setup(self):
@@ -20,6 +26,33 @@ def time_set_index(self):
         self.df.index = self.cur_index
 
 
+class SeriesArrayAttribute:
+
+    params = [["numeric", "object", "category", "datetime64", "datetime64tz"]]
+    param_names = ["dtype"]
+
+    def setup(self, dtype):
+        if dtype == "numeric":
+            self.series = pd.Series([1, 2, 3])
+        elif dtype == "object":
+            self.series = pd.Series(["a", "b", "c"], dtype=object)
+        elif dtype == "category":
+            self.series = pd.Series(["a", "b", "c"], dtype="category")
+        elif dtype == "datetime64":
+            self.series = pd.Series(pd.date_range("2013", periods=3))
+        elif dtype == "datetime64tz":
+            self.series = pd.Series(pd.date_range("2013", periods=3, tz="UTC"))
+
+    def time_array(self, dtype):
+        self.series.array
+
+    def time_extract_array(self, dtype):
+        extract_array(self.series)
+
+    def time_extract_array_numpy(self, dtype):
+        extract_array(self.series, extract_numpy=True)
+
+
 class CacheReadonly:
     def setup(self):
         class Foo:

asv_bench/benchmarks/pandas_vb_common.py

@@ -56,7 +56,7 @@
 def setup(*args, **kwargs):
     # This function just needs to be imported into each benchmark file to
     # set up the random seed before each function.
-    # http://asv.readthedocs.io/en/latest/writing_benchmarks.html
+    # https://asv.readthedocs.io/en/latest/writing_benchmarks.html
     np.random.seed(1234)
 
 

asv_bench/benchmarks/reshape.py

@@ -161,6 +161,9 @@ def time_pivot_table_categorical_observed(self):
             observed=True,
         )
 
+    def time_pivot_table_margins_only_column(self):
+        self.df.pivot_table(columns=["key2", "key3"], margins=True)
+
 
 class Crosstab:
     def setup(self):

asv_bench/benchmarks/tslibs/timedelta.py

@@ -10,6 +10,11 @@
 
 
 class TimedeltaConstructor:
+    def setup(self):
+        self.nptimedelta64 = np.timedelta64(3600)
+        self.dttimedelta = datetime.timedelta(seconds=3600)
+        self.td = Timedelta(3600, unit="s")
+
     def time_from_int(self):
         Timedelta(123456789)
 
@@ -28,10 +33,10 @@ def time_from_components(self):
         )
 
     def time_from_datetime_timedelta(self):
-        Timedelta(datetime.timedelta(days=1, seconds=1))
+        Timedelta(self.dttimedelta)
 
     def time_from_np_timedelta(self):
-        Timedelta(np.timedelta64(1, "ms"))
+        Timedelta(self.nptimedelta64)
 
     def time_from_string(self):
         Timedelta("1 days")
@@ -42,6 +47,9 @@ def time_from_iso_format(self):
     def time_from_missing(self):
         Timedelta("nat")
 
+    def time_from_pd_timedelta(self):
+        Timedelta(self.td)
+
 
 class TimedeltaProperties:
     def setup_cache(self):

asv_bench/benchmarks/tslibs/timestamp.py

@@ -1,12 +1,19 @@
 import datetime
 
 import dateutil
+import numpy as np
 import pytz
 
 from pandas import Timestamp
 
 
 class TimestampConstruction:
+    def setup(self):
+        self.npdatetime64 = np.datetime64("2020-01-01 00:00:00")
+        self.dttime_unaware = datetime.datetime(2020, 1, 1, 0, 0, 0)
+        self.dttime_aware = datetime.datetime(2020, 1, 1, 0, 0, 0, 0, pytz.UTC)
+        self.ts = Timestamp("2020-01-01 00:00:00")
+
     def time_parse_iso8601_no_tz(self):
         Timestamp("2017-08-25 08:16:14")
 
@@ -28,6 +35,18 @@ def time_fromordinal(self):
     def time_fromtimestamp(self):
         Timestamp.fromtimestamp(1515448538)
 
+    def time_from_npdatetime64(self):
+        Timestamp(self.npdatetime64)
+
+    def time_from_datetime_unaware(self):
+        Timestamp(self.dttime_unaware)
+
+    def time_from_datetime_aware(self):
+        Timestamp(self.dttime_aware)
+
+    def time_from_pd_timestamp(self):
+        Timestamp(self.ts)
+
 
 class TimestampProperties:
     _tzs = [None, pytz.timezone("Europe/Amsterdam"), pytz.UTC, dateutil.tz.tzutc()]

azure-pipelines.yml

@@ -4,7 +4,7 @@ jobs:
 - template: ci/azure/posix.yml
   parameters:
     name: macOS
-    vmImage: xcode9-macos10.13
+    vmImage: macOS-10.14
 
 - template: ci/azure/posix.yml
   parameters:

ci/deps/azure-37-locale.yaml

@@ -34,3 +34,6 @@ dependencies:
   - xlsxwriter
   - xlwt
   - pyarrow>=0.15
+  - pip
+  - pip:
+    - pyxlsb

ci/deps/azure-macos-36.yaml

@@ -33,3 +33,4 @@ dependencies:
   - pip
   - pip:
     - pyreadstat
+    - pyxlsb

ci/deps/azure-windows-37.yaml

@@ -35,3 +35,6 @@ dependencies:
   - xlsxwriter
   - xlwt
   - pyreadstat
+  - pip
+  - pip:
+    - pyxlsb

ci/deps/travis-36-cov.yaml

@@ -51,3 +51,4 @@ dependencies:
     - coverage
     - pandas-datareader
     - python-dateutil
+    - pyxlsb

doc/cheatsheet/README.txt

@@ -5,4 +5,4 @@ and pick "PDF" as the format.
 This cheat sheet was inspired by the RStudio Data Wrangling Cheatsheet[1], written by Irv Lustig, Princeton Consultants[2].
 
 [1]: https://www.rstudio.com/wp-content/uploads/2015/02/data-wrangling-cheatsheet.pdf
-[2]: http://www.princetonoptimization.com/
+[2]: https://www.princetonoptimization.com/

doc/source/conf.py

@@ -408,7 +408,7 @@
         "py": ("https://pylib.readthedocs.io/en/latest/", None),
         "python": ("https://docs.python.org/3/", None),
         "scipy": ("https://docs.scipy.org/doc/scipy/reference/", None),
-        "statsmodels": ("http://www.statsmodels.org/devel/", None),
+        "statsmodels": ("https://www.statsmodels.org/devel/", None),
     }
 
 # extlinks alias
@@ -625,10 +625,10 @@ def linkcode_resolve(domain, info):
     fn = os.path.relpath(fn, start=os.path.dirname(pandas.__file__))
 
     if "+" in pandas.__version__:
-        return f"http://github.com/pandas-dev/pandas/blob/master/pandas/{fn}{linespec}"
+        return f"https://github.com/pandas-dev/pandas/blob/master/pandas/{fn}{linespec}"
     else:
         return (
-            f"http://github.com/pandas-dev/pandas/blob/"
+            f"https://github.com/pandas-dev/pandas/blob/"
             f"v{pandas.__version__}/pandas/{fn}{linespec}"
         )
 
@@ -695,7 +695,7 @@ def rstjinja(app, docname, source):
     """
     Render our pages as a jinja template for fancy templating goodness.
     """
-    # http://ericholscher.com/blog/2016/jul/25/integrating-jinja-rst-sphinx/
+    # https://www.ericholscher.com/blog/2016/jul/25/integrating-jinja-rst-sphinx/
     # Make sure we're outputting HTML
     if app.builder.format != "html":
         return