TST: warning on non-existant s3 file

[jreback]

#17057 fixes all but this warning. I guess something is not getting closed in s3fs.

(pandas) bash-3.2$ pytest pandas/tests/io/parser/
=========================================================================================== test session starts ===========================================================================================
platform darwin -- Python 3.6.1, pytest-3.1.2, py-1.4.34, pluggy-0.4.0
rootdir: /Users/jreback/pandas, inifile: setup.cfg
plugins: cov-2.3.1, xdist-1.16.0
collected 837 items 

pandas/tests/io/parser/test_network.py ..........................
pandas/tests/io/parser/test_parsers.py .......................s............................................................................s...............................................................................................s....................................................s.............................s............................................................................s...............................................................................................s....................................................s..................................................................................s...............................................................................................s...............................................................s......
pandas/tests/io/parser/test_read_fwf.py ........................
pandas/tests/io/parser/test_textreader.py .........................
pandas/tests/io/parser/test_unsupported.py .......

============================================================================================ warnings summary =============================================================================================
pandas/tests/io/parser/test_network.py::TestS3::()::test_s3_fails
  /Users/jreback/pandas/pandas/tests/io/parser/test_network.py:174: ResourceWarning: unclosed <ssl.SSLSocket fd=15, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.1.15', 60648), raddr=('52.216.32.16', 443)>
    read_csv('s3://nyqpug/asdf.csv')

-- Docs: http://doc.pytest.org/en/latest/warnings.html
=========================================================================== 826 passed, 11 skipped, 1 warnings in 65.08 seconds ===========================================================================

This is doing this internally.

In [7]: import s3fs

In [8]: s3fs.__version__
Out[8]: '0.1.1'

In [5]: fs = s3fs.S3FileSystem(anon=True)

In [6]: fs.open('nyqpug/asdf.csv')
---------------------------------------------------------------------------
ClientError                               Traceback (most recent call last)
/Users/jreback/miniconda3/envs/pandas/lib/python3.6/site-packages/s3fs/core.py in info(self, path, refresh, **kwargs)
    393                 out = self._call_s3(self.s3.head_object,
--> 394                                     kwargs, Bucket=bucket, Key=key, **self.req_kw)
    395                 out = {'ETag': out['ETag'], 'Key': '/'.join([bucket, key]),

/Users/jreback/miniconda3/envs/pandas/lib/python3.6/site-packages/s3fs/core.py in _call_s3(self, method, *akwarglist, **kwargs)
    167         additional_kwargs.update(kwargs)
--> 168         return method(**additional_kwargs)
    169 

/Users/jreback/miniconda3/envs/pandas/lib/python3.6/site-packages/botocore/client.py in _api_call(self, *args, **kwargs)
    252             # The "self" in this scope is referring to the BaseClient.
--> 253             return self._make_api_call(operation_name, kwargs)
    254 

/Users/jreback/miniconda3/envs/pandas/lib/python3.6/site-packages/botocore/client.py in _make_api_call(self, operation_name, api_params)
    543             error_class = self.exceptions.from_code(error_code)
--> 544             raise error_class(parsed_response, operation_name)
    545         else:

ClientError: An error occurred (404) when calling the HeadObject operation: Not Found

During handling of the above exception, another exception occurred:

FileNotFoundError                         Traceback (most recent call last)
<ipython-input-6-7628c3834aad> in <module>()
----> 1 fs.open('nyqpug/asdf.csv')

/Users/jreback/miniconda3/envs/pandas/lib/python3.6/site-packages/s3fs/core.py in open(self, path, mode, block_size, acl, fill_cache, **kwargs)
    289                                       " and manage bytes" % (mode[0] + 'b'))
    290         return S3File(self, path, mode, block_size=block_size, acl=acl,
--> 291                       fill_cache=fill_cache, s3_additional_kwargs=kwargs)
    292 
    293     def _lsdir(self, path, refresh=False):

/Users/jreback/miniconda3/envs/pandas/lib/python3.6/site-packages/s3fs/core.py in __init__(self, s3, path, mode, block_size, acl, fill_cache, s3_additional_kwargs)
    927         else:
    928             try:
--> 929                 self.size = self.info()['Size']
    930             except (ClientError, ParamValidationError):
    931                 raise IOError("File not accessible", path)

/Users/jreback/miniconda3/envs/pandas/lib/python3.6/site-packages/s3fs/core.py in info(self, **kwargs)
    936     def info(self, **kwargs):
    937         """ File information about this path """
--> 938         return self.s3.info(self.path, **kwargs)
    939 
    940     def metadata(self, refresh=False, **kwargs):

/Users/jreback/miniconda3/envs/pandas/lib/python3.6/site-packages/s3fs/core.py in info(self, path, refresh, **kwargs)
    398                 return out
    399             except (ClientError, ParamValidationError):
--> 400                 raise FileNotFoundError(path)
    401 
    402     _metadata_cache = {}

FileNotFoundError: nyqpug/asdf.csv

[jreback]

cc @TomAugspurger @martindurant

[martindurant]

boto/boto3#454 unresolved bug

[jreback]

thanks @martindurant ok, out of our hands then.