Merge pull request #53 from NREL/better_docs

Better docs

CHANGES.txt

@@ -1,3 +1,8 @@
+v1.1.0, 10/09/18 -- fixes to_dataframes and to_gdx memory leaks, respects 
+                    GAMS_DIR environment variable, additional bug and warning 
+                    fixes. GdxSymbol.dataframe can now be set just using the 
+                    GdxSymbol.dims columns; in that case the value columns are 
+                    filled in with defaults
 v1.0.4, 03/02/18 -- make sure set 'Value' column fix works even when dims are all named '*'
 v1.0.3, 02/15/18 -- fix up set 'Value' column when write to GDX
 v1.0.2, 01/19/18 -- instructions and additional filepaths for running pytest on 

LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2017, Alliance for Sustainable Energy.
+Copyright (c) 2018, Alliance for Sustainable Energy.
 All rights reserved.
 
 Redistribution and use in source and binary forms, 

README.md

@@ -1,112 +1,10 @@
 # gdx-pandas
 [![PyPI](https://img.shields.io/pypi/v/gdxpds.svg)](https://pypi.python.org/pypi/gdxpds/)
+[![Documentation](https://img.shields.io/badge/docs-ready-blue.svg)](http://nrel.github.io/gdx-pandas)
 
-Python package to translate between gdx (GAMS data) and pandas. 
+gdx-pandas is a python package to translate between gdx (GAMS data) and pandas. 
 
-[Install](#install) | [Use](#use) | [Uninstall](#uninstall)
-
-## Use
-
-There are two main ways to use gdxpds. The first use case is the one that was 
-initially supported: direct conversion between GDX files on disk and pandas 
-DataFrames or a csv version thereof. The Version 1.0.0 rewrite intoduces a 
-second style of use, that is, interfacing with GDX files and symbols via the 
-`gdxpds.gdx.GdxFile` and `gdxpds.gdx.GdxSymbol` classes.
-
-[Direct Conversion](#direct-conversion) | [Backend Classes](#backend-classes)
-
-### Direct Conversion
-
-The two primary points of reference for the direct conversion utilities are GDX 
-files on disk and python dicts of {symbol_name: pandas.DataFrame}, where 
-each pandas.DataFrame contains data for a single set, parameter, equation, or 
-variable. For sets and parameters, the last column of the DataFrame is assumed to
-contain the value of the element, which for sets should be `True`, and for 
-parameters should be a `float` (or one of the `gdxpds.gdx.NUMPY_SPECIAL_VALUES`). 
-Equations and variables have additional 'value' columns, in particular a level, 
-a marginal value, a lower bound, an upper bound, and a scale, as enumerated in 
-`gdxpds.gdx.GamsValueType`. These values are all assumed to be found in the last 
-five columns of the DataFrame, also see `gdxpds.gdx.GAMS_VALUE_COLS_MAP`.
-
-The basic interface to convert from GDX to DataFrames is:
-
-```python
-import gdxpds
-
-gdx_file = 'C:\path_to_my_gdx\data.gdx'
-dataframes = gdxpds.to_dataframes(gdx_file)
-for symbol_name, df in dataframes.items():
-    print("Doing work with {}.".format(symbol_name))
-```
-
-And vice-versa:
-
-```python
-import gdxpds
-
-# assume we have a DataFrame df with last column 'value'
-data_ready_for_GAMS = { 'symbol_name': df }
-
-gdx_file = 'C:\path_to_my_output_gdx\data_to_send_to_gams.gdx'
-gdx = gdxpds.to_gdx(data_ready_for_GAMS, gdx_file)
-```
-
-Note that providing a gdx_file is optional, and the returned gdx is an object of 
-type `gdxpds.gdx.GdxFile`.
-
-Additional functions include:
-
-- `gdxpds.list_symbols`
-- `gdxpds.to_dataframe` (If the call to this method includes 
-   old_interface=False, then the return value will be a plain DataFrame, not a 
-   {'symbol_name': df} dict.)
-
-The package also includes command line utilities for converting between GDX and 
-CSV, see
-
-```bash
-python C:\your_python_path\Scripts\gdx_to_csv.py --help
-python C:\your_python_path\Scripts\csv_to_gdx.py --help
-```
-
-### Backend Classes
-
-The basic functionalities described above can also be achieved with direct use
-of the backend classes now available in `gdxpds.gdx`. To duplicate the GDX read 
-functionality shown above one would write:
-
-```python
-import gdxpds
-
-gdx_file = 'C:\path_to_my_gdx\data.gdx'
-with gdxpds.gdx.GdxFile(lazy_load=False) as f:
-    f.read(gdx_file)
-    for symbol in f:
-        symbol_name = symbol.name
-        df = symbol.dataframe
-        print("Doing work with {}:\n{}".format(symbol_name,df.head()))
-```
-
-The backend especially gives more control over creating new data in GDX format. 
-For example:
-
-```python
-import gdxpds
-
-out_file = 'my_new_gdx_data.gdx'
-with gdxpds.gdx.GdxFile() as gdx:
-    # Create a new set with one dimension
-    gdx.append(gdxpds.gdx.GdxSymbol('my_set',gdxpds.gdx.GamsDataType.Set,dims=['u']))
-    data = pds.DataFrame([['u' + str(i)] for i in range(1,11)])
-    data['Value'] = True
-    gdx[-1].dataframe = data
-    # Create a new parameter with one dimension
-    gdx.append(gdxpds.gdx.GdxSymbol('my_parameter',gdxpds.gdx.GamsDataType.Parameter,dims=['u']))
-    data = pds.DataFrame([['u' + str(i), i*100] for i in range(1,11)],
-                         columns=(gdx[-1].dims + gdx[-1].value_col_names))
-    gdx[-1].dataframe = data
-    gdx.write(out_file)
-```
+[Install](#install) | [Uninstall](#uninstall)
 
 ## Install
 
@@ -115,8 +13,6 @@ with gdxpds.gdx.GdxFile() as gdx:
 - Python 2.6 or higher 2.X; Python 3.4 or higher 3.X
 - pandas (In general you will want the SciPy stack. Anaconda comes with it, or see [my notes for Windows](http://elainethale.wordpress.com/programming-notes/python-environment-set-up/).)
 - For Python versions < 3.4, enum34. Also **uninstall the enum package** if it is installed.
-- psutil (optional--for monitoring memory use)
-- pytest (optional--for running tests)
 - GAMS Python bindings
     - See GAMS/win64/XX.X/apifiles/readme.txt on Windows, 
       GAMS/gamsXX.X_osx_x64_64_sfx/apifiles/readme.txt on Mac, or 
@@ -151,7 +47,7 @@ pip install gdxpds
 or
 
 ```bash
-pip install git+https://github.com/NREL/gdx-pandas.git@v1.0.4
+pip install git+https://github.com/NREL/gdx-pandas.git@v1.1.0
 ```
 
 or

bin/csv_to_gdx.py

@@ -1,43 +1,39 @@
-'''
-[LICENSE]
-
-Copyright (c) 2017, Alliance for Sustainable Energy.
-All rights reserved.
-
-Redistribution and use in source and binary forms, 
-with or without modification, are permitted provided 
-that the following conditions are met:
-
-1. Redistributions of source code must retain the above 
-copyright notice, this list of conditions and the 
-following disclaimer.
-
-2. Redistributions in binary form must reproduce the 
-above copyright notice, this list of conditions and the 
-following disclaimer in the documentation and/or other 
-materials provided with the distribution.
-
-3. Neither the name of the copyright holder nor the 
-names of its contributors may be used to endorse or 
-promote products derived from this software without 
-specific prior written permission.
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND 
-CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, 
-INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 
-MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 
-DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR 
-CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 
-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, 
-BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 
-SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 
-HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 
-CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE 
-OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-[/LICENSE]
-'''
+# [LICENSE]
+# Copyright (c) 2018, Alliance for Sustainable Energy.
+# All rights reserved.
+# 
+# Redistribution and use in source and binary forms, 
+# with or without modification, are permitted provided 
+# that the following conditions are met:
+# 
+# 1. Redistributions of source code must retain the above 
+# copyright notice, this list of conditions and the 
+# following disclaimer.
+# 
+# 2. Redistributions in binary form must reproduce the 
+# above copyright notice, this list of conditions and the 
+# following disclaimer in the documentation and/or other 
+# materials provided with the distribution.
+# 
+# 3. Neither the name of the copyright holder nor the 
+# names of its contributors may be used to endorse or 
+# promote products derived from this software without 
+# specific prior written permission.
+# 
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND 
+# CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, 
+# INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 
+# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 
+# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR 
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, 
+# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 
+# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 
+# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE 
+# OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# [/LICENSE]
 
 import argparse
 import os

bin/gdx_to_csv.py

@@ -1,43 +1,39 @@
-'''
-[LICENSE]
-
-Copyright (c) 2017, Alliance for Sustainable Energy.
-All rights reserved.
-
-Redistribution and use in source and binary forms, 
-with or without modification, are permitted provided 
-that the following conditions are met:
-
-1. Redistributions of source code must retain the above 
-copyright notice, this list of conditions and the 
-following disclaimer.
-
-2. Redistributions in binary form must reproduce the 
-above copyright notice, this list of conditions and the 
-following disclaimer in the documentation and/or other 
-materials provided with the distribution.
-
-3. Neither the name of the copyright holder nor the 
-names of its contributors may be used to endorse or 
-promote products derived from this software without 
-specific prior written permission.
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND 
-CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, 
-INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 
-MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 
-DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR 
-CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 
-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, 
-BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 
-SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 
-HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 
-CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE 
-OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-[/LICENSE]
-'''
+# [LICENSE]
+# Copyright (c) 2018, Alliance for Sustainable Energy.
+# All rights reserved.
+# 
+# Redistribution and use in source and binary forms, 
+# with or without modification, are permitted provided 
+# that the following conditions are met:
+# 
+# 1. Redistributions of source code must retain the above 
+# copyright notice, this list of conditions and the 
+# following disclaimer.
+# 
+# 2. Redistributions in binary form must reproduce the 
+# above copyright notice, this list of conditions and the 
+# following disclaimer in the documentation and/or other 
+# materials provided with the distribution.
+# 
+# 3. Neither the name of the copyright holder nor the 
+# names of its contributors may be used to endorse or 
+# promote products derived from this software without 
+# specific prior written permission.
+# 
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND 
+# CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, 
+# INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 
+# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 
+# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR 
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, 
+# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 
+# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 
+# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE 
+# OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# [/LICENSE]
 
 import argparse
 import logging