diff --git a/resources/en/docs/user_manual/managing_data_source/addpostgistables.png b/resources/en/docs/user_manual/managing_data_source/addpostgistables.png new file mode 100644 index 00000000000..b40a7040d7c Binary files /dev/null and b/resources/en/docs/user_manual/managing_data_source/addpostgistables.png differ diff --git a/resources/en/docs/user_manual/working_with_vector/addvectorlayerdialog.png b/resources/en/docs/user_manual/managing_data_source/addvectorlayerdialog.png similarity index 100% rename from resources/en/docs/user_manual/working_with_vector/addvectorlayerdialog.png rename to resources/en/docs/user_manual/managing_data_source/addvectorlayerdialog.png diff --git a/resources/en/docs/user_manual/qgis_browser/browser_panels.png b/resources/en/docs/user_manual/managing_data_source/browser_panels.png similarity index 100% rename from resources/en/docs/user_manual/qgis_browser/browser_panels.png rename to resources/en/docs/user_manual/managing_data_source/browser_panels.png diff --git a/resources/en/docs/user_manual/working_with_vector/create_virtual_layers.png b/resources/en/docs/user_manual/managing_data_source/create_virtual_layers.png similarity index 100% rename from resources/en/docs/user_manual/working_with_vector/create_virtual_layers.png rename to resources/en/docs/user_manual/managing_data_source/create_virtual_layers.png diff --git a/resources/en/docs/user_manual/introduction/delimited_text_dialog.png b/resources/en/docs/user_manual/managing_data_source/delimited_text_dialog.png similarity index 100% rename from resources/en/docs/user_manual/introduction/delimited_text_dialog.png rename to resources/en/docs/user_manual/managing_data_source/delimited_text_dialog.png diff --git a/resources/en/docs/user_manual/working_with_vector/editNewSpatialite.png b/resources/en/docs/user_manual/managing_data_source/editNewSpatialite.png similarity index 100% rename from resources/en/docs/user_manual/working_with_vector/editNewSpatialite.png rename to resources/en/docs/user_manual/managing_data_source/editNewSpatialite.png diff --git a/resources/en/docs/user_manual/working_with_vector/editNewVector.png b/resources/en/docs/user_manual/managing_data_source/editNewVector.png similarity index 100% rename from resources/en/docs/user_manual/working_with_vector/editNewVector.png rename to resources/en/docs/user_manual/managing_data_source/editNewVector.png diff --git a/resources/en/docs/user_manual/managing_data_source/newpostgisconnection.png b/resources/en/docs/user_manual/managing_data_source/newpostgisconnection.png new file mode 100644 index 00000000000..f3225a17a9a Binary files /dev/null and b/resources/en/docs/user_manual/managing_data_source/newpostgisconnection.png differ diff --git a/resources/en/docs/user_manual/managing_data_source/saveasraster.png b/resources/en/docs/user_manual/managing_data_source/saveasraster.png new file mode 100644 index 00000000000..b497b0be199 Binary files /dev/null and b/resources/en/docs/user_manual/managing_data_source/saveasraster.png differ diff --git a/resources/en/docs/user_manual/managing_data_source/saveasvector.png b/resources/en/docs/user_manual/managing_data_source/saveasvector.png new file mode 100644 index 00000000000..1758cfcac00 Binary files /dev/null and b/resources/en/docs/user_manual/managing_data_source/saveasvector.png differ diff --git a/resources/en/docs/user_manual/working_with_vector/shapefileloaded.png b/resources/en/docs/user_manual/managing_data_source/shapefileloaded.png similarity index 100% rename from resources/en/docs/user_manual/working_with_vector/shapefileloaded.png rename to resources/en/docs/user_manual/managing_data_source/shapefileloaded.png diff --git a/resources/en/docs/user_manual/working_with_vector/shapefileopendialog.png b/resources/en/docs/user_manual/managing_data_source/shapefileopendialog.png similarity index 100% rename from resources/en/docs/user_manual/working_with_vector/shapefileopendialog.png rename to resources/en/docs/user_manual/managing_data_source/shapefileopendialog.png diff --git a/resources/en/docs/user_manual/working_with_vector/vectorNotWrapping.png b/resources/en/docs/user_manual/managing_data_source/vectorNotWrapping.png similarity index 100% rename from resources/en/docs/user_manual/working_with_vector/vectorNotWrapping.png rename to resources/en/docs/user_manual/managing_data_source/vectorNotWrapping.png diff --git a/resources/en/docs/user_manual/working_with_vector/vectorWrapping.png b/resources/en/docs/user_manual/managing_data_source/vectorWrapping.png similarity index 100% rename from resources/en/docs/user_manual/working_with_vector/vectorWrapping.png rename to resources/en/docs/user_manual/managing_data_source/vectorWrapping.png diff --git a/source/docs/user_manual/grass_integration/grass_integration.rst b/source/docs/user_manual/grass_integration/grass_integration.rst index b7831b17651..5fab3fdead5 100644 --- a/source/docs/user_manual/grass_integration/grass_integration.rst +++ b/source/docs/user_manual/grass_integration/grass_integration.rst @@ -215,8 +215,8 @@ exercises in the following GRASS-related sections. It is useful to download and install the dataset on your computer (see :ref:`label_sampledata`). #. Start QGIS and make sure the GRASS plugin is loaded. -#. Visualize the :file:`alaska.shp` shapefile (see section - :ref:`vector_loading_file`) from the QGIS Alaska dataset (see :ref:`label_sampledata`). +#. Visualize the :file:`alaska.shp` shapefile (see section :ref:`loading_file`) + from the QGIS Alaska dataset (see :ref:`label_sampledata`). #. In the GRASS toolbar, click on the |grassNewMapset| :sup:`New mapset` icon to bring up the :guilabel:`MAPSET` wizard. #. Select an existing GRASS database (GISDBASE) folder :file:`grassdata`, or create diff --git a/source/docs/user_manual/index.rst b/source/docs/user_manual/index.rst index 336e145cb1e..be359bebcb1 100644 --- a/source/docs/user_manual/index.rst +++ b/source/docs/user_manual/index.rst @@ -21,7 +21,7 @@ QGIS User Guide introduction/general_tools introduction/qgis_configuration working_with_projections/working_with_projections - qgis_browser/qgis_browser + managing_data_source/index working_with_vector/index working_with_raster/index print_composer/index diff --git a/source/docs/user_manual/introduction/general_tools.rst b/source/docs/user_manual/introduction/general_tools.rst index f6fb70d6186..efa6a86cd26 100644 --- a/source/docs/user_manual/introduction/general_tools.rst +++ b/source/docs/user_manual/introduction/general_tools.rst @@ -1648,119 +1648,6 @@ files. For more information see :ref:`authentication_index`. A master password needs to be set up when initializing the authentication system and its portable database. -.. index:: Save layer -.. _general_saveas: - -Save layer into file -===================== - -Layers (raster or vector) can be saved in another format with the -:guilabel:`Save As...` feature in the layer contextual menu (by right-clicking -in the layer in the layer tree) or in the :menuselection:`Layer --> Save As...` -menu. - -Common parameters ------------------ - -The :guilabel:`Save As` dialog shows several parameters to change the behavior -when saving the layer. Common parameters (raster and vector) are: - -* :guilabel:`Format` -* :guilabel:`File name` -* :guilabel:`CRS` -* :guilabel:`Add saved file to map` to add the new layer to the canvas -* :guilabel:`Extent` (possible values are **layer**, **Map view** or **user-defined** - extent) - -However, some parameters are specific to raster and vector formats: - -Raster specific parameters --------------------------- - -* :guilabel:`Output mode` (it can be **raw data** or **rendered image**) -* :guilabel:`Resolution` -* :guilabel:`Create Options`: advanced options (file compression, block sizes, colorimetry...) - to fine tune the output file. See the `gdal-ogr `_ driver documentation. -* :guilabel:`Pyramids` creation -* :guilabel:`VRT Tiles` -* :guilabel:`No data values` - - -Vector specific parameters --------------------------- - -Depending on the format of export, some of these options are available or not: - -* :guilabel:`Encoding` -* :guilabel:`Save only selected features` -* :guilabel:`Select fields to export and their export options`. In case you set your - fields behavior with some :ref:`Edit widgets `, e.g. ``value - map``, you can keep the displayed values in the layer by checking |checkbox| - :guilabel:`Replace all selected raw fields values by displayed values`. -* :guilabel:`Symbology export`: can be used mainly for DXF export and for all file - formats who manage OGR feature styles (see note below) as DXF, KML, tab - file formats: - - * **No symbology**: default style of the application that reads the data - * **Feature symbology**: save style with OGR Feature Styles (see note below) - * **Symbol Layer symbology**: save with OGR Feature Styles (see note below) - but export the same geometry multiple times if there are multiple symbology - symbol layers used - * A **Scale** value can be applied to the latest options. - -.. _ogr_features_note: - -.. note:: *OGR Feature Styles* are a way to store style directly in - the data as a hidden attribute. Only some formats can handle this kind of - information. KML, DXF and TAB file formats are such formats. For advanced - users, you can read the `OGR Feature Styles specification - `_ document. - -* :guilabel:`Geometry`: you can configure the geometry capabilities of the output layer - - * :guilabel:`geometry type`: keep the original geometry of the features when set to - **Automatic**, otherwise removes or overrides it with any type. You can add - an empty geometry column to an attribute table, remove the geometry column - of a spatial layer. - * :guilabel:`Force multi-type`: force creation of multi-geometry features in the layer - * :guilabel:`Include z-dimension` to geometries. - -.. tip:: - - Overriding layer geometry type makes it possible to do things like save a - geometryless table (e.g. :file:`.csv` file) into a shapefile WITH any type of - geometry (point, line, polygon), so that geometries can then be manually added - to rows with the |addPart| :sup:`Add Part` tool . - -* :guilabel:`Datasources Options`, :guilabel:`Layer Options` or :guilabel:`Custom Options` - which allow you to configure some advanced parameters. See the `gdal-ogr `_ - driver documentation. - -.. index:: Overwrite file, Append features - -When saving a vector layer into an existing file, depending on the capabilities -of the output format (Geopackage, SpatiaLite, FileGDB...), the user can -decide whether to: - -* overwrite the whole file -* overwrite only the target layer (the layer name is configurable) -* append features to the existing target layer -* append features, add new fields if there are any. - -For formats like Shapefile, MapInfo .tab, feature append is also available. - -.. note:: **About DXF files** - - Vector layers can be exported to DXF files using another tool, the - :guilabel:`DXF Export...` in :menuselection:`Project`. The windows allow the - user to choose the layer file, the symbology mode (see the `OGR Feature Styles - `_ note), the symbology scale, the encoding, the visibility - preset and the layers to include in the DXF file. - - - As an option, you can |checkbox| :guilabel:`Use the layer title as name if set` - or :guilabel:`Export features intersecting the current map extent`. - .. index:: Variables, Expressions .. _`general_tools_variables`: diff --git a/source/docs/user_manual/introduction/qgis_gui.rst b/source/docs/user_manual/introduction/qgis_gui.rst index 4ab13bf7b21..4686c39bed3 100644 --- a/source/docs/user_manual/introduction/qgis_gui.rst +++ b/source/docs/user_manual/introduction/qgis_gui.rst @@ -259,7 +259,7 @@ Vector ============================================================== ==================== ========================================== =============================== Menu Option Shortcut Reference Toolbar ============================================================== ==================== ========================================== =============================== -:menuselection:`OpenStreetMap -->` \ see :ref:`open_street_map` \ +:menuselection:`OpenStreetMap -->` \ see :ref:`openstreetmap` \ |analysis| :menuselection:`Analysis Tools -->` \ see :ref:`vector_menu` \ |sampling| :menuselection:`Research Tools -->` \ see :ref:`vector_menu` \ |geoprocessing| :menuselection:`Geoprocessing Tools -->` \ see :ref:`vector_menu` \ @@ -463,7 +463,7 @@ Also called **Map canvas**, this is the "business end" of QGIS --- maps are displayed in this area. The map displayed in this window will depend on the vector and raster layers you have chosen to load. -When you add a layer (see e.g. :ref:`vector_loading_file`), QGIS automatically +When you add a layer (see e.g. :ref:`opening_data`), QGIS automatically looks for its Coordinate Reference System (CRS) and zooms to its extent if you work in a blank QGIS project. The layer's CRS is then applied to the project. If there are already layers in the project, and in the case the new layer has diff --git a/source/docs/user_manual/managing_data_source/create_layers.rst b/source/docs/user_manual/managing_data_source/create_layers.rst new file mode 100644 index 00000000000..6a37c207ef0 --- /dev/null +++ b/source/docs/user_manual/managing_data_source/create_layers.rst @@ -0,0 +1,483 @@ +.. only:: html + + |updatedisclaimer| + +.. _creating_layers: + +***************** + Creating Layers +***************** + +.. only:: html + +.. contents:: + :local: + + +Creating a layer can be processed in many ways in QGIS; it can be done: + +* from scratch, creating an empty layer; +* from an existing layer; +* from the clipboard; +* as a result of an SQL-like query based on one or many layers: the + :ref:`virtual layer `. + +QGIS also provides particular tools to import/export different formats. + +.. index:: Create new layers +.. index:: Shapefile, SpatiaLite, GPX + +.. _sec_create_vector: + +Creating new Vector layers +========================== + +QGIS allows you to create new shapefile layers, new SpatiaLite layers, new +GPX layers and New Temporary Scratch Layers. Creation of a new GRASS layer +is supported within the GRASS plugin. +Please refer to section :ref:`creating_new_grass_vectors` for more information +on creating GRASS vector layers. + +Creating a new Shapefile layer +------------------------------ + +To create a new shape layer for editing, choose :menuselection:`Create +Layer -->` |newVectorLayer| :menuselection:`New Shapefile Layer...` from the +:menuselection:`Layer` menu. The :guilabel:`New Vector Layer` dialog will be +displayed as shown in figure_create_shapefile_. Choose the type of layer +(point, line or polygon) and the CRS (coordinate reference system). + +Note that QGIS does not yet support creation of 2.5D features (i.e., features +with X,Y,Z coordinates). + +.. _figure_create_shapefile: + +.. figure:: /static/user_manual/managing_data_source/editNewVector.png + :align: center + + Creating a new Shapefile layer dialog + +To complete the creation of the new shapefile layer, add the desired attributes +by clicking on the **[Add to attributes list]** button and specifying a name and +type for the attribute. A first 'id' column is added as default but can be +removed, if not wanted. Only :guilabel:`Type: real` |selectString|, +:guilabel:`Type: integer` |selectString|, :guilabel:`Type: string` +|selectString| and :guilabel:`Type:date` |selectString| attributes are +supported. Additionally and according to the attribute type, you can also define +the width and precision of the new attribute column. Once you are happy with the +attributes, click **[OK]** and provide a name for the shapefile. QGIS will +automatically add a :file:`.shp` extension to the name you specify. Once the +layer has been created, it will be added to the map, and you can edit it in the +same way as described in section :ref:`sec_edit_existing_layer` above. + +.. index:: New SpatiaLite layer + +.. _vector_create_spatialite: + +Creating a new SpatiaLite layer +------------------------------- + +To create a new SpatiaLite layer for editing, choose :menuselection:`New -->` +|newSpatiaLiteLayer| :menuselection:`New SpatiaLite Layer...` from the +:menuselection:`Layer` menu. The :guilabel:`New SpatiaLite Layer` dialog will +be displayed as shown in Figure_create_spatialite_. + +.. _figure_create_spatialite: + +.. figure:: /static/user_manual/managing_data_source/editNewSpatialite.png + :align: center + + Creating a New SpatiaLite layer dialog + +The first step is to select an existing SpatiaLite database or to create a new +SpatiaLite database. This can be done with the browse button |browseButton| to +the right of the database field. Then, add a name for the new layer, define +the layer type, and specify the coordinate reference system with **[Specify CRS]**. +If desired, you can select |checkbox| :guilabel:`Create an autoincrementing primary key`. + +To define an attribute table for the new SpatiaLite layer, add the names of +the attribute columns you want to create with the corresponding column type, and +click on the **[Add to attribute list]** button. Once you are happy with the +attributes, click **[OK]**. QGIS will automatically add the new layer to the +legend, and you can edit it in the same way as described in section +:ref:`sec_edit_existing_layer` above. + +Further management of SpatiaLite layers can be done with the DB Manager. See +:ref:`dbmanager`. + + +.. index:: New GPX layer +.. _vector_create_gpx: + +Creating a new GPX layer +------------------------- + +To create a new GPX file, you need to load the GPS plugin first. :menuselection:`Plugins -->` +|showPluginManager| :menuselection:`Plugin Manager...` opens the Plugin Manager Dialog. +Activate the |checkbox| :guilabel:`GPS Tools` checkbox. + +When this plugin is loaded, choose :menuselection:`New -->` |createGPX| +:menuselection:`Create new GPX Layer...` from the :menuselection:`Layer` menu. +In the :guilabel:`Save new GPX file as` dialog, you can choose where to save the +new GPX layer. + +.. index:: New Temporary Scratch layer +.. _vector_new_scratch_layer: + +Creating a new Temporary Scratch Layer +-------------------------------------- + +Empty, editable memory layers can be defined using :menuselection:`Layer --> +Create Layer --> New Temporary Scratch Layer`. Here you can even create +|radioButtonOff|:guilabel:`Multipoint`, |radioButtonOff|:guilabel:`Multiline` +and |radioButtonOff|:guilabel:`Multipolygon` Layers beneath +|radioButtonOn|:guilabel:`Point`, |radioButtonOff|:guilabel:`Line` and +|radioButtonOff|:guilabel:`Polygon` Layers. Temporary Scratch Layers are not +saved and will be discarded when QGIS is closed. See also :ref:`paste_into_layer`. + + +.. index:: Save layer +.. _general_saveas: + +Save layer from an existing file +================================ + +Layers (raster or vector) or subset of layers can be saved in another format +with the :guilabel:`Save As...` feature in the layer contextual menu (by +right-clicking in the layer in the layer tree) or in the :menuselection:`Layer +--> Save As...` menu. + +Common parameters +----------------- + +The :guilabel:`Save As` dialog shows several parameters to change the behavior +when saving the layer. Common parameters (raster and vector) are: + +* :guilabel:`Format` +* :guilabel:`File name` +* :guilabel:`CRS` +* :guilabel:`Add saved file to map` to add the new layer to the canvas +* :guilabel:`Extent` (possible values are **layer**, **Map view** or + **user-defined** extent) + +However, some parameters are specific to raster and vector formats: + +Raster specific parameters +-------------------------- + +* :guilabel:`Output mode` (it can be **raw data** or **rendered image**) +* :guilabel:`Resolution` +* :guilabel:`Create Options`: advanced options (file compression, block sizes, + colorimetry...) to fine tune the output file. See the `gdal-ogr + `_ driver documentation. +* :guilabel:`Pyramids` creation +* :guilabel:`VRT Tiles` +* :guilabel:`No data values` + +.. _figure_save_raster: + +.. figure:: /static/user_manual/managing_data_source/saveasraster.png + :align: center + + Saving as a new raster layer + +Vector specific parameters +-------------------------- + +Depending on the format of export, some of these options are available or not: + +* :guilabel:`Encoding` +* :guilabel:`Save only selected features` +* :guilabel:`Select fields to export and their export options`. In case you set + your fields behavior with some :ref:`Edit widgets `, e.g. + ``value map``, you can keep the displayed values in the layer by checking + |checkbox| :guilabel:`Replace all selected raw fields values by displayed + values`. +* :guilabel:`Symbology export`: can be used mainly for DXF export and for all + file formats who manage OGR feature styles (see note below) as DXF, KML, tab + file formats: + + * **No symbology**: default style of the application that reads the data + * **Feature symbology**: save style with OGR Feature Styles (see note below) + * **Symbol Layer symbology**: save with OGR Feature Styles (see note below) + but export the same geometry multiple times if there are multiple symbology + symbol layers used + * A **Scale** value can be applied to the latest options. + +.. _ogr_features_note: + +.. note:: *OGR Feature Styles* are a way to store style directly in + the data as a hidden attribute. Only some formats can handle this kind of + information. KML, DXF and TAB file formats are such formats. For advanced + users, you can read the `OGR Feature Styles specification + `_ document. + +* :guilabel:`Geometry`: you can configure the geometry capabilities of the + output layer + + * :guilabel:`geometry type`: keep the original geometry of the features when + set to **Automatic**, otherwise removes or overrides it with any type. You + can add an empty geometry column to an attribute table, remove the geometry + column of a spatial layer. + * :guilabel:`Force multi-type`: force creation of multi-geometry features in + the layer + * :guilabel:`Include z-dimension` to geometries. + +.. tip:: + + Overriding layer geometry type makes it possible to do things like save a + geometryless table (e.g. :file:`.csv` file) into a shapefile WITH any type of + geometry (point, line, polygon), so that geometries can then be manually added + to rows with the |addPart| :sup:`Add Part` tool . + +* :guilabel:`Datasources Options`, :guilabel:`Layer Options` or + :guilabel:`Custom Options` which allow you to configure some advanced + parameters. See the `gdal-ogr `_ driver documentation. + +.. _figure_save_vector: + +.. figure:: /static/user_manual/managing_data_source/saveasvector.png + :align: center + + Saving as a new vector layer + +.. index:: Overwrite file, Append features + +When saving a vector layer into an existing file, depending on the capabilities +of the output format (Geopackage, SpatiaLite, FileGDB...), the user can +decide whether to: + +* overwrite the whole file +* overwrite only the target layer (the layer name is configurable) +* append features to the existing target layer +* append features, add new fields if there are any. + +For formats like ESRI Shapefile, MapInfo .tab, feature append is also available. + +.. index:: DXF Export +.. _create_dxf_files: + +Create DXF files +================ + +Besides the :guilabel:`Save As...` dialog which provides options to export a +single layer to another format, including :file:`*.DXF`, QGIS provides another +tool to export multiple layers as a single DXF layers. It's accessible in the +:menuselection:`Project --> DXF Export...` menu. + +The :guilabel:`DXF Export` dialog allows the user to: + +* indicate the destination layer file; +* choose the symbology mode and scale (see the `OGR Feature Styles + `_ note); +* select the encoding and CRS; +* check the loaded layers to include in the DXF files or pick them from an + existing :ref:`visibility preset `. + + For each layer, you can choose a field whose values are used to split features + in generated destination layers in the DXF output. You can also choose to + |checkbox| :guilabel:`Use the layer title as name if set` and keep features + grouped. +* choose to only :guilabel:`Export features intersecting the current map extent`. + + +.. _paste_into_layer: + +Create layer from a clipboard +============================= + +Features that are on the clipboard can be pasted into a new layer. To do this, +Select some features, copy them to the clipboard, and then paste them into a +new layer using :menuselection:`Edit --> Paste Features as -->` and choosing: + +* :menuselection:`New Vector Layer...`: you need to select the layer CRS, poping + up the :guilabel:`Save vector layer as...` dialog from which you can select + any supported data format (see :ref:`general_saveas` for parameters); +* or :menuselection:`Temporary Scratch Layer...`: you need to select the layer + CRS and give a name. + +A new layer, filled with selected features and their attributes is created and +added to map canvas if asked. + +.. note:: Creating layers from clipboard applies to features selected and copied + within QGIS and also to features from another source defined using well-known + text (WKT). + + +.. index:: Virtual layers +.. _vector_virtual_layers: + +Virtual layers +============== + +A special kind of vector layer allows you to define a layer as the result of an +advanced query, using the SQL language on any number of other vector layers that +QGIS is able to open. These layers are called virtual layers: they do not carry +data by themselves and can be seen as views to other layers. + +Creating a virtual layer +------------------------ + +Open the virtual layer creation dialog by clicking on +:guilabel:`Add Virtual Layer` in the :guilabel:`Layer` menu or from the +corresponding toolbar. + +The dialog allows you to specify a :guilabel:`Layer name` and a SQL +:guilabel:`Query`. The query can use the name (or id) of loaded vector +layers as tables, as well as their fields' name as columns. + +For example, if you have a layer called ``airports``, you can create a new +virtual layer called ``public_airports`` with an SQL query like: + +.. code-block:: sql + + SELECT * + FROM airports + WHERE USE = "Civilian/Public" + +The SQL query will be executed, whatever the underlying provider of the +``airports`` layer is and even if this provider does not directly support SQL +queries. + +.. figure:: /static/user_manual/managing_data_source/create_virtual_layers.png + :align: center + + Create virtual layers dialog + +Joins and complex queries can also be created simply by directly using the +names of the layers that are to be joined. + +.. note:: + + It's also possible to create virtual layers using the SQL window of + :ref:`dbmanager`. + +Embedded layers +--------------- + +Besides the vector layers available in the map canvas, the user can add layers +to the :guilabel:`Embedded layers` list, which he can use in queries +without the need to have them showing in the map canvas or Layers panel. + +To embed a layer, click :guilabel:`Add` and provide the :guilabel:`Local name`, +:guilabel:`Provider`, :guilabel:`Encoding` and the path to the +:guilabel:`Source`. + +The :guilabel:`Import` button allows adding layers loaded in the map canvas into +the Embedded layers list. This allows to later remove those layers from the +Layers panel without breaking any existent query. + +Supported language +------------------ + +The underlying engine uses SQLite and SpatiaLite to operate. + +It means you can use all of the SQL your local installation of SQLite +understands. + +Functions from SQLite and spatial functions from SpatiaLite +can also be used in a virtual layer query. For instance, creating a point +layer out of an attribute-only layer can be done with a query similar to: + +.. code-block:: sql + + SELECT id, MakePoint(x, y, 4326) as geometry + FROM coordinates + +:ref:`Functions of QGIS expressions` can also be used in a +virtual layer query. + +To refer the geometry column of a layer, use the name ``geometry``. + +Contrary to a pure SQL query, all the fields of a virtual layer query must +be named. Don't forget to use the ``as`` keyword to name your columns if they +are the result of a computation or function call. + +Performance issues +------------------ + +With default parameters set, the virtual layer engine will try its best to +detect the type of the different columns of the query, including the type of the +geometry column if one is present. + +This is done by introspecting the query when possible or by fetching the first +row of the query (LIMIT 1) at last resort. +Fetching the first row of the result just to create the layer may be undesirable +for performance reasons. + +The creation dialog allows to specify different parameters: + +* :guilabel:`Unique identifier column`: this option allows specifying which + field of the query represents unique integer values that QGIS can use as row + identifiers. By default, an autoincrementing integer value is used. + Defining a unique identifier column allows to speed up the selection of + rows by id. + +* :guilabel:`No geometry`: this option forces the virtual layer to ignore + any geometry field. The resulting layer is an attribute-only layer. + +* Geometry :guilabel:`Column`: this option allows to specify the name + of the column that is to be used as the geometry of the layer. + +* Geometry :guilabel:`Type`: this option allows to specify the type + of the geometry of the virtual layer. + +* Geometry :guilabel:`CRS`: this option allows to specify the + coordinate reference system of the virtual layer. + +Special comments +---------------- + +The virtual layer engine tries to determine the type of each column of the +query. If it fails, the first row of the query is fetched to determine +column types. + +The type of a particular column can be specified directly in the query by +using some special comments. + +The syntax is the following: ``/*:type*/``. It has to be placed just after +the name of a column. ``type`` can be either ``int`` for integers, ``real`` +for floating point numbers or ``text``. + +For instance: + +.. code-block:: sql + + SELECT id+1 as nid /*:int*/ + FROM table + +The type and coordinate reference system of the geometry column can also be set +thanks to special comments with the following syntax ``/*:gtype:srid*/`` where +``gtype`` is the geometry type (``point``, ``linestring``, ``polygon``, +``multipoint``, ``multilinestring`` or ``multipolygon``) and ``srid`` an +integer representing the EPSG code of a coordinate reference system. + +Use of indexes +-------------- + +When requesting a layer through a virtual layer, indexes of this source layer +will be used in the following ways: + +* if an ``=`` predicate is used on the primary key column of the layer, the + underlying data provider will be asked for a particular id (FilterFid) + +* for any other predicates (``>``, ``<=``, ``!=``, etc.) or on a column without + a primary key, a request built from an expression will be used to request the + underlying vector data provider. It means indexes may be used on database + providers if they exist. + +A specific syntax exists to handle spatial predicates in requests and triggers +the use of a spatial index: a hidden column named ``_search_frame_`` exists +for each virtual layer. This column can be compared for equality to a bounding +box. Example: + +.. code-block:: sql + + SELECT * + FROM vtab + WHERE _search_frame_=BuildMbr(-2.10,49.38,-1.3,49.99,4326) + +Spatial binary predicates like ``ST_Intersects`` are significantly sped up when +used in conjunction with this spatial index syntax. + + diff --git a/source/docs/user_manual/managing_data_source/index.rst b/source/docs/user_manual/managing_data_source/index.rst new file mode 100644 index 00000000000..e8a9786e932 --- /dev/null +++ b/source/docs/user_manual/managing_data_source/index.rst @@ -0,0 +1,17 @@ +.. only:: html + + |updatedisclaimer| + +.. _manage_data_source: + +********************** + Managing Data Source +********************** + +.. toctree:: + :maxdepth: 2 + + opening_data + create_layers + supported_data + diff --git a/source/docs/user_manual/managing_data_source/opening_data.rst b/source/docs/user_manual/managing_data_source/opening_data.rst new file mode 100644 index 00000000000..77c59f4df4c --- /dev/null +++ b/source/docs/user_manual/managing_data_source/opening_data.rst @@ -0,0 +1,803 @@ +.. only:: html + + |updatedisclaimer| + +.. index:: Vector, OGR, Raster, GDAL, Data, Format +.. index:: PostGreSQL, PostGIS, GeoPackage, SpatiaLite, GRASS, DXF +.. index:: ArcInfo Binary Grid, ArcInfo ASCII Grid, GeoTIFF, Erdas Imagine + +.. _opening_data: + +************** + Opening Data +************** + +.. only:: html + + .. contents:: + :local: + +As part of an Open Source Software ecosystem, QGIS is built upon different +libraries that, combined with its own providers, offer capabilities to read +and often write a lot of formats: + +* Vector data formats include ESRI formats (shapefiles, geodatabases...), + MapInfo and MicroStation file formats, AutoCAD DWG/DXF, GeoPackage, GeoJSON, + GRASS, GPX, KML, Comma Separated Values, and many more... + Read the complete list of `OGR vector supported formats + `_; +* Raster data formats include ArcInfo Binary Grid, ArcInfo ASCII Grid, JPEG, + GeoTIFF, ERDAS IMAGINE, MBTiles, R or Idrisi rasters, ASCII Gridded XYZ, + GDAL Virtual, SRTM, Sentinel Data, and many more... + Read the complete list of `raster supported formats + `_; +* Database formats include PostgreSQL/PostGIS, SQLite/SpatiaLite, Oracle, DB2 + or MSSQL Spatial, MySQL...; +* Support of web data services (WM(T)S, WFS, WCS, CSW, ArcGIS Servers...) is + also handled by QGIS providers (see :ref:`working_with_ogc`); +* You can also read supported files from archived folders and use QGIS native + formats such as virtual and memory layers. + +As of the date of this document, more than 80 vector and 140 raster formats are +supported by the `GDAL/OGR `_ and QGIS native providers. + +.. note:: + + Not all of the listed formats may work in QGIS for various reasons. For + example, some require external proprietary libraries, or the GDAL/OGR + installation of your OS may not have been built to support the format you + want to use. To have a list of available formats, run the command line + ``ogrinfo --formats`` (for vector) or check :menuselection:`settings --> + Options --> GDAL` menu (for raster) in QGIS. + +.. let's use ogrinfo until a list of vector formats is provided in a (GDAL/)OGR tab + + +.. index:: Browse data, Add layers +.. _browser_panel: + +The Browser Panel +================= + +QGIS Browser is one of the main panels of QGIS that lets you quickly and easily +add your data to projects. It helps you navigate in your filesystem and manage +geodata, regardless the type of layer (raster, vector, table), or the datasource +format (plain or compressed files, database, web services). + +To add a layer into a project: + +#. right-click on QGIS toolbar and check |checkbox| :guilabel:`Browser Panel` + to activate it or select it from the menu :menuselection:`View --> Panels` + (or |kde| :menuselection:`Settings --> Panels`); +#. a browser tree with your filesystem, databases and web services is displayed; +#. find the layer in the list; +#. right-click on its name and select **Add selected layer(s)**. Your layer is + now added to the :ref:`Layers Panel ` and can be viewed in the + :ref:`map canvas `. + +.. note:: + + You can also add a layer or open a QGIS project directly from the Browser + panel by double-clicking its name or by drag-and-drop into the map canvas. + +Once a file is loaded, you can zoom around it using the map navigation tools. +To change the style of a layer, open the :guilabel:`Layer Properties` dialog +by double clicking on the layer name or by right-clicking on the name in the +legend and choosing :menuselection:`Properties` from the context menu. See +section :ref:`vector_style_menu` for more information on setting symbology of +vector layers. + + +At the top of the Browser panel, you find some icons that help you to: + +* |addLayer| :sup:`Add Selected Layers`: you can also add data into the map + canvas by selecting **Add selected layer(s)** from the layer's context menu; +* |draw| :sup:`Refresh` the browser tree; +* |filterMap| :sup:`Filter Browser` to search for specific data. Enter a search + word or wildcard and the browser will filter the tree to only show paths to + matching DB tables, filenames or folders -- other data or folders won't be + displayed. See the Browser Panel(2) example on the figure_browser_panels_. + The comparison can be case-sensitive or not. It can also be set to: + + * **normal**: return any item containing the search text; + * using **wildcard(s)**: fine tune the search using ``?`` and/or ``*`` + characters to specify the position of the search text; + * using a **regular expression**. + +* |collapseTree| :sup:`Collapse All` the whole tree; +* |propertiesWidget| :sup:`Enable/disable properties widget`: when toggled on, + a new widget is added at the bottom of the panel showing, if applicable, + metadatas of the selected item. + +Right-click an item in the browser tree helps you to: + +* in case of file or table, display its metadata or open it in your project. + Tables can even be renamed, deleted or truncated; +* in case of folder, bookmark it into your favourites, hide it from the browser + tree. Hidden folders can be managed from the :menuselection:`Settings --> + Options --> Data Sources` tab; +* create connection to databases or web servers; +* refresh, rename or delete schema. + +You can also import files into databases or copy tables from one schema/database +to another one with a simple drag-and-drop. There is a second browser panel +available to avoid long scrolling while dragging. Just select the file and +drag-and-drop from one panel to the other. + +.. _figure_browser_panels: + +.. figure:: /static/user_manual/managing_data_source/browser_panels.png + :align: center + + QGIS Browser panels side-by-side + + +.. tip:: **Add layers to QGIS by simple drag-and-drop from your OS file browser** + + You can also add file(s) to the project by drag-and-dropping them from your + operating system file browser to the :guilabel:`Layers Panel` or the map + canvas. + +.. index:: DB Manager + +The DB Manager +============== + +The :guilabel:`DB Manager` Plugin is another one of the main and native tools +to integrate and manage spatial database formats supported by +QGIS (PostGIS, SpatiaLite, GeoPackage, Oracle Spatial, MSSQL, DB2, Virtual +layers) in one user interface. It can be activated from the +:menuselection:`Plugins --> Manage and Install Plugins...` menu. + +The |dbManager| :sup:`DB Manager` Plugin provides several features: + +* connect to databases and display its structure and contents; +* preview tables of databases; +* add layers to map canvas, either by double-click or drag-and-drop; +* add layers to a database from the QGIS Browser or from another database; +* create and add output of SQL queries to the map canvas; +* create :ref:`virtual layers `. + +More information on DB Manager capabilities are exposed in :ref:`dbmanager`. + +.. _figure_db_manager_bis: + +.. figure:: /static/user_manual/plugins/db_manager.png + :align: center + + DB Manager dialog + + +Provider-based loading tools +============================= + +Beside Browser Panel and DB Manager, the main tools provided by QGIS to add +layers regardless the format, you'll also find tools that are specific to data +providers. + +.. note:: + + Some :ref:`external plugins ` also propose tools to open specific + format files in QGIS. + +.. index:: Loading vector, Loading raster +.. _loading_file: + +Loading a layer from a file +--------------------------- + +To load a layer from a file, you can: + +* for vector data (like Shapefile, Mapinfo or dxf layer), click on + |addOgrLayer| :sup:`Add Vector Layer` toolbar button, select the + :menuselection:`Layer --> Add Layer -->` |addOgrLayer|:guilabel:`Add Vector + Layer` menu option or type :kbd:`Ctrl+Shift+V`. + This will bring up a new window (see figure_vector_add_) from which you can + check |radioButtonOn| :guilabel:`File` and click on **[Browse]**. You can + also select the encoding for the file if desired. + + .. _figure_vector_add: + + .. figure:: /static/user_manual/managing_data_source/addvectorlayerdialog.png + :align: center + + Add Vector Layer Dialog + +* for raster layers, click on the |addRasterLayer| :sup:`Add Raster Layer` icon, + select the :menuselection:`Layer --> Add Layer -->` |addRasterLayer| + :guilabel:`Add Raster Layer` menu option or type :kbd:`Ctrl+Shift+R`. + +That will bring up a standard open file dialog (see figure_vector_open_), which +allows you to navigate the file system and load a shapefile, a geotiff or other +supported data source. The selection box :guilabel:`Filter` |selectString| +allows you to preselect some supported file formats. Only the formats that have +been well tested appear in the list. Other untested formats can be loaded by +selecting ``All files (*.*)``. + + +.. _figure_vector_open: + +.. figure:: /static/user_manual/managing_data_source/shapefileopendialog.png + :align: center + + Open an OGR Supported Vector Layer Dialog + +Selecting a file from the list and clicking **[Open]** loads it into QGIS. +More than one layer can be loaded at the same time by holding down the +:kbd:`Ctrl` or :kbd:`Shift` key and clicking on multiple items in the dialog. +Figure_vector_loaded_ shows QGIS after loading the :file:`alaska.shp` file. + +.. _figure_vector_loaded: + +.. figure:: /static/user_manual/managing_data_source/shapefileloaded.png + :align: center + + QGIS with Shapefile of Alaska loaded + + +.. note:: + + Because some formats like MapInfo (e.g., :file:`.tab`) or Autocad (:file:`.dxf`) + allow mixing different types of geometry in a single file, loading such format + in QGIS opens a dialog to select geometries to use in order to have one + geometry per layer. + +.. index:: ArcInfo Binary Coverage, Tiger Format, UK National Transfer Format +.. index:: US Census Bureau + +Using the |addOgrLayer| :sup:`Add Vector Layer` tool, you can also load +specific format like ArcInfo Binary Coverage, UK. National Transfer Format, as +well as the raw TIGER format of the US Census Bureau or OpenfileGDB. To do that, +you'd need to select |radioButtonOn| :guilabel:`Directory` as :guilabel:`Source +type`. + + +.. _tip_load_from_external_drive_OSX: + +.. tip:: **Load layers and projects from mounted external drives on macOS** + + On macOS, portable drives that are mounted beside the primary hard drive + do not show up as expected under :menuselection:`File --> Open Project`. + We are working on a more macOS-native open/save dialog to fix this. + As a workaround, you can type ``/Volumes`` in the :guilabel:`File name` box + and press :kbd:`Enter`. Then you can navigate to external drives and network + mounts. + + +.. index:: CSV, Delimited text files + see: Comma Separated Values; CSV +.. _vector_loading_csv: + +Importing a delimited text file +------------------------------- + +Delimited text file (e.g. :file:`.csv`, :file:`.txt`) can be loaded in QGIS +using the tools described above. However, loaded this way, it'll show up like a +simple table data. Sometimes, delimited text files can contain geometric data +you'd want to visualize; this is what the |delimitedText| :guilabel:`Add +Delimited Text Layer` is designed for. + +Click the toolbar icon |delimitedText| :sup:`Add Delimited Text Layer` in the +:guilabel:`Manage layers` toolbar to open the :guilabel:`Create a Layer from a +Delimited Text File` dialog, as shown in figure_delimited_text_. + +.. _figure_delimited_text: + +.. figure:: /static/user_manual/managing_data_source/delimited_text_dialog.png + :align: center + + Delimited Text Dialog + +First, select the file to import (e.g., :file:`qgis_sample_data/csv/elevp.csv`) +by clicking on the **[Browse]** button. Once the file is selected, QGIS attempts +to parse the file with the most recently used delimiter. To enable QGIS to +properly parse the file, it is important to select the correct delimiter. You +can specify a delimiter by activating: + +* |radioButtonOn|:guilabel:`CSV (comma separated values)`; +* |radioButtonOff|:guilabel:`Custom delimiters`, choosing among some predefined + delimiters like ``comma``, ``space``, ``tab``, ``semicolon``...; +* or |radioButtonOff|:guilabel:`Regular expression delimiter` and entering text + into the :guilabel:`Expression` field. For example, to change the delimiter to + tab, use ``\t`` (this is a regular expression for the tab character). + +Once the file is parsed, set :guilabel:`Geometry definition` to +|radioButtonOn|:guilabel:`Point coordinates` and choose the ``X`` and ``Y`` +fields from the dropdown lists. If the coordinates are defined as +degrees/minutes/seconds, activate the |checkbox| :guilabel:`DMS coordinates` +checkbox. + +Finally, enter a layer name (e.g., :file:`elevp`), as shown in +figure_delimited_text_. To add the layer to the map, click **[OK]**. The +delimited text file now behaves as any other map layer in QGIS. + +There is also a helper option that allows you to trim leading and trailing +spaces from fields --- |checkbox| :guilabel:`Trim fields`. Also, it is possible +to |checkbox| :guilabel:`Discard empty fields`. If necessary, you can force a +comma to be the decimal separator by activating |checkbox| :guilabel:`Decimal +separator is comma`. + +If spatial information is represented by WKT, activate the |radioButtonOn| +:guilabel:`Well Known Text` option and select the field with the WKT definition +for point, line or polygon objects. If the file contains non-spatial data, +activate |radioButtonOn| :guilabel:`No geometry (attribute only table)` and it +will be loaded as an ordinal table. + +Additionally, you can enable: + +* |checkbox| :guilabel:`Use spatial index` to improve the performance of + displaying and spatially selecting features; +* |checkbox| :guilabel:`Use subset index`; +* |checkbox| :guilabel:`Watch file` to watch for changes to the file by other + applications while QGIS is running. + + +Importing a DXF or DWG file +--------------------------- + +DXF files can be added to QGIS by simple drag-and-drop from the common +Browser Panel. You'll be prompted to select the sublayers you'd like to add +to the project. Layers are added with random style properties. + +.. note:: DXF files containing several geometry types (point, line and/or + polygon), the name of the layer will be made from + * entities *. + +.. need to be tested with dwg. How does dwg format behave when added to QGIS? + +To keep the dxf/dwg structure and its symbology in QGIS, you may want to +use the dedicated :menuselection:`DWG/DXF Import...` tool. + +.. TODO: Add here the fix for https://github.com/qgis/QGIS-Documentation/issues/1579 + +.. index:: OSM (OpenStreetMap) +.. _openstreetmap: + +Importing OpenStreetMap Vectors +------------------------------- + +In recent years, the OpenStreetMap project has gained popularity because in many +countries no free geodata such as digital road maps are available. The objective +of the OSM project is to create a free editable map of the world from GPS data, +aerial photography or local knowledge. To support this objective, QGIS +provides support for OSM data. + +Using the :guilabel:`Browser Panel`, you can load a :file:`.osm` file to the +map canvas, in which case you'll get a dialog to select sublayers based on the +geometry type. The loaded layers will contain all the data of that geometry type +in the file and keep the :file:`osm` file data structure. + +To avoid working with a such complex data structure, and be able to select only +features you need based on their tags, QGIS provides a core and fully integrated +OpenStreetMap import tool: + +* To connect to the OSM server and download data, open the menu + :menuselection:`Vector --> OpenStreetMap --> Download data...`. You can skip + this step if you already obtained an :file:`.osm` XML file using JOSM, + Overpass API or any other source; +* The menu :menuselection:`Vector --> OpenStreetMap --> Import Topology from + XML...` will convert your :file:`.osm` file into a SpatiaLite database + and create a corresponding database connection; +* The menu :menuselection:`Vector --> OpenStreetMap --> Export Topology to + SpatiaLite...` then allows you to open the database connection, select the + type of data you want (points, lines, or polygons) and choose tags to import. + This creates a SpatiaLite geometry layer that you can add to your + project by clicking on the |addSpatiaLiteLayer| + :sup:`Add SpatiaLite Layer` toolbar button or by selecting the + |addSpatiaLiteLayer| :menuselection:`Add SpatiaLite Layer...` option + from the :menuselection:`Layer` menu (see section :ref:`label_spatialite`). + + +GPS +--- + +Loading GPS data in QGIS can be done using the core plugin: ``GPS Tools``. +Instructions are described in Section :ref:`plugin_gps`. + + +GRASS +----- + +Working with GRASS vector data is described in Section :ref:`sec_grass`. + + +.. index:: Spatialite, SQLite +.. _label_spatialite: + +SpatiaLite Layers +----------------- + +|addSpatiaLiteLayer| The first time you load data from a SpatiaLite +database, begin by: + +* clicking on the |addSpatiaLiteLayer| :sup:`Add SpatiaLite Layer` toolbar + button; +* selecting the |addSpatiaLiteLayer| :menuselection:`Add SpatiaLite Layer...` + option from the :menuselection:`Layer --> Add Layer` menu; +* or by typing :kbd:`Ctrl+Shift+L`. + +This will bring up a window that will allow you either to connect to a +SpatiaLite database already known to QGIS, which you can choose from the +drop-down menu, or to define a new connection to a new database. To define a +new connection, click on **[New]** and use the file browser to point to +your SpatiaLite database, which is a file with a :file:`.sqlite` extension. + +QGIS also supports editable views in SpatiaLite. + + +.. index:: Database tools, MSSQL Spatial +.. _db_tools: + +Database related tools +---------------------- + +.. index:: Connecting to database +.. _vector_create_stored_connection: + +Creating a stored Connection +............................ + +In order to read and write tables from the many database formats QGIS supports +you'll need to create a connection to that database. While :ref:`QGIS Browser +Panel ` is the simplest and recommanded way to connect and use +databases within, QGIS provides specific tools you can use to connect to each +of them and load their tables: + +* |addPostgisLayer| :menuselection:`Add PostGIS Layer...` or by typing + :kbd:`Ctrl+Shift+D` +* |addMssqlLayer| :menuselection:`Add MSSQL Spatial Layer` or by typing + :kbd:`Ctrl+Shift+M` +* |addOracleLayer| :menuselection:`Add Oracle Spatial Layer...` or typing + :kbd:`Ctrl+Shift+O` +* |addDb2Layer| :menuselection:`Add DB2 Spatial Layer...` or typing + :kbd:`Ctrl+Shift+2` + +These tools are accessible either from the :guilabel:`Manage Layers Toolbar` or +the :menuselection:`Layer --> Add Layer -->` menu. Connecting to SpatiaLite +database is described at :ref:`label_spatialite`. + +.. tip:: **Create connection to database from the QGIS Browser Panel** + + Select the corresponding database format in the Browser tree, right-click + and choose connect will provide you with the database connection dialog. + +Most of the connection dialogs follow a common basis that will be described +below using the PostGreSQL database tool as example. + +The first time you use a PostGIS data source, you must create a connection to a +database that contains the data. Begin by clicking the appropriate button as +exposed above, opening an :guilabel:`Add PostGIS Table(s)` dialog +(see figure_add_postgis_tables_). +To access the connection manager, click on the **[New]** button to display the +:guilabel:`Create a New PostGIS Connection` dialog. + +.. _figure_new_postgis_connection: + +.. figure:: /static/user_manual/managing_data_source/newpostgisconnection.png + :align: center + + Create a New PostGIS Connection Dialog + + +The parameters required for a PostGIS connection are (for other database types, +see the differences at :ref:`db_requirements`): + +* **Name**: A name for this connection. It can be the same as *Database*. +* **Service**: Service parameter to be used alternatively to hostname/port (and + potentially database). This can be defined in :file:`pg_service.conf`. + Check the :ref:`pg-service-file` section for more details. +* **Host**: Name of the database host. This must be a resolvable host name + such as would be used to open a TCP/IP connection or ping the host. If the + database is on the same computer as QGIS, simply enter *localhost* here. +* **Port**: Port number the PostgreSQL database server listens on. The default + port for PostGIS is ``5432``. +* **Database**: Name of the database. +* **SSL mode**: How the SSL connection will be negotiated with the server. Note + that massive speed-ups in PostGIS layer rendering can be achieved by disabling + SSL in the connection editor. The following options are available: + + * *Disable*: Only try an unencrypted SSL connection; + * *Allow*: Try a non-SSL connection. If that fails, try an SSL connection; + * *Prefer* (the default): Try an SSL connection. If that fails, try a + non-SSL connection; + * *Require*: Only try an SSL connection. + +* **Username**: User name used to log in to the database. +* **Password**: Password used with *Username* to connect to the database. + +Optionally, depending on the type of database, you can activate the following +checkboxes: + +* |checkbox| :guilabel:`Save Username` +* |checkbox| :guilabel:`Save Password` +* |checkbox| :guilabel:`Only show layers in the layer registries` +* |checkbox| :guilabel:`Don't resolve type of unrestricted columns (GEOMETRY)` +* |checkbox| :guilabel:`Only look in the 'public' schema` +* |checkbox| :guilabel:`Also list tables with no geometry` +* |checkbox| :guilabel:`Use estimated table metadata` + + +.. warning:: **QGIS User Settings and Security** + + In the :guilabel:`Authentication` tab, saving **username** and **password** + will keep unprotected credentials in the connection configuration. Those + **credentials will be visible** if, for instance, you shared the project file + with someone. Therefore, it's advisable to save your credentials in a + *Authentication configuration* instead (:guilabel:`Configurations` tab - + See :ref:`authentication_index` for more details) or in a service connection + file (see :ref:`pg-service-file` for example). + + +.. tip:: **Use estimated table metadata to speed up operations** + + When initializing layers, various queries may be needed to establish the + characteristics of the geometries stored in the database table. When the + :guilabel:`Use estimated table metadata` option is checked, these queries + examine only a sample of the rows and use the table statistics, rather than + the entire table. This can drastically speed up operations on large datasets, + but may result in incorrect characterization of layers (eg. the feature count + of filtered layers will not be accurately determined) and may even cause + strange behaviour in case columns that are supposed to be unique actually + are not. + +Once all parameters and options are set, you can test the connection by +clicking on the **[Test connection]** button or apply it hitting **[OK]**. +From the :guilabel:`Add PostGIS Table(s)`, click now on **[Connect]** and the +dialog is filled with tables from the selected database (as shown in +figure_add_postgis_tables_). + + +.. _db_requirements: + +Particular Connection requirements +.................................. + +Because of database type particularities, provided options are all the same for +all the databases. Below are exposed these connection specificities. + +.. _pg-service-file: + +PostgreSQL Service connection file +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The service connection file allows PostgreSQL connection parameters to be +associated with a single service name. That service name can then be specified +by a client and the associated settings will be used. + +It's called :file:`.pg_service.conf` under \*nix systems (GNU/Linux, macOS etc.) +and :file:`pg_service.conf` on Windows. + +The service file looks like:: + + [water_service] + host=192.168.0.45 + port=5433 + dbname=gisdb + user=paul + password=paulspass + + [wastewater_service] + host=dbserver.com + dbname=water + user=waterpass + +.. note:: There are two services in the above example: ``water_service`` + and ``wastewater_service``. You can use these to connect from QGIS, + pgAdmin etc. by specifying only the name of the service you want to + connect to (without the enclosing brackets). + If you want to use the service with ``psql`` you need to do something + like ``export PGSERVICE=water_service`` before doing your psql commands. + +.. note:: You can find all the parameters `here + `_ + +.. note:: If you don't want to save the passwords in the service file you can + use the `.pg_pass `_ + option. + + +On \*nix operating systems (GNU/Linux, macOS etc.) you can save the +:file:`.pg_service.conf` file in the user's home directory and +the PostgreSQL clients will automatically be aware of it. +For example, if the logged user is ``web``, :file:`.pg_service.conf` should +be saved in the :file:`/home/web/` directory in order to directly work (without +specifying any other environment variables). + +You can specify the location of the service file by creating a ``PGSERVICEFILE`` +environment variable (e.g. run the ``export PGSERVICEFILE=/home/web/.pg_service.conf`` +command under your \*nix OS to temporarily set the ``PGSERVICEFILE`` variable) + +You can also make the service file available system-wide (all users) either by +placing it at ``pg_config --sysconfdir``**/.pg_service.conf** or by adding the +``PGSYSCONFDIR`` environment variable to specify the directory containing +the service file. If service definitions with the same name exist in the user +and the system file, the user file takes precedence. + +.. warning:: + + There are some caveats under Windows: + + * The service file should be saved as :file:`pg_service.conf` + and not as :file:`.pg_service.conf`. + * The service file should be saved in Unix format in order to work. + One way to do it is to open it with `Notepad++ `_ + and :menuselection:`Edit --> EOL Conversion --> UNIX Format --> File save`. + * You can add environmental variables in various ways; a tested one, known to + work reliably, is :menuselection:`Control Panel --> System and Security --> + System --> Advanced system settings --> Environment Variables` adding + ``PGSERVICEFILE`` and the path of the type :file:`C:\Users\John\pg_service.conf` + * After adding an environment variable you may also need to restart the computer. + + +.. _create_oracle_connection: + +Connecting to Oracle Spatial +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The spatial features in Oracle Spatial aid users in managing geographic and +location data in a native type within an Oracle database. +In addition to some of the options in :ref:`vector_create_stored_connection`, +the connection dialog proposes: + +* **Database**: SID or SERVICE_NAME of the Oracle instance; +* **Port**: Port number the Oracle database server listens on. The default + port is ``1521``; +* **Workspace**: Workspace to switch to. + +Optionally, you can activate following checkboxes: + +* |checkbox| :guilabel:`Only look in metadata table`: restricts the displayed + tables to those that are in the ``all_sdo_geom_metadata`` view. This can + speed up the initial display of spatial tables; +* |checkbox| :guilabel:`Only look for user's tables`: when searching for spatial + tables, restrict the search to tables that are owned by the user; +* |checkbox| :guilabel:`Also list tables with no geometry`: indicates that + tables without geometry should also be listed by default; +* |checkbox| :guilabel:`Use estimated table statistics for the layer metadata`: + when the layer is set up, various metadata are required for the Oracle table. + This includes information such as the table row count, geometry type and + spatial extents of the data in the geometry column. If the table contains a + large number of rows, determining this metadata can be time-consuming. By + activating this option, the following fast table metadata operations are + done: Row count is determined from ``all_tables.num_rows``. Table extents + are always determined with the SDO_TUNE.EXTENTS_OF function, even if a layer + filter is applied. Table geometry is determined from the first 100 + non-null geometry rows in the table; +* |checkbox| :guilabel:`Only existing geometry types`: only list the existing + geometry types and don't offer to add others; +* |checkbox| :guilabel:`Include additional geometry attributes`. + +.. _tip_ORACLE_Spatial_layers: + +.. tip:: **Oracle Spatial Layers** + + Normally, an Oracle Spatial layer is defined by an entry in the + **USER_SDO_METADATA** table. + + +.. _create_db2_connection: + +Connecting to DB2 Spatial +^^^^^^^^^^^^^^^^^^^^^^^^^ + +In addition to some of the options described in +:ref:`vector_create_stored_connection`, the connection to a DB2 database (see +:ref:`label_db2_spatial` for more information) can be specified using either a +Service/DSN name defined to ODBC or using the driver, host and port information. + +An ODBC **Service/DSN** connection requires the service name defined to ODBC. + +A driver/host/port connection requires: + +* **Driver**: Name of the DB2 driver. Typically this would be IBM DB2 ODBC DRIVER. +* **DB2 Host**: Name of the database host. This must be a resolvable host name + such as would be used to open a TCP/IP connection or ping the host. If the + database is on the same computer as QGIS, simply enter *localhost* here. +* **DB2 Port**: Port number the DB2 database server listens on. The default + DB2 LUW port is ``50000``. The default DB2 z/OS port is ``446``. + +.. _tip_db2_Spatial_layers: + +.. tip:: **DB2 Spatial Layers** + + A DB2 Spatial layer is defined by a row in the **DB2GSE.ST_GEOMETRY_COLUMNS** + view. + +.. note:: + + In order to work effectively with DB2 spatial tables in QGIS, it is important + that tables have an INTEGER or BIGINT column defined as PRIMARY KEY and if new + features are going to be added, this column should also have the GENERATED + characteristic. + + It is also helpful for the spatial column to be registered with a specific + spatial reference identifier (most often 4326 for WGS84 coordinates). + A spatial column can be registered by calling the ST_Register_Spatial_Column + stored procedure. + + +.. _create_mssql_connection: + +Connecting to MSSQL Spatial +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In addition to some of the options in :ref:`vector_create_stored_connection`, +creating a new MSSQL connection dialog proposes you to fill a **Provider/DSN** +name. You can also display available databases. + + +.. _vector_loading_postgis: + +Loading a Database Layer +........................ + +Once you have one or more connections defined to a database (see section +vector_create_stored_connection_), you can load layers from it. Of course, this +requires having available data. See e.g. section :ref:`vector_import_data_in_postgis` +for a discussion on importing data into a PostGIS database. + +To load a layer from a database, you can perform the following steps: + +#. Open the "Add table(s)" dialog + (see :ref:`vector_create_stored_connection`), +#. Choose the connection from the drop-down list and click **[Connect]**. +#. Select or unselect |checkbox| :guilabel:`Also list tables with no geometry`. +#. Optionally, use some |checkbox| :guilabel:`Search Options` to reduce the + list of tables to those matching your search. You can also set this option + before you hit the **[Connect]** button, speeding this way the database + fetching. +#. Find the layer(s) you wish to add in the list of available layers. +#. Select it by clicking on it. You can select multiple layers by holding + down the :kbd:`Shift` key while clicking. +#. If applicable, use the **[Set Filter]** button (or double-click the layer) + to start the :guilabel:`Query builder` dialog (See section + :ref:`vector_query_builder`) and define which features to load from the + selected layer. The filter expression appears in the ``sql`` column. + This restriction can be removed or edited in the :menuselection:`Layer + Properties --> General --> Provider Feature Filter` frame. +#. The checkbox in the ``Select at id`` column that is activated by default + gets the features ids without the attributes and speed in most cases the + data loading. +#. Click on the **[Add]** button to add the layer to the map. + + +.. _figure_add_postgis_tables: + +.. figure:: /static/user_manual/managing_data_source/addpostgistables.png + :align: center + + Add PostGIS Table(s) Dialog + + +.. tip:: **Load database table(s) from the Browser Panel** + + Like simple files, connected database are also listed in the + :guilabel:`Browser Panel`. Hence, you can load tables from databases using + the Browser: + + #. Find the layer to use with the |filterMap| :sup:`Filter Browser` tool at + the top the browser panel (see :ref:`browser_panel` for the search options); + #. select and drag-and-drop it in the map canvas. + + +QGIS Custom formats +=================== + +QGIS proposes two custom formats you can load in the application using their own +loading tool: + +* Temporary Scratch Layer: a memory layer that is bound to the project it's + opened with (see :ref:`vector_new_scratch_layer` for more information) +* Virtual Layers: a layer resulting from a query on other layer(s) + (see :ref:`vector_virtual_layers` for more information) + + +Connecting to web services +========================== + +With QGIS you can have access to different types of OGC web services (WM(T)S, +WFS(-T), CSW ...). Thanks to QGIS Server, you can also publish these services. +Description of these capabilities and how-to are provided in chapter +:ref:`sec_ogc`. + + diff --git a/source/docs/user_manual/managing_data_source/supported_data.rst b/source/docs/user_manual/managing_data_source/supported_data.rst new file mode 100644 index 00000000000..59c31b7766f --- /dev/null +++ b/source/docs/user_manual/managing_data_source/supported_data.rst @@ -0,0 +1,572 @@ +.. only:: html + + |updatedisclaimer| + +.. _supported_format: + +*********************************** + Exploring Data Formats and Fields +*********************************** + +.. only:: html + + .. contents:: + :local: + +.. The aim of this chapter is to describe and add information on particular + formats read/written by QGIS. Also their characteristics (particular geometry + type, fields type...) would be exposed. The idea is to give keys to the + reader to understand what he should be aware of when working with these + formats or how he could improve working with them in QGIS. + + +Raster data +=========== + +Raster data in GIS are matrices of discrete cells that represent features on, +above or below the earth's surface. Each cell in the raster grid has the same +size, and cells are usually rectangular (in QGIS they will always be +rectangular). Typical raster datasets include remote sensing data, such as +aerial photography, or satellite imagery and modelled data, such as an elevation +matrix. + +Unlike vector data, raster data typically do not have an associated database +record for each cell. They are geocoded by pixel resolution and the x/y +coordinate of a corner pixel of the raster layer. This allows QGIS to position +the data correctly in the map canvas. + +QGIS makes use of georeference information inside the raster layer (e.g., +:index:`GeoTiff`) or in an appropriate world file to properly display the data. + +.. if there are particularities for some raster formats that are worth mention, + put them here. Maybe some comments on working with vrt, landsat data...? + + +Vector Data +=========== + +Many of the features available in QGIS work the same, regardless the vector +data source. However, because of the differences in formats specifications +(ESRI shapefiles, MapInfo and MicroStation file formats, AutoCAD DXF, PostGIS, +SpatiaLite, DB2, Oracle Spatial and MSSQL Spatial databases, and many more), +QGIS may handle differently some of their properties. +This section describes how to work with these specificities. + +.. note:: + + QGIS supports (multi)point, (multi)line, (multi)polygon, CircularString, + CompoundCurve, CurvePolygon, MultiCurve, MultiSurface feature types, all + with Z and/or M values. + + You should note also that some drivers don't support some of these feature + types like CircularString, CompoundCurve, CurvePolygon, MultiCurve, + MultiSurface feature type. QGIS will convert them to (multi)polygon feature. + + +.. index:: ESRI, Shapefile, OGR +.. _vector_shapefiles: + +ESRI Shapefiles +--------------- + +The ESRI shapefile is still one of the most used vector file format in QGIS. +However, this file format has some limitation that some other file format have +not (like Geopackage, spatialite). Support is provided by the +`OGR Simple Feature Library `_. + +A shapefile actually consists of several files. The following three are +required: + +#. :file:`.shp` file containing the feature geometries +#. :file:`.dbf` file containing the attributes in dBase format +#. :file:`.shx` index file + +Shapefiles also can include a file with a :file:`.prj` suffix, which contains +the projection information. While it is very useful to have a projection file, +it is not mandatory. A shapefile dataset can contain additional files. For +further details, see the ESRI technical specification at +http://www.esri.com/library/whitepapers/pdfs/shapefile.pdf. + +**Improving Performance for Shapefiles** + +To improve the performance of drawing a shapefile, you can create a spatial +index. A spatial index will improve the speed of both zooming and panning. +Spatial indexes used by QGIS have a :file:`.qix` extension. + +Use these steps to create the index: + +* Load a shapefile (see :ref:`browser_panel`); +* Open the :guilabel:`Layer Properties` dialog by double-clicking on the + shapefile name in the legend or by right-clicking and choosing + :menuselection:`Properties` from the context menu. +* In the :guilabel:`General` tab, click the **[Create Spatial Index]** button. + +**Problem loading a shape .prj file** + +If you load a shapefile with a :file:`.prj` file and QGIS is not able to read the +coordinate reference system from that file, you will need to define the proper +projection manually within the :guilabel:`General` tab of the +:guilabel:`Layer Properties` dialog of the layer by clicking the +**[Specify...]** button. This is due to the fact that :file:`.prj` files +often do not provide the complete projection parameters as used in QGIS and +listed in the :guilabel:`CRS` dialog. + +For the same reason, if you create a new shapefile with QGIS, two different +projection files are created: a :file:`.prj` file with limited projection +parameters, compatible with ESRI software, and a :file:`.qpj` file, providing +the complete parameters of the used CRS. Whenever QGIS finds a :file:`.qpj` +file, it will be used instead of the :file:`.prj`. + +.. index:: CSV, Delimited text files + see: Comma Separated Values; CSV +.. _vector_csv: + +Delimited Text Files +-------------------- + +Tabular data is a very common and widely used format because of its simplicity +and readability -- data can be viewed and edited even in a plain text editor. +A delimited text file is an attribute table with each column separated by a +defined character and each row separated by a line break. The first row usually +contains the column names. A common type of delimited text file is a CSV +(Comma Separated Values), with each column separated by a comma. + +Such data files can also contain positional information in two main forms: + +* As point coordinates in separate columns +* As well-known text (WKT) representation of geometry + +QGIS allows you to load a delimited text file as a layer or ordinal table. But +first check that the file meets the following requirements: + +#. The file must have a delimited header row of field names. This must be the + first line in the text file. +#. The header row must contain field(s) with geometry definition. These field(s) + can have any name. +#. The X and Y coordinates (if geometry is defined by coordinates) must be + specified as numbers. The coordinate system is not important. +#. If you have any data that is not a string (text) and the file is a CSV file, + you must have a CSVT file (see section :ref:`csvt_files`). + +As an example of a valid text file, we import the elevation point data file +:file:`elevp.csv` that comes with the QGIS sample dataset (see section +:ref:`label_sampledata`): + +:: + + X;Y;ELEV + -300120;7689960;13 + -654360;7562040;52 + 1640;7512840;3 + [...] + +Some items to note about the text file: + +#. The example text file uses ``;`` (semicolon) as delimiter. Any character can + be used to delimit the fields. +#. The first row is the header row. It contains the fields ``X``, ``Y`` and + ``ELEV``. +#. No quotes (``"``) are used to delimit text fields. +#. The X coordinates are contained in the ``X`` field. +#. The Y coordinates are contained in the ``Y`` field. + + +.. index:: CSV, CSVT +.. _csvt_files: + +CSVT Files +.......... + +When loading CSV files, the OGR driver assumes all fields are strings (i.e. text) +unless it is told otherwise. You can create a CSVT file to tell OGR (and QGIS) +what data type the different columns are: + + +.. csv-table:: + :header: "Type", "Name", "Example" + + "Whole number", "Integer", 4 + "Decimal number", "Real", 3.456 + "Date", "Date (YYYY-MM-DD)", 2016-07-28 + "Time", "Time (HH:MM:SS+nn)", 18:33:12+00 + "Date & Time", "DateTime (YYYY-MM-DD HH:MM:SS+nn)", 2016-07-28 18:33:12+00 + +The CSVT file is a **ONE line** plain text file with the data types in quotes +and separated by commas, e.g.:: + +"Integer","Real","String" + +You can even specify width and precision of each column, e.g.:: + +"Integer(6)","Real(5.5)","String(22)" + +This file is saved in the same folder as the :file:`.csv` file, with the same +name, but :file:`.csvt` as the extension. + +*You can find more information at* `GDAL CSV Driver `_. + +Others valuable informations for advanced users +............................................... + +Features with curved geometries (CircularString, CurvePolygon and CompoundCurve) are +supported. Here are three examples of such geometry types as a delimited text +with WKT geometries:: + + Label;WKT_geom + CircularString;CIRCULARSTRING(268 415,227 505,227 406) + CurvePolygon;CURVEPOLYGON(CIRCULARSTRING(1 3, 3 5, 4 7, 7 3, 1 3)) + CompoundCurve;COMPOUNDCURVE((5 3, 5 13), CIRCULARSTRING(5 13, 7 15, + 9 13), (9 13, 9 3), CIRCULARSTRING(9 3, 7 1, 5 3)) + +Delimited Text supports also Z and M coordinates in geometries:: + + LINESTRINGM(10.0 20.0 30.0, 11.0 21.0 31.0) + + +.. index:: PostGIS, PostgreSQL +.. _label_postgis: + +PostGIS Layers +-------------- + +PostGIS layers are stored in a PostgreSQL database. The advantages of PostGIS +are its spatial indexing, filtering and querying capabilities it provides. Using +PostGIS, vector functions such as select and identify work more accurately than +they do with OGR layers in QGIS. + + +.. _tip_postgis_layers: + +.. tip:: **PostGIS Layers** + + Normally, a PostGIS layer is defined by an entry in the geometry_columns + table. QGIS can load layers that do not have an entry in the geometry_columns + table. This includes both tables and views. Defining a spatial view provides + a powerful means to visualize your data. Refer to your PostgreSQL manual for + information on creating views. + +This section contains some details on how QGIS accesses PostgreSQL layers. +Most of the time, QGIS should simply provide you with a list of database +tables that can be loaded, and it will load them on request. However, if you +have trouble loading a PostgreSQL table into QGIS, the information below may +help you understand any QGIS messages and give you direction on changing +the PostgreSQL table or view definition to allow QGIS to load it. + +Primary key +........... + +QGIS requires that PostgreSQL layers contain a column that can be used +as a unique key for the layer. For tables, this usually means that the table +needs a primary key, or a column with a unique constraint on it. In QGIS, +this column needs to be of type int4 (an integer of size 4 bytes). +Alternatively, the ctid column can be used as primary key. If a table lacks +these items, the oid column will be used instead. Performance will be +improved if the column is indexed (note that primary keys are automatically +indexed in PostgreSQL). + +QGIS offers a checkbox **Select at id** that is activated by default. This +option gets the ids without the attributes which is faster in most cases. + +View +.... + +If the PostgreSQL layer is a view, the same requirement exists, but views +do not always have primary keys or columns with unique constraints on them. You +have to define a primary key field (has to be integer) in the QGIS dialog before +you can load the view. If a suitable column does not exist in the view, QGIS +will not load the layer. If this occurs, the solution is to alter the view so +that it does include a suitable column (a type of integer and either a primary +key or with a unique constraint, preferably indexed). + +As for table, a checkbox **Select at id** is activated by default (see above +for the meaning of the checkbox). It can make sense to disable this option when +you use expensive views. + +.. _layer_style_backup: + +QGIS layer_style table and database backup +.......................................... + +If you want to make a backup of your PostGIS database using the :file:`pg_dump` and +:file:`pg_restore` commands, and the default layer styles as saved by QGIS fail to +restore afterwards, you need to set the XML option to :file:`DOCUMENT` before the +restore command: + +.. code-block:: sql + + SET XML OPTION DOCUMENT; + + +Filter database side +.................... + +QGIS allows to filter features already on server side. Check the +:menuselection:`Settings --> Options --> Data Sources -->` |checkbox| +:menuselection:`Execute expressions on postgres server-side if possible` +checkbox to do so. Only supported expressions will be +sent to the database. Expressions using unsupported operators or functions will +gracefully fallback to local evaluation. + +Support of PostgreSQL data types +................................ + +Most of common data types are supported by the PostgreSQL provider: integer, float, +varchar, geometry, timestamp and array. + +.. index:: shp2pgsql + single: PostGIS; shp2pgsql +.. _vector_import_data_in_postgis: + +Importing Data into PostgreSQL +------------------------------ + +Data can be imported into PostgreSQL/PostGIS using several tools, including the +DB Manager plugin and the command line tools shp2pgsql and ogr2ogr. + +DB Manager +.......... + +QGIS comes with a core plugin named |dbManager| :sup:`DB Manager`. It can +be used to load shapefiles and other data formats, and it includes support for +schemas. See section :ref:`dbmanager` for more information. + +shp2pgsql +......... + +PostGIS includes an utility called **shp2pgsql** that can be used to import +shapefiles into a PostGIS-enabled database. For example, to import a +shapefile named :file:`lakes.shp` into a PostgreSQL database named +``gis_data``, use the following command: + +:: + + shp2pgsql -s 2964 lakes.shp lakes_new | psql gis_data + +This creates a new layer named ``lakes_new`` in the ``gis_data`` database. +The new layer will have a spatial reference identifier (SRID) of 2964. +See section :ref:`label_projections` for more information on spatial +reference systems and projections. + +.. index:: pgsql2shp + +.. _tip_export_from_postgis: + +.. tip:: **Exporting datasets from PostGIS** + + Like the import tool **shp2pgsql**, there is also a tool to export + PostGIS datasets as shapefiles: **pgsql2shp**. This is shipped within + your PostGIS distribution. + +.. index:: ogr2ogr + single: PostGIS; ogr2ogr + +ogr2ogr +....... + +Besides **shp2pgsql** and **DB Manager**, there is another tool for feeding geodata +in PostGIS: **ogr2ogr**. This is part of your GDAL installation. + +To import a shapefile into PostGIS, do the following: +:: + + ogr2ogr -f "PostgreSQL" PG:"dbname=postgis host=myhost.de user=postgres + password=topsecret" alaska.shp + +This will import the shapefile :file:`alaska.shp` into the PostGIS database +*postgis* using the user *postgres* with the password *topsecret* on host +server *myhost.de*. + +Note that OGR must be built with PostgreSQL to support PostGIS. +You can verify this by typing (in |nix|) +:: + + ogrinfo --formats | grep -i post + + +If you prefer to use PostgreSQL's **COPY** command instead of the default +**INSERT INTO** method, you can export the following environment variable +(at least available on |nix| and |osx|): +:: + + export PG_USE_COPY=YES + +**ogr2ogr** does not create spatial indexes like **shp2pgsl** does. You +need to create them manually, using the normal SQL command **CREATE INDEX** +afterwards as an extra step (as described in the next section +:ref:`vector_improving_performance`). + +.. index:: Spatial index; GiST index + single: PostGIS; Spatial index +.. _vector_improving_performance: + +Improving Performance +..................... + +Retrieving features from a PostgreSQL database can be time-consuming, especially +over a network. You can improve the drawing performance of PostgreSQL layers by +ensuring that a PostGIS spatial index exists on each layer in the +database. PostGIS supports creation of a GiST (Generalized Search Tree) +index to speed up spatial searches of the data (GiST index information is taken +from the PostGIS documentation available at http://postgis.net). + +.. tip:: You can use the DBManager to create an index to your layer. You should + first select the layer and click on :menuselection:`Table --> Edit table`, go + to :menuselection:`Indexes` tab and click on **[Add spatial index]**. + +The syntax for creating a GiST index is: +:: + + + CREATE INDEX [indexname] ON [tablename] + USING GIST ( [geometryfield] GIST_GEOMETRY_OPS ); + + +Note that for large tables, creating the index can take a long time. Once the +index is created, you should perform a ``VACUUM ANALYZE``. See the PostGIS +documentation (POSTGIS-PROJECT :ref:`literature_and_web`) for more information. + +The following is an example of creating a GiST index: +:: + + gsherman@madison:~/current$ psql gis_data + Welcome to psql 8.3.0, the PostgreSQL interactive terminal. + + Type: \copyright for distribution terms + \h for help with SQL commands + \? for help with psql commands + \g or terminate with semicolon to execute query + \q to quit + + gis_data=# CREATE INDEX sidx_alaska_lakes ON alaska_lakes + gis_data-# USING GIST (the_geom GIST_GEOMETRY_OPS); + CREATE INDEX + gis_data=# VACUUM ANALYZE alaska_lakes; + VACUUM + gis_data=# \q + gsherman@madison:~/current$ + +.. index:: PostGIS; ST_Shift_Longitude + +Vector layers crossing 180 |degrees| longitude +---------------------------------------------- + +Many GIS packages don't wrap vector maps with a geographic reference system +(lat/lon) crossing the 180 degrees longitude line +(http://postgis.refractions.net/documentation/manual-2.0/ST\_Shift\_Longitude.html). +As result, if we open such a map in QGIS, we will see two far, distinct locations, +that should appear near each other. In Figure_vector_crossing_, the tiny point on the far +left of the map canvas (Chatham Islands) should be within the grid, to the right of the +New Zealand main islands. + +.. _figure_vector_crossing: + +.. figure:: /static/user_manual/managing_data_source/vectorNotWrapping.png + :align: center + + Map in lat/lon crossing the 180 |degrees| longitude line + +A work-around is to transform the longitude values using PostGIS and the +**ST_Shift_Longitude** function. This function reads every point/vertex in every +component of every feature in a geometry, and if the longitude coordinate is +< 0 |degrees|, it adds 360 |degrees| to it. The result is a 0 |degrees| - 360 |degrees| +version of the data to be plotted in a 180 |degrees|-centric map. + +.. _figure_vector_crossing_map: + +.. figure:: /static/user_manual/managing_data_source/vectorWrapping.png + :align: center + :width: 25em + + Crossing 180 |degrees| longitude applying the **ST_Shift_Longitude** + function + +Usage +..... + +* Import data into PostGIS (:ref:`vector_import_data_in_postgis`) using, + for example, the DB Manager plugin. +* Use the PostGIS command line interface to issue the following command + (in this example, "TABLE" is the actual name of your PostGIS table): + ``gis_data=# update TABLE set the_geom=ST_Shift_Longitude(the_geom);`` +* If everything went well, you should receive a confirmation about the + number of features that were updated. Then you'll be able to load the + map and see the difference (Figure_vector_crossing_map_). + +.. index:: Spatialite, SQLite +.. _spatialite_data: + +SpatiaLite Layers +----------------- + +If you want to save a vector layer to SpatiaLite format, you can do this by +right clicking the layer in the legend. Then, click on :menuselection:`Save as...`, +define the name of the output file, and select 'SpatiaLite' as format and the CRS. +Also, you can select 'SQLite' as format and then add ``SPATIALITE=YES`` in the +OGR data source creation option field. This tells OGR to create a SpatiaLite +database. See also http://www.gdal.org/ogr/drv_sqlite.html. + +QGIS also supports editable views in SpatiaLite. + +If you want to create a new SpatiaLite layer, please refer to section +:ref:`vector_create_spatialite`. + +.. index:: QSpatiaLite, Spatialite manager, DB Manager + +.. _tip_spatialite_management_plugin: + +.. tip:: **SpatiaLite data management Plugins** + + For SpatiaLite data management, you can also use several Python plugins: + QSpatiaLite, SpatiaLite Manager or :ref:`DB Manager ` (core plugin, + recommended). If necessary, they can be downloaded and installed with the + Plugin Installer. + +.. index:: DB2 Spatial +.. _label_db2_spatial: + +DB2 Spatial Layers +------------------ + +IBM DB2 for Linux, Unix and Windows (DB2 LUW), IBM DB2 for z/OS (mainframe) +and IBM DashDB products allow +users to store and analyse spatial data in relational table columns. +The DB2 provider for QGIS supports the full range of visualization, analysis +and manipulation of spatial data in these databases. + +.. _DB2 z/OS KnowledgeCenter: https://www.ibm.com/support/knowledgecenter/en/SSEPEK_11.0.0/spatl/src/tpc/spatl_db2sb03.html +.. _DB2 LUW KnowledgeCenter: http://www.ibm.com/support/knowledgecenter/SSEPGG_11.1.0/com.ibm.db2.luw.spatial.topics.doc/doc/db2sb03.html +.. _DB2 DashDB KnowledgeCenter: https://www.ibm.com/support/knowledgecenter/SS6NHC/com.ibm.db2.luw.spatial.topics.doc/doc/csbp1001.html +.. _DB2 Spatial Tutorial: https://www.ibm.com/developerworks/data/tutorials/dm-1202db2spatialdata1/ + +User documentation on these capabilities can be found at the +`DB2 z/OS KnowledgeCenter`_, `DB2 LUW KnowledgeCenter`_ +and `DB2 DashDB KnowledgeCenter`_. + +For more information about working with the DB2 spatial capabilities, check out +the `DB2 Spatial Tutorial`_ on IBM DeveloperWorks. + + +The DB2 provider currently only supports the Windows environment through the +Windows ODBC driver. + +The client running QGIS needs to have one of the following installed: + +* DB2 LUW +* IBM Data Server Driver Package +* IBM Data Server Client + +If you are accessing a DB2 LUW database on the same machine or using DB2 LUW as +a client, the DB2 executables and supporting files need to be included in the +Windows path. This can be done by creating a batch file like the following with +the name **db2.bat** and including it in the directory **%OSGEO4W_ROOT%/etc/ini**. + +:: + + @echo off + REM Point the following to where DB2 is installed + SET db2path=C:\Program Files (x86)\sqllib + REM This should usually be ok - modify if necessary + SET gskpath=C:\Program Files (x86)\ibm\gsk8 + SET Path=%db2path%\BIN;%db2path%\FUNCTION;%gskpath%\lib64;%gskpath%\lib;%path% + + diff --git a/source/docs/user_manual/qgis_browser/qgis_browser.rst b/source/docs/user_manual/qgis_browser/qgis_browser.rst deleted file mode 100644 index fd8ebbef648..00000000000 --- a/source/docs/user_manual/qgis_browser/qgis_browser.rst +++ /dev/null @@ -1,70 +0,0 @@ -.. only:: html - - |updatedisclaimer| - -.. index:: Browse data, Add layers -.. _browser_panel: - -************* -QGIS Browser -************* - - -QGIS Browser is a panel in QGIS Desktop that lets you easily navigate in your -filesystem and manage geodata. You can have access to common vector files -(e.g., ESRI shapefiles or MapInfo files), databases (e.g., PostGIS, Oracle, -SpatiaLite, GeoPackage or MSSQL Spatial) and OWS/WCS|WMS|WFS connections. You -can also view your GRASS data (to get the data into QGIS, see :ref:`sec_grass`). - - -To activate QGIS Browser, right-click on QGIS toolbar and check |checkbox| -:guilabel:`Browser Panel` or select it from :menuselection:`View --> Panels` -(or |kde| :menuselection:`Settings --> Panels`). -In the :guilabel:`Browser` panel, you can now browse in your filesystem, -databases and web services and get your data into the map view with a -simple drag-and-drop or double-click. - -You can also open a QGIS project directly from the Browser panel by double-clicking -its name or by drag-and-drop into the map view. - -At the top of the panel, you find some icons that help you to: - -* |addLayer| Add Selected Layers. You can also add data into the map view - by selecting **Add selected layer(s)** in the context menu. -* |draw| Refresh the browser tree -* |filterMap| search for specific data. Enter a search word or wildcard - and the browser will filter the tree to only show paths to matching DB tables, filenames - or folders -- other data or folders won't be displayed. See the Browser Panel(2) - example on the figure_browser_panels_. The comparison can be case-sensitive or not. - It can also be set to: - - * normal: return any item containing the search text - * using wildcard(s): fine tune the search using ``?`` and/or ``*`` characters to - specify the position of the search text - * using a regular expression - -* |collapseTree| Collapse the whole tree -* |propertiesWidget| Enable and disable properties widget. When toggled on, - a new widget is added at the bottom of the panel showing, if applicable, - metadatas of the selected item - -Right-click an item in the browser tree helps you to: - -* in case of file or table, display its metadata or open it in your project. - Tables can even be renamed, deleted or truncated -* in case of folder, bookmark it into your favourites, hide it from the browser tree. - Hidden folders can be managed from the :menuselection:`Settings --> Options - --> Data Sources` tab -* refresh, rename or delete schema. - -You can also import files into databases or copy tables from one schema/database -to another one with a simple drag-and-drop. There is a second browser panel -available to avoid long scrolling while dragging. Just select the file and -drag-and-drop from one panel to the other. - -.. _figure_browser_panels: - -.. figure:: /static/user_manual/qgis_browser/browser_panels.png - :align: center - - QGIS Browser panels side-by-side diff --git a/source/docs/user_manual/working_with_raster/index.rst b/source/docs/user_manual/working_with_raster/index.rst index aaa9243380b..2fa925042ae 100644 --- a/source/docs/user_manual/working_with_raster/index.rst +++ b/source/docs/user_manual/working_with_raster/index.rst @@ -11,9 +11,6 @@ Working with Raster Data .. toctree:: :maxdepth: 2 - supported_data raster_properties raster_analysis - - diff --git a/source/docs/user_manual/working_with_raster/supported_data.rst b/source/docs/user_manual/working_with_raster/supported_data.rst deleted file mode 100644 index 38b48ee0e07..00000000000 --- a/source/docs/user_manual/working_with_raster/supported_data.rst +++ /dev/null @@ -1,95 +0,0 @@ -.. only:: html - - |updatedisclaimer| - -.. index:: Raster, GDAL -.. index:: ArcInfo Binary Grid, ArcInfo ASCII Grid, GeoTIFF -.. index:: Erdas Imagine - -************************* -Working with Raster Data -************************* - -.. only:: html - - .. contents:: - :local: - - -This section describes how to visualize and set raster layer properties. -QGIS uses the :index:`GDAL` library to read and write raster data formats, -including ArcInfo Binary Grid, ArcInfo ASCII Grid, GeoTIFF, ERDAS IMAGINE, -and many more. GRASS raster support is supplied by a native QGIS data provider -plugin. The raster data can also be loaded in read mode from zip and gzip -archives into QGIS. - -As of the date of this document, more than 100 raster formats are supported by the -GDAL library (see GDAL-SOFTWARE-SUITE in :ref:`literature_and_web`). A complete -list is available at http://www.gdal.org/formats_list.html. - -.. note:: - Not all of the listed formats may work in QGIS for various reasons. For example, - some require external commercial libraries, or the GDAL installation of your OS - may not have been built to support the format you want to use. Only those formats that - have been well tested will appear in the list of file types when loading a - raster into QGIS. Other untested formats can be loaded by selecting the - ``[GDAL] All files (*)`` filter. - -Working with GRASS raster data is described in section :ref:`sec_grass`. - - -What is raster data? -==================== - -Raster data in GIS are matrices of discrete cells that represent features on, -above or below the earth's surface. Each cell in the raster grid is the same -size, and cells are usually rectangular (in QGIS they will always be -rectangular). Typical raster datasets include remote sensing data, such as -aerial photography, or satellite imagery and modelled data, such as an elevation -matrix. - -Unlike vector data, raster data typically do not have an associated database -record for each cell. They are geocoded by pixel resolution and the x/y -coordinate of a corner pixel of the raster layer. This allows QGIS to position -the data correctly in the map canvas. - -QGIS makes use of georeference information inside the raster layer (e.g., :index:`GeoTiff`) -or in an appropriate world file to properly display the data. - -.. index:: Loading raster - -.. _load_raster: - -Loading raster data in QGIS -=========================== - -Raster layers are loaded either by clicking on the |addRasterLayer| -:sup:`Add Raster Layer` icon or by selecting the :menuselection:`Layer -->` -|addRasterLayer| :guilabel:`Add Raster Layer` menu option. More than one -layer can be loaded at the same time by holding down the :kbd:`Ctrl` or -:kbd:`Shift` key and clicking on multiple items in the -:guilabel:`Open a GDAL Supported Raster Data Source` dialog. - - -Once a raster layer is loaded in the map legend, you can click on the layer name -with the right mouse button to select and activate layer-specific features or to -open a dialog to set raster properties for the layer. - -**Right mouse button menu for raster layers** - -* :menuselection:`Zoom to Layer Extent` -* :menuselection:`Zoom to Best Scale (100\%)` -* :menuselection:`Stretch Using Current Extend` -* :menuselection:`Show in Overview` -* :menuselection:`Remove` -* :menuselection:`Duplicate` -* :menuselection:`Set Layer CRS` -* :menuselection:`Set Project CRS from Layer` -* :menuselection:`Save as ...` -* :menuselection:`Properties` -* :menuselection:`Rename` -* :menuselection:`Copy Style` -* :menuselection:`Add New Group` -* :menuselection:`Expand all` -* :menuselection:`Collapse all` -* :menuselection:`Update Drawing Order` diff --git a/source/docs/user_manual/working_with_vector/attribute_table.rst b/source/docs/user_manual/working_with_vector/attribute_table.rst index 012bbedef21..46fa03814a4 100644 --- a/source/docs/user_manual/working_with_vector/attribute_table.rst +++ b/source/docs/user_manual/working_with_vector/attribute_table.rst @@ -421,21 +421,6 @@ and CRS (see section :ref:`general_saveas`). To save the selection ensure that the |checkbox| :menuselection:`Save only selected features` is selected. It is also possible to specify OGR creation options within the dialog. -.. _paste_into_layer: - -Paste into new layer ---------------------- - -Features that are on the clipboard may be pasted into a new -layer. To do this, first make a layer editable. Select some features, copy -them to the clipboard, and then paste them into a new layer using -:menuselection:`Edit --> Paste Features as` and choosing -:menuselection:`New vector layer` or :menuselection:`New memory -layer`. - -This applies to features selected and copied within QGIS and -also to features from another source defined using well-known text (WKT). - .. index:: Field Calculator, Derived Fields, Virtual Fields, Fields edit .. _calculate_fields_values: diff --git a/source/docs/user_manual/working_with_vector/editing_geometry_attributes.rst b/source/docs/user_manual/working_with_vector/editing_geometry_attributes.rst index 2c857dfc052..9f0323ebc51 100644 --- a/source/docs/user_manual/working_with_vector/editing_geometry_attributes.rst +++ b/source/docs/user_manual/working_with_vector/editing_geometry_attributes.rst @@ -1227,113 +1227,3 @@ and angle entered. Repeating the process, several points can be added. Points at given distance and angle - -.. index:: Create new layers -.. index:: Shapefile, SpatiaLite, GPX - -.. _sec_create_vector: - -Creating new Vector layers -========================== - -QGIS allows you to create new shapefile layers, new SpatiaLite layers, new -GPX layers and New Temporary Scratch Layers. Creation of a new GRASS layer -is supported within the GRASS plugin. -Please refer to section :ref:`creating_new_grass_vectors` for more information -on creating GRASS vector layers. - -Creating a new Shapefile layer ------------------------------- - -To create a new shape layer for editing, choose :menuselection:`New -->` -|newVectorLayer| :menuselection:`New Shapefile Layer...` from the -:menuselection:`Layer` menu. The :guilabel:`New Vector Layer` dialog will be -displayed as shown in figure_create_shapefile_. Choose the type of layer -(point, line or polygon) and the CRS (coordinate reference system). - -Note that QGIS does not yet support creation of 2.5D features (i.e., features -with X,Y,Z coordinates). - -.. _figure_create_shapefile: - -.. figure:: /static/user_manual/working_with_vector/editNewVector.png - :align: center - - Creating a new Shapefile layer dialog - -To complete the creation of the new shapefile layer, add the desired attributes -by clicking on the **[Add to attributes list]** button and specifying a name and -type for the attribute. A first 'id' column is added as default but can be -removed, if not wanted. Only :guilabel:`Type: real` |selectString|, -:guilabel:`Type: integer` |selectString|, :guilabel:`Type: string` -|selectString| and :guilabel:`Type:date` |selectString| attributes are -supported. Additionally and according to the attribute type, you can also define -the width and precision of the new attribute column. Once you are happy with the -attributes, click **[OK]** and provide a name for the shapefile. QGIS will -automatically add a :file:`.shp` extension to the name you specify. Once the -layer has been created, it will be added to the map, and you can edit it in the -same way as described in section :ref:`sec_edit_existing_layer` above. - -.. index:: New SpatiaLite layer - -.. _vector_create_spatialite: - -Creating a new SpatiaLite layer -------------------------------- - -To create a new SpatiaLite layer for editing, choose :menuselection:`New -->` -|newSpatiaLiteLayer| :menuselection:`New SpatiaLite Layer...` from the -:menuselection:`Layer` menu. The :guilabel:`New SpatiaLite Layer` dialog will -be displayed as shown in Figure_create_spatialite_. - -.. _figure_create_spatialite: - -.. figure:: /static/user_manual/working_with_vector/editNewSpatialite.png - :align: center - - Creating a New SpatiaLite layer dialog - -The first step is to select an existing SpatiaLite database or to create a new -SpatiaLite database. This can be done with the browse button |browseButton| to -the right of the database field. Then, add a name for the new layer, define -the layer type, and specify the coordinate reference system with **[Specify CRS]**. -If desired, you can select |checkbox| :guilabel:`Create an autoincrementing primary key`. - -To define an attribute table for the new SpatiaLite layer, add the names of -the attribute columns you want to create with the corresponding column type, and -click on the **[Add to attribute list]** button. Once you are happy with the -attributes, click **[OK]**. QGIS will automatically add the new layer to the -legend, and you can edit it in the same way as described in section -:ref:`sec_edit_existing_layer` above. - -Further management of SpatiaLite layers can be done with the DB Manager. See -:ref:`dbmanager`. - -.. index:: New GPX layer -.. _vector_create_gpx: - -Creating a new GPX layer -------------------------- - -To create a new GPX file, you need to load the GPS plugin first. :menuselection:`Plugins -->` -|showPluginManager| :menuselection:`Plugin Manager...` opens the Plugin Manager Dialog. -Activate the |checkbox| :guilabel:`GPS Tools` checkbox. - -When this plugin is loaded, choose :menuselection:`New -->` |createGPX| -:menuselection:`Create new GPX Layer...` from the :menuselection:`Layer` menu. -In the :guilabel:`Save new GPX file as` dialog, you can choose where to save the -new GPX layer. - -.. index:: New Temporary Scratch layer -.. _vector_new_scratch_layer: - -Creating a new Temporary Scratch Layer --------------------------------------- - -Empty, editable memory layers can be defined using :menuselection:`Layer --> -Create Layer --> New Temporary Scratch Layer`. Here you can even create -|radioButtonOff|:guilabel:`Multipoint`, |radioButtonOff|:guilabel:`Multiline` -and |radioButtonOff|:guilabel:`Multipolygon` Layers beneath |radioButtonOn|:guilabel:`Point`, -|radioButtonOff|:guilabel:`Line` and |radioButtonOff|:guilabel:`Polygon` Layers. -Temporary Scratch Layers are not saved and will be discarded when QGIS is closed. -See also :ref:`paste_into_layer`. diff --git a/source/docs/user_manual/working_with_vector/index.rst b/source/docs/user_manual/working_with_vector/index.rst index 731dfbf8476..c2556de2d5d 100644 --- a/source/docs/user_manual/working_with_vector/index.rst +++ b/source/docs/user_manual/working_with_vector/index.rst @@ -11,11 +11,9 @@ .. toctree:: :maxdepth: 2 - supported_data style_library vector_properties expression attribute_table editing_geometry_attributes - virtual_layers diff --git a/source/docs/user_manual/working_with_vector/supported_data.rst b/source/docs/user_manual/working_with_vector/supported_data.rst deleted file mode 100644 index a0c5c2beaa8..00000000000 --- a/source/docs/user_manual/working_with_vector/supported_data.rst +++ /dev/null @@ -1,1147 +0,0 @@ -.. only:: html - - |updatedisclaimer| - -.. _supported_format: - -Supported Data Formats -====================== - -.. only:: html - - .. contents:: - :local: - -QGIS uses the OGR library to read and write vector data formats, -including ESRI shapefiles, MapInfo and MicroStation file formats, AutoCAD DXF, -PostGIS, SpatiaLite, DB2, Oracle Spatial and MSSQL Spatial databases, and many more. -GRASS vector and PostgreSQL support is supplied by native QGIS data provider -plugins. Vector data can also be loaded in read mode from zip and gzip archives -into QGIS. As of the date of this document, 69 vector formats are supported by -the OGR library (see OGR-SOFTWARE-SUITE in :ref:`literature_and_web`). The -complete list is available at http://www.gdal.org/ogr/ogr_formats.html. - -.. note:: - - Not all of the listed formats may work in QGIS for various reasons. For - example, some require external commercial libraries, or the GDAL/OGR - installation of your OS may not have been built to support the format you - want to use. Only those formats that have been well tested will appear in - the list of file types when loading a vector into QGIS. Other untested - formats can be loaded by selecting ``*.*``. - -Working with GRASS vector data is described in Section :ref:`sec_grass`. - -This section describes how to work with several common formats: ESRI -shapefiles, PostGIS layers, SpatiaLite layers, OpenStreetMap vectors, and Comma -Separated data (CSV). Many of the features available in QGIS work the same, -regardless of the vector data source. This is by design, and it includes the -identify, select, labelling and attributes functions. - -.. note:: - - QGIS supports (multi)point, (multi)line, (multi)polygon, CircularString, - CompoundCurve, CurvePolygon, MultiCurve, MultiSurface feature types, all - with Z and/or M values. - - You should note also that some drivers don't support some of these feature - types like CircularString, CompoundCurve, CurvePolygon, MultiCurve, - MultiSurface feature type. QGIS will convert them to (multi)polygon feature. - -.. index:: MapInfo, Vector file, Load a shapefile, Shapefile -.. _vector_loading_file: - -Loading a layer from a file ---------------------------- - -|addOgrLayer| To load a layer from a file (like a Shapefile, a Mapinfo or a dxf -layer), click on the |addOgrLayer| :sup:`Add Vector Layer` toolbar button; or -type :kbd:`Ctrl+Shift+V`. This will bring up a new window (see -figure_vector_add_). - -.. _figure_vector_add: - -.. figure:: /static/user_manual/working_with_vector/addvectorlayerdialog.png - :align: center - - Add Vector Layer Dialog - -From the available options check |radioButtonOn| :guilabel:`File`. Click on -**[Browse]**. That will bring up a standard open file dialog -(see figure_vector_open_), which allows you to navigate the file system and load a -shapefile or other supported data source. The selection box :guilabel:`Filter` -|selectString| allows you to preselect some OGR-supported file formats. - -You can also select the encoding for the file if desired. - -.. _figure_vector_open: - -.. figure:: /static/user_manual/working_with_vector/shapefileopendialog.png - :align: center - - Open an OGR Supported Vector Layer Dialog - -Selecting a file from the list and clicking **[Open]** loads it into QGIS. -Figure_vector_loaded_ shows QGIS after loading the :file:`alaska.shp` file. - -.. _figure_vector_loaded: - -.. figure:: /static/user_manual/working_with_vector/shapefileloaded.png - :align: center - - QGIS with Shapefile of Alaska loaded - -.. tip:: **Layer Colors** - - When you add a layer to the map, it is assigned a random color. When adding - more than one layer at a time, different colors are assigned to each layer. - -Once a file is loaded, you can zoom around it using the map navigation tools. -To change the style of a layer, open the :guilabel:`Layer Properties` dialog -by double clicking on the layer name or by right-clicking on the name in the -legend and choosing :menuselection:`Properties` from the context menu. See -section :ref:`vector_style_menu` for more information on setting symbology of -vector layers. - -.. _tip_load_from_external_drive_OSX: - -.. tip:: **Load layer and project from mounted external drives on macOS** - - On macOS, portable drives that are mounted beside the primary hard drive - do not show up as expected under :menuselection:`File --> Open Project`. - We are working on a more macOS-native open/save dialog to fix this. - As a workaround, you can type ``/Volumes`` in the :guilabel:`File name` box - and press :kbd:`Enter`. Then you can navigate to external drives and network - mounts. - -.. note:: DXF files containing several geometry types (point, line and/or - polygon), the name of the layer will be made from * entities - *. - -.. note:: You can also drag and drop the file(s) into the :guilabel:`Layers - Panel` from either the files browser or the QGIS Browser panel. If the layer - contains several geometry types, a new windows will ask you to select the - sublayer. This often occurs with GPX, Mapinfo or DXF files format. - -.. index:: ArcInfo Binary Coverage, Tiger Format, UK National Transfer Format, US Census Bureau -.. _vector_loading_directory_based_layer: - -Loading specific directory based layer -...................................... - -|addOgrLayer| To load some specific format like ArcInfo Binary Coverage, UK. -National Transfer Format, as well as the raw TIGER format of the US Census -Bureau or OpenfileGDB, click on the |addOgrLayer| :sup:`Add Vector Layer` -toolbar button or press :kbd:`Ctrl+Shift+V` to open the -:guilabel:`Add Vector Layer` dialog. Select |radioButtonOn| -:guilabel:`Directory` as :guilabel:`Source type`. Change the file type filter -:guilabel:`Files of type` |selectString| to the format you want to open, for -example 'Arc/Info Binary Coverage'. Navigate to the directory that contains the -coverage file or the file, and select it. - -.. index:: ESRI, Shapefile, OGR -.. _vector_shapefiles: - -ESRI Shapefiles -................ - -The ESRI shapefile is still one of the most used vector file format in QGIS. -However, this file format has some limitation that some other file format have -not (like Geopackage, spatialite). Support is provided by the -`OGR Simple Feature Library `_. - -A shapefile actually consists of several files. The following three are -required: - -#. :file:`.shp` file containing the feature geometries -#. :file:`.dbf` file containing the attributes in dBase format -#. :file:`.shx` index file - -Shapefiles also can include a file with a :file:`.prj` suffix, which contains -the projection information. While it is very useful to have a projection file, -it is not mandatory. A shapefile dataset can contain additional files. For -further details, see the ESRI technical specification at -http://www.esri.com/library/whitepapers/pdfs/shapefile.pdf. - -**Improving Performance for Shapefiles** - -To improve the performance of drawing a shapefile, you can create a spatial -index. A spatial index will improve the speed of both zooming and panning. -Spatial indexes used by QGIS have a :file:`.qix` extension. - -Use these steps to create the index: - -* Load a shapefile by clicking on the |addOgrLayer| :sup:`Add Vector Layer` - toolbar button or pressing :kbd:`Ctrl+Shift+V`. -* Open the :guilabel:`Layer Properties` dialog by double-clicking on the - shapefile name in the legend or by right-clicking and choosing - :menuselection:`Properties` from the context menu. -* In the :guilabel:`General` tab, click the **[Create Spatial Index]** button. - -**Problem loading a shape .prj file** - -If you load a shapefile with a :file:`.prj` file and QGIS is not able to read the -coordinate reference system from that file, you will need to define the proper -projection manually within the :guilabel:`General` tab of the -:guilabel:`Layer Properties` dialog of the layer by clicking the -**[Specify...]** button. This is due to the fact that :file:`.prj` files -often do not provide the complete projection parameters as used in QGIS and -listed in the :guilabel:`CRS` dialog. - -For the same reason, if you create a new shapefile with QGIS, two different -projection files are created: a :file:`.prj` file with limited projection -parameters, compatible with ESRI software, and a :file:`.qpj` file, providing -the complete parameters of the used CRS. Whenever QGIS finds a :file:`.qpj` -file, it will be used instead of the :file:`.prj`. - -.. index:: CSV, Delimited text files - see: Comma Separated Values; CSV -.. _vector_csv: - -Delimited Text Files --------------------- - -Tabular data is a very common and widely used format because of its simplicity -and readability -- data can be viewed and edited even in a plain text editor. -A delimited text file is an attribute table with each column separated by a -defined character and each row separated by a line break. The first row usually -contains the column names. A common type of delimited text file is a CSV -(Comma Separated Values), with each column separated by a comma. - -Such data files can also contain positional information in two main forms: - -* As point coordinates in separate columns -* As well-known text (WKT) representation of geometry - -QGIS allows you to load a delimited text file as a layer or ordinal table. But -first check that the file meets the following requirements: - -#. The file must have a delimited header row of field names. This must be the - first line in the text file. -#. The header row must contain field(s) with geometry definition. These field(s) - can have any name. -#. The X and Y coordinates (if geometry is defined by coordinates) must be - specified as numbers. The coordinate system is not important. -#. If you have any data that is not a string (text) and the file is a CSV file, - you must have a CSVT file (see section :ref:`csvt_files`). - -As an example of a valid text file, we import the elevation point data file -:file:`elevp.csv` that comes with the QGIS sample dataset (see section -:ref:`label_sampledata`): - -:: - - X;Y;ELEV - -300120;7689960;13 - -654360;7562040;52 - 1640;7512840;3 - [...] - -Some items to note about the text file: - -#. The example text file uses ``;`` (semicolon) as delimiter. Any character can - be used to delimit the fields. -#. The first row is the header row. It contains the fields ``X``, ``Y`` and - ``ELEV``. -#. No quotes (``"``) are used to delimit text fields. -#. The X coordinates are contained in the ``X`` field. -#. The Y coordinates are contained in the ``Y`` field. - -.. _vector_loading_csv: - -Loading a delimited text file -............................. - -Click the toolbar icon |delimitedText| :sup:`Add Delimited Text Layer` in the -:guilabel:`Manage layers` toolbar to open the :guilabel:`Create a Layer from a -Delimited Text File` dialog, as shown in figure_delimited_text_. - -.. _figure_delimited_text: - -.. figure:: /static/user_manual/introduction/delimited_text_dialog.png - :align: center - - Delimited Text Dialog - -First, select the file to import (e.g., :file:`qgis_sample_data/csv/elevp.csv`) -by clicking on the **[Browse]** button. Once the file is selected, QGIS -attempts to parse the file with the most recently used delimiter. To enable QGIS to properly parse the -file, it is important to select the correct delimiter. You can specify a -delimiter by activating |radioButtonOn| :guilabel:`Custom delimiters`, or by activating -|radioButtonOn| :guilabel:`Regular expression delimiter` and entering -text into the :guilabel:`Expression` field. For example, to -change the delimiter to tab, use ``\t`` (this is a regular expression for the -tab character). - -Once the file is parsed, set :guilabel:`Geometry definition` to -|radioButtonOn|:guilabel:`Point coordinates` and choose the ``X`` and ``Y`` -fields from the dropdown lists. If the coordinates are defined as -degrees/minutes/seconds, activate the |checkbox| :guilabel:`DMS coordinates` -checkbox. - -Finally, enter a layer name (e.g., :file:`elevp`), as shown in -figure_delimited_text_. To add the layer to the map, click **[OK]**. The -delimited text file now behaves as any other map layer in QGIS. - -There is also a helper option that allows you to trim leading and trailing -spaces from fields --- |checkbox| :guilabel:`Trim fields`. Also, it is possible -to |checkbox| :guilabel:`Discard empty fields`. If necessary, you can force a comma -to be the decimal separator by activating |checkbox| :guilabel:`Decimal separator is -comma`. - -If spatial information is represented by WKT, activate the |radioButtonOn| -:guilabel:`Well Known Text` option and select the field with the WKT definition for -point, line or polygon objects. If the file contains non-spatial data, activate -|radioButtonOn| :guilabel:`No geometry (attribute only table)` and it will be -loaded as an ordinal table. - -Additionally, you can enable: - -* |checkbox| :guilabel:`Use spatial index` to improve the performance of displaying - and spatially selecting features. -* |checkbox| :guilabel:`Use subset index`. -* |checkbox| :guilabel:`Watch file` to watch for changes to the file by other - applications while QGIS is running. - -.. index:: CSV, CSVT -.. _csvt_files: - -CSVT Files -.......... - -When loading CSV files, the OGR driver assumes all fields are strings (i.e. text) -unless it is told otherwise. You can create a CSVT file to tell OGR (and QGIS) -what data type the different columns are: - - -.. csv-table:: - :header: "Type", "Name", "Example" - - "Whole number", "Integer", 4 - "Decimal number", "Real", 3.456 - "Date", "Date (YYYY-MM-DD)", 2016-07-28 - "Time", "Time (HH:MM:SS+nn)", 18:33:12+00 - "Date & Time", "DateTime (YYYY-MM-DD HH:MM:SS+nn)", 2016-07-28 18:33:12+00 - -The CSVT file is a **ONE line** plain text file with the data types in quotes -and separated by commas, e.g.:: - -"Integer","Real","String" - -You can even specify width and precision of each column, e.g.:: - -"Integer(6)","Real(5.5)","String(22)" - -This file is saved in the same folder as the :file:`.csv` file, with the same -name, but :file:`.csvt` as the extension. - -*You can find more information at* `GDAL CSV Driver `_. - -Others valuable informations for advanced users -............................................... - -Features with curved geometries (CircularString, CurvePolygon and CompoundCurve) are -supported. Here are three examples of such geometry types as a delimited text -with WKT geometries:: - - Label;WKT_geom - CircularString;CIRCULARSTRING(268 415,227 505,227 406) - CurvePolygon;CURVEPOLYGON(CIRCULARSTRING(1 3, 3 5, 4 7, 7 3, 1 3)) - CompoundCurve;COMPOUNDCURVE((5 3, 5 13), CIRCULARSTRING(5 13, 7 15, - 9 13), (9 13, 9 3), CIRCULARSTRING(9 3, 7 1, 5 3)) - -Delimited Text supports also Z and M coordinates in geometries:: - - LINESTRINGM(10.0 20.0 30.0, 11.0 21.0 31.0) - -.. index:: OSM (OpenStreetMap) -.. _vector_osm: - -OpenStreetMap data ------------------- - -In recent years, the OpenStreetMap project has gained popularity because in many -countries no free geodata such as digital road maps are available. The objective -of the OSM project is to create a free editable map of the world from GPS data, -aerial photography or local knowledge. To support this objective, QGIS -provides support for OSM data. - -.. _open_street_map: - -Loading OpenStreetMap Vectors -............................. - -QGIS integrates OpenStreetMap import as a core functionality. - -* To connect to the OSM server and download data, open the menu - :menuselection:`Vector --> Openstreetmap --> Load data`. You can skip this - step if you already obtained an :file:`.osm` XML file using JOSM, Overpass API or - any other source. -* The menu :menuselection:`Vector --> Openstreetmap --> Import topology from - an XML file` will convert your :file:`.osm` file into a SpatiaLite database - and create a corresponding database connection. -* The menu :menuselection:`Vector --> Openstreetmap --> Export topology to - SpatiaLite` then allows you to open the database connection, select the type - of data you want (points, lines, or polygons) and choose tags to import. - This creates a SpatiaLite geometry layer that you can add to your - project by clicking on the |addSpatiaLiteLayer| - :sup:`Add SpatiaLite Layer` toolbar button or by selecting the - |addSpatiaLiteLayer| :menuselection:`Add SpatiaLite Layer...` option - from the :menuselection:`Layer` menu (see section :ref:`label_spatialite`). - -.. index:: PostGIS, PostgreSQL -.. _label_postgis: - -PostGIS Layers --------------- - -PostGIS layers are stored in a PostgreSQL database. The advantages of PostGIS -are its spatial indexing, filtering and querying capabilities it provides. Using -PostGIS, vector functions such as select and identify work more accurately than they do -with OGR layers in QGIS. - -.. _vector_create_stored_connection: - -Creating a stored Connection -............................ - -|addPostgisLayer| The first time you use a PostGIS data source, you must -create a connection to the PostgreSQL database that contains the data. Begin by -clicking on the |addPostgisLayer| :sup:`Add PostGIS Layer` toolbar -button, selecting the |addPostgisLayer| :menuselection:`Add PostGIS Layer...` -option from the :menuselection:`Layer` menu, or typing :kbd:`Ctrl+Shift+D`. You -can also open the :guilabel:`Add Vector Layer` dialog and select -|radioButtonOn| :guilabel:`Database`. The :guilabel:`Add PostGIS Table(s)` -dialog will be displayed. To access the connection manager, click on the -**[New]** button to display the :guilabel:`Create a New PostGIS Connection` -dialog. The parameters required for a connection are: - -* **Name**: A name for this connection. It can be the same as *Database*. -* **Service**: Service parameter to be used alternatively to hostname/port (and - potentially database). This can be defined in :file:`pg_service.conf`. - Check the :ref:`pg-service-file` section for more details. -* **Host**: Name of the database host. This must be a resolvable host name - such as would be used to open a telnet connection or ping the host. If the - database is on the same computer as QGIS, simply enter *'localhost'* here. -* **Port**: Port number the PostgreSQL database server listens on. The default - port is 5432. -* **Database**: Name of the database. -* **SSL mode**: How the SSL connection will be negotiated with the server. Note - that massive speed-ups in PostGIS layer rendering can be achieved by disabling - SSL in the connection editor. The following options are available: - - * Disable: Only try an unencrypted SSL connection. - * Allow: Try a non-SSL connection. If that fails, try an SSL connection. - * Prefer (the default): Try an SSL connection. If that fails, try a - non-SSL connection. - * Require: Only try an SSL connection. - -* **Username**: User name used to log in to the database. -* **Password**: Password used with *Username* to connect to the database. - -Optionally, you can activate the following checkboxes: - -* |checkbox| :guilabel:`Save Username` -* |checkbox| :guilabel:`Save Password` -* |checkbox| :guilabel:`Only look in the geometry_columns table` -* |checkbox| :guilabel:`Don't resolve type of unrestricted columns (GEOMETRY)` -* |checkbox| :guilabel:`Only look in the 'public' schema` -* |checkbox| :guilabel:`Also list tables with no geometry` -* |checkbox| :guilabel:`Use estimated table metadata` - -Once all parameters and options are set, you can test the connection -by clicking on the **[Test Connect]** button. - -.. tip:: **Use estimated table metadata to speed up operations** - - When initializing layers, various queries may be needed to establish the - characteristics of the geometries stored in the database table. When the - :guilabel:`Use estimated table metadata` option is checked, these queries - examine only a sample of the rows and use the table statistics, rather than - the entire table. This can drastically speed up operations on large datasets, - but may result in incorrect characterization of layers (eg. the feature count - of filtered layers will not be accurately determined) and may even cause strange - behaviour in case columns that are supposed to be unique actually are not. - -.. _vector_loading_postgis: - -Loading a PostGIS Layer -....................... - -|addPostgisLayer| Once you have one or more connections defined, you can -load layers from the PostgreSQL database. Of course, this requires having data in -PostgreSQL. See section :ref:`vector_import_data_in_postgis` for a discussion on -importing data into the database. - -To load a layer from PostGIS, perform the following steps: - -* If the :guilabel:`Add PostGIS layers` dialog is not already open, - selecting the |addPostgisLayer| :menuselection:`Add PostGIS Layer...` - option from the :menuselection:`Layer` menu or typing :kbd:`Ctrl+Shift+D` - opens the dialog. -* Choose the connection from the drop-down list and click **[Connect]**. -* Select or unselect |checkbox| :guilabel:`Also list tables with no geometry`. -* Optionally, use some |checkbox| :guilabel:`Search Options` to define - which features to load from the layer, or use the **[Build query]** button - to start the :guilabel:`Query builder` dialog. -* Find the layer(s) you wish to add in the list of available layers. -* Select it by clicking on it. You can select multiple layers by holding - down the :kbd:`Shift` key while clicking. See section - :ref:`vector_query_builder` for information on using the PostgreSQL - Query Builder to further define the layer. -* Click on the **[Add]** button to add the layer to the map. - -.. _tip_postgis_layers: - -.. tip:: **PostGIS Layers** - - Normally, a PostGIS layer is defined by an entry in the geometry_columns - table. QGIS can load layers that do not have an entry in the geometry_columns - table. This includes both tables and views. Defining a spatial view provides - a powerful means to visualize your data. Refer to your PostgreSQL manual for - information on creating views. - -.. _pg-service-file: - -Service connection file -^^^^^^^^^^^^^^^^^^^^^^^ - -The service connection file allows PostgreSQL connection parameters to be -associated with a single service name. That service name can then be specified -by a client and the associated settings will be used. - -It's called :file:`.pg_service.conf` under \*nix systems (GNU/Linux, macOS etc.) and -:file:`pg_service.conf` on Windows. - -The service file looks like:: - - [water_service] - host=192.168.0.45 - port=5433 - dbname=gisdb - user=paul - password=paulspass - - [wastewater_service] - host=dbserver.com - dbname=water - user=waterpass - -.. note:: There are two services in the above example: ``water_service`` - and ``wastewater_service``. You can use these to connect from QGIS, - pgAdmin etc. by specifying only the name of the service you want to - connect to (without the enclosing brackets). - If you want to use the service with ``psql`` you need to do something - like ``export PGSERVICE=water_service`` before doing your psql commands. - -.. note:: You can find all the parameters `here - `_ - -.. note:: If you don't want to save the passwords in the service file you can - use the `.pg_pass `_ - option. - - -On \*nix operating systems (GNU/Linux, macOS etc.) you can save the -:file:`.pg_service.conf` file in the user's home directory and -the PostgreSQL clients will automatically be aware of it. -For example, if the logged user is ``web``, :file:`.pg_service.conf` should -be saved in the :file:`/home/web/` directory in order to directly work (without -specifying any other environment variables). - -You can specify the location of the service file by creating a ``PGSERVICEFILE`` -environment variable (e.g. run the ``export PGSERVICEFILE=/home/web/.pg_service.conf`` -command under your \*nix OS to temporarily set the ``PGSERVICEFILE`` variable) - -You can also make the service file available system-wide (all users) either by -placing it at ``pg_config --sysconfdir``**/.pg_service.conf** or by adding the -``PGSYSCONFDIR`` environment variable to specify the directory containing -the service file. If service definitions with the same name exist in the user -and the system file, the user file takes precedence. - -.. warning:: - - There are some caveats under Windows: - - * The service file should be saved as :file:`pg_service.conf` - and not as :file:`.pg_service.conf`. - * The service file should be saved in Unix format in order to work. - One way to do it is to open it with `Notepad++ `_ - and :menuselection:`Edit --> EOL Conversion --> UNIX Format --> File save`. - * You can add environmental variables in various ways; a tested one, known to work reliably, - is :menuselection:`Control Panel --> System and Security --> System --> - Advanced system settings --> Environment Variables` adding ``PGSERVICEFILE`` and - the path of the type :file:`C:\Users\John\pg_service.conf` - * After adding an environment variable you may also need to restart the computer. - - -.. _sec_postgis_details: - -Some details about PostgreSQL layers -.................................... - -This section contains some details on how QGIS accesses PostgreSQL layers. -Most of the time, QGIS should simply provide you with a list of database -tables that can be loaded, and it will load them on request. However, if you have -trouble loading a PostgreSQL table into QGIS, the information below may -help you understand any QGIS messages and give you direction on changing -the PostgreSQL table or view definition to allow QGIS to load it. - -Primary key -^^^^^^^^^^^^ - -QGIS requires that PostgreSQL layers contain a column that can be used -as a unique key for the layer. For tables, this usually means that the table -needs a primary key, or a column with a unique constraint on it. In QGIS, -this column needs to be of type int4 (an integer of size 4 bytes). -Alternatively, the ctid column can be used as primary key. If a table lacks -these items, the oid column will be used instead. Performance will be -improved if the column is indexed (note that primary keys are automatically -indexed in PostgreSQL). - -QGIS offers a checkbox **Select at id** that is activated by default. This -option gets the ids without the attributes which is faster in most cases. - -View -^^^^ - -If the PostgreSQL layer is a view, the same requirement exists, but views -do not always have primary keys or columns with unique constraints on them. You -have to define a primary key field (has to be integer) in the QGIS dialog before -you can load the view. If a suitable column does not exist in the view, QGIS -will not load the layer. If this occurs, the solution is to alter the view so -that it does include a suitable column (a type of integer and either a primary -key or with a unique constraint, preferably indexed). - -As for table, a checkbox **Select at id** is activated by default (see above -for the meaning of the checkbox). It can make sense to disable this option when -you use expensive views. - -.. _`layer_style_backup`: - -QGIS layer_style table and database backup -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If you want to make a backup of your PostGIS database using the :file:`pg_dump` and -:file:`pg_restore` commands, and the default layer styles as saved by QGIS fail to -restore afterwards, you need to set the XML option to :file:`DOCUMENT` before the -restore command: - -.. code-block:: sql - - SET XML OPTION DOCUMENT; - -.. %FIXME: Add missing information -.. % When dealing with views, QGIS parses the view definition and - -Filter database side -^^^^^^^^^^^^^^^^^^^^ - -QGIS allows to filter features already on server side. Check the -:menuselection:`Settings --> Options --> Data Sources -->` |checkbox| -:menuselection:`Execute expressions on postgres server-side if possible` -checkbox to do so. Only supported expressions will be -sent to the database. Expressions using unsupported operators or functions will -gracefully fallback to local evaluation. - -Support of PostgreSQL data types -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Most of common data types are supported by the PostgreSQL provider: integer, float, -varchar, geometry, timestamp and array. - -.. index:: shp2pgsql - single: PostGIS; shp2pgsql -.. _vector_import_data_in_postgis: - -Importing Data into PostgreSQL ------------------------------- - -Data can be imported into PostgreSQL/PostGIS using several tools, including the -DB Manager plugin and the command line tools shp2pgsql and ogr2ogr. - -DB Manager -.......... - -QGIS comes with a core plugin named |dbManager| :sup:`DB Manager`. It can -be used to load shapefiles and other data formats, and it includes support for -schemas. See section :ref:`dbmanager` for more information. - -shp2pgsql -......... - -PostGIS includes an utility called **shp2pgsql** that can be used to import -shapefiles into a PostGIS-enabled database. For example, to import a -shapefile named :file:`lakes.shp` into a PostgreSQL database named -``gis_data``, use the following command: - -:: - - shp2pgsql -s 2964 lakes.shp lakes_new | psql gis_data - -This creates a new layer named ``lakes_new`` in the ``gis_data`` database. -The new layer will have a spatial reference identifier (SRID) of 2964. -See section :ref:`label_projections` for more information on spatial -reference systems and projections. - -.. index:: pgsql2shp - -.. _tip_export_from_postgis: - -.. tip:: **Exporting datasets from PostGIS** - - Like the import tool **shp2pgsql**, there is also a tool to export - PostGIS datasets as shapefiles: **pgsql2shp**. This is shipped within - your PostGIS distribution. - -.. index:: ogr2ogr - single: PostGIS; ogr2ogr - -ogr2ogr -....... - -Besides **shp2pgsql** and **DB Manager**, there is another tool for feeding geodata -in PostGIS: **ogr2ogr**. This is part of your GDAL installation. - -To import a shapefile into PostGIS, do the following: -:: - - ogr2ogr -f "PostgreSQL" PG:"dbname=postgis host=myhost.de user=postgres - password=topsecret" alaska.shp - -This will import the shapefile :file:`alaska.shp` into the PostGIS database -*postgis* using the user *postgres* with the password *topsecret* on host -server *myhost.de*. - -Note that OGR must be built with PostgreSQL to support PostGIS. -You can verify this by typing (in |nix|) -:: - - ogrinfo --formats | grep -i post - - -If you prefer to use PostgreSQL's **COPY** command instead of the default -**INSERT INTO** method, you can export the following environment variable -(at least available on |nix| and |osx|): -:: - - export PG_USE_COPY=YES - -**ogr2ogr** does not create spatial indexes like **shp2pgsl** does. You -need to create them manually, using the normal SQL command **CREATE INDEX** -afterwards as an extra step (as described in the next section -:ref:`vector_improving_performance`). - -.. index:: Spatial index; GiST index - single: PostGIS; Spatial index -.. _vector_improving_performance: - -Improving Performance -..................... - -Retrieving features from a PostgreSQL database can be time-consuming, especially -over a network. You can improve the drawing performance of PostgreSQL layers by -ensuring that a PostGIS spatial index exists on each layer in the -database. PostGIS supports creation of a GiST (Generalized Search Tree) -index to speed up spatial searches of the data (GiST index information is taken -from the PostGIS documentation available at http://postgis.net). - -.. tip:: You can use the DBManager to create an index to your layer. You should - first select the layer and click on :menuselection:`Table --> Edit table`, go to - :menuselection:`Indexes` tab and click on **[Add spatial index]**. - -The syntax for creating a GiST index is: -:: - - - CREATE INDEX [indexname] ON [tablename] - USING GIST ( [geometryfield] GIST_GEOMETRY_OPS ); - - -Note that for large tables, creating the index can take a long time. Once the -index is created, you should perform a ``VACUUM ANALYZE``. See the PostGIS -documentation (POSTGIS-PROJECT :ref:`literature_and_web`) for more information. - -The following is an example of creating a GiST index: -:: - - gsherman@madison:~/current$ psql gis_data - Welcome to psql 8.3.0, the PostgreSQL interactive terminal. - - Type: \copyright for distribution terms - \h for help with SQL commands - \? for help with psql commands - \g or terminate with semicolon to execute query - \q to quit - - gis_data=# CREATE INDEX sidx_alaska_lakes ON alaska_lakes - gis_data-# USING GIST (the_geom GIST_GEOMETRY_OPS); - CREATE INDEX - gis_data=# VACUUM ANALYZE alaska_lakes; - VACUUM - gis_data=# \q - gsherman@madison:~/current$ - -.. index:: PostGIS; ST_Shift_Longitude - -Vector layers crossing 180 |degrees| longitude ----------------------------------------------- - -Many GIS packages don't wrap vector maps with a geographic reference system -(lat/lon) crossing the 180 degrees longitude line -(http://postgis.refractions.net/documentation/manual-2.0/ST\_Shift\_Longitude.html). -As result, if we open such a map in QGIS, we will see two far, distinct locations, -that should appear near each other. In Figure_vector_crossing_, the tiny point on the far -left of the map canvas (Chatham Islands) should be within the grid, to the right of the -New Zealand main islands. - -.. _figure_vector_crossing: - -.. figure:: /static/user_manual/working_with_vector/vectorNotWrapping.png - :align: center - - Map in lat/lon crossing the 180 |degrees| longitude line - -A work-around is to transform the longitude values using PostGIS and the -**ST_Shift_Longitude** function. This function reads every point/vertex in every -component of every feature in a geometry, and if the longitude coordinate is -< 0 |degrees|, it adds 360 |degrees| to it. The result is a 0 |degrees| - 360 |degrees| -version of the data to be plotted in a 180 |degrees|-centric map. - -.. _figure_vector_crossing_map: - -.. figure:: /static/user_manual/working_with_vector/vectorWrapping.png - :align: center - :width: 25em - - Crossing 180 |degrees| longitude applying the **ST_Shift_Longitude** - function - -Usage -..... - -* Import data into PostGIS (:ref:`vector_import_data_in_postgis`) using, - for example, the DB Manager plugin. -* Use the PostGIS command line interface to issue the following command - (in this example, "TABLE" is the actual name of your PostGIS table): - ``gis_data=# update TABLE set the_geom=ST_Shift_Longitude(the_geom);`` -* If everything went well, you should receive a confirmation about the - number of features that were updated. Then you'll be able to load the - map and see the difference (Figure_vector_crossing_map_). - -.. index:: Spatialite, SQLite -.. _label_spatialite: - -SpatiaLite Layers ------------------ - -|addSpatiaLiteLayer| The first time you load data from a SpatiaLite -database, begin by clicking on the |addSpatiaLiteLayer| -:sup:`Add SpatiaLite Layer` toolbar button, or by selecting the -|addSpatiaLiteLayer| :menuselection:`Add SpatiaLite Layer...` option -from the :menuselection:`Layer` menu, or by typing :kbd:`Ctrl+Shift+L`. -This will bring up a window that will allow you either to connect to a -SpatiaLite database already known to QGIS, which you can choose from the -drop-down menu, or to define a new connection to a new database. To define a -new connection, click on **[New]** and use the file browser to point to -your SpatiaLite database, which is a file with a :file:`.sqlite` extension. - -If you want to save a vector layer to SpatiaLite format, you can do this by -right clicking the layer in the legend. Then, click on :menuselection:`Save as..`, -define the name of the output file, and select 'SpatiaLite' as format and the CRS. -Also, you can select 'SQLite' as format and then add ``SPATIALITE=YES`` in the -OGR data source creation option field. This tells OGR to create a SpatiaLite -database. See also http://www.gdal.org/ogr/drv_sqlite.html. - -QGIS also supports editable views in SpatiaLite. - -Creating a new SpatiaLite layer -............................... - -If you want to create a new SpatiaLite layer, please refer to section -:ref:`vector_create_spatialite`. - -.. index:: QSpatiaLite, Spatialite manager, DB Manager - -.. _tip_spatialite_management_plugin: - -.. tip:: **SpatiaLite data management Plugins** - - For SpatiaLite data management, you can also use several Python plugins: - QSpatiaLite, SpatiaLite Manager or :ref:`DB Manager ` (core plugin, recommended). - If necessary, they can be downloaded and installed with the Plugin Installer. - -.. index:: MSSQL Spatial -.. _label_mssql: - -MSSQL Spatial Layers --------------------- - -|addMssqlLayer| QGIS also provides native MS SQL support. The first -time you load MSSQL Spatial data, begin by clicking on the -|addMssqlLayer| :sup:`Add MSSQL Spatial Layer` toolbar button or by -selecting the |addMssqlLayer| :menuselection:`Add MSSQL Spatial Layer...` -option from the :menuselection:`Layer` menu, or by typing :kbd:`Ctrl+Shift+M`. - -.. index:: Oracle Spatial -.. _label_oracle_spatial: - -Oracle Spatial Layers ---------------------- - -The spatial features in Oracle Spatial aid users in managing geographic and -location data in a native type within an Oracle database. QGIS now has -support for such layers. - -.. _create_oracle_connection: - -Creating a stored Connection -............................ - -|addOracleLayer| The first time you use an Oracle Spatial data source, -you must create a connection to the database that contains the data. Begin by -clicking on the |addOracleLayer| :sup:`Add Oracle Spatial Layer` toolbar -button, selecting the |addOracleLayer| :menuselection:`Add Oracle -Spatial Layer...` option from the :menuselection:`Layer` menu, or typing -:kbd:`Ctrl+Shift+O`. To access the connection manager, click on the **[New]** -button to display the :guilabel:`Create a New Oracle Spatial Connection` dialog. -The parameters required for a connection are: - -* **Name**: A name for this connection. It can be the same as *Database* -* **Database**: SID or SERVICE_NAME of the Oracle instance. -* **Host**: Name of the database host. This must be a resolvable host name - such as would be used to open a telnet connection or ping the host. If the - database is on the same computer as QGIS, simply enter *'localhost'* here. -* **Port**: Port number the Oracle database server listens on. The default - port is 1521. -* **Username**: Username used to login to the database. -* **Password**: Password used with *Username* to connect to the database. -* **Workspace**: Workspace to switch to. - -Optionally, you can activate following checkboxes: - -* |checkbox| :guilabel:`Save Username` Indicates whether to save the database - username in the connection configuration. -* |checkbox| :guilabel:`Save Password` Indicates whether to save the database - password in the connection settings. -* |checkbox| :guilabel:`Only look in meta data table` Restricts the displayed - tables to those that are in the all_sdo_geom_metadata view. This can speed - up the initial display of spatial tables. -* |checkbox| :guilabel:`Only look for user's tables` When searching for spatial - tables, restrict the search to tables that are owned by the user. -* |checkbox| :guilabel:`Also list tables with no geometry` Indicates that - tables without geometry should also be listed by default. -* |checkbox| :guilabel:`Use estimated table statistics for the layer metadata` - When the layer is set up, various metadata are required for the Oracle table. - This includes information such as the table row count, geometry type and - spatial extents of the data in the geometry column. If the table contains a - large number of rows, determining this metadata can be time-consuming. By - activating this option, the following fast table metadata operations are - done: Row count is determined from ``all_tables.num_rows``. Table extents - are always determined with the SDO_TUNE.EXTENTS_OF function, even if a layer - filter is applied. Table geometry is determined from the first 100 - non-null geometry rows in the table. -* |checkbox| :guilabel:`Only existing geometry types` Only list the existing - geometry types and don't offer to add others. - -.. warning:: - - In the :guilabel:`Authentication` tab, saving **username** and **password** - will keep unprotected credentials in the connection configuration. Those - **credentials will be visible** if, for instance, you shared the project file - with someone. Therefore, it's advisable to save your credentials in a - *Authentication configuration* instead (:guilabel:`configurations` tab). - See :ref:`authentication_index` for more details. - -Once all parameters and options are set, you can test the connection by -clicking on the **[Test Connect]** button. - -.. _tip_settings_security: - -.. tip:: **QGIS User Settings and Security** - - Depending on your computing environment, storing passwords in your QGIS - settings may be a security risk. Passwords are saved in clear text in the - system configuration and in the project files! - Your customized settings for QGIS are stored based on the operating system: - - * |nix| The settings are stored in your home directory in :file:`~/.qgis2`. - * |win| The settings are stored in the registry. - -.. _load_oracle_layer: - -Loading an Oracle Spatial Layer -................................ - -|addOracleLayer| Once you have one or more connections defined, you can -load layers from the Oracle database. Of course, this requires having data in -Oracle. - -To load a layer from Oracle Spatial, perform the following steps: - -* If the :guilabel:`Add Oracle Spatial layers` dialog is not already open, - click on the |addOracleLayer| :sup:`Add Oracle Spatial Layer` toolbar - button. -* Choose the connection from the drop-down list and click **[Connect]**. -* Select or unselect |checkbox| :guilabel:`Also list tables with no geometry`. -* Optionally, use some |checkbox| :guilabel:`Search Options` to define - which features to load from the layer or use the **[Build query]** button - to start the :guilabel:`Query builder` dialog. -* Find the layer(s) you wish to add in the list of available layers. -* Select it by clicking on it. You can select multiple layers by holding - down the :kbd:`Shift` key while clicking. See section - :ref:`vector_query_builder` for information on using the Oracle - Query Builder to further define the layer. -* Click on the **[Add]** button to add the layer to the map. - -.. _tip_ORACLE_Spatial_layers: - -.. tip:: **Oracle Spatial Layers** - - Normally, an Oracle Spatial layer is defined by an entry in the - **USER_SDO_METADATA** table. - -.. index:: DB2 Spatial -.. _label_db2_spatial: - -DB2 Spatial Layers -------------------- - -IBM DB2 for Linux, Unix and Windows (DB2 LUW), IBM DB2 for z/OS (mainframe) -and IBM DashDB products allow -users to store and analyse spatial data in relational table columns. -The DB2 provider for QGIS supports the full range of visualization, analysis -and manipulation of spatial data in these databases. - -.. _DB2 z/OS KnowledgeCenter: https://www.ibm.com/support/knowledgecenter/en/SSEPEK_11.0.0/spatl/src/tpc/spatl_db2sb03.html -.. _DB2 LUW KnowledgeCenter: http://www.ibm.com/support/knowledgecenter/SSEPGG_11.1.0/com.ibm.db2.luw.spatial.topics.doc/doc/db2sb03.html -.. _DB2 DashDB KnowledgeCenter: https://www.ibm.com/support/knowledgecenter/SS6NHC/com.ibm.db2.luw.spatial.topics.doc/doc/csbp1001.html -.. _DB2 Spatial Tutorial: https://www.ibm.com/developerworks/data/tutorials/dm-1202db2spatialdata1/ - -User documentation on these capabilities can be found at the -`DB2 z/OS KnowledgeCenter`_, `DB2 LUW KnowledgeCenter`_ -and `DB2 DashDB KnowledgeCenter`_. - -For more information about working with the DB2 spatial capabilities, check out -the `DB2 Spatial Tutorial`_ on IBM DeveloperWorks. - -Configuring QGIS for DB2 -......................... - -The DB2 provider currently only supports the Windows environment through the -Windows ODBC driver. - -The client running QGIS needs to have one of the following installed: - -* DB2 LUW -* IBM Data Server Driver Package -* IBM Data Server Client - -If you are accessing a DB2 LUW database on the same machine or using DB2 LUW as -a client, the DB2 executables and supporting files need to be included in the -Windows path. This can be done by creating a batch file like the following with -the name **db2.bat** and including it in the directory **%OSGEO4W_ROOT%/etc/ini**. - -:: - - @echo off - REM Point the following to where DB2 is installed - SET db2path=C:\Program Files (x86)\sqllib - REM This should usually be ok - modify if necessary - SET gskpath=C:\Program Files (x86)\ibm\gsk8 - SET Path=%db2path%\BIN;%db2path%\FUNCTION;%gskpath%\lib64;%gskpath%\lib;%path% - -.. _create_db2_connection: - -Creating a stored Connection -............................ - -|addDb2Layer| The DB2 provider uses ODBC to connect to a DB2 database. -Windows includes ODBC by default. - -The first time you use an DB2 Spatial data source, -you must create a connection to the database that contains the data. -A connection can be created by: - -* Right-clicking on |db2| :menuselection:`DB2` in the QGIS Browser panel - and selecting :menuselection:`New connection` - -or - -* Selecting the |addDb2Layer| :menuselection:`Add DB2 - Spatial Layer...` option from the :menuselection:`Layer` menu. - To access the connection manager, click on the **[New]** - button to display the :guilabel:`Create a New DB2 Connection` dialog. - -The connection can be specified using either a Service/DSN name defined to ODBC -or using the driver, host and port information. - -All connections require: - -* **Connection Name**: A name for this connection. It can be the same as *Database* -* **Database**: The DB2 database name. -* User name and password. See more information below. - -An ODBC Service/DSN connection requires in addition: - -* **Service/DSN**: The service name defined to ODBC - -A driver / host / host connection requires in addition: - -* **Driver**: Name of the DB2 driver. Typically this would be IBM DB2 ODBC DRIVER. -* **DB2 Host**: Name of the database host. This must be a resolvable host name - such as would be used to open a telnet connection or ping the host. If the - database is on the same computer as QGIS, simply enter *'localhost'* here. -* **DB2 Port**: Port number the DB2 database server listens on. The default - DB2 LUW port is 50000. The default DB2 z/OS port is 446. - -.. warning:: - - In the :guilabel:`Authentication` tab, saving **username** and **password** - will keep unprotected credentials in the connection configuration. Those - **credentials will be visible** if, for instance, you shared the project file - with someone. Therefore, it's advisable to save your credentials in a - *Authentication configuration* instead (:guilabel:`configurations` tab). - See :ref:`authentication_index` for more details. - -Once all parameters and options are set, you can test the connection by -clicking on the **[Test connection]** button. - -.. _load_db2_layer: - -Loading a DB2 Spatial Layer -............................. - -|addDb2Layer| Once you have one or more connections defined, you can -load layers from the DB2 database. A DB2 Spatial layer is defined by a row in the -**DB2GSE.ST_GEOMETRY_COLUMNS** view. - -To load a layer from DB2 Spatial, perform the following steps: - -* If the :guilabel:`Add DB2 Spatial layers` dialog is not already open, - click on the |addDb2Layer| :sup:`Add DB2 Spatial Layer` toolbar - button. -* Choose the connection from the drop-down list and click **[Connect]**. -* Optionally, use some |checkbox| :guilabel:`Search Options` to define - which features to load from the layer or use the **[Build query]** button - to start the :guilabel:`Query builder` dialog. -* Find the layer(s) you wish to add in the list of available layers. -* Select it by clicking on it. You can select multiple layers by holding - down the :kbd:`Shift` key while clicking. See section - :ref:`vector_query_builder` for information on using the - Query Builder to further define the layer. -* Click on the **[Add]** button to add the layer to the map. - -Or more simply, expand the |db2| :menuselection:`DB2` connection in the QGIS Browser panel -and double-click the name of the layer. - -.. note:: - - In order to work effectively with DB2 spatial tables in QGIS, it is important that - tables have an INTEGER or BIGINT column defined as PRIMARY KEY and if new features - are going to be added, this column should also have the GENERATED characteristic. - - It is also helpful for the spatial column to be registered with a specific spatial - reference identifier (most often 4326 for WGS84 coordinates). - A spatial column can be registered by calling the ST_Register_Spatial_Column stored - procedure. - diff --git a/source/docs/user_manual/working_with_vector/virtual_layers.rst b/source/docs/user_manual/working_with_vector/virtual_layers.rst deleted file mode 100644 index 3d9077dffda..00000000000 --- a/source/docs/user_manual/working_with_vector/virtual_layers.rst +++ /dev/null @@ -1,185 +0,0 @@ -.. only:: html - - |updatedisclaimer| - -.. index:: Virtual_Layers - -.. _vector_virtual_layers: - -Virtual layers -============== - -.. only:: html - - .. contents:: - :local: - -A special kind of vector layer allows you to define a layer as the result of an -advanced query, using the SQL language on any number of other vector layers that -QGIS is able to open. These layers are called virtual layers: they do not carry -data by themselves and can be seen as views to other layers. - -Creating a virtual layer ------------------------- - -Open the virtual layer creation dialog by clicking on -:guilabel:`Add Virtual Layer` in the :guilabel:`Layer` menu or from the -corresponding toolbar. - -The dialog allows you to specify a :guilabel:`Layer name` and a SQL -:guilabel:`Query`. The query can use the name (or id) of loaded vector -layers as tables, as well as their fields' name as columns. - -For example, if you have a layer called ``airports``, you can create a new -virtual layer called ``public_airports`` with an SQL query like: - -.. code-block:: sql - - SELECT * - FROM airports - WHERE USE = "Civilian/Public" - -The SQL query will be executed, whatever the underlying provider of the -``airports`` layer is and even if this provider does not directly support SQL -queries. - -.. figure:: /static/user_manual/working_with_vector/create_virtual_layers.png - :align: center - - Create virtual layers dialog - -Joins and complex queries can also be created simply by directly using the -names of the layers that are to be joined. - -.. note:: - - It's also possible to create virtual layers using the SQL window of - :ref:`dbmanager`. - -Embedded layers ---------------- - -Besides the vector layers available in the map canvas, the user can add layers -to the :guilabel:`Embedded layers` list, which he can use in queries -without the need to have them showing in the map canvas or Layers panel. - -To embed a layer, click :guilabel:`Add` and provide the :guilabel:`Local name`, -:guilabel:`Provider`, :guilabel:`Encoding` and the path to the -:guilabel:`Source`. - -The :guilabel:`Import` button allows adding layers loaded in the map canvas into -the Embedded layers list. This allows to later remove those layers from the -Layers panel without breaking any existent query. - -Supported language ------------------- - -The underlying engine uses SQLite and Spatialite to operate. - -It means you can use all of the SQL your local installation of SQLite -understands. - -Functions from SQLite and spatial functions from Spatialite -can also be used in a virtual layer query. For instance, creating a point -layer out of an attribute-only layer can be done with a query similar to: - -.. code-block:: sql - - SELECT id, MakePoint(x, y, 4326) as geometry - FROM coordinates - -:ref:`Functions of QGIS expressions` can also be used in a -virtual layer query. - -To refer the geometry column of a layer, use the name ``geometry``. - -Contrary to a pure SQL query, all the fields of a virtual layer query must -be named. Don't forget to use the ``as`` keyword to name your columns if they -are the result of a computation or function call. - -Performance issues ------------------- - -With default parameters set, the virtual layer engine will try its best to -detect the type of the different columns of the query, including the type of the -geometry column if one is present. - -This is done by introspecting the query when possible or by fetching the first -row of the query (LIMIT 1) at last resort. -Fetching the first row of the result just to create the layer may be undesirable -for performance reasons. - -The creation dialog allows to specify different parameters: - -* :guilabel:`Unique identifier column`: this option allows specifying which - field of the query represents unique integer values that QGIS can use as row - identifiers. By default, an autoincrementing integer value is used. - Defining a unique identifier column allows to speed up the selection of - rows by id. - -* :guilabel:`No geometry`: this option forces the virtual layer to ignore - any geometry field. The resulting layer is an attribute-only layer. - -* Geometry :guilabel:`Column`: this option allows to specify the name - of the column that is to be used as the geometry of the layer. - -* Geometry :guilabel:`Type`: this option allows to specify the type - of the geometry of the virtual layer. - -* Geometry :guilabel:`CRS`: this option allows to specify the - coordinate reference system of the virtual layer. - -Special comments ----------------- - -The virtual layer engine tries to determine the type of each column of the -query. If it fails, the first row of the query is fetched to determine -column types. - -The type of a particular column can be specified directly in the query by -using some special comments. - -The syntax is the following: ``/*:type*/``. It has to be placed just after -the name of a column. ``type`` can be either ``int`` for integers, ``real`` -for floating point numbers or ``text``. - -For instance: - -.. code-block:: sql - - SELECT id+1 as nid /*:int*/ - FROM table - -The type and coordinate reference system of the geometry column can also be set -thanks to special comments with the following syntax ``/*:gtype:srid*/`` where -``gtype`` is the geometry type (``point``, ``linestring``, ``polygon``, -``multipoint``, ``multilinestring`` or ``multipolygon``) and ``srid`` an -integer representing the EPSG code of a coordinate reference system. - -Use of indexes --------------- - -When requesting a layer through a virtual layer, indexes of this source layer -will be used in the following ways: - -* if an ``=`` predicate is used on the primary key column of the layer, the - underlying data provider will be asked for a particular id (FilterFid) - -* for any other predicates (``>``, ``<=``, ``!=``, etc.) or on a column without - a primary key, a request built from an expression will be used to request the - underlying vector data provider. It means indexes may be used on database - providers if they exist. - -A specific syntax exists to handle spatial predicates in requests and triggers -the use of a spatial index: a hidden column named ``_search_frame_`` exists -for each virtual layer. This column can be compared for equality to a bounding -box. Example: - -.. code-block:: sql - - SELECT * - FROM vtab - WHERE _search_frame_=BuildMbr(-2.10,49.38,-1.3,49.99,4326) - -Spatial binary predicates like ``ST_Intersects`` are significantly sped up when -used in conjunction with this spatial index syntax. \ No newline at end of file