Oasis - Prepare for charge station input

[CodeBear801]

Subtask of #231

Input

For charge station data, or POI data, they usually recorded in database. First step would be dump this database into csv format, each line represent information for a single charge station.

csv

Processing flow

Telenav's Database(postGIS) 
       -> csv 
               -> adjust column name 
                       -> convert csv to json 
                                 -> decoding json

More details could go to here: #237 (comment)

Target format

Electricity charge stations, expect input in JSON format follow OSM's tag definition about node and charging_station, example:

{
   "id": 12345,
   "lat": 51.5173639,
   "lon": -0.140043,
   "amenity": "charging_station",
   "operator": "NOFT",
   "fee": "yes",
   "capacity":4,
}

• Later could implement more input connector: pbf, csv, etc.
• We want to make this json format as protocol for following processing
• A sample JSON record could be found here
• Json record sample:

[{"id":12345,"vendor_code":"NOFT","vendor_poi_id":"NOFT-10_NREL99930@-@NOCO-37145190@-@NOCO-996595530","lat":0.0,"lon":0.0,"nav_lat":0.0,"nav_lon":0.0,"dis_lat":0.0,"dis_lon":0.0,"map_link_id":12345,"side_of_street":"+","country":"USA","space_id":"USA_CA","airport_code":"NULL","is_national":false,"is_state_importance":false,"is_city_importance":false,"fax":"NULL","phone":8054662325,"encoded_phone":"Prefix=1,CountryCode=1,AreaCode=805,LocalNumber=4662325,RawData=8054662325,Preferred=1,E164=+18054662325,PhoneType=phone","email":"NULL","web_url":"NULL","category_id_gather":"xxx``xxx``xxx","chain_gather":"1364@-@@-@",,"hilbert":2.32E+11,"amenity":"charging_station"},]

[CodeBear801]

Flow

Telenav's Database(postGIS) -> csv -> adjust column name -> convert csv to json -> decoding json

• For information about charge station dump, please go to Telenav wiki
• For sample JSON output, please go to Telenav file repo
• Adjust column name means convert names used internally to a common OSM name, such as PlaceID -> ID

[wangyoucao577]

Subtask of #231

Input

For charge station data, or POI data, they usually recorded in database. First step would be dump this database into csv format, each line represent information for a single charge station.

csv record sample:

[{"id":12345,"vendor_code":"NOFT","vendor_poi_id":"NOFT-10_NREL99930@-@NOCO-37145190@-@NOCO-996595530","lat":0.0,"lon":0.0,"nav_lat":0.0,"nav_lon":0.0,"dis_lat":0.0,"dis_lon":0.0,"map_link_id":12345,"side_of_street":"+","country":"USA","space_id":"USA_CA","airport_code":"NULL","is_national":false,"is_state_importance":false,"is_city_importance":false,"fax":"NULL","phone":8054662325,"encoded_phone":"Prefix=1,CountryCode=1,AreaCode=805,LocalNumber=4662325,RawData=8054662325,Preferred=1,E164=+18054662325,PhoneType=phone","email":"NULL","web_url":"NULL","category_id_gather":"xxx``xxx``xxx","chain_gather":"1364@-@@-@",,"hilbert":2.32E+11,"amenity":"charging_station"},]

Processing flow

Telenav's Database(postGIS) 
       -> csv 
               -> adjust column name 
                       -> convert csv to json 
                                 -> decoding json

More details could go to here: #237 (comment)

Target format

Electricity charge stations, expect input in JSON format follow OSM's tag definition about node and charging_station, example:

{
   "id": 12345,
   "lat": 51.5173639,
   "lon": -0.140043,
   "amenity": "charging_station",
   "operator": "NOFT",
   "fee": "yes",
   "capacity":4,
}

• Later could implement more input connector: pbf, csv, etc.
• We want to make this json format as protocol for following processing
• A sample JSON record could be found here

The original format is not csv. It's json actually (it's a json array with head [ and tail ]). It should be able to parse by json library directly. The sample json you metiond is a json with line endings for better human readable, some json libraries support it. But without line endings is more common json.

[CodeBear801]

Subtask of #231

Input

For charge station data, or POI data, they usually recorded in database. First step would be dump this database into csv format, each line represent information for a single charge station.
csv record sample:

[{"id":12345,"vendor_code":"NOFT","vendor_poi_id":"NOFT-10_NREL99930@-@NOCO-37145190@-@NOCO-996595530","lat":0.0,"lon":0.0,"nav_lat":0.0,"nav_lon":0.0,"dis_lat":0.0,"dis_lon":0.0,"map_link_id":12345,"side_of_street":"+","country":"USA","space_id":"USA_CA","airport_code":"NULL","is_national":false,"is_state_importance":false,"is_city_importance":false,"fax":"NULL","phone":8054662325,"encoded_phone":"Prefix=1,CountryCode=1,AreaCode=805,LocalNumber=4662325,RawData=8054662325,Preferred=1,E164=+18054662325,PhoneType=phone","email":"NULL","web_url":"NULL","category_id_gather":"xxx``xxx``xxx","chain_gather":"1364@-@@-@",,"hilbert":2.32E+11,"amenity":"charging_station"},]

Processing flow

Telenav's Database(postGIS) 
       -> csv 
               -> adjust column name 
                       -> convert csv to json 
                                 -> decoding json

More details could go to here: #237 (comment)

Target format

Electricity charge stations, expect input in JSON format follow OSM's tag definition about node and charging_station, example:

{
   "id": 12345,
   "lat": 51.5173639,
   "lon": -0.140043,
   "amenity": "charging_station",
   "operator": "NOFT",
   "fee": "yes",
   "capacity":4,
}

• Later could implement more input connector: pbf, csv, etc.
• We want to make this json format as protocol for following processing
• A sample JSON record could be found here

The original format is not csv. It's json actually (it's a json array with head [ and tail ]). It should be able to parse by json library directly. The sample json you metiond is a json with line endings for better human readable, some json libraries support it. But without line endings is more common json.

My bad, I put a converted json record there :(
Update the first comment.

[CodeBear801]

It is also possible to directly dump sql query result to json format.

[wangyoucao577]

Aha! Now it makes sense. But I still think the csv-to-json is not a good name since the convertion is not generic but for specified domain. How about rename it to something like poi-preprocess or poi-unify? Or maybe poi-converter that possible to support converting between csv,json,pbf, etc.