eve-sdeloader
is a tool for importing an EVE static data export from EVE
Developer Resources to a
Postgresql database.
go get github.com/lflux/eve-sdeloader
should install the tool. If this fails,
check out the repository under your $GOPATH
and use
dep to fetch the dependencies:
$ mkdir -p $GOPATH/src/github.com/lflux
$ cd $GOPATH/src/github.com/lflux
$ git clone https://github.com/lflux/eve-sdeloader.git
$ cd eve-sdeloader
$ dep ensure
$ go install .
The tool will be in $GOPATH/bin/eve-sdeloader
(by default GOPATH
is ~/go
as of Go 1.8)
Easiest way to create a database for this tool is to connect to the database as a superuser and create a role that can log in, along with a database owned by that user:
CREATE ROLE sdetest LOGIN;
CREATE DATABASE sdtest OWNER sdetest;
$ cd /tmp
$ unzip ~/Downloads/sde-20171024-TRANQUILITY.zip
$ curl -O https://raw.githubusercontent.com/fuzzysteve/yamlloader/master/invVolumes1.csv # or copy from this repo
$ curl -O https://raw.githubusercontent.com/fuzzysteve/yamlloader/master/invVolumes2.csv # or copy from this repo
# One entry needs fixing due to YAML vagaries
$ gsed -i -e 's/radius: 0.0059e18/radius: 0.0059e\+18/' sde/fsd/universe/wormhole/G-R00031/G-C00311/constellation.static
$ eve-sdeloader
By default eve-sdeloader
will try to import SDE files from under ./sde
and
./invVolumes{1,2}.csv
to the database sdetest
on localhost
with the user
sdetest
. This is all overridable with CLI flags:
Usage of ./eve-sdeloader:
-cpuprofile string
File to write cpu profile to
-dbhost string
Database host (default "localhost")
-dbname string
Database name (default "sdetest")
-dbpassword string
Database password
-dbport int
Database port (default 5432)
-dbuser string
Database username (default "sdetest")
-invvoldirectory string
Directory containing invVolumes{1,2}.csv (default ".")
-memprofile string
File to write memory profile to
-nobsd
Disable importing of BSD directory
-nouniverse
Disable importing of universe data
-sdedirectory string
Directory containing an unzipped EVE SDE YAML dump (default "./sde")
-single-file string
Import only a single FSD file
Import works with Postgres and is 99% compatible with the importer from @fuzzysteve. A full import currently takes about 4 minutes on a Macbook Pro (to contrast with 48 minutes for the python importer). The TODO file attempts to keep track of project status.
Currently tested on Go 1.9 with Postgres 9.6 on OSX 10.12 (Sierra), but it probably works all the way back to Postgres 8.4 and should work fine on Linux.
- Negative 0 floats are imported as
-0
, whereas the python importer treats both as 0 invtraits
has different traitids due to insertion order. We attempt to keep these mostly the same by sorting types and skills, but the python importer doesn't sort at all and uses whatever order python map iteration gives you.
eve-sdeloader
accepts -cpuprofile
and -memprofile
flags to dump profiling
output from runtime/pprof
into pprof files that can be consumed by other
tools.
Please raise issues on Github.
eve-sdeloader
is copyright (c) 2017 Ian Delahorne and licensed under the MIT
license, which can be found in the LICENSE file.