_load_recent_forts recent fort error

[mupinkz]

Expected Behavior

executing ./run.sh should work

Actual Behavior

crashing after login successfuly.

Your FULL config.json (remove your username, password, gmapkey and any other private info)

`{
"websocket_server": false,
"heartbeat_threshold": 10,
"enable_social": true,
"live_config_update": {
"enabled": false,
"tasks_only": false
},
"tasks": [
{
"//NOTE: This task MUST be placed on the top of task list": {},
"type": "RandomAlivePause",
"config": {
"enabled": true,
"min_duration": "00:00:10",
"max_duration": "00:10:00",
"min_interval": "00:15:00",
"max_interval": "02:30:00"
}
},
{
"type": "HandleSoftBan"
},
{
"type": "CompleteTutorial",
"config": {
"enabled": false,
"// set a name": "",
"nickname": ""
}
},
{
"type": "CollectLevelUpReward",
"config": {
"collect_reward": true,
"level_limit": -1
}
},
{
"type": "IncubateEggs",
"config": {
"enabled": true,
"infinite_longer_eggs_first": false,
"breakable_longer_eggs_first": true,
"min_interval": 120
}
},
{
"type": "UpdateLiveStats",
"config": {
"enabled": true,
"min_interval": 10,
"stats": ["username", "uptime", "level_stats", "km_walked", "stardust_earned", "xp_earned", "xp_per_hour", "stops_visited", "pokemon_stats"],
"terminal_log": true,
"terminal_title": true
}
},
{
"type": "UpdateLiveInventory",
"config": {
"enabled": true,
"min_interval": 30,
"show_all_multiple_lines": false,
"items": ["pokemon_bag", "space_info", "pokeballs", "greatballs", "ultraballs", "razzberries", "revive", "luckyegg"]
}
},
{
"type": "ShowBestPokemon",
"config": {
"enabled": true,
"min_interval": 60,
"amount": 5,
"order_by": "ivcp",
"info_to_show": ["cp", "ivcp", "dps", "hp"]
}
},
{
"type": "PokemonOptimizer",
"config": {
"enabled": true,
"// the 'transfer' parameter activate or deactivate the transfer of pokemons": {},
"// at false, no pokemon is going to be transfered, ever": {},
"// at false, you will still get the log information of what the optimizer": {},
"// would have transfered if the parameter was true": {},
"transfer": true,
"// 'transfer_wait_min' and 'transfer_wait_max' are the minimum and maximum": {},
"// time to wait when transferring a pokemon": {},
"transfer_wait_min": 3,
"transfer_wait_max": 5,
"// the 'evolve' parameter activate or deactivate the evolution of pokemons": {},
"// at false, no pokemon is going to be evolved, ever": {},
"// at false, you will still get the log information of what the": {},
"// optimizer would have evolved if the parameter was true": {},
"evolve": true,
"// Time in seconds to wait between two evolve": {},
"evolve_time": 20,
"// the 'evolve_for_xp' parameter let you choose if you want the optimizer": {},
"// to use your candies to evolve low quality pokemons in order to maximize your xp": {},
"// at false, the optimizer will still use candies to evolve your best Pokemons": {},
"evolve_for_xp": true,
"// the 'evolve_only_with_lucky_egg' parameter let you choose if you want the optimizer": {},
"// to only Evolve Pokemons when a lucky egg is available": {},
"evolve_only_with_lucky_egg": false,
"// the 'evolve_count_for_lucky_egg' parameter let you define the minimum": {},
"// number of Pokemons that must evolve before using a lucky egg": {},
"// If that number is not reached, and evolve_only_with_lucky_egg is true, evolution will be skipped": {},
"// If that number is not reached, and evolve_only_with_lucky_egg is false,": {},
"// evolution will be performed without using a lucky egg": {},
"evolve_count_for_lucky_egg": 92,
"// the 'may_use_lucky_egg' parameter let you choose if you want the optimizer": {},
"// to use a lucky egg right before evolving Pokemons. At false; the optimizer": {},
"// is free to evolve Pokemons even if you do not have any lucky egg.": {},
"may_use_lucky_egg": true,
"// the 'keep' parameter let you define what pokemons you consider are the 'best'. These Pokemons": {},
"// will be keep and evolved. Note that Pokemons are evaluated inside their whole family": {},
"// Multiple way of ranking can be defined. Following configuration let you keep the best iv,": {},
"// the best ncp and the best cp": {},
"keep": [
{
"// Following setting let you keep the best iv of the family": {},
"// the 'top' parameter allow you to define how many Pokemons you want to keep": {},
"// at the top of your ranking. If several Pokemons get the same score, they are": {},
"// considered equal. Thus, top=1 might result in keeping more than 1 Pokemon.": {},
"top": 3,
"// the 'evolve' parameter let you choose if you want to evolve the Pokemons you keep": {},
"evolve": false,
"// the 'sort' parameter define how you want to rank your pokemons": {},
"// Critera are sorted fro, the most important to the least important.": {},
"// Available criteria are:": {},
"// 'iv' = individual value": {},
"// 'ivcp' = iv weigted so that for equal iv, attack > defense > stamina": {},
"// 'cp' = combat power (can be increased with candies)": {},
"// 'cp_exact' = combat power (not rounded)": {},
"// 'ncp' (normalized cp) or 'cp_percent' = ratio cp / max_cp": {},
"// 'iv_attack' = attach component of iv": {},
"// 'iv_defense' = defense component of iv": {},
"// 'iv_stamina' = stamina component of iv": {},
"// 'dps' = raw dps based on the moves of the pokemon": {},
"// 'dps_attack' = average dps when attacking": {},
"// 'attack_perfection' = ratio dps_attack / best_dps_attack. Return same order as 'dps_attack'": {},
"// 'dps_defense' = average dps when defending": {},
"// 'defense_perfection' = ratio dps_defense / best_dps_defense. Return same order as 'dps_defense'": {},
"// 'hp' = current health points": {},
"// 'hp_max' = max health points": {},
"// Note that the more criteria you add to this list, the less likely Pokemons": {},
"// will be equals": {},
"sort": ["iv", "cp"]
},
{
"// Following setting let you keep keep the best normalized cp of the family": {},
"// That is the Pokemon with higher CP once fully evolved": {},
"top": 1,
"evolve": true,
"sort": ["ncp"]
},
{
"// Following setting let you keep keep the best cp of the family.": {},
"// But will not evolve it further (in favor of the best ncp)": {},
"// It will only applies to the family of 'Dragonite', 'Arcanine' and 'Muk'": {},
"// Other families are not following this setting": {},
"names": ["Dragonite", "Arcanine", "Muk"],
"top": 3,
"evolve": false,
"sort": ["cp"]
}
]
}
},
{
"type": "RecycleItems",
"config": {
"enabled": true,
"min_empty_space": 15,
"max_balls_keep": 150,
"max_potions_keep": 50,
"max_berries_keep": 70,
"max_revives_keep": 30,
"item_filter": {
"Pokeball": { "keep" : 100 },
"Potion": { "keep" : 10 },
"Super Potion": { "keep" : 20 },
"Hyper Potion": { "keep" : 30 },
"Revive": { "keep" : 30 },
"Razz Berry": { "keep" : 100 }
},
"recycle_wait_min": 3,
"recycle_wait_max": 5,
"recycle_force": true,
"recycle_force_min": "00:01:00",
"recycle_force_max": "00:05:00"
}
},
{
"type": "CatchPokemon",
"config": {
"enabled": true,
"catch_visible_pokemon": true,
"catch_lured_pokemon": true,
"min_ultraball_to_keep": 5,
"berry_threshold": 0.35,
"vip_berry_threshold": 0.9,
"treat_unseen_as_vip": true,
"daily_catch_limit": 800,
"catch_throw_parameters": {
"excellent_rate": 0.1,
"great_rate": 0.5,
"nice_rate": 0.3,
"normal_rate": 0.1,
"spin_success_rate" : 0.6
},
"catch_simulation": {
"flee_count": 3,
"flee_duration": 2,
"catch_wait_min": 3,
"catch_wait_max": 6,
"berry_wait_min": 3,
"berry_wait_max": 5,
"changeball_wait_min": 3,
"changeball_wait_max": 5,
"newtodex_wait_min": 20,
"newtodex_wait_max": 30
}
}
},
{
"type": "SpinFort",
"config": {
"enabled": true,
"spin_wait_min": 3,
"spin_wait_max": 5
}
},
{ "type": "UpdateWebInventory",
"config": {
"enabled": true
}
},
{
"type": "MoveToFort",
"config": {
"enabled": true,
"lure_attraction": true,
"lure_max_distance": 2000,
"log_interval": 5
}
},
{
"type": "FollowSpiral",
"config": {
"enabled": true,
"diameter": 4,
"step_size": 70
}
}
],
"map_object_cache_time": 5,
"forts": {
"avoid_circles": true,
"max_circle_size": 50,
"cache_recent_forts": true
},
"pokemon_bag": {
"// if 'show_at_start' is true, it will log all the pokemons in the bag (not eggs) at bot start": {},
"show_at_start": true,
"// if 'show_count' is true, it will show the amount of each pokemon (minimum 1)": {},
"show_count": false,
"// if 'show_candies' is true, it will show the amount of candies for each pokemon": {},
"show_candies": false,
"// 'pokemon_info' parameter define which info to show for each pokemon": {},
"// the available options are": {},
"// ['cp', 'iv_ads', 'iv_pct', 'ivcp', 'ncp', 'level', 'hp', 'moveset', 'dps']": {},
"pokemon_info": ["cp", "iv_pct"]
},
"walk_max": 4.16,
"walk_min": 2.16,
"alt_min": 500,
"alt_max": 1000,
"sleep_schedule": [
{
"time": "12:00",
"duration": "5:30",
"time_random_offset": "00:30",
"duration_random_offset": "00:30",
"wake_up_at_location": ""
},
{
"time": "17:45",
"duration": "3:00",
"time_random_offset": "01:00",
"duration_random_offset": "00:30",
"wake_up_at_location": ""
}
],
"debug": false,
"test": false,
"walker_limit_output": false,
"health_record": true,
"location_cache": true,
"distance_unit": "km",
"reconnecting_timeout": 15,
"logging": {
"color": true,
"clean": false
},
"catch": {
"any": {"always_catch": true}
},
"release": {
"any": {"keep_best_iv": 3, "release_below_cp": 0, "release_below_iv": 0, "logic": "or"},
"// Example of always releasing Rattata:": {},
"// Rattata": {"always_release": true},
"Raticate": {"always_release": true},
"Pidgeotto": {"always_release": true},
"Kakuna": {"always_release": true},
"Metapod": {"always_release": true},
"Snorlax": {"keep_best_iv": 3},
"Lapras": {"keep_best_iv": 3},
"Moltres": {"keep_best_iv": 3},
"Zapdos": {"keep_best_iv": 3},
"Articuno": {"keep_best_iv": 3},
"Muk": {"keep_best_iv": 3},
"Mewtwo": {"keep_best_iv": 3},
"Mew": {"keep_best_iv": 3},
"Pinsir": {"keep_best_iv": 2},
"Chamander": {"keep_best_iv": 3},
"// Example of keeping 3 stronger (based on CP) Pidgey:": {},
"// Pidgey": {"keep_best_cp": 3},
"// Example of keeping 2 stronger (based on IV) Zubat:": {},
"// Zubat": {"keep_best_iv": 2},
"// Keep no more than 3 best IV pokemon for every pokemon type": {},
"// any": {"keep_best_iv": 3},
"// Discard all pokemon in bag except 100 pokemon with best CP": {},
"// all": {"keep_best_cp": 100},
"// Example of keeping the 2 strongest (based on CP) and 3 best (based on IV) Zubat:": {},
"// Zubat": {"keep_best_cp": 2, "keep_best_iv": 3}
},
"vips" : {
"Any pokemon put here directly force to use Berry & Best Ball to capture, to secure the capture rate": {},
"any": {"catch_above_cp": 1200, "catch_above_iv": 0.9, "logic": "or" },
"Lapras": {},
"Moltres": {},
"Zapdos": {},
"Articuno": {},

    "// S-Tier pokemons (if pokemon can be evolved into tier, list the representative)": {},
    "Mewtwo": {},
    "Dragonite": {},
    "Snorlax": {},
    "// Mew evolves to Mewtwo": {},
    "Mew": {},
    "Arcanine": {},
    "Vaporeon": {},
    "Gyarados": {},
    "Exeggutor": {},
    "Muk": {},
    "Weezing": {},
    "Flareon": {}

}

}`

Output when issue occurred

Branch master have an update. Run ./setup.sh -u to update. 2016-08-30 18:36:35,654 [ cli] [INFO] PokemonGO Bot v1.0 2016-08-30 18:36:35,664 [ cli] [INFO] commit: ab5fb3540ac9312b5320bd0904e7faea170d7824 2016-08-30 18:36:35,669 [ cli] [INFO] Configuration initialized 2016-08-30 18:36:35,669 [pokemongo_bot.health_record.bot_event] [INFO] Health check is enabled. For more information: 2016-08-30 18:36:35,669 [pokemongo_bot.health_record.bot_event] [INFO] https://github.com/PokemonGoF/PokemonGo-Bot/tree/dev#analytics 2016-08-30 18:36:35,746 [requests.packages.urllib3.connectionpool] [INFO] Starting new HTTP connection (1): www.google-analytics.com 2016-08-30 18:36:36,535 [SleepSchedule] [INFO] [next_sleep] Next sleep at 2016-08-31 11:49:55.535766 2016-08-30 18:36:37,318 [PokemonGoBot] [INFO] [set_start_location] Setting start location. 2016-08-30 18:36:37,319 [PokemonGoBot] [INFO] Favorite location found: majorratchayothin ([u'13.829186', u'100.568974']) 2016-08-30 18:36:37,319 [PokemonGoBot] [INFO] [location_found] Location found: MajorRatchayothin (13.829186, 100.568974, 8.0) 2016-08-30 18:36:37,319 [PokemonGoBot] [INFO] [position_update] Now at (13.829186, 100.568974, 8.0) 2016-08-30 18:36:37,320 [PokemonGoBot] [INFO] [login_started] Login procedure started. 2016-08-30 18:36:41,869 [PokemonGoBot] [INFO] [login_successful] Login successful. 2016-08-30 18:36:41,869 [PokemonGoBot] [INFO] Found encrypt.so! Platform: darwin encrypt.so directory: /Users/Me/PokemonGo-Bot 2016-08-30 18:36:41,869 [PokemonGoBot] [INFO] 2016-08-30 18:36:42,888 [ cli] [INFO] 2016-08-30 18:36:42,888 [ cli] [INFO] Ran for 0:00:06 2016-08-30 18:36:42,889 [ cli] [INFO] Total XP Earned: 0 Average: 0.00/h 2016-08-30 18:36:42,889 [ cli] [INFO] Travelled 0.00km 2016-08-30 18:36:42,889 [ cli] [INFO] Visited 0 stops 2016-08-30 18:36:42,889 [ cli] [INFO] Encountered 0 pokemon, 0 caught, 0 released, 0 evolved, 0 never seen before () 2016-08-30 18:36:42,889 [ cli] [INFO] Threw 0 pokeballs 2016-08-30 18:36:42,889 [ cli] [INFO] Earned 0 Stardust 2016-08-30 18:36:42,889 [ cli] [INFO] Hatched eggs 0 2016-08-30 18:36:42,890 [ cli] [INFO] 2016-08-30 18:36:42,890 [ cli] [INFO] Highest CP Pokemon: 2016-08-30 18:36:42,890 [ cli] [INFO] Most Perfect Pokemon: Traceback (most recent call last): File "pokecli.py", line 764, in <module> main() File "pokecli.py", line 119, in main bot = start_bot(bot,config) File "pokecli.py", line 86, in start_bot bot.start() File "/Users/Me/PokemonGo-Bot/pokemongo_bot/__init__.py", line 123, in start self._load_recent_forts() File "/Users/Me/PokemonGo-Bot/pokemongo_bot/__init__.py", line 1298, in _load_recent_forts cached_recent_forts = json.load(f) File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 291, in load **kw) File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 339, in loads return _default_decoder.decode(s) File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 364, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 382, in raw_decode raise ValueError("No JSON object could be decoded") ValueError: No JSON object could be decoded 2016-08-30 18:36:42,905 [sentry.errors] [ERROR] Sentry responded with an error: 'utf8' codec can't decode byte 0x9c in position 1: invalid start byte (url: https://app.getsentry.com/api/90254/store/) Traceback (most recent call last): File "/Users/Me/PokemonGo-Bot/lib/python2.7/site-packages/raven/transport/threaded.py", line 174, in send_sync super(ThreadedHTTPTransport, self).send(data, headers) File "/Users/Me/PokemonGo-Bot/lib/python2.7/site-packages/raven/transport/http.py", line 47, in send ca_certs=self.ca_certs, File "/Users/Me/PokemonGo-Bot/lib/python2.7/site-packages/raven/utils/http.py", line 66, in urlopen return opener.open(url, data, timeout) File "/Users/Me/PokemonGo-Bot/lib/python2.7/site-packages/future/backports/urllib/request.py", line 494, in open response = self._open(req, data) File "/Users/Me/PokemonGo-Bot/lib/python2.7/site-packages/future/backports/urllib/request.py", line 512, in _open '_open', req) File "/Users/Me/PokemonGo-Bot/lib/python2.7/site-packages/future/backports/urllib/request.py", line 466, in _call_chain result = func(*args) File "/Users/Me/PokemonGo-Bot/lib/python2.7/site-packages/raven/utils/http.py", line 46, in https_open return self.do_open(ValidHTTPSConnection, req) File "/Users/Me/PokemonGo-Bot/lib/python2.7/site-packages/future/backports/urllib/request.py", line 1284, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1057, in request self._send_request(method, url, body, headers) File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1097, in _send_request self.endheaders(body) File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1053, in endheaders self._send_output(message_body) File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 895, in _send_output msg += message_body File "/Users/Me/PokemonGo-Bot/lib/python2.7/encodings/utf_8.py", line 16, in decode return codecs.utf_8_decode(input, errors, True) UnicodeDecodeError: 'utf8' codec can't decode byte 0x9c in position 1: invalid start byte 2016-08-30 18:36:42,907 [sentry.errors.uncaught] [ERROR] [u'ValueError: No JSON object could be decoded', u' File "pokecli.py", line 764, in <module>', u' File "pokecli.py", line 119, in main', u' File "pokecli.py", line 86, in start_bot', u' File "pokemongo_bot/__init__.py", line 123, in start', u' File "pokemongo_bot/__init__.py", line 1298, in _load_recent_forts', u' File "json/__init__.py", line 291, in load', u' File "json/__init__.py", line 339, in loads', u' File "json/decoder.py", line 364, in decode', u' File "json/decoder.py", line 382, in raw_decode'] Tue Aug 30 18:36:42 ICT 2016 Pokebot Stopped.

Steps to Reproduce

Run ./run.sh

Other Information

OS:

OSX El Capitan 10.11.4

Branch:

master

Git Commit:

56339fb

Python Version:

Python 2.7.12

Any other relevant files/configs (eg: path files)

[sohje]

Try to remove data/recent-forts-BOTNAME.json

[davidspeijer]

And update to the latest version...

See first line of your update: Branch master have an update. Run ./setup.sh -u to update.

[mjmadsen]

Looks like we missed a json read try.

[mjmadsen]

I think we need to check all live data json files at startup. As to why they keep having problems is the real problem. Or switch to a db so we can revert if write was unsuccessful.

[sohje]

@mjmadsen Do we really need recent-forts-botname.json? I dont see any practical usages.

[mjmadsen]

I think it was added for folks to see what the fort name is (without looking it up)? Not entirely sure.

[sohje]

Only list of fort ids inside that file. Also we have config option, right? Maybe we should remove it?

[mjmadsen]

Let me dig and see why this was even made a thing.

[sohje]

Yeah, sure.

[mjmadsen]

Oh, it's for remembering visited forts between restarts.

[sohje]

Yeap, but i dont see any usages - only write/read to json file.
update: i was wrong ;] found it inside move_to_fort task

[mjmadsen]

#3556 Is the initial merge. Supposed to make sure max_circle_size functions properly between boots.

I feel a bit like max_circle_size is a bit depreciated. It would seem to be implemented when we were trying to solve problems that are simpler than the current state of the bot.

Speaking of that, do we really have any purpose of using avoid circles? It seems very nonhuman to tour an entire city when there are forts available for spin near you.

[davidspeijer]

Moving around is making sure that you cover more ground to find different (types of) Pokemon? And isnt the avoid circle making sure that you don't walk up and down between 2 forts? If you want to spin more recent forts you can lower the value to 1 in your config?

[mjmadsen]

I suppose I'd instead prefer we had a feature to sleep for a bit then move us to a new cluster. As though we're a player who walks for a bit at the park then drives to the beach. But you bring up a good point.

[mjmadsen]

@davidspeijer Would you be satisfied if it sometimes revisited recent forts after a restart? We can probably fix it to at least start a new file if it can't be read properly.

[davidspeijer]

I think we need to check if the file isn't corrupt at loading the bot. I don't think it can breakdown during running (at least not without restart).
If corrupt create new?

[mjmadsen]

#4943 Should accomplish that. Though, I am rather tired.

[sohje]

Why we cant use data from api.get_map_objects()? There is already 'cooldown_complete_timestamp_ms' nope?

[mjmadsen]

@sohje I'll take a deeper look at the code tomorrow. Perhaps we can refactor and remove this bit.

[mjmadsen]

#5160 should solve this @mupinkz @sohje

[julienlavergne]

@sohje I checked and it seems that, sometimes, when calling api right after spinning a fort, the spun fort is still marked with the old cool down.
Which means we have to maintain the list of fort spun internally and rely on that rather than on the api.

[mupinkz]

The issue is solved. Thank you all.