1.5: stop_replica does not work

[ghost]

chameleon.py stop_replica --config switchv2 does not work with 1.5:

I could only stop by CTRL-C the chameleon.py start_replica --config switchv2 process

here the log:

Jul 18 15:03:11: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:03:19: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:03:23: [DEBUG] - mysql_lib.py (513): getting table metadata
Jul 18 15:03:24: [INFO] - global_lib.py (273): waiting for replica process to stop
Jul 18 15:03:27: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:03:35: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:03:43: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:03:51: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:03:58: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:04:06: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:04:14: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:04:22: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:04:30: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:04:38: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:04:46: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:04:54: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:05:02: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:05:10: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:05:19: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:05:27: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:05:35: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:05:44: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:05:52: [DEBUG] - pg_lib.py (1037): Batch loop value True
Jul 18 15:06:00: [ERROR] - global_lib.py (413): An error occurred during the replica. (<class 'SystemError'>, SystemError('<built-in function utf_8_decode> returned a result with an error set',), <traceback object at 0x7ff151295d08>)
Jul 18 15:06:03: [INFO] - global_lib.py (280): replica process stopped

[the4thdoctor]

the pg_lib.py (1037): Batch loop value True means the process is replaying the data collected from mysql. the replica will stop when all the batch data has been replayed.

You should be patient :)
If the replay is taking too much increase the value of replica_batch_size. By default is 1000, very low.
I've found a value of 50000 is a good compromise between speed and data moved across the network.

[ghost]

ok - maybe a different message would better explain this to the non-experts?

I had assumed that as soon as the "total events exceeded" messages are finished the replication is finished.
(I was actually starting to report an additional bug with regards to not handling a log-switch)

I now start to realize that it works a bit differently:
mysql-binlogs->t_log_replica_* tables->final_location

[the4thdoctor]

I agree. I will change the message as it's definitely not clear.

The caveats part explains the process
http://pgchameleon.org/documents/readme.html#caveats

[ghost]

thanks for your efforts - this is much nicer to read!

Jul 28 09:44:07: [INFO] - pg_lib.py (1047): Still working on batch for source  switchv2 replay size 200000 rows