Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attempt to modify a readonly table script #238

Open
jarnoux opened this issue Nov 21, 2022 · 3 comments
Open

Attempt to modify a readonly table script #238

jarnoux opened this issue Nov 21, 2022 · 3 comments

Comments

@jarnoux
Copy link

jarnoux commented Nov 21, 2022

Hi Folks,
I'm getting the error below when trying to run tasks. It does look like a bug or a version compatibility issue. I tried flushing all redis, without success.
If anything, I'm at least not able to understand my mistake from the logs:

import tasktiger

tiger = tasktiger.TaskTiger(
    connection=StrictRedis(
        host=os.environ.get("REDIS_HOST"),
        port=int(os.environ.get("REDIS_PORT")),
        username=os.environ.get("REDIS_USERNAME", None),
        password=os.environ.get("REDIS_PASSWORD", None),
        db=0,
        decode_responses=True,
    ),
)

def schedule_miner_task(miner_id, delay=0):
    task = tasktiger.Task(tiger, miner_task, args=[miner_id])
    task.delay(when=delay)


@tiger.task(unique=True)
def miner_task(miner_id):
    miner = _find_miner_by_id(miner_id)

    if miner is not None:
        handle_process_miner(miner)

Redis version: 7.0.3
tasktiger version: 0.16.1

2022-11-21 06:47.15 [info     ] ready                          exclude_queues=[] id=30eec0d9-ad7d-4728-80dd-95c946898bfc max_workers=None pid=1 queues=[] single_worker_queues=[]
 2022-11-21 06:48.15 [info     ] stats                          pid=1 time_busy=0 time_total=60.00238370895386 utilization=0.0
 2022-11-21 06:49.15 [info     ] stats                          pid=1 time_busy=0 time_total=60.008546590805054 utilization=0.0
 2022-11-21 06:49.43 [debug    ] new queue                      pid=1 queue=default
 2022-11-21 06:49.43 [debug    ] moved tasks                    dest_queue=active pid=1 qty=1 queue=default src_queue=queued
 2022-11-21 06:49.43 [info     ] processing                     child_pid=16 func=miners.task:miner_task params={'args': ['637b12468e1139bdf913c98e'], 'kwargs': {}} pid=1 queue=default task_id=0413d2b4defb6f8a744e1c3efb75bf9edaca3aed99bc645473025e67423ca2f7
 2022-11-21 06:49.43 [debug    ] processed                      attempted=1 pid=1 processed=1 queue=default
 2022-11-21 06:49.43 [error    ] task error                     exception_name=builtins:KeyError func=miners.task:miner_task pid=1 processing_duration=0.0664057731628418 queue=default task_args=['637b12468e1139bdf913c98e'] task_id=0413d2b4defb6f8a744e1c3efb75bf9edaca3aed99bc645473025e67423ca2f7 task_kwargs={} time_failed=1669013383.275227 traceback=Traceback (most recent call last)

## My stack trace with my error in handle_process_miner
## then...

2022-11-21 06:49.43 [error    ] exception                      pid=1
Traceback (most recent call last):
   File "/app/server.py", line 47, in <module>
     tiger.run_worker()
   File "/usr/local/lib/python3.9/site-packages/tasktiger/tasktiger.py", line 385, in run_worker
     worker.run()
   File "/usr/local/lib/python3.9/site-packages/tasktiger/worker.py", line 1219, in run
     self._worker_run()
   File "/usr/local/lib/python3.9/site-packages/tasktiger/worker.py", line 1099, in _worker_run
     task_ids, processed_count = self._process_from_queue(queue)
   File "/usr/local/lib/python3.9/site-packages/tasktiger/worker.py", line 877, in _process_from_queue
     processed_count = self._process_queue_tasks(
   File "/usr/local/lib/python3.9/site-packages/tasktiger/worker.py", line 815, in _process_queue_tasks
     self._finish_task_processing(queue, task, success, now)
   File "/usr/local/lib/python3.9/site-packages/tasktiger/worker.py", line 1079, in _finish_task_processing
     task._move(from_state=ACTIVE, to_state=state, when=when)
   File "/usr/local/lib/python3.9/site-packages/tasktiger/task.py", line 321, in _move
     scripts.execute_pipeline(pipeline)
   File "/usr/local/lib/python3.9/site-packages/tasktiger/redis_scripts.py", line 544, in execute_pipeline
     raw_results = self._execute_pipeline(
   File "/usr/local/lib/python3.9/site-packages/redis-3.5.3-py3.9.egg/redis/client.py", line 4073, in __call__
     return client.evalsha(self.sha, len(keys), *args)
   File "/usr/local/lib/python3.9/site-packages/redis-3.5.3-py3.9.egg/redis/client.py", line 3143, in evalsha
     return self.execute_command('EVALSHA', sha, numkeys, *keys_and_args)
   File "/usr/local/lib/python3.9/site-packages/redis-3.5.3-py3.9.egg/redis/client.py", line 901, in execute_command
     return self.parse_response(conn, command_name, **options)
   File "/usr/local/lib/python3.9/site-packages/redis-3.5.3-py3.9.egg/redis/client.py", line 915, in parse_response
     response = connection.read_response()
   File "/usr/local/lib/python3.9/site-packages/redis-3.5.3-py3.9.egg/redis/connection.py", line 756, in read_response
     raise response
 redis.exceptions.ResponseError: user_script:50: Attempt to modify a readonly table script: 2020188d5e7d3d717c70cee65ed1674041eab216, on @user_script:50.
@AlecRosenbaum
Copy link
Contributor

It looks like we only run tests on redis versions 4/5/6, so it's possible there's a breaking change in redis 7 that we haven't tested yet

redis-version: [4, 5, "6.2.6"]

@AlecRosenbaum
Copy link
Contributor

AlecRosenbaum commented Nov 21, 2022

It's also possible that the breaking change is one of these, as tests started failing with 6.2.7:
https://github.com/redis/redis/blob/e6f67092f8d4d81761a60c46011d1ff1dc3c2628/00-RELEASENOTES#L34

@nsaje
Copy link
Contributor

nsaje commented Mar 14, 2023

This error is caused by redis/redis#10651 and is affecting Redis versions >= 6.2.7.

I created a new issue to track the resolution #268

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants