forked from celery/kombu
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathChangelog
2156 lines (1366 loc) · 58.5 KB
/
Changelog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
.. _changelog:
================
Change history
================
.. _version-3.0.0:
3.0.0
=====
:release-date: TBA
- No longer supports Python 2.5
- pickle, yaml and msgpack deserialization is now disabled by default.
To enable insecure serializers you have to call:
>>> from kombu import enable_insecure_serializers
>>> enable_insecure_serializers()
.. _version-2.5.11:
2.5.11
======
:release-date: TBA
- Adds ``passive`` option to :class:`~kombu.Exchange`.
Contributed by Rafal Malinowski
- librabbitmq: Now raises :exc:`NotImplementedError`
if ``ssl`` option enabled.
The librabbitmq library does not support ssl,
but you can use stunnel or change to the ``pyamqp://`` transport
instead.
Fix contributed by Dan LaMotte.
- eventio: select implementation now removes bad file descriptors.
.. _version-2.5.10:
2.5.10
======
:release-date: 2013-04-11 18:10 P.M BST
Note about upcoming changes for Kombu 3.0
-----------------------------------------
Kombu 3 consumers will no longer accept pickle/yaml or msgpack
by default, and you will have to explicitly enable untrusted deserializers
either globally using :func:`kombu.enable_insecure_serializers`, or
using the ``accept`` argument to :class:`~kombu.Consumer`.
Changes
-------
- New utility function to disable/enable untrusted serializers.
- :func:`kombu.disable_insecure_serializers`
- :func:`kombu.enable_insecure_serializers`.
- Consumer: ``accept`` can now be used to specify a whitelist
of content types to accept.
If the accept whitelist is set and a message is received
with a content type that is not in the whitelist then a
:exc:`~kombu.exceptions.ContentDisallowed` exception
is raised. Note that this error can be handled by the already
existing `on_decode_error` callback
Examples::
Consumer(accept=['application/json'])
Consumer(accept=['pickle', 'json'])
- Now depends on amqp 1.0.11
- pidbox: Mailbox now supports the ``accept`` argument.
- Redis: More friendly error for when keys are missing.
- Connection URLs: The parser did not work well when there were
multiple '+' tokens.
.. _version-2.5.9:
2.5.9
=====
:release-date: 2013-04-08 05:07 P.M BST
- Pidbox: Now warns if there are multiple nodes consuming from
the same pidbox.
- Adds :attr:`Queue.on_declared <kombu.Queue.on_declared>`
A callback to be called when the queue is declared,
with signature ``(name, messages, consumers)``.
- Now uses fuzzy matching to suggest alternatives to typos in transport
names.
- SQS: Adds new transport option ``queue_prefix``.
Contributed by j0hnsmith.
- pyamqp: No longer overrides verify_connection.
- SQS: Now specifies the ``driver_type`` and ``driver_name``
attributes.
Fix contributed by Mher Movsisyan.
- Fixed bug with ``kombu.utils.retry_over_time`` when no errback
specified.
.. _version-2.5.8:
2.5.8
=====
:release-date: 2013-03-21 04:00 P.M UTC
- Now depends on :mod:`amqp` 1.0.10 which fixes a Python 3 compatibility error.
- Redis: Fixed a possible race condition (Issue #171).
- Redis: Ack emulation/visibility_timeout can now be disabled
using a transport option.
Ack emulation adds quite a lot of overhead to ensure data is safe
even in the event of an unclean shutdown. If data loss do not worry
you there is now an `ack_emulation` transport option you can use
to disable it::
Connection('redis://', transport_options={'ack_emulation': False})
- SQS: Fixed :mod:`boto` v2.7 compatibility (Issue #207).
- Exchange: Should not try to re-declare default exchange (``""``)
(Issue #209).
- SQS: Long polling is now disabled by default as it was not
implemented correctly, resulting in long delays between receiving
messages (Issue #202).
- Fixed Python 2.6 incompatibility depending on ``exc.errno``
being available.
Fix contributed by Ephemera.
.. _version-2.5.7:
2.5.7
=====
:release-date: 2013-03-08 01:00 P.M UTC
- Now depends on amqp 1.0.9
- Redis: A regression in 2.5.6 caused the redis transport to
ignore options set in ``transport_options``.
- Redis: New ``socket_timeout`` transport option.
- Redis: ``InconsistencyError`` is now regarded as a recoverable error.
- Resource pools: Will no longer attempt to release resource
that was never acquired.
- MongoDB: Now supports the ``ssl`` option.
Contributed by Sebastian Pawlus.
.. _version-2.5.6:
2.5.6
=====
:release-date: 2013-02-08 01:00 P.M UTC
- Now depends on amqp 1.0.8 which works around a bug found on some
Python 2.5 installations where 2**32 overflows to 0.
.. _version-2.5.5:
2.5.5
=====
:release-date: 2013-02-07 17:00 P.M UTC
SQS: Now supports long polling (Issue #176).
The polling interval default has been changed to 0 and a new
transport option (``wait_time_seconds``) has been added.
This parameter specifies how long to wait for a message from
SQS, and defaults to 20 seconds, which is the maximum
value currently allowed by Amazon SQS.
Contributed by James Saryerwinnie.
- SQS: Now removes unpickleable fields before restoring messages.
- Consumer.__exit__ now ignores exceptions occurring while
cancelling the consumer.
- Virtual: Routing keys can now consist of characters also used
in regular expressions (e.g. parens) (Issue #194).
- Virtual: Fixed compression header when restoring messages.
Fix contributed by Alex Koshelev.
- Virtual: ack/reject/requeue now works while using ``basic_get``.
- Virtual: Message.reject is now supported by virtual transports
(requeue depends on individual transport support).
- Fixed typo in hack used for static analyzers.
Fix contributed by Basil Mironenko.
.. _version-2.5.4:
2.5.4
=====
:release-date: 2012-12-10 12:35 P.M UTC
- Fixed problem with connection clone and multiple URLs (Issue #182).
Fix contributed by Dane Guempel.
- zeromq: Now compatible with libzmq 3.2.x.
Fix contributed by Andrey Antukh.
- Fixed Python 3 installation problem (Issue #187).
.. _version-2.5.3:
2.5.3
=====
:release-date: 2012-11-29 12:35 P.M UTC
- Pidbox: Fixed compatibility with Python 2.6
2.5.2
=====
:release-date: 2012-11-29 12:35 P.M UTC
.. _version-2.5.2:
2.5.2
=====
:release-date: 2012-11-29 12:35 P.M UTC
- [Redis] Fixed connection leak and added a new 'max_connections' transport
option.
.. _version-2.5.1:
2.5.1
=====
:release-date: 2012-11-28 12:45 P.M UTC
- Fixed bug where return value of Queue.as_dict could not be serialized with
JSON (Issue #177).
.. _version-2.5.0:
2.5.0
=====
:release-date: 2012-11-27 04:00 P.M UTC
- `py-amqp`_ is now the new default transport, replacing ``amqplib``.
The new `py-amqp`_ library is a fork of amqplib started with the
following goals:
- Uses AMQP 0.9.1 instead of 0.8
- Support for heartbeats (Issue #79 + Issue #131)
- Automatically revives channels on channel errors.
- Support for all RabbitMQ extensions
- Consumer Cancel Notifications (Issue #131)
- Publisher Confirms (Issue #131).
- Exchange-to-exchange bindings: ``exchange_bind`` / ``exchange_unbind``.
- API compatible with :mod:`librabbitmq` so that it can be used
as a pure-python replacement in environments where rabbitmq-c cannot
be compiled. librabbitmq will be updated to support all the same
features as py-amqp.
- Support for using multiple connection URL's for failover.
The first argument to :class:`~kombu.Connection` can now be a list of
connection URLs:
.. code-block:: python
Connection(['amqp://foo', 'amqp://bar'])
or it can be a single string argument with several URLs separated by
semicolon:
.. code-block:: python
Connection('amqp://foo;amqp://bar')
There is also a new keyword argument ``failover_strategy`` that defines
how :meth:`~kombu.Connection.ensure_connection`/
:meth:`~kombu.Connection.ensure`/:meth:`kombu.Connection.autoretry` will
reconnect in the event of connection failures.
The default reconnection strategy is ``round-robin``, which will simply
cycle through the list forever, and there's also a ``shuffle`` strategy
that will select random hosts from the list. Custom strategies can also
be used, in that case the argument must be a generator yielding the URL
to connect to.
Example:
.. code-block:: python
Connection('amqp://foo;amqp://bar')
- Now supports PyDev, PyCharm, pylint and other static code analysis tools.
- :class:`~kombu.Queue` now supports multiple bindings.
You can now have multiple bindings in the same queue by having
the second argument be a list:
.. code-block:: python
from kombu import binding, Queue
Queue('name', [
binding(Exchange('E1'), routing_key='foo'),
binding(Exchange('E1'), routing_key='bar'),
binding(Exchange('E2'), routing_key='baz'),
])
To enable this, helper methods have been added:
- :meth:`~kombu.Queue.bind_to`
- :meth:`~kombu.Queue.unbind_from`
Contributed by Rumyana Neykova.
- Custom serializers can now be registered using Setuptools entry-points.
See :ref:`serialization-entrypoints`.
- New :class:`kombu.common.QoS` class used as a thread-safe way to manage
changes to a consumer or channels prefetch_count.
This was previously an internal class used in Celery now moved to
the :mod:`kombu.common` module.
- Consumer now supports a ``on_message`` callback that can be used to process
raw messages (not decoded).
Other callbacks specified using the ``callbacks`` argument, and
the ``receive`` method will be not be called when a on message callback
is present.
- New utility :func:`kombu.common.ignore_errors` ignores connection and
channel errors.
Must only be used for cleanup actions at shutdown or on connection loss.
- Support for exchange-to-exchange bindings.
The :class:`~kombu.Exchange` entity gained ``bind_to``
and ``unbind_from`` methods:
.. code-block:: python
e1 = Exchange('A')(connection)
e2 = Exchange('B')(connection)
e2.bind_to(e1, routing_key='rkey', arguments=None)
e2.unbind_from(e1, routing_key='rkey', arguments=None)
This is currently only supported by the ``pyamqp`` transport.
Contributed by Rumyana Neykova.
.. _version-2.4.10:
2.4.10
======
:release-date: 2012-11-22 06:00 P.M UTC
- The previous versions connection pool changes broke Redis support so that
it would always connect to localhost (default setting) no matter what
connection parameters were provided (Issue #176).
.. _version-2.4.9:
2.4.9
=====
:release-date: 2012-11-21 03:00 P.M UTC
- Redis: Fixed race condition that could occur while trying to restore
messages (Issue #171).
Fix contributed by Ollie Walsh.
- Redis: Each channel is now using a specific connection pool instance,
which is disconnected on connection failure.
- ProducerPool: Fixed possible dead-lock in the acquire method.
- ProducerPool: ``force_close_all`` no longer tries to call the non-existent
``Producer._close``.
- librabbitmq: Now implements ``transport.verify_connection`` so that
connection pools will not give back connections that are no longer working.
- New and better ``repr()`` for Queue and Exchange objects.
- Python3: Fixed problem with running the unit test suite.
- Python3: Fixed problem with JSON codec.
.. _version-2.4.8:
2.4.8
=====
:release-date: 2012-11-02 05:00 P.M UTC
- Redis: Improved fair queue cycle implementation (Issue #166).
Contributed by Kevin McCarthy.
- Redis: Unacked message restore limit is now unlimited by default.
Also, the limit can now be configured using the ``unacked_restore_limit``
transport option:
.. code-block:: python
Connection('redis://', transport_options={
'unacked_restore_limit': 100,
})
A limit of 100 means that the consumer will restore at most 100
messages at each pass.
- Redis: Now uses a mutex to ensure only one consumer restores messages at a
time.
The mutex expires after 5 minutes by default, but can be configured
using the ``unacked_mutex_expire`` transport option.
- LamportClock.adjust now returns the new clock value.
- Heartbeats can now be specified in URLs.
Fix contributed by Mher Movsisyan.
- Kombu can now be used with PyDev, PyCharm and other static analysis tools.
- Fixes problem with msgpack on Python 3 (Issue #162).
Fix contributed by Jasper Bryant-Greene
- amqplib: Fixed bug with timeouts when SSL is used in non-blocking mode.
Fix contributed by Mher Movsisyan
.. _version-2.4.7:
2.4.7
=====
:release-date: 2012-09-18 03:00 P.M BST
- Virtual: Unknown exchanges now default to 'direct' when sending a message.
- MongoDB: Fixed memory leak when merging keys stored in the db (Issue #159)
Fix contributed by Michael Korbakov.
- MongoDB: Better index for MongoDB transport (Issue #158).
This improvement will create a new compund index for queue and _id in order
to be able to use both indexed fields for getting a new message (using
queue field) and sorting by _id. It'll be necessary to manually delete
the old index from the collection.
Improvement contributed by rmihael
.. _version-2.4.6:
2.4.6
=====
:release-date: 2012-09-12 03:00 P.M BST
- Adds additional compatibility dependencies:
- Python <= 2.6:
- importlib
- ordereddict
- Python <= 2.5
- simplejson
.. _version-2.4.5:
2.4.5
=====
:release-date: 2012-08-30 03:36 P.M BST
- Last version broke installtion on PyPy and Jython due
to test requirements clean-up.
.. _version-2.4.4:
2.4.4
=====
:release-date: 2012-08-29 04:00 P.M BST
- amqplib: Fixed a bug with asynchronously reading large messages.
- pyamqp: Now requires amqp 0.9.3
- Cleaned up test requirements.
.. _version-2.4.3:
2.4.3
=====
:release-date: 2012-08-25 10:30 P.M BST
- Fixed problem with amqp transport alias (Issue #154).
.. _version-2.4.2:
2.4.2
=====
:release-date: 2012-08-24 05:00 P.M BST
- Having an empty transport name broke in 2.4.1.
.. _version-2.4.1:
2.4.1
=====
:release-date: 2012-08-24 04:00 P.M BST
- Redis: Fixed race condition that could cause the consumer to crash (Issue #151)
Often leading to the error message ``"could not convert string to float"``
- Connection retry could cause an inifite loop (Issue #145).
- The ``amqp`` alias is now resolved at runtime, so that eventlet detection
works even if patching was done later.
.. _version-2.4.0:
2.4.0
=====
:release-date: 2012-08-17 08:00 P.M BST
- New experimental :mod:`ZeroMQ <kombu.transport.zmq` transport.
Contributed by John Watson.
- Redis: Ack timed-out messages were not restored when using the eventloop.
- Now uses pickle protocol 2 by default to be cross-compatible with Python 3.
The protocol can also now be changed using the :envvar:`PICKLE_PROTOCOL`
environment variable.
- Adds ``Transport.supports_ev`` attribute.
- Pika: Queue purge was not working properly.
Fix contributed by Steeve Morin.
- Pika backend was no longer working since Kombu 2.3
Fix contributed by Steeve Morin.
.. _version-2.3.2:
2.3.2
=====
:release-date: 2012-08-01 06:00 P.M BST
- Fixes problem with deserialization in Python 3.
.. _version-2.3.1:
2.3.1
=====
:release-date: 2012-08-01 04:00 P.M BST
- librabbitmq: Can now handle messages that does not have a
content_encoding/content_type set (Issue #149).
Fix contributed by C Anthony Risinger.
- Beanstalk: Now uses localhost by default if the URL does not contain a host.
.. _version-2.3.0:
2.3.0
=====
:release-date: 2012-07-24 03:50 P.M BST
- New ``pyamqp://`` transport!
The new `py-amqp`_ library is a fork of amqplib started with the
following goals:
- Uses AMQP 0.9.1 instead of 0.8
- Should support all RabbitMQ extensions
- API compatible with :mod:`librabbitmq` so that it can be used
as a pure-python replacement in environments where rabbitmq-c cannot
be compiled.
.. _`py-amqp`: http://amqp.readthedocs.org/
If you start using use py-amqp instead of amqplib you can enjoy many
advantages including:
- Heartbeat support (Issue #79 + Issue #131)
- Consumer Cancel Notifications (Issue #131)
- Publisher Confirms
amqplib has not been updated in a long while, so maintaining our own fork
ensures that we can quickly roll out new features and fixes without
resorting to monkey patching.
To use the py-amqp transport you must install the :mod:`amqp` library::
$ pip install amqp
and change the connection URL to use the correct transport::
>>> conn = Connection('pyamqp://guest:guest@localhost//')
The ``pyamqp://`` transport will be the default fallback transport
in Kombu version 3.0, when :mod:`librabbitmq` is not installed,
and librabbitmq will also be updated to support the same features.
- Connection now supports heartbeat argument.
If enabled you must make sure to manually maintain heartbeats
by calling the ``Connection.heartbeat_check`` at twice the rate
of the specified heartbeat interval.
E.g. if you have ``Connection(heartbeat=10)``,
then you must call ``Connection.heartbeat_check()`` every 5 seconds.
if the server has not sent heartbeats at a suitable rate then
the heartbeat check method must raise an error that is listed
in ``Connection.connection_errors``.
The attribute ``Connection.supports_heartbeats`` has been added
for the ability to inspect if a transport supports heartbeats
or not.
Calling ``heartbeat_check`` on a transport that does
not support heartbeats results in a noop operation.
- SQS: Fixed bug with invalid characters in queue names.
Fix contributed by Zach Smith.
- utils.reprcall: Fixed typo where kwargs argument was an empty tuple by
default, and not an empty dict.
.. _version-2.2.6:
2.2.6
=====
:release-date: 2012-07-10 17:00 P.M BST
- Adds ``kombu.messaging.entry_to_queue`` for compat with previous versions.
.. _version-2.2.5:
2.2.5
=====
:release-date: 2012-07-10 17:00 P.M BST
- Pidbox: Now sets queue expire at 10 seconds for reply queues.
- EventIO: Now ignores ``ValueError`` raised by epoll unregister.
- MongoDB: Fixes Issue #142
Fix by Flavio Percoco Premoli
.. _version-2.2.4:
2.2.4
=====
:release-date: 2012-07-05 16:00 P.M BST
- Support for msgpack-python 0.2.0 (Issue #143)
The latest msgpack version no longer supports Python 2.5, so if you're
still using that you need to depend on an earlier msgpack-python version.
Fix contributed by Sebastian Insua
- :func:`~kombu.common.maybe_declare` no longer caches entities with the
``auto_delete`` flag set.
- New experimental filesystem transport.
Contributed by Bobby Beever.
- Virtual Transports: Now support anonymous queues and exchanges.
.. _version-2.2.3:
2.2.3
=====
:release-date: 2012-06-24 17:00 P.M BST
- ``BrokerConnection`` now renamed to ``Connection``.
The name ``Connection`` has been an alias for a very long time,
but now the rename is official in the documentation as well.
The Connection alias has been available since version 1.1.3,
and ``BrokerConnection`` will still work and is not deprecated.
- ``Connection.clone()`` now works for the sqlalchemy transport.
- :func:`kombu.common.eventloop`, :func:`kombu.utils.uuid`,
and :func:`kombu.utils.url.parse_url` can now be
imported from the :mod:`kombu` module directly.
- Pidbox transport callback ``after_reply_message_received`` now happens
in a finally block.
- Trying to use the ``librabbitmq://`` transport will now show the right
name in the :exc:`ImportError` if :mod:`librabbitmq` is not installed.
The librabbitmq falls back to the older ``pylibrabbitmq`` name for
compatibility reasons and would therefore show ``No module named
pylibrabbitmq`` instead of librabbitmq.
.. _version-2.2.2:
2.2.2
=====
:release-date: 2012-06-22 02:30 P.M BST
- Now depends on :mod:`anyjson` 0.3.3
- Json serializer: Now passes :class:`buffer` objects directly,
since this is supported in the latest :mod:`anyjson` version.
- Fixes blocking epoll call if timeout was set to 0.
Fix contributed by John Watson.
- setup.py now takes requirements from the :file:`requirements/` directory.
- The distribution directory :file:`contrib/` is now renamed to :file:`extra/`
.. _version-2.2.1:
2.2.1
=====
:release-date: 2012-06-21 01:00 P.M BST
- SQS: Default visibility timeout is now 30 minutes.
Since we have ack emulation the visibility timeout is
only in effect if the consumer is abrubtly terminated.
- retry argument to ``Producer.publish`` now works properly,
when the declare argument is specified.
- Json serializer: didn't handle buffer objects (Issue #135).
Fix contributed by Jens Hoffrichter.
- Virtual: Now supports passive argument to ``exchange_declare``.
- Exchange & Queue can now be bound to connections (which will use the default
channel):
>>> exchange = Exchange('name')
>>> bound_exchange = exchange(connection)
>>> bound_exchange.declare()
- ``SimpleQueue`` & ``SimpleBuffer`` can now be bound to connections (which
will use the default channel).
- ``Connection.manager.get_bindings`` now works for librabbitmq and pika.
- Adds new transport info attributes::
- ``Transport.driver_type``
Type of underlying driver, e.g. "amqp", "redis", "sql".
- ``Transport.driver_name``
Name of library used e.g. "amqplib", "redis", "pymongo".
- ``Transport.driver_version()``
Version of underlying library.
.. _version-2.2.0:
2.2.0
=====
:release-date: 2012-06-07 3:10 P.M BST
:by: Ask Solem
.. _v220-important:
Important Notes
---------------
- The canonical source code repository has been moved to
http://github.com/celery/kombu
- Pidbox: Exchanges used by pidbox are no longer auto_delete.
Auto delete has been described as a misfeature,
and therefore we have disabled it.
For RabbitMQ users old exchanges used by pidbox must be removed,
these are named ``mailbox_name.pidbox``,
and ``reply.mailbox_name.pidbox``.
The following command can be used to clean up these exchanges::
VHOST=/ URL=amqp:// python -c'import sys,kombu;[kombu.Connection(
sys.argv[-1]).channel().exchange_delete(x)
for x in sys.argv[1:-1]]' \
$(sudo rabbitmqctl -q list_exchanges -p "$VHOST" \
| grep \.pidbox | awk '{print $1}') "$URL"
The :envvar:`VHOST` variable must be set to the target RabbitMQ virtual host,
and the :envvar:`URL` must be the AMQP URL to the server.
- The ``amqp`` transport alias will now use :mod:`librabbitmq`
if installed.
`py-librabbitmq`_ is a fast AMQP client for Python
using the librabbitmq C library.
It can be installed by::
$ pip install librabbitmq
It will not be used if the process is monkey patched by eventlet/gevent.
.. _`py-librabbitmq`: https://github.com/celery/librabbitmq
.. _v220-news:
News
----
- Redis: Ack emulation improvements.
Reducing the possibility of data loss.
Acks are now implemented by storing a copy of the message when the message
is consumed. The copy is not removed until the consumer acknowledges
or rejects it.
This means that unacknowledged messages will be redelivered either
when the connection is closed, or when the visibility timeout is exceeded.
- Visibility timeout
This is a timeout for acks, so that if the consumer
does not ack the message within this time limit, the message
is redelivered to another consumer.
The timeout is set to one hour by default, but
can be changed by configuring a transport option:
>>> Connection('redis://', transport_options={
... 'visibility_timeout': 1800, # 30 minutes
... })
**NOTE**: Messages that have not been acked will be redelivered
if the visibility timeout is exceeded, for Celery users
this means that ETA/countdown tasks that are scheduled to execute
with a time that exceeds the visibility timeout will be executed
twice (or more). If you plan on using long ETA/countdowns you
should tweak the visibility timeout accordingly::
BROKER_TRANSPORT_OPTIONS = {'visibility_timeout': 18000} # 5 hours
Setting a long timeout means that it will take a long time
for messages to be redelivered in the event of a power failure,
but if so happens you could temporarily set the visibility timeout lower
to flush out messages when you start up the systems again.
- Experimental `Apache ZooKeeper`_ transport
More information is in the module reference:
:mod:`kombu.transport.zookeeper`.
Contributed by Mahendra M.
.. _`Apache ZooKeeper`: http://zookeeper.apache.org/
- Redis: Priority support.
The message's ``priority`` field is now respected by the Redis
transport by having multiple lists for each named queue.
The queues are then consumed by in order of priority.
The priority field is a number in the range of 0 - 9, where
0 is the default and highest priority.
The priority range is collapsed into four steps by default, since it is
unlikely that nine steps will yield more benefit than using four steps.
The number of steps can be configured by setting the ``priority_steps``
transport option, which must be a list of numbers in **sorted order**::
>>> x = Connection('redis://', transport_options={
... 'priority_steps': [0, 2, 4, 6, 8, 9],
... })
Priorities implemented in this way is not as reliable as
priorities on the server side, which is why
nickname the feature "quasi-priorities";
**Using routing is still the suggested way of ensuring
quality of service**, as client implemented priorities
fall short in a number of ways, e.g. if the worker
is busy with long running tasks, has prefetched many messages,
or the queues are congested.
Still, it is possible that using priorities in combination
with routing can be more beneficial than using routing
or priorities alone. Experimentation and monitoring
should be used to prove this.
Contributed by Germán M. Bravo.
- Redis: Now cycles queues so that consuming is fair.
This ensures that a very busy queue won't block messages
from other queues, and ensures that all queues have
an equal chance of being consumed from.
This used to be the case before, but the behavior was
accidentally changed while switching to using blocking pop.
- Redis: Auto delete queues that are bound to fanout exchanges
is now deleted at channel.close.
- amqplib: Refactored the drain_events implementation.
- Pidbox: Now uses ``connection.default_channel``.
- Pickle serialization: Can now decode buffer objects.
- Exchange/Queue declarations can now be cached even if
the entity is non-durable.
This is possible because the list of cached declarations
are now kept with the connection, so that the entities
will be redeclared if the connection is lost.
- Kombu source code now only uses one-level of explicit relative imports.
.. _v220-fixes:
Fixes
-----
- eventio: Now ignores ENOENT raised by ``epoll.register``, and
EEXIST from ``epoll.unregister``.
- eventio: kqueue now ignores :exc:`KeyError` on unregister.
- Redis: ``Message.reject`` now supports the ``requeue`` argument.
- Redis: Remove superfluous pipeline call.
Fix contributed by Thomas Johansson.
- Redis: Now sets redelivered header for redelivered messages.