BUG: pickle RootLogger object

[lesteve]

It seems like #94 only fixed the case for logging.Logger objects and not logging.RootLogger objects (e.g. the logger you get when calling logging.getLogger()):

To reproduce:

import logging
import cloudpickle

logger = logging.getLogger()
logging.basicConfig()
cloudpickle.dumps(logger)

Error:

TypeError: can't pickle _thread.RLock objects

Full traceback on master

TypeError                                 Traceback (most recent call last)
<ipython-input-1-7f7ba95099c6> in <module>()
      4 logger = logging.getLogger()
      5 logging.basicConfig()
----> 6 cloudpickle.dumps(logger)

/home/local/lesteve/dev/cloudpickle/cloudpickle/cloudpickle.py in dumps(obj, protocol)
    889     try:
    890         cp = CloudPickler(file, protocol=protocol)
--> 891         cp.dump(obj)
    892         return file.getvalue()
    893     finally:

/home/local/lesteve/dev/cloudpickle/cloudpickle/cloudpickle.py in dump(self, obj)
    266         self.inject_addons()
    267         try:
--> 268             return Pickler.dump(self, obj)
    269         except RuntimeError as e:
    270             if 'recursion' in e.args[0]:

/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in dump(self, obj)
    407         if self.proto >= 4:
    408             self.framer.start_framing()
--> 409         self.save(obj)
    410         self.write(STOP)
    411         self.framer.end_framing()

/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save(self, obj, save_persistent_id)
    519 
    520         # Save the reduce() output and finally memoize the object
--> 521         self.save_reduce(obj=obj, *rv)
    522 
    523     def persistent_id(self, obj):

/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save_reduce(self, func, args, state, listitems, dictitems, obj)
    632 
    633         if state is not None:
--> 634             save(state)
    635             write(BUILD)
    636 

/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save(self, obj, save_persistent_id)
    474         f = self.dispatch.get(t)
    475         if f is not None:
--> 476             f(self, obj) # Call unbound method with explicit self
    477             return
    478 

/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save_dict(self, obj)
    819 
    820         self.memoize(obj)
--> 821         self._batch_setitems(obj.items())
    822 
    823     dispatch[dict] = save_dict

/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in _batch_setitems(self, items)
    845                 for k, v in tmp:
    846                     save(k)
--> 847                     save(v)
    848                 write(SETITEMS)
    849             elif n:

/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save(self, obj, save_persistent_id)
    474         f = self.dispatch.get(t)
    475         if f is not None:
--> 476             f(self, obj) # Call unbound method with explicit self
    477             return
    478 

/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save_list(self, obj)
    779 
    780         self.memoize(obj)
--> 781         self._batch_appends(obj)
    782 
    783     dispatch[list] = save_list

/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in _batch_appends(self, items)
    806                 write(APPENDS)
    807             elif n:
--> 808                 save(tmp[0])
    809                 write(APPEND)
    810             # else tmp is empty, and we're done

/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save(self, obj, save_persistent_id)
    519 
    520         # Save the reduce() output and finally memoize the object
--> 521         self.save_reduce(obj=obj, *rv)
    522 
    523     def persistent_id(self, obj):

/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save_reduce(self, func, args, state, listitems, dictitems, obj)
    632 
    633         if state is not None:
--> 634             save(state)
    635             write(BUILD)
    636 

/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save(self, obj, save_persistent_id)
    474         f = self.dispatch.get(t)
    475         if f is not None:
--> 476             f(self, obj) # Call unbound method with explicit self
    477             return
    478 

/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save_dict(self, obj)
    819 
    820         self.memoize(obj)
--> 821         self._batch_setitems(obj.items())
    822 
    823     dispatch[dict] = save_dict

/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in _batch_setitems(self, items)
    845                 for k, v in tmp:
    846                     save(k)
--> 847                     save(v)
    848                 write(SETITEMS)
    849             elif n:

/home/local/lesteve/miniconda3/lib/python3.6/pickle.py in save(self, obj, save_persistent_id)
    494             reduce = getattr(obj, "__reduce_ex__", None)
    495             if reduce is not None:
--> 496                 rv = reduce(self.proto)
    497             else:
    498                 reduce = getattr(obj, "__reduce__", None)

TypeError: can't pickle _thread.RLock objects

[lesteve]

I tried to use pytest.mark.parametrize but I could not make it work with test classes deriving from unittest.TestCase (see this SO answer which seems to indicate that this is not possible)

[codecov-io]

Codecov Report

Merging #160 into master will increase coverage by 0.09%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #160      +/-   ##
==========================================
+ Coverage   83.73%   83.82%   +0.09%     
==========================================
  Files           1        1              
  Lines         535      538       +3     
  Branches       96       96              
==========================================
+ Hits          448      451       +3     
  Misses         63       63              
  Partials       24       24

Impacted Files	Coverage Δ
cloudpickle/cloudpickle.py	83.82% <100%> (+0.09%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f14d19d...8551606. Read the comment docs.

[lesteve]

Since logging.RootLogger derives from logging.Logger I was wondering whether it would be worth to walk the class __mro__ in order to find a custom reduce function. I went for the simpler option.

[pitrou]

This looks fine to me, thanks.

[rgbkrk]

Given the last few, do you want to make another release @HyukjinKwon?

[lesteve]

Great to see this merged, thanks everyone!

[HyukjinKwon]

@rgbkrk Sure, will do soon later. I am a bit busy for relocating now :-).