Proper encoding of load_url

[maeb]

Formatting of load_url does not encode the url parameter properly if it ends up in the query string of the configured url_field (replay_url):

pywb/pywb/warcserver/index/indexsource.py

Lines 160 to 162 in 843fe28

	cdx[self.url_field] = self.replay_url.format(url=cdx['url'],
	timestamp=cdx['timestamp'],
	src_coll=source_coll)

Some url's does not survive query parameter parsing unscaded when the url parameter is part of the query string of the load_url.

This seems to fix the issue:

        cdx[self.url_field] = res_template(self.replay_url, dict(url=cdx['url'],
                                                                 timestamp=cdx['timestamp'],
                                                                 src_coll=source_coll))

I believe this is a proper fix without breaking changes, but I am not sure. Shall I post a PR?

[maeb]

Referencing my previous issue #656 here. That issue concerned encoding of the url parameter in the query string of the request between the frontend and the backend. This issue concerns encoding of the same parameter in the request between the backend and the warcserver (as configured via the replay_url).

[ikreymer]

Just to confirm, this was for use with OutbackCDX, right? Or some other configuration?

[maeb]

We use our own indexer and loader backend https://github.com/nlnwa/gowarcserver.

[maeb]

Our config looks something like:

collections:                                                                                                          
  veidemann:                                                                                                          
    index:                                                                                                            
      type: cdx                                                                                                       
      api_url: http://gowarcserver:9999/warcserver/all/index?url={url}&closest={closest}                        
      replay_url: http://gowarcserver:9999/warcserver/all/resource?url={url}&closest={timestamp}&output=content