URLs containing certain character codes break routing (reprise)

[jcoyne]

(moved from rails/rails#30811 at the behest @pixeltrix )

This was originally raised in rails/rails#4450 and I see that some code was created to address this:

https://github.com/econsultancy/rails/commit/7c385d085f131346e617b8bbdecf05d006429576
(I'm not seeing this code in the rails master branch though)

When this url is called:
https://example.com/assets/%F8%80%80%80%AE%F8%80%80%80%AE/%F8%80%80%80%AE%F8%80%80%80%AE/%F8%80%80%80%AE%F8%80%80%80%AE/%F8%80%80%80%AE%F8%80%80%80%AE/%F8%80%80%80%AE%F8%80%80%80%AE/%F8%80%80%80%AE%F8%80%80%80%AE/%F8%80%80%80%AE%F8%80%80%80%AE/%F8%80%80%80%AE%F8%80%80%80%AE/%F8%80%80%80%AE%F8%80%80%80%AE/%F8%80%80%80%AE%F8%80%80%80%AE/%F8%80%80%80%AE%F8%80%80%80%AE/%F8%80%80%80%AE%F8%80%80%80%AE/etc/passwd

ArgumentError: invalid byte sequence in UTF-8

    [GEM_ROOT]/gems/sprockets-3.7.1/lib/sprockets/server.rb:289 :in `[]`
    [GEM_ROOT]/gems/sprockets-3.7.1/lib/sprockets/server.rb:289 :in `path_fingerprint`
    [GEM_ROOT]/gems/sprockets-3.7.1/lib/sprockets/server.rb:36 :in `call`
    [GEM_ROOT]/gems/actionpack-5.1.4/lib/action_dispatch/routing/mapper.rb:17 :in `block in <class:Constraints>`
    [GEM_ROOT]/gems/actionpack-5.1.4/lib/action_dispatch/routing/mapper.rb:46 :in `serve`
    [GEM_ROOT]/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:50 :in `block in serve`

    [GEM_ROOT]/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:33 :in `each`
    [GEM_ROOT]/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:33 :in `serve`
    [GEM_ROOT]/gems/actionpack-5.1.4/lib/action_dispatch/routing/route_set.rb:834 :in `call`
    [GEM_ROOT]/gems/rack-2.0.3/lib/rack/etag.rb:25 :in `call` 

[schneems]

I"m not sure what we want to do here. @pixeltrix what kind of exception would Rails pick up and interpret as a 404? I can add validation encoding check to the server.rb for sprockets. That will "fix" the issue locally, but the problem will still exist in production in Rails.

[schneems]

Totally missed #514 never mind