Render a generic icon for shop=*

[dkniffin]

https://trac.openstreetmap.org/ticket/3043

I'm working on this right now. I will submitting a pull request shortly.

[gravitystorm]

I'm happy to see all shops rendered, and I'm in favour of an approach that doesn't involve hundreds of new icons!

However, I'm opposed to "catch-all" rendering, generally of the form "foo is not null" in the queries. I'm slowly removing them, since they've been shown over the years to be A Bad Idea.

• catch-alls are very hard to customize. Rendering the names of everything with a name, for example, means that if you don't want underground pipeline names showing up, when you remove the catch-all then 90% of the other names disappear too. If there turns out to be some kind of niche use of the shop tag, e.g. street vendors or youth club chocolate bar sellers, we end up rendering them and it's hard for downstream cartographers to be selective.
• catch-alls promote bad tagging. Given the tight feedback loop between the style and mapping verification, and the lengths we've gone to elsewhere to avoid it (see Resolve the landcover ordering problem #15 (comment) ) I don't want to render shop=bakres or shop=spuermarket

I'm going to give a few technical comments on the separate pull request too.

[dkniffin]

Ok, I think I understand, but I want to make sure. Instead of doing "shop is not null" in the query, I should instead do "shop in ('supermarket','clothes', [etc])" and include everything we want rendered? I'm fine with that, but it will make the query about twice as long. I guess I'd include everything from http://wiki.openstreetmap.org/wiki/Key:shop and then if there's others, they'd get added over time.

[jremillard]

Use tag info, shows all of the values for shop that are actually used.

http://taginfo.openstreetmap.org/keys/shop#values

[dkniffin]

Right, but then which of those do I add? I certainly can't (and don't want to) add all 7551.

[jremillard]

Whatever you think makes sense. How about anything over 1000 instances, or the top 50?

[dkniffin]

That sounds reasonable. I think I'll include all the ones on the wiki, and anything over 1000 uses in taginfo. Of course, in the future, the ones "over 1000" will be different, but it can be updated later.

[dkniffin]

so, that becomes: ('alcohol', 'anime', 'antiques', 'art', 'baby_goods', 'bag', 'bakery', 'bathroom_furnishing', 'beauty', 'bed', 'beverages', 'bicycle', 'books', 'boutique', 'butcher', 'candles', 'car', 'car_parts', 'carpet', 'car_repair', 'charity', 'chemist', 'clothes', 'computer', 'confectionery', 'convenience', 'copyshop', 'craft', 'curtain', 'deli', 'department_store', 'dive', 'doityourself', 'drugstore', 'dry_cleaning', 'electronics', 'energy', 'erotic', 'estate_agent', 'fabric', 'farm', 'fishing', 'fishmonger', 'florist', 'frame', 'funeral_directors', 'furnace', 'furniture', 'garden_centre', 'gas', 'general', 'gift', 'glaziery', 'greengrocer', 'hairdresser', 'hardware', 'hearing_aids', 'herbalist', 'hifi', 'hunting', 'interior_decoration', 'jewelry', 'kiosk', 'kitchen', 'laundry', 'mall', 'massage', 'mobile_phone', 'money_lender', 'motorcycle', 'music', 'musical_instrument', 'newsagent', 'no', 'optician', 'organic', 'outdoor', 'paint', 'pawnbroker', 'pet', 'pharmacy', 'photo', 'pyrotechnics', 'radiotechnics', 'religion', 'seafood', 'second_hand', 'shoes', 'sports', 'stationery', 'supermarket', 'tailor', 'tattoo', 'ticket', 'tobacco', 'toys', 'trade', 'travel_agency', 'tyres', 'vacant', 'vacuum_cleaner', 'variety_store', 'video', 'video_games', 'window_blind', 'yes')

[dkniffin]

I'll take shop=no out. any others?

[jremillard]

Is including erotic going to be an issue?

[dkniffin]

I removed it

[dkniffin]

I also removed vacant. I think it'd cause too much confusion

[RicoElectrico]

I think that shop=no should also be excluded.

[matthijsmelissen]

What is exactly the reason to remove shop=erotic? Google Maps does render them (with a generic shop icon), both in the US and Europe. Wikipedia has a strict 'no self-censorship' policy, what is our policy? Also, if it becomes ultimately the only type of shop that is not rendered, people might start working around if (tagging it as shop=gift? ;)).

[matthijsmelissen]

The tag drugstore is discouraged on the wiki and rarely used, I would remove it.

[matthijsmelissen]

There exist office=estate_agent and shop=estate_agent. The second is a bit more popular. We might want to render only one of them, in order to promote consistent tagging.

[jremillard]

1. It is below 1000 instances
2. It is offensive to some people.

There are ~6500 other shop types that are also not getting rendered because they are below a 1000 instances.

[matthijsmelissen]

I understand the problem of course. But not including data on the map that might offend people can get tricky easily. For example, shop=alcohol and shop=hunting (or a potential shop=guns) are also offensive to some people (and suffer equally from an age limit problem). I think we should be really careful here what argumentation we use, because it might set a precedent for later.

[jremillard]

For me, taginfo is most important, since erotic is basically right at the cutoff. It is probably ok to leave it off. Including it, and not other tags above it would be a big problem. I would rather not be part of another talk like this one at next years state of the map.

http://vimeopro.com/openstreetmapus/state-of-the-map-us-2013/video/68098504

[dieterdreist]

2013/8/29 math1985 [email protected]

There exist office=estate_agent and shop=estate_agent. The second is a bit
more popular. We might want to render only one of them, in order to promote
consistent tagging.

I'd render both of them in order to promote consistent tagging ;-)
an office is a place not intended for occasional customers to drop by while
a shop is a place open to the general public.

cheers,
Martin

[dieterdreist]

2013/8/28 oddityoverseer13 [email protected]

so, that becomes: ('alcohol', 'anime', 'antiques', 'art', 'baby_goods',
'bag', 'bakery', 'bathroom_furnishing', 'beauty', 'bed', 'beverages',
'bicycle', 'books', 'boutique', 'butcher', 'candles', 'car', 'car_parts',
'carpet', 'car_repair', 'charity', 'chemist', 'clothes', 'computer',
'confectionery', 'convenience', 'copyshop', 'craft', 'curtain', 'deli',
'department_store', 'dive', 'doityourself', 'drugstore', 'dry_cleaning',
'electronics', 'energy', 'erotic', 'estate_agent', 'fabric', 'farm',
'fishing', 'fishmonger', 'florist', 'frame', 'funeral_directors',
'furnace', 'furniture', 'garden_centre', 'gas', 'general', 'gift',
'glaziery', 'greengrocer', 'hairdresser', 'hardware', 'hearing_aids',
'herbalist', 'hifi', 'hunting', 'interior_decoration', 'jewelry', 'kiosk',
'kitchen', 'laundry', 'mall', 'massage', 'mobile_phone', 'money_lender',
'motorcycle', 'music', 'musical_instrument', 'newsagent', 'no', 'optician',
'organic', 'outdoor', 'paint', 'pawnbroker', 'pet', 'pharmacy ', 'phot o',
'pyrotechnics', 'radiotechnics', 'religion', 'seafood', 'second_hand',
'shoes', 'sports', 'stationery', 'supermarket', 'tailor', 'tattoo',
'ticket', 'tobacco', 'toys', 'trade', 'travel_agency', 'tyres', 'vacant',
'vacuum_cleaner', 'variety_store', 'video', 'video_games', 'window_blind',
'yes')

I propose to add antiques, insurance, ticket, tattoo, hearing_aids, frame,
bed, wine, curtain, pawnbroker, carpet, baby_goods, betting,
bathroom_furnishing, tea, shoe_repair, cosmetics, solarium, cheese,
perfumery, herbalist, bag, or in other words what is documented or above a
few hundred. As you can see these are actually not so many that would have
to be added and they don't seem to be covered by what you already have
above.

cheers,
Martin

[vincentdephily]

I agree with the additional included values, as there are some in there I added myself, and the uncommon shops are actually the ones that one might do an internet search for.

Did you just take the values from taginfo ? What cutoff value did you use; it'd be easyer to see if the list was sorted by frequency instead of alphabetically (it might even have a small performance advantage).

Here's an ugly shell script that spits out a list of all shops used at least 100 times, excluding "no" and "fixme":

wget -qO- 'http://taginfo.openstreetmap.org/api/4/key/values?key=shop&sortname=count_all&sortorder=desc'|sed 's/{"value":/\n/g'|grep 'count":[0-9][0-9][0-9]'|cut -d, -f1|grep -vE '"(no|fixme)"'|sed "s/\"/'/g;s/$/,/g"|tr -d "\n"

And the resulting list:

'supermarket', 'convenience', 'clothes', 'bakery', 'hairdresser', 'car_repair', 'car', 'kiosk', 'doityourself', 'butcher', 'florist', 'mall', 'bicycle', 'furniture', 'alcohol', 'books', 'electronics', 'shoes', 'department_store', 'yes', 'hardware', 'jewelry', 'optician', 'chemist', 'gift', 'garden_centre', 'greengrocer', 'mobile_phone', 'beverages', 'computer', 'beauty', 'sports', 'laundry', 'toys', 'confectionery', 'travel_agency', 'stationery', 'hifi', 'motorcycle', 'variety_store', 'newsagent', 'dry_cleaning', 'vacant', 'pet', 'car_parts', 'boutique', 'outdoor', 'tyres', 'video', 'deli', 'copyshop', 'estate_agent', 'funeral_directors', 'music', 'fishmonger', 'art', 'organic', 'fabric', 'general', 'photo', 'pharmacy', 'second_hand', 'kitchen', 'tailor', 'farm', 'seafood', 'tobacco', 'charity', 'massage', 'paint', 'musical_instrument', 'interior_decoration', 'trade', 'energy', 'antiques', 'insurance', 'erotic', 'ticket', 'tattoo', 'hearing_aids', 'frame', 'bed', 'wine', 'curtain', 'pawnbroker', 'carpet', 'food', 'ice_cream', 'grocery', 'baby_goods', 'bookmaker', 'betting', 'cosmetics', 'glaziery', 'bathroom_furnishing', 'fish', 'tea', 'winery', 'fashion', 'shoe_repair', 'photo_studio', 'solarium', 'craft', 'medical_supply', 'locksmith', 'hunting', 'cheese', 'perfumery', 'market', 'bag', 'coffee', 'household', 'discount', 'radiotechnics', 'money_lender', 'gas', 'other', 'appliance', 'wholesale', 'herbalist', 'lottery', 'fishing', 'vacuum_cleaner', 'video_games', 'communication', 'bags', 'houseware', 'builder', 'gallery', 'hobby', 'shopping_centre', 'salon', 'office_supplies', 'pets', 'musical_instruments', 'watches', 'dive', 'furnace', 'scuba_diving', 'gambling', 'leather', 'photography', 'pottery', 'motorcycle_repair', 'printing', 'medical', 'lighting'

Concerning 'erotic', I really don't see a good reason for excluding it. We've included rarer shops and I don't feel like opening Pandora's Censorship Box. People who are shocked by such shop should lobby them out of the neighborhood instead of out of the map.

[dieterdreist]

2013/8/29 vincentdephily [email protected]

I agree with the additional included values, as there are some in there I
added myself, and the uncommon shops are actually the ones that one might
do an internet search for.

Did you just take the values from taginfo ?

yes, I copied those of the less used values that seemed not to be disputed
or alternatives of what is already there.

Some of the often used values aren't unambiguous or are unsuitable or
duplicates (IMHO), e.g. from your taginfo list: general, yes, other, art
(wouldn't that be an art_gallery instead of a shop?), farm (better you
something like farm_produce?), vacant, charity (is this really a shop or
would office be better, if not, what kind of stuff do they sell, maybe
better use second_hand?), trade (maybe I simply don't know what this is),
ice_cream (isn't the common tag amenity=ice_cream for take away places and
amenity=cafe with cuisine=ice_cream for inplace icecream eating?), market
(what is this? convenience?), communication (what is this, telcos? mobile
phones? envelopes for letters? or pigeons?), glaziery (isn't this better
put in craft?), fashion (is this clothes? maybe a subtype might be more
suitable, see also boutique), craft (what is intended, handycraft?),
household (what do they sell, kitchen tools? cutlery? duvet covers?
furniture? food?), medical and finally there is winery vs. wine.

What cutoff value did you use

I went without real systematics, looking to add some of the values I
personally like and that I thought to be undisputed with 200-1000
occurences to the 1000+ list. ;-)

I agree that 100 seems a better limit than 1000.

cheers,
Martin

[matthijsmelissen]

Some comments on some your tags:

• art: I would tag it as a shop when the main purpose of the place is to sell the art. For example, some galleries tag their art works with a price tag. In that case, I would say it's rather a shop than, let's say, a museum.
• charity: Charity shops are a well-known concept in the UK, see https://en.wikipedia.org/wiki/Charity_shop . Brits always refer to these shops as 'charity shops' instead of 'second hand shops'. I think they don't really exist in the rest of Europe.
• trade: I use it for places that sell, for example, bricks. I also don't like the tag, though.
• glaziery: this is a shop that sells window panes. Is glaziery a term well-understood by natives? i had to look it up. I think if it has a showroom, then shop might be better than craft?
• craft: the shop where you buy stuff for sewing, painting etc.
• household: I like this one. In the Netherlands, there are many general household shops (kitchen tools, cleaning tools, maybe even some decorational stuff). I noticed that these don't occur that much in other countries (in Luxembourg, I would buy most stuff in hypermarkets instead). It has overlap with shop=interior_decoration, though.

I think I agree with the rest of your points.

If I have some time next month, I might try to adapt the wiki to the current situation and perhaps even discuss with the community on how to settle on some of the disputed tags. I have tagged all missing shops in Luxembourg-city, so I have a quite good of the problems currently encountered when tagging.

[dieterdreist]

2013/8/29 math1985 [email protected]

Some comments on some your tags:

• art: I would tag it as a shop when the main purpose of the place is
  to sell the art. For example, some galleries tag their art works with a
  price tag. In that case, I would say it's rather a shop than, let's say, a
  museum.

I would never tag a private art gallery as museum - or maybe almost never
(and these edge cases are "private museums" and not "art galleries").
Usually the idea of selling the exposed works is part of the concept of a
private art_gallery (not saying its the only one reason), even if they
don't put prices besides the art work (more commonly there is a price list
you can get if you ask for) but that's not necessarily an argument for shop.

• charity: Charity shops are a well-known concept in the UK, see
  https://en.wikipedia.org/wiki/Charity_shop . Brits always refer to
  these shops as 'charity shops' instead of 'second hand shops'. I think they
  don't really exist in the rest of Europe.

they do, at least in Berlin it is almost crowded with them ;-)

• trade: I use it for places that sell, for example, bricks. I also
  don't like the tag, though.

ah OK, my guess is this is really an insider term for natives ;-), would
suggest to use something like shop=building_material (but that's maybe not
very distinct and could refer to lots of different shops?), maybe
mason's_material?

• craft: the shop where you buy stuff for sewing, painting etc.

in here (Italy and Germany) these are two distinct kinds of shops (one
where you get wool, sewing, knitting material/tools etc. and another type
that is specialized in artists (different kind of pencils, paint, canvas,
paper, brushes, sometimes specialized in one or more techniques, in other
cases they try to offer universal artist supply)

• household: I like this one. In the Netherlands, there are many
  general household shops (kitchen tools, cleaning tools, maybe even some
  decorational stuff). I noticed that these don't occur that much in other
  countries (in Luxembourg, I would buy most stuff in hypermarkets instead).
  It's similar to shop

I seriously wasn't sure what the intended meaning was, as in Germany what
we call "Haushaltswaren" would usually be cups, plates, cutlery but not
cleaning tools and rarely electric kitchen tools, e.g.
http://www.dinkel-porzellan.de/haushaltswaren-tuebingen or this declare
themselves "Haushaltwaren": http://www.emsa.com/

but there are also others that sell household appliances (vacuumcleaners,
flattening irons, toothbrushs, shavers) and could be called "household" in
some way, also those that sell detergents, plastic cutlery, toilet paper
and the like...

cheers,
Martin

[matthijsmelissen]

Here is a folder that gives an overview of what Dutch household shops sell:
http://view.publitas.com/de-blokker-folder/blokker-folder-35/page/1
Do you have shops like that in Germany? How would you propose tagging something like this? I think it's smaller than a department store, although some of them are tagged like that.

[dieterdreist]

2013/8/29 math1985 [email protected]

Here is a folder of what Dutch household shops sell:
http://view.publitas.com/de-blokker-folder/blokker-folder-35/page/20
Do you not have shops like that in Germany? How would you propose tagging
something like this? I think it's smaller than a department store, although
some of them are tagged like that.

yes, I'd say this is a kind of a department store, they have a bit of
everything (electronics, knives, garden stuff) (in German: "Kaufhaus").

cheers,
Martin

[dkniffin]

So, since it seems like this needs a larger discussion, I've created a wiki page: http://wiki.openstreetmap.org/wiki/Users:Oddityoverseer/Shops_to_Render

The page explains itself.

[matthijsmelissen]

Good idea. I added a section for tags under discussion on the page.

This is an example of the household shop I mentioned: http://www.ijsselmonde-online.nl/nex/sites/default/files/profielen/Virano/blokker.jpg
The shop is usually not much larger than this. I think it is too small for a department store, and it doesn't really have departments.

[jremillard]

In the interest of getting something merged in the near future, It would be prudent to just pick some threshold on the taginfo (1000, 200, 100, I don't care which), and render things above that limit, unless we know for sure a tag should not be rendered (like no, vacant, etc). Then let the refinements to the initial list be different pull requests. I could see this turning into a bunch of long "conversations" as people lobby for and against specific tags. Lets not let perfect get in the way of good.

[SK53]

In this case I was thinking of large appliances, white & brown goods (resp. Washing Machines, Fridges, Dishwashers vs. TVs, HiFis ...), rather than smaller appliances. There may be some ambiguity in how these different shops are tagged. My case is really that we can recognise a relatively small number of categories to simplify rendering and other processes such as finding a suitable tag: it's perhaps a bit harder to decide exactly what belongs in each category. (I thought easily my most controversial assignment was shop=bicycle to hobby :-).

[dieterdreist]

2013/9/13 SK53 [email protected]

In this case I was thinking of large appliances, white & brown goods
(resp. Washing Machines, Fridges, Dishwashers vs. TVs, HiFis ...), rather
than smaller appliances.

I see. My comment was because some time ago I was looking for a tag for a
shop that sold and repaired only small appliances and accessory (flattening
iron, hair dryer, light bulb etc.) where the common "electronics" would not
have hit it (because although some of these might have electronic parts in
the meantime, traditionally they don't need them as opposed to tv, hifis,
dvd players, gaming consoles, ...). You know, these old style small village
shops. I think the shops for bigger appliances are mostly mapped as
"electronics".
+1 to grouping in the style sheet, -1 to normalizing the data, but this
grouping could also be done by variating a similar icon, not necessarily
has it to look identical. Anyways, for the moment also a generic icon for
all shops that haven't got theirs would be fine, e.g. at zoom 18 and/or 19.

[fuddl]

I think shop=erotic totally should be visible on the map

[pnorman]

catch-alls promote bad tagging. Given the tight feedback loop between the style and mapping verification, and the lengths we've gone to elsewhere to avoid it (see #15 (comment) ) I don't want to render shop=bakres or shop=spuermarket

While I agree in many cases, I'm not sure that is an issue here. If someone tags as those they're getting feedback by the use of a generic icon instead of the supermarket/bakery.

We'd also be encouraging inappropriate tagging still. Picking a random local example, take shop=locksmith. It's low enough usage that it hasn't fallen into any of the lists so by not having it render.

On the other hand, this doesn't exactly help the proliferation of shop values.

[vincentdephily]

I like the idea of multi-purpose icon, although this might be out of scope for this bug.

I dont think that we should try to de-bias / normalize the data :

1. you'll never find unbiased data. A fully-maped town is likely an european one, biasing against Africa/etc. Even a fully-maped town is biased (see recent childcare-is-all-the-same controversy).
2. it needs an obscure query (why is value foo more important ?) that is harder to maintain an probably slower.
3. it's not the right place to solve maping bias: so what if shop=erotic is overrepresented compared to shop=beautician ? I think it's more important for this style to be useful to current contributors rather than demographicaly correct. Encourage less-biased maping, extract the list of shops automatically and regularly from taginfo, and the problem disapears.

[vincentdephily]

@pnorman re including rare values VS excluding bad values :
I agree that we want to map every correct value, no matter how rare. But there is a simple issue of available manpower to review and exclude bad values as we start including the long tail.

I think the proposed threshold of 100 is good for a start (come on, merge that already ! :p). If we've got more time later, we can lower the threshold and sift through the values.

I also think, if that wasn't clear already, that we shouldn't commit the list of values but rather the script that generates it (fetching from taginfo and applying a blacklist). I know this style is currently a list of static files and no Makefile, but I think the change would be worth it.

[brycenesbitt]

@pnorman agreed completely. Setting a threshold encourages mapping to the rendering. I'll admit I've even remapped a greengrocer as a convenience just to get it to show up (sorry). And I searched hard to find a suitable rendered tag that would work for produce stands before filing https://trac.openstreetmap.org/ticket/3906 .

Thus I'd say:

• A small list of shops that have unique icons (supermarket, tobacco, bookstores, etc)
• A small list of exclusions (no, vacant, fixme, asdf).
• A small list of zoom exceptions (farm stands show up in empty areas)
• A future grouping key for shop classes (e.g. eating / growing / recreation / building supplies / electronics / other ).
• A generic icon for the rest.

I think it's up to editing tools to discourage 'shope=spuermarket'. In general I would appreciate, not be annoyed, by a prompt 'spuermarket has not been used yet, did you mean supermarket?'.

If the objection raised by @gravitystorm on exclusions is too strong, then put shop= on some sort of cron tagwatch process that looks for new tag values, and assigns http://maproulette.org/ tasks to go in and see if they can be made more regular.

Let's also not forget about the semicolon (e.g. shop=guns;alcohol at http://taginfo.openstreetmap.org/search?q=+shop%3Dguns%3Balcohol ).

++++++++++++++++++++++
Example: there's a dog collar store in Berkeley California, selling nothing but dog collars: http://pacocollars.com/stores/
There can't be more than a few dozen of those on the entire planet. That makes it all the more interesting to map.

There's a relationship here with what the Open Directory Project does in categorizing physical locations, but that project is not nearly as vibrant as OSM. That said, they vet the website portion of local shop listings and have an insanely strict ontology at places like:
http://www.dmoz.org/Regional/North_America/United_States/California/Localities/B/Berkeley/Recreation_and_Sports/Crafts/

[jremillard]

I thought this was considered a feature request and therefor needed to wait until the re-factoring/bug fixing is complete.

[brycenesbitt]

@jremillard is it not always appropriate to discuss, even if implementation is delayed?

[brycenesbitt]

Rendering shops could make it just a tad harder for people wanting to spam online shops into OSM:
https://lists.openstreetmap.org/pipermail/talk/2013-December/068735.html

[HolgerJeromin]

please cleanup the shop=fish things in the list. The "right" Tag is
http://wiki.openstreetmap.org/wiki/Tag:shop%3Dseafood
Right now fishmonger, fish and the right seafood is not rendered. So please do not start rendering the "wrong" ones :-)

[matthijsmelissen]

See also http://wiki.openstreetmap.org/wiki/Project_of_the_week/2010/Jul_25 .

[daganzdaanda]

Hi, any development on this?

[matthijsmelissen]

I think this is a very important issue, and I would like to see this implemented as soon as possible. I don't know if @oddityoverseer13 is around, but if not, I'm sure someone else can take over.

@gravitystorm Would you be willing to merge this (once a suitable pull-request has been created) before we are at the 'add new features' stage? Out of all missing features, I think this is one of the most desirable ones,

[gravitystorm]

Sure

[dkniffin]

Just to update this issue, I have created a new pull request #604 that has the changes mentioned. See the PR for more info.

[matthijsmelissen]

Looks great! I tested this in Tilemill and checked the code, and don't see any problems, so I recommend this to be merged.

[matthijsmelissen]

Closed by #604.