Leeds council performace

[robbrad]

Seaons greetings @dp247 - the Leeds council takes a long time to run - downloading the address file etc each time.

Any thoughts on how to remove the need to do this?

What if it was a pre-req to find out your UPRN before run? - or do these change?

[dp247]

I like how github didn't send me an email for that ping 😅.

It could def be changed to eliminate one of the CSVs by asking users to find their property id in the dataset. I'll have a play around.

[dp247]

Wrote a tool that can parse the premises.csv file with an inputted postcode and paon/saon and return the correct premises ID for a property:

So the scraper can be updated to take the ID as UPRN and reduce the load time. I haven't written any docs yet, but the script is here (lots more user-friendly than downloading the CSV file and finding the ID manually I reckon!)

[robbrad]

@dp247 when you get a second - can you check the current performace on the Leeds council - I think we are in a good place on this now?

[robbrad]

@dp247 - im gonna close for now

[dp247]

@dp247 - im gonna close for now

Didn't actually see this until today, but yeah, its working much quicker than I remember. It will probably never be massively speedy because the job file it parses is ~139MB lol

[robbrad]

I think making this Selenium would make a big difference

https://www.leeds.gov.uk/residents/bins-and-recycling/check-your-bin-day