Megan Taylor

front-end dev, volunteacher, news & data junkie, bibliophile, Flyers fan, sci-fi geek and kitteh servant

Visualizing Farmers Markets in NYC

<Stream of Consciousness>

Problem: GrowNYC does not have an at-a-glance visualization of what farmers market are open and where they are.

Solutions: Calendar and map visualizations, to answer both “What’s open now” and “What’s open near me.”

Implementation: I’ve never worked with .ical files before, but I would imagine you could scrape the data off the GrowNYC site and run a script to format it to be added to Google Calendar or whatever. I’m more interested in the map visualization. I’ve done some work with KML and GIS before, so I know it would be possible to scrape the data off the site and build a map with different icons showing year-round, open and closed markets.

Sub-Problem: I know how to scrape a single page of data, but have no idea how to go about scraping a series of static pages.

Solution: Google that!

I would like to scrape the data into a Google Spreadsheet, this would enable a lot of versatility and less maintenance (maybe?). Then set the scraper up to run once a week or so to update the spreadsheet. Spreadsheet could easily be used as the basis for both maps and calendars of various types.

Data needed: Each market’s location, active days of the week and hours, special closed days?, location, special events occurring?.

List of markets

Each market then has a page of its own (which contains the data needed!) like so:

97th St Greenmarket

Probably could all be done with Python. Use Beautiful Soup for scraping? How to get scraping results into Google Spreadsheet then? Google Spreadsheets also has a scraping language, I think. Could try to figure that out.

</Stream of Consciousness>

Yes, this is actually the way I thought this through on the way to work this morning.


  1. Interesting. Are you going to actually do it? Because that would be really cool and very helpful. Going green never goes out of style and I would love to a site that could tell me where the closest farmers market is from my place.

  2. I might make it a weekend project for January.