A few months ago we announced a partnership with Stamen Design, the best online mapping and data visualization company we know. It’s time to announce Polymaps, the first public milestone in that partnership, at http://polymaps.org.
We’ve been working with Stamen to provide visual analysis of the huge datasets that we’re working with, and how people can communicate this data in sophisticated ways. A first step toward that goal is the release of a free and open-source set of tools and map engines allowing people to perform relatively sophisticated operations on their data in the browser.
The project has been online for a while at http://github.com/simplegeo/polymaps, and you can download the source code there; what’s new is the addition of a series of example maps so you can demonstrate what’s going on, and human-readable documentation so you can use them for your own projects.
Some of the examples are straightforward, letting you do things like group points into clusters and drop scaled gradients on to map locations. Others are more robust, letting you do things like change which direction is north by rotating the map and visualize the quality of street surfaces in San Francisco.
Stamen has also been consulting with us on our API development and serving as alpha consumers of the datasets that we’re working with. The Polymaps project has helped us get clear on what parts of the API we’re ready to support in the near term, and which are going to require more work. Stamen is helping us identify new needs that the SimpleGeo service could fulfill; basically fleshing out the API methods that projects like http://crimespotting.org use to feed richly interactive maps over time, divide them up by administrative district, and so on.
The kinds of datasets that SimpleGeo works with are often on the scale of (say) a million Crimespotting projects, so the ability to do the kind of queries that come naturally to smaller projects doesn’t necessarily come free.
Working with Stamen at this early stage is helping us clarify and answer questions that developers will want to be able to ask when our service is out in the wild.
Starting August 15th, we will start to enforcing users to accept a Terms of Service for access to every layer in the SimpleGeo Marketplace. This means that any layers you are currently using for which you haven’t added through the marketplace, particularly the com.simplegeo.global.* layers, will stop working. Any layer you’ve created privately will not be affected.
For the past few months, the Foursquare, Flickr, Twitter, Brightkite, Fwix, and Bump layers have been available but not advertised. We are actively working on licensing deals with these companies to make this data available to you soon.
As we build out the marketplace and re-launch our website in the coming months, the process of creating a publicly accessible layer and enabling a TOS will be more clear.
If you have data that you want to showcase in our marketplace, send us an email at firstname.lastname@example.org.
Next week, we will be participating in Geo-Loco 2010, an event that focuses on the business and future of geo-location services. Use the code simplegeo20 to get a 20% discount on a ticket to the event.
As part of the festivities, we will be hosting a networking event on Tuesday night at our new San Francisco headquarters, located at 41 Decatur St.
There will be a guest list, so please make sure to register soon as capacity is limited.
On Thursday July 22nd, we will be having open office hours that you can sign up for to pick our brains and learn about our API and product offerings. If you are interested in scheduling a time to come by, please send an email to email@example.com, and we will coordinate with you on a time to meet.
Last week, we showed you how to grab all of your checkins from Foursquare, and visualize them on a SimpleGeo layer.
The script is available to download on Github. The Twitter API by default only returns 20 tweets at a time. You can add the count=200 parameter to max out the request. If you wanted to grab the last 1600 tweets, you could use tweepy’s pagination functions.
I often like to joke that the dozen or so years I’ve spent working in and helping to build startups is the longest case study in how not to run a startup. Since starting SimpleGeo Matt and I have obsessed about culture. We spend a lot of time attempting to make SimpleGeo a great company to work for filled with wonderfully talented people working on difficult problems.
Recently, while working on our mission statement, Jeffrey, Rob, and I were brainstorming words we felt represented SimpleGeo’s products, goals, and mission. Two things ended up happening:
- We kept interrupting each other. Finally, Jeffrey had had enough and we decided that we’d allow each other to fully express their opinions before retorting.
- I shared some random link from the internet and started a completely unrelated discussion. All of the sudden Rob starts making a rowing motion with his arms. I asked, “What the hell does that mean?” Rob said it was an old joke from another company he worked at where they referred to people getting off-track in meetings as “kayaking.” From that point on all three of us were making rowing motions whenever one of us got off-track.
These two simple rules got me thinking. What if we codified a set of principles for SimpleGeo? What if our employees had guiding principles that they could point at during the hiring process, while building products, while arguing, etc.? After further thought, I sent off a Basecamp message with my initial thoughts on what would eventually become SimpleGeo’s “Cultural Creed.”
Being an agile shop, we iterated from my initial ideas into a set of 10, well 9, principles that we believe embodied SimpleGeo. Here they are:
- Approach people and ideas with courage and respect.
- Be attentive to customers and their needs.
- Embrace and learn from failure.
- Be curious.
- Celebrate the bizarre.
- Codify best practices.
- Be adaptable; things change.
- Repay technical debt.
- Embrace and drive change.
There you have it. The SimpleGeo Cultural Creed. I think too often in the race to raise money, hire more engineers, and build products that startups forget that their companies are an organism that’s growing and changing over time. As such, it’s important to remember your DNA.
CTO & Co-founder
About a week ago Joe and I gathered the entire SimpleGeo team at our San Francisco offices for a week of team building, camaraderie, and hacking. It was also a time for our executive team to hammer out the basics of what SimpleGeo will be as a company moving forward. One thing quickly became clear during that week: SimpleGeo would be better off if Joe and I were in San Francisco full-time.
This was not an easy conclusion to come to, especially for myself. While Joe is more than happy to return to San Francisco, I’m finding it difficult to embrace leaving Boulder. Boulder has been my home, my springboard, and a source of great joy since I moved here for TechStars. Don’t worry, though, both Joe and I will be returning frequently and are committed to supporting the great startup community in Boulder moving forward.
That being said, Joe and I want to be perfectly clear: SimpleGeo will maintain an active presence in Boulder moving forward. Boulder continues to be a great source of engineering talent, especially for location-based services such as us. The fact is, the Colorado Front Range has been the Silicon Valley of location for years thanks to the mining and oil industries here. We’d be remiss to ignore the talent lurking about Boulder. Or, as Joe often says, “We’d go to Antarctica to find great engineers if we had to.”
What a difference a year can make. See you all in San Francisco!
Thanks, Matt Galligan, Co-Founder & CEO
If you are a Foursquare user, have you ever wanted to visualize all of your checkins on a map? Using your Foursquare RSS feed and a simple Python script, you can import all of your checkins to SimpleGeo’s API and do spatial queries on them.
Step 1: Login to your Foursquare account and visit the Feeds page.
Take note of your RSS feed. In my case, it was http://feeds.foursquare.com/history/026d45564de36937b02de38a01095620.rss. You can add a count parameter to the end of that URL to get more items. I used count=2000 to get all of my checkins.
Step 3: Create a new layer, and name it com.foursquare.[your_foursquare_username].checkins.
Step 4: Download this Python script and update your SimpleGeo OAuth token and secret, the name of the layer you want to store your checkins, and your Foursquare RSS feed. Make sure you have the SimpleGeo Python client installed.
Step 5: Once you’ve updated those custom fields in Step 3, open the Python interactive shell and type the following:
import parse and then
The script adds records to the SimpleGeo API 90 at a time, to avoid being rate-limited. If you login to our Dashboard and click on the layer you just created, you can then see all of your checkins through the data explorer.
If you have any questions about how to use this script, feel free to send me an email at andrew [at] simplegeo.com.
Update: Bill Shupp has written an excellent version of this script in PHP.
We’ve got a lot of data at SimpleGeo. Sadly, our data is wandering through our server farms aimlessly in search of the meaning within itself. You know, the usual existentialist questions that data sets seem to have:
- What do I mean?
- Why was I written to disk?
- Will I just end up as an input to some larger aggregate?
- Should I wear XML or JSON today?
We’re becoming a bit worried here at SimpleGeo. Our data has taken to wearing a black beret, smoking cigarettes, and reading La Nausée in dimly lit coffee shops while it searches for the meaning of itself.
As a result we’re actively looking for a Data Scientist to help us and our data find meaning in our data’s life. Besides weaning our data off of caffeine and nicotine, we’re hoping you exhibit the following traits and desires:
- Live near, or are willing to relocate to, San Francisco, CA.
- Advanced degree (MS, PhD) or applicable experience in something resembling applied mathematics, computer science, machine learning, economics, or physics.
- Demonstrable experience in extracting useful insights from large datasets.
- Experience in working with large datasets using Hadoop is a huge plus.
- Experience with Java and/or Python.
- Experience with data analysis tools, such as MATLAB and/or R, a plus.
- A strong desire to work with a small, but highly technical team.
If this sounds interesting to you, or sounds like someone you know, please send us your resume (PDF or TXT) or LinkedIn profile to firstname.lastname@example.org.
The dataset is updated by the USGS every few minutes, but for the purposes of this experiment, we’re just going to download the latest version of the XML file and parse it. This data is from the past 7 days.
We used Python to hack together this script that adds records to the SimpleGeo API. There are three parts to the script:
Here, we are initializing the imports and setting up our authentication. Grab your OAuth key and secret from the Account Settings page.
The USGS returns the timestamp of an earthquake in a weird format, so here is some code to generate a standard time object for Python:
And here is the actual parser, using lxml. It’s pretty straight forward; we are basically creating Record objects and injecting them into the API 90 at a time.
When it’s all said and done, there are about 1800 records in the API now.
The next step is to write some sort of script to update the XML file every few minutes so the records are up to date.
If you want to play around with this data, activate the layer in the Marketplace.
The SimpleGeo Marketplace is an place for developers to share their data with the rest of the community.
The layer includes U.S. campgrounds, parks, museums, historic sites, recreation sites, and other nature-related points of interest. There are roughly 9,000 records in this layer.
To start using this layer, login to your SimpleGeo account, and visit the layer’s page. Click “Review Terms and Activate Layer”, and accept the terms. Then you can visit the Data Visualizer page and pan around to see the records on the map.
This is just the beginning. The engineering team has a data wishlist internally, and during our hack week in San Francisco, we are adding to it.
We would love to know what other data you want to see featured in our Marketplace? Let us know by pinging us on Twitter.