Happy Mapping: GIS Discussion Thread

[Dirx]

I was prompted in another thread to start a thread about GIS. So, uh, here it is.


In short, GIS (Geographic Information System) encompasses all sorts of ways to collect and manage data with a spatial/geographic component. And the more you get into it, the more you learn that just about everything can be manipulated in GIS to some degree.

I'm no know-it-all by any means, but I do use it professionally, so I'm happy to discuss it and dispense any advice I can.


I and a large percentage of other users use ArcGIS software from ESRI. It's like the Microsoft Office of GIS software. There are certainly other options (like the free and open source QGIS), but you're probably going to get the best out-of-the-box functionality and integration with ESRI's software suite. Because I'm hooked and like to nerd out, I got myself a personal use subscription to ArcGIS, so I can dabble in it at home. This will run you about $100/year, so really it's no more expensive than the supplies you'd buy for many other hobbies. Hell, I'm pretty sure Netflix is more expensive at this point.


So how did I get into GIS? Most of you know me as the dinosaur guy. I even had an old "Ask the Paleontologist" thread around here somewhere. Well, it turns out, I got hired (as a paleontologist) onto an archaeological consulting company. There were a couple large projects involving fossils that I got to work with. After those wrapped up, however, there wasn't really any paleo left, so I did what the other archaeologists and archeo techs did: write boring reports for cell towers and make maps of the nearby cultural resources.* Turns out, I caught on to the mapping much more than the others, and ended up specializing in it. When the main GIS person left, I took over and did it exclusively.** Six years later, almost to the day, I took a job as the first-ever GIS Specialist for a nearby city. That was just five months ago. I'm loving every minute of it, and learning more and more every day. GIS has a myriad array of uses and specializations, and you could spend a lifetime mastering just a few.

And to think, back in grad school, when my best friends were taking GIS courses, I teased them and said "But anyone can make a map!"

Well, it's true. Anyone can make a map. But GIS can help you make a damn good one, and do so much more. And it can appeal to so many different types of personalities. Got a head for numbers and other data? You can do tons of that without ever even making a map. Are you artsy-fartsy and like making pretty things? Cartography is totally where you're at--grab some ready-made data and go to town! Like community outreach, service, and education? Make resources for public use via intuitive online map/app designers. It's all there.


Anyway, I don't want to write a novel here in the OP, so I'll invite questions from those who are interested in GIS, and just let the thread go on from there. I also invite anyone else who uses GIS--be it professional or personal or whathaveyou--to chime in. I'm hoping for this to be more of a discussion thread than a straight up "Ask the..." thread.



*"No, this tower is not built on top of a burial mound." "Yes, this tower would be visible from the picturesque historic district, but it doesn't really count so it's cool...-ish."

**Well, I did still do a spot of paleo here and there, and I also got roped into endangered species evaluations, but those are outside the scope of this thread.

[SNFaulkner]

I'm still not sure I get what it is. What's a normal day at work like for you?

[SmartAleq]

Here's a neat bit of mapwork that conclusively explains why Mt St Helens is so very far out of line with the other Cascadian volcanoes. I have to admit I've wondered why this is the case--now I know!

[Dirx]

I do all kinds of stuff, but one of may primary goals right now is to get the city's water/sewer system mapped into our database. Sure, we have various staking sheets, as-built plans, CAD maps, and the ever-present "So-and-so says it's right here" methods of locating things, but who's going to carry all that around all the time?

I've been using a fancy, centimeter-accuracy GPS unit to record water features (valves etc), and compiling all the above resources together into one large database. In this database, you can click on any given feature and find out all sorts of information about it. On a segment of water main, for example, you can see not just the location, length, and direction, but you can record the material, diameter, water type, flow, and more. Think of having a whole spreadsheet of data attached to every individual feature on a map.

Not only that, but an add-on I've been working with can intuitively map flow within the water network. So, let's say there's a leak in the main in a particular neighborhood. I can point to that segment in the database and run a tool that will quickly point out which valves need to be shut off in order to isolate that segment.

I can publish our water data online for the rest of the city, so when Public Works personnel need to find someone's curb stop valve, they no longer have to look for a blue-painted marker that may or may not still be there, or wave their metal detector around for 15 minutes trying to find it. They can instead bring up the data on their smart phone and be taken right to it.

I can then go one step further, and allow that same person to update data from their phone, and have that update translate back to the server-stored database. "Installed new water line with valves--new point and line features are recorded and uploading now." Or "Existing valve is broken; stuck in 'On' position; updating condition status."

And with a feature called "Dashboard," I can set up an always-on display of the water data, with maps, graphs, and indicators that reflect real-time data on the mains meters, storage tank levels, active service orders, and more.

And this is all just for water. For a local government, GIS is often used in planning/zoning, emergency responses, community involvement/reporting, and more.

Even without all of the bells and whistles, GIS is still, at the very least, a digital map studio. Anything you find on a paper map can be done within a digital GIS environment, and can be combined or overlaid for convenience or further analysis. Have three different utility projects at one intersection, all done at different times by different companies? Instead of flipping back and forth between three or more maps, you can have those layers all visible within the same interactive map, allowing you to change their appearance, toggle them on or off, select and inspect attributes of individual features, and so on. You can then export this map, with whatever labels, annotation, etc., at any scale, at any size, to various formats. Need a poster of your resources to keep on the office wall? Maybe just a page-sized PDF to distribute to the public? It's as simple as any Office program to do so.



Eventually, we'll be doing more work with Planning & Zoning, and then seeing if any other departments want in (for example, it's not uncommon for police departments to use some GIS as well). Not only that, but in the other thread, SmartAleq mentioned using it to see real estate data. A lot of state, county, and city governments publish public map servers online. These typically include parcel data, school/voting districts, park locations, flood zones, and more. Where I live, Pennington County has a few different maps set up for public use. Try googling "[your county] GIS" and see what you find. All the data that goes into that has to be managed by one or more individuals. It's curated, documented, organized, and maintained, just like physical artifacts in a museum's collection. And because it's all data, it can be manipulated and reproduced and combined to create new data. With the right data sets and software tools, you can ask a map to compare multiple layers and point out, say, "all the residential lots that are appraised at X value, outside the 100 year flood plain, and within Y distance of a bus stop or commuter rail station." At my previous company, it was very common to search for historic and archaeological resources within X radius of a proposed cell tower (dependent on tower height), which we then filtered by additional criteria such as age, current condition, and status on the National Register of Historic Places. We'd then generate maps of this data for the engineering companies that hired us, so they could make sure their project was in compliance with federal guidelines and whatnot.

In short: if it's on a map, or can go on a map, it can go in a digital map. I'll find some good online examples of some of the things I talked about, to help illustrate, but for now I should get some sleep. I hope I helped answer your question!

[Dirx]

Originally Posted by SmartAleq

Here's a neat bit of mapwork that conclusively explains why Mt St Helens is so very far out of line with the other Cascadian volcanoes. I have to admit I've wondered why this is the case--now I know!

Aww yeah, mapping + geology + answering puzzling questions =

[saves link for later 'cause I'm too tired to process it right now]

[SmartAleq]

Portland Maps is one of the better GIS map systems I've navigated--put in an address and you'll have access to aerial photos of the property, assessor's information, where the utility lines run, the lot lines, a list of building permits issued, tax info, owner of record, just a crapton of information about every piece of property in the city. You can change the type of information displayed on the map by toggling various layers and there are usually links to stuff like assessor's parcel maps and surveyor's maps and just all kinds of fun shit for nosy people.

[SNFaulkner]

Originally Posted by Dirx

I do all kinds of stuff, but one of may primary goals right now is to get the city's water/sewer system mapped into our database. Sure, we have various staking sheets, as-built plans, CAD maps, and the ever-present "So-and-so says it's right here" methods of locating things, but who's going to carry all that around all the time?

I've been using a fancy, centimeter-accuracy GPS unit to record water features (valves etc), and compiling all the above resources together into one large database. In this database, you can click on any given feature and find out all sorts of information about it. On a segment of water main, for example, you can see not just the location, length, and direction, but you can record the material, diameter, water type, flow, and more. Think of having a whole spreadsheet of data attached to every individual feature on a map.

Not only that, but an add-on I've been working with can intuitively map flow within the water network. So, let's say there's a leak in the main in a particular neighborhood. I can point to that segment in the database and run a tool that will quickly point out which valves need to be shut off in order to isolate that segment.

I can publish our water data online for the rest of the city, so when Public Works personnel need to find someone's curb stop valve, they no longer have to look for a blue-painted marker that may or may not still be there, or wave their metal detector around for 15 minutes trying to find it. They can instead bring up the data on their smart phone and be taken right to it.

I can then go one step further, and allow that same person to update data from their phone, and have that update translate back to the server-stored database. "Installed new water line with valves--new point and line features are recorded and uploading now." Or "Existing valve is broken; stuck in 'On' position; updating condition status."

And with a feature called "Dashboard," I can set up an always-on display of the water data, with maps, graphs, and indicators that reflect real-time data on the mains meters, storage tank levels, active service orders, and more.

And this is all just for water. For a local government, GIS is often used in planning/zoning, emergency responses, community involvement/reporting, and more.

Even without all of the bells and whistles, GIS is still, at the very least, a digital map studio. Anything you find on a paper map can be done within a digital GIS environment, and can be combined or overlaid for convenience or further analysis. Have three different utility projects at one intersection, all done at different times by different companies? Instead of flipping back and forth between three or more maps, you can have those layers all visible within the same interactive map, allowing you to change their appearance, toggle them on or off, select and inspect attributes of individual features, and so on. You can then export this map, with whatever labels, annotation, etc., at any scale, at any size, to various formats. Need a poster of your resources to keep on the office wall? Maybe just a page-sized PDF to distribute to the public? It's as simple as any Office program to do so.



Eventually, we'll be doing more work with Planning & Zoning, and then seeing if any other departments want in (for example, it's not uncommon for police departments to use some GIS as well). Not only that, but in the other thread, SmartAleq mentioned using it to see real estate data. A lot of state, county, and city governments publish public map servers online. These typically include parcel data, school/voting districts, park locations, flood zones, and more. Where I live, Pennington County has a few different maps set up for public use. Try googling "[your county] GIS" and see what you find. All the data that goes into that has to be managed by one or more individuals. It's curated, documented, organized, and maintained, just like physical artifacts in a museum's collection. And because it's all data, it can be manipulated and reproduced and combined to create new data. With the right data sets and software tools, you can ask a map to compare multiple layers and point out, say, "all the residential lots that are appraised at X value, outside the 100 year flood plain, and within Y distance of a bus stop or commuter rail station." At my previous company, it was very common to search for historic and archaeological resources within X radius of a proposed cell tower (dependent on tower height), which we then filtered by additional criteria such as age, current condition, and status on the National Register of Historic Places. We'd then generate maps of this data for the engineering companies that hired us, so they could make sure their project was in compliance with federal guidelines and whatnot.

In short: if it's on a map, or can go on a map, it can go in a digital map. I'll find some good online examples of some of the things I talked about, to help illustrate, but for now I should get some sleep. I hope I helped answer your question!

It did. Sounds fun!

[SmartAleq]

GIS systems are one of the better examples of the sort of thing tax dollars should be spent on--you have no idea how hard it used to be to look up site information for real estate appraisal purposes back before there was an internet. You had to physically go to the courthouse and be very proficient with microfilm and microfiche viewers--back in the day being a good appraisal researcher was as much an art as a science and most people just didn't know how to cast the right sized net in the proper direction to get the info needed on the first try. These young whippersnappers don't know how good they have it--they'll never know how to find the right block of FEMA flood map, unfold the entire thing across a conference table to find the correct location of the property on the unmarked maps then how to fold the map to A) get a photocopy of the right spot then B) get it back into the proper slot in the file without ripping it to shreds. THE STRUGGLE WAS REAL Y'ALL.

[Jaglavak]

Thaks for starting this. I'm running the free app GRASS GIS. I'm a total noob, and am interested in using GIS to screen for real estate. That would include at minimum the topo, hydrology, climate, and boundaries and jurisdictions. Also possibly stuff like soil types, vegetation, and local political leanings. I would like to do things like filter for elevation, slope, azimuth of the fall line, and rainfall.

Right now I'm having trouble just getting a map on the screen. I downloaded shape files from a Washington state database which unpacked as a bunch of prj, shx, dbf, and shp files. I fire up GRASS, I select the data directory, I create a test project, and it opens up a layer manager window and a map display window. Both of which are blank white screen. I've tried zooming out to map extents just in case I was focused on a mole hill, but no dice. Where's the map?

[Dirx]

To be honest, I've never used GRASS. I just booted it up, and am having trouble adding and displaying data myself. If you can, I would try QGIS instead; it's quite a bit more user-friendly, and still free.

One issue I was running into with GRASS just now was the projections. I had to set a projection for my project to start with, and then it didn't like it when I tried adding data that wasn't in that projection. QGIS (and ArcGIS) will reproject "on the fly," making it easier to add different data sets in the same map.


Projections define how a spherical Earth is projected onto a flat map. We've all seen various world maps with different projections, each with a different distortion of size and/or direction. I believe QGIS, like ArcGIS, will set its map projection to match the first data you load into a project, and then reproject anything else you add (if it's different) on the fly, so as to match. It slows things down a little to have these on the fly projections happening, but not terribly so in most cases.


Anyway, in QGIS, adding data to your map is easy, and can be done in a few ways. My preference is to use the Browser Panel--basically a file tree just like File Manager--to navigate to your downloaded data. Right click the shapefile and select "Add Layer," and it will show up in your map as well as in your Layers Panel. (If you don't see one or both panels, just go to View > Panels and turn both of those on).

[C2H5OH]

Originally Posted by Dirx

Try googling "[your county] GIS" and see what you find.

I tried that. Everything on the first page of results was either a broken piece of crap or required a login.

[Dirx]

Hm.. I just checked,* and your county's online GIS map seems to be working fine. Sometimes they don't play nice with certain browsers. I'm using Firefox, but an older legacy version that can still run Flash etc. I'd suggest trying a couple different browsers. I'll PM a link to the map I found, to make sure I am looking in the right place.



*Not a stalker Just still had your address from the postcard exchange.

[Jaglavak]

Loading QGIS now, thank you.

[Jaglavak]

Sweet, that worked. Next question, what are your favorite sources of data? Also this talk of regexp script queries sounds interesting.

[Jules]

I did some rudimentary GIS mapping of small water systems last year when I was working for a nonprofit, and I really enjoyed it. I found myself very frustrated with my lack of knowledge, time, and funding to make it do what I knew it could do - basically what you just said you were doing, really. As an operator, thank you, thank you, thank you! I ran a system for a long time based on one printed set of maps from 1960 and my own knowledge that I supplemented with a commercial GPS unit. Very frustrating.

[Dirx]

Originally Posted by Jaglavak

Sweet, that worked. Next question, what are your favorite sources of data? Also this talk of regexp script queries sounds interesting.

First thing first is to see what, if anything, is available from your city, state, and county. Instead of searching just "[locality] GIS", which will often take you an interactive web map with little or no download capability, try some of these terms:
"___ GIS data"
"___ GIS shapefiles"*
"___ GIS gateway"
"___ GIS clearinghouse:
"___ GIS download"

*QGIS and ArcGIS can use geodatabases too (though as a search term, 'shapefiles' will usually generate better hits). These are kinda like zipped directories (extension .gdb) containing the equivalent of shapefiles and other things. Within the GIS software, you just navigate to the gdb as if it were any other folder, and add its data just like I described in my other post.

A really great resource is the NRCS Geospatial Data Gateway. (Click the big green Get Data button on the right). Lots of fun data to play with. Note that some of it is raster data, which is basically an image file, so it can't quite be manipulated quite like a shapefile, but it still provides a bunch of info.

Otherwise, finding other good data sets depends on what you're looking for. Search around online and see what's available. Government agencies, non-profits, and universities have the best chances of making data freely available; you will inevitably run into some data sets that are behind a paywall here and there.

You'd be surprised what you can find. When I was looking for examples of water network data, I came across a shapefile of fire hydrants at an Antarctic research station!

[HookerChemical]

I work with a fair amount of land data like elevation maps and land use data, which meet the definition of GIS, but it's rarely discussed in the same terms. It's also not nearly as interesting. I've never had to use Arcview or similar programs, though I sometimes wish I knew how to use them better.

One of the clients I do work for is doing a similar project of mapping old utility infrastructure. They're private, so their data isn't generally available. One of the issues they have is that some locations have pipe that's been in the ground for 100 years and nobody really knows what's there or where it is, only that something exists between the main and the customer.

How much of an issue is the legacy infrastructure for the utility work you do?

What's the best way to start learning how to use that kind of software?